JP2005234074A - Apparatus and method for information processing, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide more suitable information for a user by effectively using an environmental sound as to an apparatus and method for information processing enabled to effectively use the environmental sound generally regarded as noise, a recording medium, and a program. <P>SOLUTION: A sound source specification part 15 specifies the sound source of a collected sound and an information acquisition part 16 acquires information relating to the collected sound from an information database 17 based upon information representing the sound source specified by the sound source specification part 15. An image editing part 18 edits a photographed image by using the information that the information acquisition part 16 acquires and a recording control part 19 records the image edited by the image editing part 18 in a recording part 20 together with the collected sound. Based upon an inputted sound, information relating to the inputted sound can be acquired to provide more suitable information with the user. For example, this invention is applicable to a video camera which records a photographed image and a collected sound. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information processing device, an information processing method, a recording medium, and a program, and in particular, an information processing device, an information processing method, and a recording that enable effective use of environmental sound that is generally regarded as noise. The present invention relates to a medium and a program.

  For example, in language speech recognition, noise is removed from input speech, acoustic analysis such as spectrum analysis is performed, and speech recognition is performed using an acoustic model and a language model. The acoustic model is usually a statistical model of a voice feature amount pattern distribution in units of phonemes, and a Hidden Malkov Model is mainstream. The language model defines connection relationships between words. The language model generally used is a statistical model in which word chain statistics are extracted from a text database, and the connection relationship between words is given as a probability value (see, for example, Non-Patent Document 1).

  In addition, as a method for removing noise from speech, that is, a method for distinguishing noise from speech speech, for example, there is a method using LPC (linear prediction) analysis.

  On the other hand, recently, image editing can be performed using, for example, an image editing function installed in a PC (Personal Computer) or the like. Further, it is possible to edit an image recorded in a flash memory or the like using an image editing function installed in the video camera body.

  However, as a precondition, the user himself / herself has to learn the function and the editing operation method of the apparatus. Furthermore, since it takes time to perform the editing work, it may be troublesome for the user.

  Therefore, a camera device that synthesizes text data obtained by voice recognition and a photographed image as image editing has been proposed (see, for example, Patent Document 1).

IPSJ MAGAZINE Vol.41 No.4 Apr.2000 P436-439

JP 2001-275031 A

  In conventional speech recognition, speech recognition is mainly used for speech, and speech other than speech is treated as noise and is subject to removal, so speech other than speech (environmental sound) is not effectively used. It was.

  This invention is made | formed in view of such a condition, and makes it possible to provide a user with more suitable information by utilizing an environmental sound effectively.

  An information processing apparatus according to the present invention includes a sound source specifying unit that specifies a sound source of an input sound, an information acquisition unit that acquires information related to sound based on sound source information indicating the sound source specified by the sound source specifying unit, It is characterized by providing.

  The information processing apparatus according to the present invention may further include a recording unit that records the information acquired by the information acquisition unit together with the voice.

  The information processing apparatus according to the present invention further includes language speech recognition means for recognizing language speech included in speech and outputting text data corresponding to the language speech, and the recording means records the text data together with the speech. can do.

  An information processing apparatus according to the present invention includes an imaging unit that captures an image, an image editing unit that edits an image acquired by the imaging unit using information acquired by the information acquisition unit, and editing by the image editing unit. And a recording means for recording the edited image together with the sound.

  The information processing apparatus of the present invention further includes language speech recognition means for recognizing language speech included in speech and outputting text data corresponding to the language speech, and the image editing means uses the image as text data. Can be edited.

  The information processing apparatus according to the present invention may further include a mail creating unit that creates data of electronic mail using the information acquired by the information acquiring unit.

  The information processing apparatus of the present invention further includes language speech recognition means for recognizing language speech included in speech and outputting text data corresponding to the language speech, and the mail creation means converts the email data into text data. Can also be created using.

  The information processing apparatus of the present invention can further include display means for displaying information acquired by the information acquisition means.

  The sound source specifying means can specify the sound source by causing an external device connected via the network to process the sound.

The information acquisition means can acquire information by communicating via a network.

  The information processing method of the present invention includes a sound source specifying step for specifying a sound source of input sound, and an information acquisition step for acquiring information related to sound based on sound source information indicating the sound source specified by the sound source specifying process. It is characterized by including.

  The program recorded in the recording medium of the present invention includes a sound source specifying step for specifying a sound source of input sound, and sound-related information based on sound source information indicating the sound source specified by the processing of the sound source specifying step. And an information acquisition step of acquiring.

  The program of the present invention includes a sound source specifying step for specifying a sound source of input sound, an information acquisition step for acquiring information related to sound based on sound source information indicating the sound source specified by the processing of the sound source specifying step, Is executed by a computer.

  In the present invention, a sound source of the input sound is specified, and information related to the input sound is acquired based on sound source information indicating the sound source.

  Note that a network is a mechanism in which at least two devices are connected and information can be transmitted from one device to another device. The devices that communicate via the network may be independent devices, or may be internal blocks that constitute one device.

  The communication is not only wireless communication and wired communication, but also communication in which wireless communication and wired communication are mixed, that is, wireless communication is performed in a certain section and wired communication is performed in another section. May be. Further, communication from one device to another device may be performed by wired communication, and communication from another device to one device may be performed by wireless communication.

  ADVANTAGE OF THE INVENTION According to this invention, the information relevant to the input audio | voice can be acquired based on the input audio | voice, and more suitable information can be provided with respect to a user.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Therefore, even if there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

  The information processing apparatus according to claim 1 is based on sound source specifying means (for example, the sound source specifying unit 15 in FIG. 1) for specifying the sound source of the input sound and sound source information indicating the sound source specified by the sound source specifying means. And information acquisition means (for example, the information acquisition unit 16 in FIG. 1) for acquiring information related to voice.

  The information processing apparatus according to claim 2 includes a recording unit (for example, the recording control unit 19 in FIG. 1) that records the information acquired by the information acquisition unit (for example, the information acquisition unit 16 in FIG. 1) together with the voice. It is further provided with the feature.

  The information processing apparatus according to claim 3 further includes a linguistic speech recognition unit (for example, the linguistic speech recognition unit 14 in FIG. 1) that recognizes linguistic speech included in speech and outputs text data corresponding to the linguistic speech. The recording means (for example, the recording control unit 19 in FIG. 1) records the text data together with the voice.

  The information processing apparatus according to claim 4 includes an imaging unit that captures an image (for example, the image input unit 12 in FIG. 1), and an information acquisition unit (for example, the information acquisition unit in FIG. Image editing means (for example, the image editing section 18 in FIG. 1) that edits using the information acquired by the section 16), and recording means (for example, FIG. 1) that records the edited image obtained by editing by the image editing means together with sound. 1 recording control unit 19).

  The information processing apparatus according to claim 5 further includes language speech recognition means (for example, the language speech recognition unit 14 in FIG. 1) that recognizes language speech included in speech and outputs text data corresponding to the language speech. The image editing means (for example, the image editing unit 18 in FIG. 1) edits the image using text data.

  The information processing apparatus according to claim 6 is a mail creation unit (for example, FIG. 11) that creates e-mail data using information acquired by the information acquisition unit (for example, the information acquisition unit 16 in FIG. 11). It is further characterized by further comprising a mail creating unit 51).

  The information processing apparatus according to claim 7 further includes a linguistic voice recognition unit (for example, the linguistic voice recognition unit 14 in FIG. 11) that recognizes linguistic voice included in the voice and outputs text data corresponding to the linguistic voice. The mail creation means (for example, the mail creation unit 51 in FIG. 11) creates e-mail data using text data.

  The information processing apparatus according to claim 8 further includes a display unit (for example, the display control unit 81 in FIG. 17) that displays information acquired by the information acquisition unit (for example, the information acquisition unit 16 in FIG. 17). Features.

  The information processing apparatus according to claim 9 specifies a sound source by a sound source specifying unit (for example, the sound source specifying unit 15 in FIG. 1) by causing an external device connected via a network to process the sound. It is characterized by that.

  The information processing apparatus according to claim 10 is characterized in that information acquisition means (for example, the information acquisition unit 16 in FIG. 1) acquires information by performing communication via a network.

  The information processing method according to claim 11 is a sound source specifying step (for example, the process of step S3 in FIG. 8) for specifying the sound source of the input voice and the sound source information indicating the sound source specified by the process of the sound source specifying step. The information acquisition step (for example, the process of step S4 of FIG. 8) which acquires the information relevant to audio | voice based on this is characterized by the above-mentioned.

  The program recorded on the recording medium according to claim 12 and the program according to claim 13 include a sound source specifying step (for example, the process of step S3 in FIG. 8) for specifying the sound source of the input sound, The information acquisition step (for example, the process of step S4 of FIG. 8) which acquires the information relevant to a sound based on the sound source information which shows the sound source identified by the process of a sound source identification step is characterized by the above-mentioned.

  This program can be recorded on a recording medium (for example, the magnetic disk 111 in FIG. 20).

  FIG. 1 is a block diagram showing a configuration example of an embodiment of a video camera to which the present invention is applied.

  The video camera 10 includes an audio input unit 11, an image input unit 12, an audio recognition unit 13, an information acquisition unit 16, an information database 17, an image editing unit 18, a recording control unit 19, and a recording medium 20.

  The voice input unit 11 acquires voice data by collecting voice and supplies the voice data to the language sound recognition unit 14, the sound source identification unit 15, and the recording control unit 19. The voice input unit 11 is composed of, for example, a microphone.

  The image input unit 12 acquires the captured image data by capturing an image of the capturing target, and supplies the captured image data to the image editing unit 18. The image input unit 12 includes, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) imager, or the like.

  The voice recognition unit 13 includes a language sound recognition unit 14 and a sound source identification unit 15.

  The language sound recognition unit 14 performs speech recognition processing on the language sound included in the sound data supplied from the sound input unit 11 and supplies text data corresponding to the language sound to the image editing unit 18.

  The sound source specifying unit 15 specifies a sound source of sound included in the sound data supplied from the sound input unit 11 and supplies sound source information representing the sound source to the information acquisition unit 16.

  Based on the sound source information supplied from the sound source specifying unit 15, the information acquisition unit 16 acquires information related to the sound input to the sound input unit 11 from the information database 17 and supplies the information to the image editing unit 18.

  The information database 17 includes information related to various sounds (for example, images and text information directly related to a specific sound (sound source), and an image (for example, an effect) that effectively represents a scene in which the specific sound occurs. (Image or text information indirectly related to a specific sound, such as an image to be applied (effect image)) is stored in association with sound source information (for example, a keyword such as “jet coaster”).

  The image editing unit 18 edits the captured image data supplied from the image input unit 12. That is, this combines the captured image data from the image editing unit 18 with the text data supplied from the language / speech recognition unit 14 or the information supplied from the information acquisition unit 16, and the resulting composite image data is obtained. , And supplied to the recording control unit 19.

  The recording control unit 19 controls to record the audio data supplied from the audio input unit 11 and the composite image data supplied from the image editing unit 18 on the recording medium 20.

  The recording medium 20 is composed of, for example, a magnetic tape, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like.

  FIG. 2 shows a configuration example of the sound source identification unit 15 of FIG. The sound source identification unit 15 includes an FFT processing unit 31, a data comparison unit 32, and a sound source identification database 33.

  The FFT processing unit 31 obtains a frequency spectrum by performing FFT (Fast Fourier Transform) processing on the audio data supplied from the audio input unit 11 and supplies the frequency spectrum to the data comparison unit 32.

  The data comparison unit 32 compares the frequency spectrum supplied from the FFT processing unit 31 with the frequency spectrum stored in the sound source identification database 33 and represents sound source information representing the sound source of the audio data supplied from the audio input unit 11. Is output.

  The sound source identification database 33 stores frequency spectrum models (frequency spectrum models) corresponding to various sound sources. In addition, the sound source specifying unit 33 may also store data related to the reflected sound in order to specify other places (environments) such as indoors, outdoors, and in the car.

  Here, the outline of the processing of the sound source identification unit 15 will be described with reference to FIG. For example, when a roller coaster is photographed by the video camera 10 in FIG. 1 at an amusement park, the sound input unit 11 collects the wheel sound and other sounds of the roller coaster. The sound source specifying unit 15 uses, as a sound source, the sound collected by the sound input unit 11 (hereinafter referred to as sound collection sound as appropriate), for example, the wheel sound of a roller coaster as shown in FIG. It is specified that the amusement park BGM, other sounds of the amusement park, and the narration comment of the photographer are included, and the sound source information representing each sound source is supplied to the information acquisition unit 16.

  The sound source specifying unit 15 specifies a sound source by, for example, comparing frequency spectra as described below.

  That is, the sound source specifying unit 15 uses the frequency spectrum (model) stored in the sound source specifying database 33 as a frequency spectrum obtained by performing FFT (Fast Fourier Transform) processing on the sound data from the sound input unit 11. The sound source is identified by comparing with.

  FIG. 4 shows a frequency axis spectrum obtained by performing FFT processing on the sound of a certain sound source. In FIG. 4, the horizontal axis represents frequency and the vertical axis represents intensity. For example, the sound source specifying unit 15 specifies a sound source by comparing the frequency components protruding in the frequency spectrum shown in FIG. 4 with frequency spectra corresponding to various sound sources stored in the sound source specifying database 33. .

  Note that the comparison between frequency components (frequency spectra) may be performed simply, for example, by dividing the audible frequency band into 16 or 32, and so on for each divided frequency band.

  The sound source specifying unit 15 can also specify a sound source by a time change of the frequency spectrum. The sound source specifying unit 15 extracts, for example, a feature in which a frequency at a specific level rises or falls with time from the sound from the sound input unit 11, and the feature is extracted from the sound source specifying database 33. The sound source can be specified by comparing with the frequency spectrum stored in. Here, FIG. 5 shows a frequency spectrum in which the frequency at a specific level increases.

  Further, in the sound source specifying unit 15, the time change of the frequency spectrum of the sound from the sound input unit 11 as shown in FIG. 6, and the distribution and intensity of the harmonic component thereof are stored in the sound source specifying database 33. The sound source can be specified by comparing with a frequency spectrum.

  Further, in the sound source specifying unit 15, the time change of the frequency spectrum of the sound from the sound input unit 11 and the attenuation characteristics with the passage of time as shown in FIG. 7 are stored in the frequency spectrum stored in the sound source specifying database 33. A sound source can be specified by comparison.

  FIG. 8 is a flowchart for explaining shooting and recording processing of the video camera 10 of FIG.

  In step S <b> 1, the voice input unit 11 acquires voice data by collecting voice and supplies the voice data to the language voice recognition unit 14, the sound source identification unit 15, and the recording control unit 19. In addition, the image input unit 12 acquires captured image data by capturing a subject to be captured, and supplies the captured image data to the image editing unit 18.

  Proceeding from step S1 to step S2, the language speech recognition unit 14 converts the speech data supplied from the speech input unit 11 into corresponding text data using a general language speech recognition technology, and the image editing unit 18. To step S3.

  In step S3, the sound source identification unit 15 (FIG. 2) performs FFT processing on the audio data supplied from the audio input unit 11 in the FFT processing unit 31, converts the audio data into a frequency spectrum, and performs data processing. It supplies to the comparison part 32.

  Further, in step S3, the data comparison unit 32 compares the frequency spectrum supplied from the FFT processing unit 31 with frequency spectra (models) corresponding to various sound sources stored in the sound source identification database 33, and compares them. Based on the result, the sound source of the sound input to the sound input unit 11 is specified. The data comparison unit 32 supplies sound source information indicating the identified sound source to the information acquisition unit 16, and proceeds to step S4.

  In step S4, the information acquisition unit 16 collects sound from the information database 17 based on the sound source information supplied from the sound source identification unit 15 (the data comparison unit 32) (the sound input to the sound input unit 11). Is obtained and supplied to the image editing unit 18.

  Proceeding from step S4 to step S5, the image editing unit 18 supplies the text data supplied from the language sound recognition unit 14 and the information acquisition unit 16 to the captured image data supplied from the image input unit 12. The information is combined, the combined image data is supplied to the recording control unit 19, and the process proceeds to step S6.

  In step S6, the recording control unit 19 records both the audio data supplied from the audio input unit 11 and the composite image data supplied from the image editing unit 18 on the recording medium 20, and returns to step S1 to perform shooting recording processing. repeat.

  In the above-described case, the composite image data obtained by combining the captured image data as an edit is recorded. However, information or text (telop) data to be combined with the captured image data is recorded in the captured image data. In addition, recording may be performed in association with audio data, and may be combined with captured image data during reproduction. In this case, reproduction of captured image data or composite image data can be selected by a user operation.

  Further, the process of acquiring information related to the collected sound and the process of synthesizing the acquired information with captured image data can be performed after shooting (recording).

  FIG. 9 is a diagram showing an outline of the first processing of the video camera 10 of FIG.

  A photographed image 41 shows an image obtained by photographing with the video camera 10. That is, the photographed image 41 is an image obtained by photographing a roller coaster at an amusement park, for example.

  The effect image 42 indicates an image as information acquired from the information database 17 based on sound source information indicating a sound source of sound included in the collected sound when the captured image 41 is captured by the video camera 10. The effect image 42 is an image that gives an image of fear when riding the roller coaster displayed in the captured image 41, for example.

  In the video camera 10 of FIG. 1, the effect image 42 is synthesized with the captured image 41, and as a result, a synthesized image 43 is obtained.

  FIG. 10 is a diagram showing an outline of the second processing of the video camera 10 of FIG.

  The photographed image 46 shows an image obtained by photographing with the video camera 10. That is, the photographed image 46 is an image obtained by photographing a racing car in an F1 (Formula 1) race, for example.

  The information image 47 shows an image as information acquired from the information database 17 based on the sound source information indicating the sound source of the sound included in the collected sound by the video camera 10. The information image 47 includes, for example, a machine image specified from the engine sound of the racing car displayed in the captured image 46, text information describing detailed information, and the like.

  In the video camera 10 of FIG. 1, the information image 47 is synthesized with the captured image 46 to obtain a synthesized image 48.

  As described above, the video camera 10 performs image editing such as synthesizing the effect image and the information image with the captured image in accordance with the sound collected at the time of shooting, so that the user can save time and effort for editing.

  In addition, at the time of viewing, editing that is not predicted by the user is added to the captured image, so that even more enjoyment can be given to the user.

  Furthermore, since the image and text information related to the specific sound included in the collected voice are synthesized, more useful information can be provided to the user.

  Note that the above-described processing can acquire images and sound in the same manner as a video camera. For example, the processing can be performed by a digital still camera, a television receiver, or the like. Can be obtained.

  In the above-described case, the voice recognition unit 13, the information acquisition unit 16, and the image editing unit 18 are mounted in the video camera 10, but the voice recognition unit 13, the information acquisition unit 16, or the image editing unit 18 is mounted. One or more of them are mounted on an external device (external device), and the video camera 10 communicates with the external device via a network, so that the voice recognition unit 13, the information acquisition unit 16, or the image editing is performed. The processing performed by the unit 18 may be executed by an external device. The external device is, for example, a PC or a server.

  In the above case, the information acquisition unit 16 acquires information from the information database 17 built in the video camera 10 body. However, the information acquisition unit 16 receives information from an external device via a network. The information may be acquired, or information may be acquired from a removable medium that can be attached to and detached from the video camera 10, for example, a flash memory.

  FIG. 11 is a block diagram showing a configuration example of an embodiment of a mobile phone to which the present invention is applied. 11, parts corresponding to those in the video camera 10 of FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The cellular phone 50 includes a voice input unit 11, a voice recognition unit 13, an information acquisition unit 16, an information database 17, a mail creation unit 51, and a communication unit 52.

  The language voice recognition unit 14 supplies text data obtained by voice recognition of the voice data supplied from the voice input unit 11 to the mail creation unit 51.

  The information acquisition unit 16 supplies information obtained from the information database 17 to the mail creation unit 51 based on the sound source information supplied by the sound source identification unit 15.

  The mail creation unit 51 creates a mail (e-mail) based on the text data supplied from the language speech recognition unit 14 and the information supplied from the information acquisition unit 16, and sends the mail to the communication unit 52. Supply data.

  The communication unit 52 transmits the mail data supplied from the mail creation unit 51 to the outside (call partner) via a network such as the Internet.

  Here, in the present embodiment of FIG. 11, the voice recognition unit 13, the information acquisition unit 16, and the mail creation unit 51 are implemented in the mobile phone 50, but the voice recognition unit 13, the information acquisition unit 16, or The mail creation unit 51 can be mounted on an external device, for example, a server installed in a data center (base station or control center) of a mobile phone company. In this case, necessary data may be transmitted from the mobile phone 50 to the data center, and the processing performed by the voice recognition unit 13, the information acquisition unit 16, or the mail creation unit 51 may be performed in the data center.

  In the present embodiment of FIG. 11, the information acquisition unit 16 acquires information from the information database 17 installed in the mobile phone 50. For example, a data center (base station or control) of a mobile phone company is used. Information may be acquired from a server installed in the center) or information may be acquired from a removable medium that can be attached to and detached from the mobile phone 50, for example, a flash memory.

  FIG. 12 is a flowchart for explaining processing in the mobile phone 50 of FIG.

  In step S21, the voice input unit 11 acquires voice data, supplies the voice data to the voice recognition unit 13, and proceeds to step S22. The voice data is supplied to the voice recognition unit 13 and transmitted to the base station as telephone voice data.

  The process according to the flowchart of FIG. 12 is started when, for example, the mobile phone 50 starts communication with another mobile phone. That is, in step S <b> 22, the language speech recognition unit 14 recognizes the speech data supplied from the speech input unit 11, converts it into text data, and supplies it to the mail creation unit 51.

  Thereafter, in step S23, as in step S3 of FIG. 8, the sound source specifying unit 15 specifies the sound source of the sound input to the sound input unit 11, and the sound source information representing the sound source is obtained from the information acquisition unit 16. To proceed to step S24. In step S24, the information acquisition unit 16 acquires an image related to the collected sound (sound input to the sound input unit 11) from the information database 17 based on the sound source information supplied from the sound source specifying unit 15. To the mail creation unit 51.

  Proceeding from step S24 to step S25, the mail creating unit 51 determines whether or not the call has ended. If it is determined in step S25 that the call is in progress (the call has not ended), the process returns to step S21, and the processes in steps S21 to S25 are repeated.

  If it is determined in step S25 that the call has ended, the process proceeds to step S26, where the mail creation unit 51 receives the text data supplied from the language speech recognition unit 14 and the image supplied from the information acquisition unit 16. A mail is originally created and the mail data of the mail is supplied to the communication unit 52.

  Here, the mail creation unit 51 uses the text data from the language speech recognition unit 14 as, for example, the mail body of the mail. The mail creation unit 51 also uses the image supplied from the information acquisition unit 16 as an email attachment file. For example, an image that is an attached file can be used as a background image of a mail text when a mail is opened.

  Proceeding from step S26 to step S27, the communication unit 52 transmits the mail data supplied from the mail creation unit 51 to the other party via the network, and ends the process.

  In the embodiment shown in FIG. 12, the mail is created using the voice collected during the call. However, when the user performs the operation of creating the mail, the voice is input to the voice input unit 11. An image related to the voice may be acquired based on the voice, and the image may be attached to an email created by the user.

  FIG. 13 shows a display example of the mail on a mobile phone (hereinafter, appropriately referred to as a receiver) that has received the mail (mail data) transmitted by the mobile phone 50 of FIG.

  The leftmost part of FIG. 13 is a display example of the screen of the receiver that has received the mail when the user of the mobile phone 50 of FIG. 11 makes a call or creates a mail at the amusement park.

  In the far left of Figure 13, along with the text data of "Hello! Today has come to play in the amusement park!", As its background, amusement park of the image is displayed. Therefore, the user of the receiver can recognize that the user of the mobile phone 50 is in the amusement park.

  The second (center) from the left in FIG. 13 is a display example of the screen of the receiver that has received the mail when the user of the mobile phone 50 in FIG. 11 makes a call or creates a mail at the banquet hall. .

  In the second from the left in FIG. 13, an image of the banquet hall is displayed as the background along with the text data “Please come to the party now!”. Therefore, the user of the receiver can recognize that the user of the mobile phone 50 wants the user of the receiver to come to the party.

  The third from the left in FIG. 13 (the rightmost) is the screen of the receiver that has received the mail when the user of the mobile phone 50 in FIG. Is a display example.

  In the third part from the left in FIG. 13, along with text data “I will get there in 15 minutes”, an image of the road viewed from the inside of the running car is displayed as the background. Therefore, the user of the receiver can recognize that the user of the mobile phone 50 is in the traveling car.

  Thus, the mobile phone 50 can transmit information to the communication partner more effectively.

  FIG. 14 is an external view showing a configuration example of an embodiment of an IC recorder system to which the present invention is applied.

  The IC recorder 60 collects sound and specifies the sound source of the collected sound. Further, the IC recorder 60 acquires an image 62 related to the collected sound based on the sound source information indicating the specified sound source. Further, the IC recorder 60 recognizes a language voice included in the collected voice, converts the voice to text data, and records it together with the image 62 and the collected voice.

  Here, as the image 62, for example, an image (for example, an image of a conference room) representing an environment in which sound is collected by an IC recorder is acquired.

  When the information recorded by the recording process of the IC recorder 60 is output to the PC 61, for example, the display of the PC 61 recognizes the image 62 related to the sound collected by the IC recorder 60 and the collected sound by voice recognition. A text screen 63 on which the obtained text data is displayed is displayed.

  FIG. 15 is a block diagram illustrating an internal configuration example of the IC recorder 60 of FIG. 15, parts corresponding to those in the video camera 10 of FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The IC recorder 60 includes a voice input unit 11, a voice recognition unit 13, an information acquisition unit 16, an information database 17, a recording control unit 66, and a recording medium 67.

  The voice input unit 11 acquires voice data by collecting voice and supplies the voice data to the language sound recognition unit 14, the sound source identification unit 15, and the recording control unit 66.

  The language voice recognition unit 14 supplies text data obtained by voice recognition of the voice data supplied from the voice input unit 11 to the recording control unit 66.

  The information acquisition unit 16 supplies the information acquired from the information database 17 to the recording control unit 66 based on the sound source information supplied by the sound source identification unit 15.

  The recording control unit 66 performs control to record the voice data supplied from the voice input unit 11 on the recording medium 67 together with the text data supplied from the language voice recognition unit 14 and the information supplied from the information acquisition unit 16. . As the recording medium 67, for example, a flash memory or the like can be used.

  In the embodiment of FIG. 15, the voice recognition unit 13 and the information acquisition unit 16 are mounted in the IC recorder 60, but the voice recognition unit 13 or the information acquisition unit 16 is an external device (external It is also possible to cause the external device to execute processing performed by the voice recognition unit 13 or the information acquisition unit 16 when the IC recorder 60 communicates with the external device via a network. The external device is, for example, a PC or a server.

  In the embodiment of FIG. 15, the information acquisition unit 16 acquires information from the information database 17 built in the main body of the IC recorder 60. However, the information acquisition unit 16 is connected to an external device via a network. Information may be acquired, or information may be acquired from a removable medium that can be attached to and detached from the IC recorder 60, such as a flash memory.

  FIG. 16 is a flowchart for explaining the recording process of the IC recorder 60.

  In step S41, the voice input unit 11 acquires voice data, supplies the voice data to the voice recognition unit 13 and the recording control unit 66, and proceeds to step S42.

  In step S <b> 42, the language voice recognition unit 14 converts the voice data supplied from the voice input unit 11 into text data by voice recognition, and supplies the text data to the recording control unit 66.

  Thereafter, in step S43, as in step S3 of FIG. 8, the sound source specifying unit 15 specifies the sound source of the sound input to the sound input unit 11, and the sound source information representing the sound source is sent to the information acquisition unit 16. The process proceeds to step S44. In step S44, based on the sound source information supplied from the sound source specifying unit 15, the information acquisition unit 16 collects sound from the information database 17 such as an image related to the collected sound (for example, an image of a station home). An image that gives an image of the environment) and supplies it to the information recording unit 66.

  Proceeding from step S44 to step S45, the recording control unit 66 stores the text data supplied from the language speech recognition unit 14 and the image data supplied from the information acquisition unit 16 in the recording medium 67 in association with the voice data, Returning to step S41, the recording process is repeated.

  As described above, the IC recorder 60 records the collected sound together with the image that makes the sound collection environment image, so that the user can easily grasp the contents and the recording situation when the user confirms the recorded contents later. .

  Further, in the IC recorder 60, for example, when the state during the meeting is recorded, the recorded content can be output to the PC as a minutes. In addition, the IC recorder 60 can specify a speaker based on a pre-registered voice, and can add an image representing a speaker's emotion and facial expression based on the tone of the voice at the time of speaking. It is.

  FIG. 17 is a block diagram showing a configuration example of another embodiment of a mobile phone to which the present invention is applied. 17, parts corresponding to those in the video camera 10 of FIG. 1 are denoted by the same reference numerals, and description thereof is omitted as appropriate.

  The cellular phone 80 includes a voice input unit 11, a sound source identification unit 15, an information acquisition unit 16, an information database 17, a display control unit 81, and a display unit 82.

  The information acquisition unit 16 supplies the information acquired from the information database 17 to the display control unit 81 based on the sound source information supplied from the sound source specifying unit 15.

  The display control unit 81 supplies the information supplied from the information acquisition unit 16 to the display unit 82 for display.

  The display unit 82 is configured by, for example, an LCD (Liquid Crystal Display), and displays an image from the display control unit 81 as a so-called standby screen image.

  In the embodiment of FIG. 17, the sound source identification unit 15 and the information acquisition unit 16 are mounted inside the mobile phone 80, but the sound source identification unit 15 or the information acquisition unit 16 is an external device (external device). And the mobile phone 80 may cause the external device to execute the processing performed by the sound source identification unit 15 or the information acquisition unit 16 by communicating with the external device via the network. The external device is, for example, a PC or a server.

  In the embodiment of FIG. 17, the information acquisition unit 16 acquires information from the information database 17 built in the mobile phone 80 body. However, the information acquisition unit 16 receives information from an external device via a network. Information may be acquired, or information may be acquired from a removable medium that can be attached to and detached from the mobile phone 80, such as a flash memory.

  FIG. 18 is a flowchart showing a standby screen display process for displaying a standby screen, which is performed by the mobile phone 80 of FIG.

  In step S61, the voice input unit 11 acquires voice data, supplies the voice data to the sound source specifying unit 15, and proceeds to step S62.

  After that, in step S62, as in step S3 of FIG. 8, the sound source specifying unit 15 specifies the sound source of the sound input to the sound input unit 11, and the sound source information representing the sound source is sent to the information acquisition unit 16. The process proceeds to step S63. In step S63, the information acquisition unit 16 acquires an image related to the collected sound from the information database 17 based on the sound source information supplied by the sound source specifying unit 15, and supplies the image to the display control unit 81 to be supplied to step S64. Proceed to

  In step S64, the display control unit 81 supplies the image supplied from the information acquisition unit 16 to the display unit 82 for display, and proceeds to step S65.

In step S65, the voice input unit 11 determines whether or not the current time is the set time. If it is determined in step S65 that the current time is not the set time, the time is continuously monitored. If it is determined that the current time is the set time, the process returns to step S61, and steps S61 to S65 are performed. Repeat the process.
In this case, in the mobile phone 80, the standby screen displayed on the display unit 82 is switched with the set time as a trigger (as a trigger).

  As for the set time, for example, a time period such as 1 hour cycle or morning (6:00), noon (12:00), night (18:00), midnight (0:00) can be set. it can. In the present embodiment, the standby screen display is switched according to the time. However, the voice input unit 11 is always turned on, and the standby screen display is switched in accordance with the change in the collected sound. May be.

  As described above, the mobile phone 80 can switch the display of the standby screen in response to a certain period, a set time, or a change in collected sound.

  FIG. 19 is a diagram illustrating a display example of a standby screen displayed on the display unit 82 of the mobile phone 80 of FIG.

  For example, when collecting warbler's calls at a certain point in time, the mobile phone 80 specifies that the collected voice is a warbler's call (the sound source of the collected voice is a warbler), and information related to the warbler. A certain information image 91 is acquired. Then, the cellular phone 80 displays the information image 91 on the display unit 82. Further, when the mobile phone 80 subsequently collects another bird's cry, it acquires an information image 92 that is information related to the other bird, and switches the display of the display unit 82 to the information image 92.

  In FIG. 19, the information image 91 and the information image 92 include information related to a bird such as the name (name) of the bird in addition to the bird image.

  In this case, the mobile phone 80 also plays a pictorial role.

  In the above description, the squeal of a bird has been described as an example. Of course, the same processing can be performed for a squeal other than a bird, and for example, an engine sound of an automobile or an airplane. In addition, in the mobile phone 80, for example, when the station home is at a certain time, an image of the station home can be displayed on the standby screen.

  For example, by storing the history of images displayed in a certain time range, it is possible to use the history to remember memories such as where the user was at a certain time. .

  Further, the images displayed on the display unit 82 can be limited to images belonging to a desired category based on user operations. Categories include, for example, landscapes, animals, and cars.

  The present invention is not limited to a video camera, an IC recorder, a mobile phone, a portable device having at least a sound collecting function such as a PDA (Personal Digital Assistants), a portable device having a display function, and the like. It is possible to apply.

  The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer 100 or the like.

  As shown in FIG. 20, the recording medium is distributed to provide a program to the user separately from the personal computer 100. The recording medium includes a magnetic disk 111 (including a flexible disk) and an optical disk 112 (including a flexible disk). CD-ROM (including Compact Disc-Read Only Memory) and DVD (Digital Versatile Disc)), magneto-optical disk 113 (including MD (Mini-Disc) (trademark)), or package media including semiconductor memory 114 In addition to being configured, it is configured by a ROM 102 on which a program is recorded and a hard disk included in the storage unit 108 provided to the user in a state of being preinstalled in a computer.

  The CPU 101 of the personal computer 100 controls the overall operation of the personal computer. Further, when an instruction is input from the input unit 106 such as a keyboard or a mouse from the user via the bus 104 and the input / output interface 105, the CPU 101 stores the instruction in a ROM (Read Only Memory) 102 correspondingly. Run the program. Alternatively, the CPU 101 reads a program read from the magnetic disk 111, the optical disk 112, the magneto-optical disk 113, or the semiconductor memory 114 connected to the drive 110 and installed in the storage unit 108 into a RAM (Random Access Memory) 103. To load and execute. Further, the CPU 101 controls the communication unit 109 to communicate with the outside and exchange data.

  The program for executing the series of processes described above is installed in a computer via a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via an interface such as a router or a modem as necessary. You may be made to do.

  Further, in the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

It is a block diagram which shows the structural example of one Embodiment of the video camera to which this invention is applied. It is a block diagram which shows the structural example of the sound source specific | specification part 15 of FIG. It is a figure which shows the outline | summary of the process in the sound source specific | specification part 15 of FIG. It is a figure which shows a frequency spectrum. It is a figure which shows the time change of a frequency spectrum. It is a figure which shows the time change of a frequency spectrum. It is a figure which shows the time change of a frequency spectrum. It is a flowchart explaining the imaging | photography recording process of the video camera 10 of FIG. It is a figure which shows the outline | summary of the process in the video camera 10 of FIG. It is a figure which shows the outline | summary of the process in the video camera 10 of FIG. It is a block diagram which shows the structural example of one Embodiment of the mobile telephone to which this invention is applied. 12 is a flowchart for explaining processing in the mobile phone 50 of FIG. It is a figure which shows the example of a display of the mail in the mobile telephone which received the mail which the mobile telephone 50 of FIG. 11 transmitted. 1 is an external view showing a configuration example of an embodiment of an IC recorder system to which the present invention is applied. It is a block diagram which shows the internal structural example of IC recorder 60 of FIG. It is a flowchart explaining the recording process in the IC recorder 60 of FIG. It is a block diagram which shows the structural example of other embodiment of the mobile telephone to which this invention is applied. 18 is a flowchart for explaining a standby screen display process of the mobile phone 80 of FIG. It is a figure which shows the outline | summary of a process of the mobile telephone 80 of FIG. It is a figure which shows the structural example of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Video camera, 11 Voice input part, 12 Image input part, 13 Voice recognition part, 14 Language speech recognition part, 15 Sound source specific part, 16 Information acquisition part, 17 Information database, 18 Image editing part, 19 Recording control part, 20 Recording medium, 31 FFT processing section, 32 data comparison section, 33 sound source identification database, 41 captured image, 42 effect image, 43 composite image, 46 captured image, 47 information image, 48 composite image, 50 mobile phone, 51 mail creation section , 52 communication unit, 60 IC recorder, 61 PC, 62 image, 63 text screen, 66 recording control unit, 67 recording medium, 80 mobile phone, 81 display control unit, 82 display unit, 91 information image, 92 information image, 100 Personal computer, 101 CPU, 102 ROM, 103 RAM, 104 internal bus, 105 input / output interface, 106 input section, 107 output section, 108 storage section, 109 communication section, 110 drive, 111 magnetic disk, 112 optical disk, 113 magneto-optical disk, 114 semiconductor memory

Claims (13)

A sound source identifying means for identifying the sound source of the input voice;
An information processing apparatus comprising: information acquisition means for acquiring information related to the sound based on sound source information indicating the sound source specified by the sound source specifying means. The information processing apparatus according to claim 1, further comprising a recording unit that records the information acquired by the information acquisition unit together with the sound. Language speech recognition means for recognizing language speech included in the speech and outputting text data corresponding to the language speech;
The information processing apparatus according to claim 2, wherein the recording unit records the text data together with the voice. Photographing means for photographing an image;
Image editing means for editing the image acquired by the photographing means using the information acquired by the information acquiring means;
The information processing apparatus according to claim 1, further comprising: a recording unit that records an edited image obtained by editing by the image editing unit together with the sound. Language speech recognition means for recognizing language speech included in the speech and outputting text data corresponding to the language speech;
The information processing apparatus according to claim 4, wherein the image editing unit edits the image using the text data. The information processing apparatus according to claim 1, further comprising: a mail creation unit that creates e-mail data using the information acquired by the information acquisition unit. Language speech recognition means for recognizing language speech included in the speech and outputting text data corresponding to the language speech;
The information processing apparatus according to claim 6, wherein the e-mail creating unit creates the e-mail data using the text data. The information processing apparatus according to claim 1, further comprising display means for displaying information acquired by the information acquisition means. The information processing apparatus according to claim 1, wherein the sound source specifying unit specifies the sound source by causing the external apparatus connected via a network to process the sound. The information processing apparatus according to claim 1, wherein the information acquisition unit acquires the information by performing communication via a network. A sound source identification step for identifying the sound source of the input voice;
An information acquisition method comprising: an information acquisition step of acquiring information related to the sound based on sound source information indicating the sound source specified by the sound source specifying step. A sound source identification step for identifying the sound source of the input voice;
An information acquisition step of acquiring information related to the sound based on sound source information indicating the sound source specified by the processing of the sound source specifying step. Recording medium. A sound source identification step for identifying the sound source of the input voice;
An information acquisition step of acquiring information related to the sound based on sound source information indicating the sound source specified by the processing of the sound source specifying step.

Images