JP2010266721A - Device and method for attaching label to sound data, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for attaching a label to sound data by which the label is attached to the sound data. <P>SOLUTION: The device includes: a sound data receiving unit 11 for receiving a sound data which is collected by a microphone at a position of a subject's position; a corresponding information memory 16 in which two or more pieces of corresponding information including a label and a label sound image which is a sound image for indicating intensity for each frequency of sound corresponding to the label, by relating it to each other; a sound image conversion unit 15 for converting the received sound data to the sound image for indicating intensity for each frequency; and a comparison unit 17 for comparing the sound image after conversion, with the label sound image which is stored in the corresponding information memory 16; and a label attaching unit 18 in which a label corresponding to a label sound image of highest similarity with the sound image after conversion, and which attaches the specified label to the sound data corresponding to the sound image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a sound data label attaching device that attaches a label to sound data at a position of a subject collected by a microphone.

  2. Description of the Related Art Conventionally, studies have been made to accumulate information related to individuals, such as My Life Bits and Life Log, and to use the accumulated information later. As a related technique, a system for accumulating image data obtained by photographing the periphery of an individual has been developed (for example, see Patent Document 1).

JP 2006-1115011 A

  As information related to an individual, for example, it is conceivable to store sound data around the individual. However, sound data has a problem that searchability is not as good as that of image data. For example, when a desired scene is searched from moving image data, a still image at a likely time is viewed, and it is determined whether the desired scene is earlier or later, and a moving image is determined according to the determination. A desired scene is searched by fast-forwarding or rewinding data. Judgment in that case can be made almost instantaneously by looking at the still image. On the other hand, when searching for the desired sound from the sound data, listen to the sound data at the applicable time, determine whether the desired sound is before or behind it, and fast forward the sound data according to the determination By searching or rewinding, a desired sound is searched. In this case, since it is necessary to listen to sound data for a certain time (for example, 5 seconds or 10 seconds), it takes much time. In addition, if a sound that is significant for the listener (for example, a person's voice, a telephone ringing tone, a train passing sound, etc.) is not heard, it takes more time to make the determination.

Therefore, in the case of sound data, it is preferable to give information that helps the search in advance. As the information, for example, it is effective to use information on a position where sound data is accumulated. As the position information, it may be possible to use position information of latitude and longitude acquired using GPS, but in that case, it is necessary to use a device that receives radio waves from GPS satellites separately. The equipment becomes a big deal. In the case of GPS, there is also a problem that it is difficult to use outdoors.
Generally speaking, there has been a demand for providing a label for helping search to sound data around a subject without using an extra device or the like.

  The present invention has been made in accordance with the above circumstances, and provides a sound data label attaching device and the like that can attach a label for helping search to sound data collected at the position of a subject. For the purpose.

  In order to achieve the above object, a sound data label applying apparatus according to the present invention includes a sound data receiving unit that receives sound data collected by a microphone at the position of a subject, and an intensity for each frequency of sound corresponding to the label and the label. A correspondence information storage unit storing two or more correspondence information, which is information having a label sound image corresponding to a sound image to be displayed, and a sound image indicating the intensity for each frequency of the sound data received by the sound data reception unit A sound image conversion unit that converts the sound image, a sound image converted by the sound image conversion unit, a comparison unit that compares the label sound image stored in the correspondence information storage unit, and a comparison result by the comparison unit, A label attaching unit that identifies a label corresponding to a label sound image having a high similarity to the sound image converted by the image conversion unit, and assigns the identified label to sound data corresponding to the sound image; Than is.

  With such a configuration, a label can be assigned to the sound data. For example, when the position of the subject is indicated by the label, the relationship between the position of the subject and the sound data can be easily known from the sound data to which the label is attached without using a device such as GPS. become able to. Accordingly, the position can be determined even indoors where it is difficult to use GPS. Further, by converting sound data into sound images and comparing them, it is possible to easily compare sound data having chords and sound data on which more complex various sounds are superimposed.

  In the sound data label assigning device according to the present invention, the sound data received by the sound data receiving unit includes environmental sound data that is sound data of the subject's environment. From the sound data received by the sound data receiving unit, The image processing apparatus may further include a separation unit that separates the sound data, and the sound image conversion unit may convert the environmental sound data separated by the separation unit into a sound image.

  With such a configuration, a label can be given using environmental sound data among sound data collected at the position of the subject. For example, when the position of the subject is indicated by the label, it is considered that a more appropriate label can be given by using the sound data of the subject's environment.

In the sound data label assigning device according to the present invention, the sound image conversion unit may convert sound data in a period having continuous frequency peaks into one sound image.
With such a configuration, for example, sound data in a period in which frequency peaks are continuous, such as a station melody or a store theme song, can be compared as one sound image. As a result, more accurate labeling can be performed.

In the sound data label assigning device according to the present invention, the comparison unit calculates similarity information that is information on the similarity between the sound image and the label sound image, and the label attaching unit The specified one or more labels and the similar information respectively corresponding to the specified one or more labels may be associated with each other and given to the sound data.
With such a configuration, for example, when two or more labels are given to sound data, the validity of each label can be determined using the similar information given to the sound data.

In the sound data label assigning device according to the present invention, the label attaching unit assigns each sound data with three or more labels and similar information corresponding to each of the three or more labels in association with each other. A position correspondence information storage unit for storing position correspondence information, which is information for associating coordinates of a sound output device that outputs a sound corresponding to a label, three or more labels and similar information given to sound data, and three or more A coordinate calculation unit that calculates coordinates corresponding to the sound data using coordinates corresponding to the labels, and a coordinate application unit that assigns the coordinates calculated by the coordinate calculation unit to the sound data corresponding to the coordinates. Further, it may be provided.
With such a configuration, it becomes possible to know a more accurate position of the subject using the sound output from the sound output device.

In the sound data label assigning device according to the present invention, the sound output from the sound output device may be a non-audible sound.
With such a configuration, the sound output from the sound output device can be prevented from becoming noise for humans.

  In the sound data label assigning device according to the present invention, the sound data storage unit storing the sound data received by the sound data receiving unit, and the erasure storing the label identification information for identifying the label corresponding to the sound data to be deleted A target label storage unit; and an erasure unit that erases, from the sound data storage unit, sound data related to the label identified by the label identification information stored in the erasure target label storage unit among the sound data to which the label is attached. Further, it may be provided.

  With such a configuration, for example, by storing the label identification information of the label corresponding to the sound data that you do not want to leave in the sound data storage unit in the deletion target label storage unit, the sound data is automatically deleted. Will come to be. For example, sound data that can be a violation of privacy can be automatically deleted.

  In the sound data label assigning device according to the present invention, the sound data storage unit storing the sound data received by the sound data receiving unit and the extraction for storing the label identification information for identifying the label corresponding to the sound data to be extracted are stored. A target label storage unit; and an extraction unit that extracts and accumulates sound data related to the label identified by the label identification information stored in the extraction target label storage unit among the sound data to which the label is attached. May be.

  With such a configuration, for example, by storing label identification information of a label corresponding to sound data to be extracted in the extraction target label storage unit, the sound data is automatically extracted.

  Note that the sound data storage unit storing the sound data to be erased and the sound data to be extracted, that is, the sound data storage unit storing the sound data received by the sound data receiving unit is the sound data receiving unit. The sound data storage unit may store the sound data after receiving the sound data. The sound data storage unit stores the sound data before the sound data reception unit receives the sound data. Also good.

  According to the sound data label assigning device or the like according to the present invention, a label can be assigned to sound data collected at the position of the subject. By using the label, for example, sound data can be searched, and desired sound data can be easily accessed.

The block diagram which shows the structure of the sound data label provision apparatus by Embodiment 1 of this invention. The flowchart which shows operation | movement of the sound data label provision apparatus by the embodiment The figure which shows an example of the correspondence information in the embodiment The figure which shows an example of the information memorize | stored in the deletion object label memory | storage part in the embodiment The figure which shows an example of the information memorize | stored in the extraction object label memory | storage part in the embodiment The figure which shows an example of the sound data memorize | stored in the sound data memory | storage part in the embodiment The figure which shows an example of the sound image in the embodiment The figure which shows an example of a response | compatibility with the time code and label in the embodiment The figure which shows an example of a response | compatibility with the time code and label in the embodiment The figure which shows an example of the sound image in which the peak exists in the embodiment The block diagram which shows the structure of the sound data label provision apparatus by Embodiment 2 of this invention. The flowchart which shows operation | movement of the sound data label provision apparatus by the embodiment The figure which shows an example of the correspondence information in the embodiment The figure which shows an example of the position corresponding | compatible information in the embodiment The figure which shows an example of a response | compatibility with the time code travel in the embodiment The figure which shows an example of a response | compatibility with the time code travel and coordinate in the embodiment The schematic diagram which shows an example of the external appearance of the computer system in each said embodiment The figure which shows an example of a structure of the computer system in each said embodiment

  Hereinafter, a sound data label applying apparatus according to the present invention will be described with reference to embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A sound data label applying apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings. The sound data label assigning apparatus according to the present embodiment attaches a label to sound data collected at the position of the subject. For example, the position of the subject is indicated by the label.

  FIG. 1 is a block diagram showing a configuration of a sound data label applying apparatus 1 according to this embodiment. The sound data label assigning apparatus 1 according to the present embodiment includes a sound data receiving unit 11, a separation unit 12, a storage unit 13, a sound data storage unit 14, a sound image conversion unit 15, and a correspondence information storage unit 16. The comparison unit 17, the label providing unit 18, the erasure target label storage unit 19, the erasure unit 20, the extraction target label storage unit 21, and the extraction unit 22 are provided.

  The sound data reception unit 11 receives sound data collected by a microphone at the position of the subject. The subject is a target from which ambient sound data is collected, and is mainly assumed to be an individual (human), but may be, for example, an animal or a robot that can act autonomously. In the present embodiment, a case where the subject is a human will be described. In addition, a microphone that collects sounds around the subject may be attached to the subject (for example, a hands-free microphone, a headset microphone, a clip microphone, a tie pin microphone, or the like), or according to the movement of the subject. It may be configured to move. The sound data collected by this microphone generally includes environmental sound data and other sound data. The environmental sound data is sound data of the subject's environment. As sound data other than the environmental sound data, for example, sound data generated by a subject (for example, speech of the subject, operation sound when the subject is a robot), sound data uttered by the subject's talking partner, etc. There is. Further, the number of microphones used for obtaining sound data may be one, or two or more. In the latter case, for example, a target microphone that collects the sound generated by the subject (for example, the microphone may be arranged at any position of the subject's mouth or the subject), and the sound generated by the subject and the environmental sound. And a target microphone (for example, may be placed on the subject's neck). In the latter case, for example, a stereo microphone may be used. The sound data receiving unit 11 may receive sound data collected by a microphone in real time, or may receive sound data collected by a microphone and recorded at once. The sound data received by the sound data receiving unit 11 is preferably digital data. This is because, unlike analog data recorded on a tape, a change caused by the elongation of the tape does not occur. In this embodiment, the case where the sound data is digital data will be described.

  As described above, the sound data receiving unit 11 may receive sound data input from a microphone, for example, or may receive sound data transmitted via a wired or wireless communication line. Sound data read from a recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be received. Note that the sound data reception unit 11 may or may not include a device (for example, a modem or a network card) for reception. The sound data receiving unit 11 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The separation unit 12 separates the environmental sound data from the sound data received by the sound data reception unit 11. This is because it is considered that an appropriate label relating to the position of the subject can be given by giving the label using the environmental sound data. Here, several methods for separating the environmental sound data will be described.

(1) Frequency cut It is generally known that a human voice has a frequency of 85 Hz to 8 kHz. Therefore, the separation unit 12 acquires environmental sound data that is sound data obtained by cutting (removing) the frequency component corresponding to the human voice out of the entire sound range of the sound data received by the sound data receiving unit 11. Also good. Needless to say, the frequency band corresponding to the human voice may be arbitrarily set.

(2) Separation by sound pressure When a microphone that collects sound data is located in the vicinity of the subject's mouth, for example, the sound pressure level of the sound uttered by the subject is considered to be high. Therefore, the separation unit 12 may remove sound data whose sound pressure exceeds the threshold value, and obtain environmental sound data that is sound data whose sound pressure is smaller than the threshold value. This threshold value may be a preset value or may be a value generated using the maximum sound pressure of the sound data received by the sound data receiving unit 11. In the latter case, the threshold value may be, for example, a value obtained by multiplying the maximum sound pressure by a value smaller than 1 (for example, 0.6 or 0.8).

(3) Separation using two channels As described above, the sound data is collected by two microphones and each of the two channels is one channel (this is referred to as “first channel”). ) Corresponds to the sound data uttered by the subject, and the other channel (referred to as “second channel”) corresponds to the sound data of the utterance of the subject and the environmental sound, the separation unit 12 Only the environmental sound data may be separated by subtracting the sound data of the first channel from the sound data of the second channel. Needless to say, at the time of the separation, the levels of the first and second channels may be appropriately adjusted so that the optimum separation can be performed.

(4) Removal of human voice In recent years, techniques for removing only human voice from music data with vocals have been developed. Therefore, the separation unit 12 acquires environmental sound data, which is sound data obtained by removing the components of the human voice, from the sound data received by the sound data receiving unit 11 using such a technique for removing the human voice. May be.

  Here, although several methods have been described in which the separation unit 12 separates the environmental sound data, it goes without saying that the environmental sound data may be separated using other methods.

  The accumulation unit 13 accumulates the sound data received by the sound data reception unit 11 in a sound data storage unit 14 to be described later. The storage unit 13 also stores sound data, which is environmental sound data separated by the separation unit 12, in the sound data storage unit 14.

  The sound data storage unit 14 stores the sound data received by the sound data reception unit 11. The sound data storage unit 14 also stores sound data that is environmental sound data separated by the separation unit 12. Further, it is preferable that a time code is associated with the sound data stored in the sound data storage unit 14. The time code may be set in advance in the sound data received by the sound data receiving unit 11 or may be set when the storage unit 13 stores the sound data in the sound data storage unit 14. Further, the time code may be set over all time zones of the sound data, or may be set only for a part of the start point, the end point, and the like. Even in the latter case, the time code can be known for all the time zones of the sound data by calculating the time of the difference from the position where the time code is set. The time code may indicate an absolute date and time or may indicate a relative time. In the former case, for example, the date and time acquired from a radio clock or the like may be used. Further, the time code of the sound data after separation and the time code of the sound data before separation are preferably synchronized (the same time code is associated with the same temporal position of the sound data). It is. As will be described later, label selection is performed using the separated sound data, and label assignment is performed on the sound data before separation. Storage in the sound data storage unit 14 may be temporary storage in a RAM or the like, or may be long-term storage. The sound data storage unit 14 is realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The sound image conversion unit 15 converts sound data, which is environmental sound data received by the sound data reception unit 11 and separated by the separation unit 12, into a sound image indicating the intensity for each frequency. For example, the sound image conversion unit 15 may convert the sound data stored in the sound data storage unit 14 into a sound image at certain time intervals. The time interval may or may not be constant. The sound image may be, for example, a spectrum image (power spectrum image) that is a result of Fourier transform of sound data, in which the horizontal axis is frequency and the vertical axis is signal intensity, the horizontal axis is time, and the vertical axis May be a spectrogram whose intensity is indicated by shading, color intensity, or the like, or an image showing the intensity of sound for other frequencies. The spectrogram may be generated using, for example, a plurality of bandpass filters, or may be generated by a short-time Fourier transform. A method for generating a sound image from sound data is already known, and a detailed description thereof will be omitted.

  In the correspondence information storage unit 16, two or more pieces of correspondence information are stored. The correspondence information is information having a label and a label sound image that is a sound image indicating the intensity of each sound frequency corresponding to the label in association with each other. The label is generally a label indicating a position (for example, “meeting room”, “corridor”, “working seat”, “break room”, etc.), but other labels may also be used. Here, “position” is not a position in a strict sense indicated by longitude, latitude, coordinates, etc., but can be considered as an attribute relating to the position. As a label indicating a position other than the position, for example, a label “erase trigger” indicating that it is to be erased by an erasure unit 20 described later, or a label “extraction trigger” indicating that it is an object of extraction by an extraction unit 22 described later. , There is a label “important” indicating that it is an important part. Since the label sound image is to be compared with the sound image converted by the sound image conversion unit 15, it is preferable that the label sound image is of the same type as the sound image. Therefore, when the sound data is converted into a sound image that is a spectrum image by the sound image conversion unit 15, the label sound image is also preferably a spectrum image. In addition, when the sound data is converted into a sound image that is a spectrogram by the sound image conversion unit 15, it is preferable that the label sound image is also a spectrogram. The label sound image is generated according to the corresponding label. For example, the label sound image corresponding to the label “meeting room” can be generated by recording the sound of the meeting room and converting the recorded sound data into a sound image. Note that the label sound image corresponding to the label other than the position such as the label “erase trigger” records a sound corresponding to the label (for example, a predetermined electronic sound, a sound of a finger, a clapping sound, etc.). The recorded sound data can be generated by converting it into a sound image.

  The process of storing two or more pieces of correspondence information in the correspondence information storage unit 16 does not matter. For example, two or more pieces of correspondence information may be stored in the correspondence information storage unit 16 via a recording medium, and two or more pieces of correspondence information transmitted via a communication line or the like may be stored in the correspondence information storage unit 16. The correspondence information may be stored, or two or more pieces of correspondence information input via the input device may be stored in the correspondence information storage unit 16. Storage in the correspondence information storage unit 16 may be temporary storage in a RAM or the like, or may be long-term storage. The correspondence information storage unit 16 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The comparison unit 17 compares the sound image converted by the sound image conversion unit 15 with the label sound image stored in the correspondence information storage unit 16. This comparison is performed in order to identify a label sound image similar to the sound image converted by the sound image conversion unit 15. Therefore, the comparison unit 17 may calculate similarity information that is information related to the similarity between the sound image converted by the sound image conversion unit 15 and the label sound image. The content of the similar information is not limited as long as it is information that shows the degree of similarity. The similarity information may be, for example, information having a larger value as the similarity is higher, or information having a larger value as the similarity is lower. A method for calculating information indicating the similarity of images is already known, and a detailed description thereof will be omitted. In the present embodiment, it is assumed that the comparison unit 17 calculates similar information between the sound image and the label sound image.

  The label assigning unit 18 specifies a label corresponding to the label sound image having a high similarity to the sound image converted by the sound image conversion unit 15 using the comparison result by the comparison unit 17, and uses the specified label as the sound image. To the sound data corresponding to. The sound data to which the label is attached is sound data received by the sound data receiving unit 11 and not separated by the separating unit 12. The “label sound image having a high similarity to the sound image” may be, for example, one label sound image having the highest similarity to the sound image, and from the one having the highest similarity to the sound image. It may be two or more label sound images, and may be one or two or more label sound images having a similarity higher than the threshold value. The threshold value may be a preset value or may be a value generated using a value indicating the maximum similarity calculated by the comparison unit 17. In the latter case, the threshold value may be, for example, a value obtained by multiplying a value indicating the maximum similarity by a value smaller than 1 (for example, 0.6 or 0.8). Thus, the label specified by the label applying unit 18 may be one, or two or more. In addition, giving a label to sound data means that it is only necessary to know the correspondence between the label and the temporal position of the sound data. Therefore, attaching a label to sound data means, for example, directly attaching the label to a corresponding portion of the sound data (for example, the sound data has a channel for labeling, and a label is set for the channel). In other words, a label is set for the time code (for example, the label “conference room” is set at the time code 00:00:00 to 00:05:35, and the time code 00:05:35 is set) ˜00: 06: 12, etc., and a time code may be set for the label (for example, the time code 00:00:00 to 00 for the label “meeting room”). : 05: 35, 01:47:56 to 02:43:45, etc.). In the present embodiment, the case where the label applied by the label applying unit 18 is stored in the sound data storage unit 14 is described, but this need not be the case. The label applied by the label applying unit 18 may be stored on another recording medium. Further, when adding one or more labels to the sound data, the label assigning unit 18 associates the one or more labels with the similar information corresponding to the one or more labels, respectively. It may be added to the sound data or not. The similarity information is calculated by the comparison unit 17. If there is no label sound image having a high similarity to the sound image, that is, if there is no label sound image having a similarity to the sound image higher than a preset threshold value, the label attaching unit 18 May add a label indicating that it is “unknown” or a label indicating that the label cannot be specified to the sound data.

  The erasure target label storage unit 19 stores label identification information for identifying a label corresponding to the erasure target sound data. The sound data corresponding to the label identified by the label identification information stored in the erasure target label storage unit 19 is erased by the erasure unit 20 described later. Therefore, the label identification information for identifying the label corresponding to the sound data to be erased for confidentiality reasons, privacy protection reasons, etc. is stored in the erasure target label storage unit 19 so that the label identification information is stored in the label identification information. The corresponding sound data can be automatically deleted. The label identification information may be any content as long as the label can be identified. For example, the label identification information may be the label itself, or a numerical value or symbol that identifies the label.

  The process in which the label identification information is stored in the erasure target label storage unit 19 does not matter. For example, the label identification information may be stored in the erasure target label storage unit 19 via a recording medium, and the label identification information transmitted via a communication line or the like is stored in the erasure target label storage unit 19. Alternatively, the label identification information input via the input device may be stored in the erasure target label storage unit 19. Storage in the erasure target label storage unit 19 may be temporary storage in a RAM or the like, or may be long-term storage. The erasure target label storage unit 19 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The erasure unit 20 erases the sound data related to the label identified by the label identification information stored in the erasure target label storage unit 19 from the sound data storage unit 14 among the sound data to which the label is attached. The sound data related to the label may be sound data in a range corresponding to the label, or sound data having a range in which at least one of the start point and the end point of the sound data range corresponding to the label is changed. . When the label “toilet” is given to the time code 00: 10: 0 to 00:15:35 and the label “toilet” is stored in the erasure target label storage unit 19, for example, the former Therefore, the sound data in the range of the time code 00: 10: 0 to 00:15:35 will be erased. In the latter case, the extended range of the time code (for example, 1 minute before and after) The sound data of the time code 00:09:00 to 00:16:35) is deleted. The degree of changing at least one of the start point and the end point in the latter case may be the same for all label identification information stored in the erasure target label storage unit 19 or may be different for each label identification information. Good. In the latter case, the degree of change may be stored in the erasure target label storage unit 19, or may be stored in another recording medium. Further, erasing the sound data may be to set the sound data to a silent level, or to delete the sound data itself from the sound data storage unit 14. However, even in the latter case, it is preferable to delete the time code itself so as not to be changed.

  The extraction target label storage unit 21 stores label identification information for identifying a label corresponding to the sound data to be extracted. The sound data corresponding to the label identified by the label identification information stored in the extraction target label storage unit 21 is extracted by the extraction unit 22 described later. Therefore, by storing in the extraction target label storage unit 21 label identification information for identifying a label corresponding to sound data to be extracted in advance, the sound data corresponding to the label identification information can be automatically extracted.

  The process in which the label identification information is stored in the extraction target label storage unit 21 does not matter. For example, label identification information may be stored in the extraction target label storage unit 21 via a recording medium, and label identification information transmitted via a communication line or the like is stored in the extraction target label storage unit 21. Alternatively, label identification information input via an input device may be stored in the extraction target label storage unit 21. Storage in the extraction target label storage unit 21 may be temporary storage in a RAM or the like, or may be long-term storage. The extraction target label storage unit 21 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, or the like).

  The extraction unit 22 extracts, from the sound data storage unit 14, the sound data related to the label identified by the label identification information stored in the extraction target label storage unit 21 from the sound data to which the label is attached, and stores it in the recording medium. accumulate. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the extraction unit 22 or may exist outside the extraction unit 22. In addition, this recording medium may or may not store sound data temporarily. The “sound data related to the label” is the same as that described in the erasing unit 20 except that the erasure target label storage unit 19 becomes the extraction target label storage unit 21.

  Note that any two or more storage units of the sound data storage unit 14, the correspondence information storage unit 16, the erasure target label storage unit 19, and the extraction target label storage unit 21 may be realized by the same recording medium. Alternatively, it may be realized by a separate recording medium. In the former case, for example, an area storing sound data may be the sound data storage unit 14, and an area storing two or more pieces of correspondence information may be the correspondence information storage unit 16.

  Further, in the sound data label assigning apparatus 1, sounds unique to each meeting room, hallway, etc. (for example, a sound of a cooling fan of a personal computer, a constant sound when a monitor is lit, an air conditioning sound, a reflected sound thereof, etc.) Therefore, a label is assigned to sound data by using it, but an environmental sound used for the label may be intentionally output. For example, by installing a sound output device that outputs a sound corresponding to a label at a position corresponding to the label, when the subject is at that position, the sound of the sound output from the sound output device You will get the data. Then, more accurate labeling can be performed using the sound data. The sound output from the sound output device is preferably a sound having a wavelength characteristic different from that of office equipment, air-conditioning sound, or the like. This is to make it distinguishable from other environmental sounds. Further, the sound output from the sound output device may or may not be a sound in a non-audible range. In the former case, the subject and other people are not bothered by the sound output from the sound output device. The sound in the non-audible range is preferably a sound on the higher frequency side than the sound in the audible range. This is because it is known that a high-frequency sound is less likely to travel farther than a low-frequency sound, and is more suitable for specifying the position. For example, the sound output device may output a sound using a speaker, or may not. In the latter case, for example, an electric bell using an electromagnet, or a motor equipped with a motor that taps a bell may be used. The sound output device may be installed on the ceiling, for example. In that case, the power for the sound output device may be acquired by using a power feeding technique for acquiring power from a fluorescent lamp by electromagnetic induction. Moreover, you may acquire the electric power for sound output devices using a solar cell. By doing so, it is not necessary to provide a power source for the sound output device. Further, a human sensor may be connected to the sound output device so that no sound is output when there is no person.

Next, the operation of the sound data label applying apparatus 1 according to the present embodiment will be described using the flowchart of FIG.
(Step S101) The sound data reception unit 11 determines whether sound data has been received. If sound data is received, the process proceeds to step S102, and if not, the process proceeds to step S104.

  (Step S102) The separation unit 12 separates the environmental sound data from the sound data received by the sound data reception unit 11.

  (Step S <b> 103) The storage unit 13 stores sound data, which is environmental sound data separated by the separation unit 12, in the sound data storage unit 14. The storage unit 13 also stores the sound data before separation received by the sound data reception unit 11 in the sound data storage unit 14. Then, the process returns to step S101.

  (Step S104) The label assigning unit 18 determines whether to apply a label. For example, it may be determined that labeling is to be performed after a series of sound data has been accumulated, and labeling is performed when the sound data label imparting apparatus 1 receives an instruction to perform labeling. It may be determined to be performed, or it may be determined that labeling is performed at other timing. If labeling is to be performed, the process proceeds to step S105, and if not, the process proceeds to step S108.

  (Step S105) The sound image conversion unit 15 reads out sound data to be converted into a sound image from the sound data storage unit 14, and converts the sound data into a sound image. The sound image may be temporarily stored in a recording medium (not shown).

  (Step S106) The comparison unit 17 compares the label sound image stored in the correspondence information storage unit 16 with the sound image converted in step S105. Specifically, the comparison unit 17 calculates similarity information between the label sound image and the sound image.

  (Step S107) Using the comparison result by the comparison unit 17, the label assigning unit 18 assigns a label corresponding to the label sound image having high similarity to the sound image to the sound data corresponding to the sound image. Then, the process returns to step S101.

  Note that the processing in steps S105 to S107 may be repeatedly performed on all sound data from the beginning to the rear end of the sound data stored in the sound data storage unit 14.

  (Step S108) The erasure unit 20 determines whether or not to erase the sound data. For example, it may be determined that the sound data is to be erased after a series of label assignments to the sound data is completed, and the sound data label attaching apparatus 1 receives an instruction to erase the sound data. It may be determined that the sound data is to be erased, or it may be determined that the sound data is to be erased at other timing. If the sound data is to be erased, the process proceeds to step S109. If not, the process proceeds to step S112.

  (Step S109) Using the label identification information stored in the erasure target label storage unit 19, the erasure unit 20 specifies sound data relating to the label identified by the label identification information.

  (Step S110) The erasure unit 20 determines whether or not at least a part of sound data is specified in step S109. And when it specifies, it progresses to step S111, and when that is not right, it returns to step S101.

  (Step S111) The erasure unit 20 erases the sound data specified in step S109. If two or more ranges of sound data are specified, each of them is deleted. Then, the process returns to step S101.

  (Step S112) The extraction unit 22 determines whether or not to extract sound data. For example, it may be determined that sound data is extracted after a series of labels are added to sound data, and the sound data label attaching apparatus 1 receives an instruction to extract sound data. The sound data may be extracted, or it may be determined that the sound data is extracted at other timing. If sound data is to be extracted, the process proceeds to step S113. If not, the process returns to step S101.

  (Step S113) Using the label identification information stored in the extraction target label storage unit 21, the extraction unit 22 specifies sound data relating to the label identified by the label identification information.

  (Step S114) The extraction unit 22 determines whether or not at least a part of sound data has been specified in step S113. And when it specifies, it progresses to step S115, and when that is not right, it returns to step S101.

(Step S115) The extraction unit 22 extracts the sound data specified in Step S113 and stores it in the recording medium. When two or more ranges of sound data are specified, each of them is extracted and stored. Then, the process returns to step S101.
In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

Next, the operation of the sound data label applying apparatus 1 according to the present embodiment will be described using a specific example.
In this specific example, it is assumed that the correspondence information storage unit 16 stores the correspondence information shown in FIG. In FIG. 3, there are labels indicating positions such as “workplace seats” and “meeting rooms” and labels not related to positions such as “extraction trigger”. The label sound image “SP001” corresponding to the label is the file name of the image data of the label sound image. The image data identified by the label sound image “SP001” or the like is a spectrum image of sound data corresponding to the corresponding label. FIG. 3 shows a case where one label sound image is associated with one label, but two or more label sound images may be associated with one label. . For example, in hallways and conference rooms, characteristic sounds may change depending on the location. Therefore, in such a case, a plurality of characteristic sound data may be recorded, a label sound image corresponding to each sound data may be generated, and registered in the correspondence information.

  Further, in this specific example, it is assumed that the label identification information shown in FIG. 4 is stored in the erasure target label storage unit 19. In FIG. 4, label identification information is information indicating the label itself. Each label identification information is associated with extended information. The extended information is information indicating how much the start point and the end point of the sound data corresponding to the label identification information are extended and deleted. For example, in the case of label identification information “toilet”, the start point and the end point of the sound data to which the label “toilet” is attached are expanded by 5 seconds and deleted. Therefore, the data is deleted for 10 seconds more than the sound data assigned with the label “toilet”. Although FIG. 4 shows the case where the extended information has a positive value, the extended information may have a negative value. In the case of a negative value, the start point and the end point are reduced and deleted by the absolute value of the value.

  In this specific example, it is assumed that the label identification information shown in FIG. 5 is stored in the extraction target label storage unit 21. Also in FIG. 5, the label identification information is information indicating the label itself. Each label identification information is associated with extended information. The extended information is the same as that described with reference to FIG. 4 except that erasure is changed to extraction, and the description thereof is omitted.

  Assume that after the subject wearing the microphone arrives at the office, the recorder is operated by operating the switch of the recorder connected to the microphone. When the subject wants to extract the sound data intentionally, the subject generates a sound corresponding to the extraction trigger so that the sound is collected by the microphone. In addition, when the test subject intentionally wants to erase the sound data, the subject generates a sound corresponding to the erase trigger so that the sound is collected by the microphone. In this specific example, it is assumed that only the sound corresponding to the extraction trigger is generated. The sound may be generated using a part of the subject's body, such as ringing a finger or clapping hands, as described above, or knocking with a specific knock-type ballpoint pen. May be generated. Then, it is assumed that after recording all day, the memory is taken out from the recorder and attached to the reader connected to the sound data label applying apparatus 1. Then, the sound data reception unit 11 reads out the sound data recorded from the memory and passes it to the separation unit 12 and the storage unit 13 (step S101). The separation unit 12 separates the environmental sound data from the received sound data and passes it to the storage unit 13 (step S102). The accumulating unit 13 accumulates the sound data received from the sound data receiving unit 11 and the sound data that is the environmental sound data received from the separating unit 12 in the sound data storage unit 14 (step S103). FIG. 6 is a diagram showing an example of the separated sound data accumulated as described above. In FIG. 6, time is indicated by the horizontal axis, and sound pressure is indicated by the vertical axis.

  When the label attaching unit 18 detects that a series of separated sound data has been accumulated, the label attaching unit 18 determines that the label is attached, and instructs the sound image converting unit 15 to convert the sound image through a route (not shown). (Step S104). Then, the sound image conversion unit 15 accesses the separated sound data stored in the sound data storage unit 14 and converts it into a sound image every 50 milliseconds. Specifically, first, the sound image conversion unit 15 performs Fourier transform on the sound data of the time code 00: 00: 00.00 to 00: 00: 00.05 to convert it into a sound image (step S105). The sound image is, for example, an image in which the horizontal axis is frequency and the vertical axis is intensity, as shown in FIG. For convenience of explanation, “frequency”, “intensity”, and the like are clearly shown in FIG. 7, but the sound image is image information of only the waveform diagram of FIG.

  Next, the comparison unit 17 calculates similar information between the sound image converted by the sound image conversion unit 15 and the label sound image included in the correspondence information shown in FIG. (Step S106). The label assigning unit 18 specifies similar information indicating the highest similarity among the similar information calculated by the comparing unit 17 and acquires a label corresponding to the similar information. In this case, it is assumed that it is a “corridor”. Then, the label assigning unit 18 assigns a label “corridor” to the sound data of the time code 00: 00: 00.00 to 00: 00: 00.05 (step S107). Thereafter, the conversion to the sound image, the comparison of the sound images, and the labeling are sequentially repeated every 50 milliseconds (steps S105 to S107). FIG. 8 is a diagram showing an example of the label attached to the sound data as described above. In FIG. 8, time codes and labels are associated with each other. The information shown in FIG. 8 is accumulated in the sound data storage unit 14 by the label assigning unit 18. Further, after giving labels to all the time codes of the sound data, the label attaching unit 18 merges the time codes of the information shown in FIG. 8 for each label to generate the information shown in FIG. It is assumed that the sound data storage unit 14 is accumulated.

  When the erasing unit 20 detects the end of the series of label assignments, it determines that the sound data is to be erased (step S108), and among the labels shown in FIG. 9, the label identified by the label identification information shown in FIG. A time code corresponding to is specified (step S109). In this case, it is assumed that the time code 00: 40: 25.20 to 00: 41: 38.10,... Corresponding to the label “toilet” is specified. Then, the erasure unit 20 determines that there is an erasure target (step S110), reads the extended information corresponding to the label “toilet” that is the erasure target, and specifies the specified time code 00: 40: 25.20-00: 41: 38.10, ... are changed according to the read extended information. Specifically, the expanded time code 00: 40: 20.20 to 00: 41: 43.10, with the start point of each time code reduced by 5 seconds and the end point increased by 5 seconds, is generated, and the time code is generated. The sound data corresponding to (this sound data is the sound data before separation) is deleted (step S111).

  Next, when the extraction unit 22 detects that the deletion of the sound data has ended, it determines that the sound data is to be extracted (step S112), and is identified by the label identification information of FIG. 5 among the labels of FIG. A time code corresponding to the label is identified (step S113). In this case, it is assumed that the time code 00: 41: 56.55 to 00: 41: 57.50 corresponding to the label “extraction trigger” is specified. Then, the extraction unit 22 determines that an extraction target exists (step S114), reads the extended information corresponding to the label “extraction trigger” that is the extraction target, and specifies the specified time code 00: 41: 56.55 to 00. : 41: 57.50 is changed according to the read extended information. The time code after the expansion is 00: 41: 56.55 to 00: 42: 57.50. Then, the extraction unit 22 extracts sound data corresponding to the expanded time code (this sound data is sound data before separation), and stores the extracted sound data in a recording medium (not shown) ( Step S115).

  In this specific example, the case where the time code indicates relative time has been described, but this need not be the case. As described above, a time code indicating an absolute date and time may be used. Further, as in this specific example, even when a time code indicating relative time is used, it corresponds to an arbitrary point (for example, a start point, an end point, or other time point) of the time code. It is preferable to specify the date and time.

  In this specific example, the case where sound data is erased or sound data is extracted using an erase trigger or an extraction trigger has been described. However, the erase trigger or extraction trigger includes an erase target or an extraction target. There may be both a trigger indicating the start point and a trigger indicating the end point. Then, the erasure unit 20 erases from the time point of the sound data corresponding to the erasure trigger indicating the start point to the time point of the sound data corresponding to the erasure trigger indicating the end point, and the extraction unit 22 corresponds to the extraction trigger indicating the start point. From the time of the sound data to be performed to the time of the sound data corresponding to the extraction trigger indicating the end point may be extracted. In this case, different labels are assigned to the trigger indicating the start point and the trigger indicating the end point.

  Moreover, when converting sound data into a sound image, a process of removing noise may be performed. For example, noise may be removed using time integration processing or other methods. A method of removing noise such as time integration processing is already known and will not be described.

  As described above, according to the sound data label applying apparatus 1 according to the present embodiment, a label indicating a position or the like can be attached to sound data. As a result, it becomes possible to search for sound data using the label, and it becomes easy to access desired sound data. For example, when it is desired to access sound data when talking with a person in the hallway, the time zone may not be stored. Even in such a case, only the sound data corresponding to the label “corridor” needs to be checked because the label is given as in the present embodiment, rather than checking all the sound data. Also, the desired sound data can be easily accessed.

  In addition, since sound data corresponding to a label specified in advance can be automatically deleted or extracted, for example, sound data related to privacy can be automatically deleted or extracted. The desired sound data can be automatically extracted.

  In the specific example of the present embodiment, the case where the sound image is converted at regular intervals (50 milliseconds) is described, but this need not be the case. For example, the sound image conversion unit 15 may convert sound data of a period having continuous frequency peaks into one sound image. As shown in FIG. 10, sound data having a frequency peak is continuous when there is a peak in the sound image (in FIG. 10, there is a peak around 1700 Hz). The sound data of the period may be converted into one sound image. In the continuation of the peak, the peak frequency may not change or the peak frequency may change. For example, the melody of the electronic sound that flows when the train departs may differ from station to station. In such a case, since the peak continues from the beginning to the end of the melody, the beginning to the end of the melody will be converted into one sound image. A label according to the position can be given. In this case, the label sound image associated with the label in the correspondence information is also preferably a sound image corresponding to sound data in a period in which peaks are continuous. Specifically, as described above, the sound image conversion unit 15 performs conversion to a sound image every 50 milliseconds, and when there is a peak in a certain sound image, The conversion to the sound image is continued until the peak does not exist in the conversion, and when the peak does not exist, the entire sound data in the period in which the peak is continuous is converted into the sound image. And the comparison part 17 compares with a label sound image using the sound image corresponding to the sound data of the period when the peak continued instead of the sound image for every 50 milliseconds which has a peak.

  Here, a method for detecting the presence of a peak will be briefly described. For example, let f (x) be the intensity (power) of a spectrum obtained by Fourier transforming sound data. Note that x is a frequency. Then, when the frequency x of f (x) is sequentially changed from the lower end frequency to the upper end frequency, if x satisfying either of the following relations 1 and 2 exists, a peak is present. You may judge.

Relationship 1: f (x) × S <f (x + Δx)
Relationship 2: f (x)> f (x + Δx) × S

  However, S is a threshold value used as a standard for peak detection. For example, S may be a value such as 1.5 or 3. Δx is a frequency increment (for example, 5 Hz). As is apparent from this peak detection method, the presence or absence of a peak is determined by the frequency x at which the absolute value of the range of f ′ (x) obtained by differentiating the intensity f (x) after Fourier transformation exceeds S. Judgment may be made based on whether or not exists. If the frequency x exists, a peak exists, and if the frequency x does not exist, no peak exists.

(Embodiment 2)
A sound data label applying apparatus 2 according to Embodiment 2 of the present invention will be described with reference to the drawings. The sound data label assigning device 2 according to the present embodiment also provides the subject's coordinates to sound data using sounds output from a plurality of sound output devices.

  FIG. 11 is a block diagram showing the configuration of the sound data label assigning apparatus 2 according to this embodiment. The sound data label assigning apparatus 2 according to the present embodiment includes a sound data receiving unit 11, a separation unit 12, a storage unit 13, a sound data storage unit 14, a sound image conversion unit 15, and a correspondence information storage unit 16. The comparison unit 17, the label addition unit 18, the erasure target label storage unit 19, the erasure unit 20, the extraction target label storage unit 21, the extraction unit 22, the position correspondence information storage unit 31, and the coordinate calculation unit 32. And a coordinate assigning unit 33. Configurations and operations other than the position correspondence information storage unit 31, the coordinate calculation unit 32, and the coordinate assignment unit 33 correspond to three labels and each label when the label assignment unit 18 assigns a label for coordinate calculation. This is the same as Embodiment 1 except that similar information is given, and the description thereof is omitted.

  The position correspondence information storage unit 31 stores three or more pieces of position correspondence information. The position correspondence information is information that associates a label with coordinates of a sound output device that outputs a sound corresponding to the label. That is, in the position correspondence information, the coordinates of the sound output device and the label that can identify the sound output device are associated with each other. The sound output device is the same as that described in the first embodiment. The coordinates of the sound output device may be local coordinates (for example, coordinates in a conference room, coordinates in an office, etc.) or global coordinates (for example, latitude and longitude). Good.

  The process of storing three or more pieces of position correspondence information in the position correspondence information storage unit 31 is not limited. For example, three or more pieces of position correspondence information may be stored in the position correspondence information storage unit 31 via a recording medium, and three or more pieces of position correspondence information transmitted via a communication line or the like may be stored. The correspondence information storage unit 31 may store the information, or three or more pieces of position correspondence information input via the input device may be stored in the position correspondence information storage unit 31. . The storage in the position correspondence information storage unit 31 may be temporary storage in a RAM or the like, or may be long-term storage. The position correspondence information storage unit 31 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  Here, “associating the label with the coordinates” means that the coordinates need only be acquired from the label. Therefore, the position correspondence information may include information including a label and coordinates as a set, or may be information that links the label and coordinates. In the latter case, the position correspondence information may be, for example, information that associates a label with a pointer or address indicating a position where coordinates are stored. In the present embodiment, the former case will be described. Further, the label and the coordinate need not be directly associated with each other. For example, the third information may correspond to the label, and the coordinates may correspond to the third information.

  The coordinate calculation unit 32 calculates the coordinates corresponding to the sound data using the three labels and similar information given to the sound data and the coordinates corresponding to the three labels. That is, the coordinate calculation unit 32 calculates the coordinates corresponding to the sound data using the three pieces of similar information corresponding to the sound data and the coordinates corresponding to the three pieces of similar information. The calculation of the coordinates can be performed by the same method as the three-point survey. Note that the sound output from one sound source is collected by four microphones (four because it is a space, but three if it is a plane), and the position of the sound source in the three-dimensional space is specified. A technique is disclosed in Japanese Patent Laid-Open No. 6-241883. What the coordinate calculation unit 32 does, conversely, identifies the position of the microphone on the two-dimensional plane by collecting sounds of the same sound pressure output from the three point sound sources with one microphone. Is. For example, the similarity indicated by the similar information corresponding to the sound source A is X%, the similarity indicated by the similar information corresponding to the sound source B is Y%, and the similarity indicated by the similar information corresponding to the sound source C is Z%. , The coordinates to be calculated are positions where the ratio of the distance to the sound source A and the distance to the sound source B is SQR (1 / X): SQR (1 / Y). The ratio between the distance to the sound source C and the distance to the sound source C is SQR (1 / Y): SQR (1 / Z), and the ratio between the distance to the sound source C and the distance to the sound source A is SQR. (1 / Z): It is a position that becomes SQR (1 / X). Therefore, using this condition and the coordinates of the sound sources A, B, and C, the coordinates corresponding to the sound data can be calculated. SQR (M) means the square root of M.

  In addition, when the sound output device corresponding to the three sound sources outputs a single frequency sound, the sound image of the sound data including the sound of the three sound sources has three sound images. The label sound image corresponding to each sound source has one peak corresponding to each sound source. Therefore, when calculating the similar information, the similar information corresponding to the ratio of the peak heights may be calculated. For example, if the peak height of the sound image is 70% of the peak height of the label sound image with respect to the frequency peak of a certain sound source, the comparison unit 17 indicates that the similarity is 70%. Information may be calculated.

  The coordinate assigning unit 33 assigns the coordinates calculated by the coordinate calculating unit 32 to the sound data corresponding to the coordinates. The sound data corresponding to the coordinates is sound data to which similar information used in the calculation of the coordinates is given. Note that the method of assigning coordinates to sound data is the same as the method of assigning labels to sound data, and a description thereof is omitted.

  As described above, in the present embodiment, the comparison unit 17 calculates similarity information that is information regarding the similarity between the sound image and the label sound image.

  In addition, when providing a label for coordinate calculation, the label attaching unit 18 associates each sound data with three labels and similar information respectively corresponding to the three labels. And However, it is assumed that the three labels are the three labels with the highest similarity.

  Further, any two or more storage units of the sound data storage unit 14, the correspondence information storage unit 16, the erasure target label storage unit 19, the extraction target label storage unit 21, and the position correspondence information storage unit 31 are the same. These recording media may be realized, or may be realized by separate recording media.

  Next, operation | movement of the sound data label provision apparatus 2 by this Embodiment is demonstrated using the flowchart of FIG. The processes other than steps S201 to S205 are the same as those in the flowchart of FIG. 2 in the first embodiment, and a description thereof will be omitted.

  (Step S201) The coordinate assigning unit 33 determines whether or not to assign coordinates. For example, it may be determined that the application of coordinates is performed after a series of labels are applied to the sound data. When the sound data label applying apparatus 1 receives an instruction to apply the coordinates, May be determined to be applied, or it may be determined that the coordinates are applied at other timing. If the coordinates are assigned, the process proceeds to step S202. If not, the process returns to step S101.

  (Step S <b> 202) The coordinate calculation unit 32 specifies sound data that is a target to which coordinates are to be assigned. Whether or not the sound data is a target to which coordinates are assigned can be determined by whether or not a label for coordinate calculation is given to the sound data.

  (Step S203) The coordinate calculation unit 32 determines whether or not at least a part of sound data has been specified in Step S202. Then, if specified, the process proceeds to step S204, and if not, the process returns to step S101.

  (Step S204) The coordinate calculation unit 32 calculates coordinates for the sound data specified in step S202. Specifically, the coordinates corresponding to the label given to the specified sound data are acquired using the position correspondence information stored in the position correspondence information storage unit 31, the acquired coordinates, and the coordinates The coordinates corresponding to the sound data are calculated using the similar information corresponding to.

  (Step S205) The coordinate assigning unit 33 assigns the coordinates calculated by the coordinate calculating unit 32 to the sound data corresponding to the coordinates. Then, the process returns to step S101.

  When calculating two or more coordinates, the processes in steps S204 and S205 may be repeated. Further, in the flowchart of FIG. 12, the processing is ended by powering off or interruption for aborting the processing.

Next, operation | movement of the sound data label provision apparatus 2 by this Embodiment is demonstrated using a specific example.
In this specific example, it is assumed that the correspondence information shown in FIG. 13 is stored in the correspondence information storage unit 16. In the correspondence information in FIG. 13, the label and the label sound image are stored in association with each other as in the correspondence information in FIG. 3. The sound source 101 or the like is a label for identifying a sound output device. That is, a label including “sound source” is a coordinate calculation label. Also, the label sound image SP101 corresponding to the sound source 101 or the like is a sound image corresponding to the sound output from the sound source 101 or the like.

  Further, in this specific example, it is assumed that the position correspondence information storage unit 31 stores the position correspondence information shown in FIG. In the position correspondence information in FIG. 14, labels and coordinates are associated with each other. The coordinates are coordinates indicating a position in a room.

Further, in this specific example, the information of FIG. 4 is stored in the erasure target label storage unit 19 and the information of FIG. 5 is stored in the extraction target label storage unit 21 is a specific example of the first embodiment. It is assumed that
In this specific example, it is assumed that the subject also moves between sound output devices arranged at the positions indicated by the position correspondence information in FIG.

  In this specific example, the process of receiving and storing sound data, the process of assigning labels, the process of erasing sound data, and the process of extracting sound data are the same as those of the specific example of the first embodiment. Is omitted. However, if the label sound image having the highest similarity with the converted sound image is a label for coordinate calculation when the label is given (that is, if the label includes “sound source”), The label assigning unit 18 assigns three labels to sound data in descending order of similarity, and also assigns similar information corresponding to the given labels. If the label sound image having the highest similarity with the converted sound image is not a coordinate calculation label, labeling is performed in the same manner as in the specific example of the first embodiment.

  Specifically, the result of labeling by the label applying unit 18 every 50 milliseconds is as shown in FIG. 15, for example. In FIG. 15, corresponding to time codes 01: 18: 40.15-01: 18: 40.20 etc., three coordinate calculation labels and similar information corresponding to each label are given. I understand that. In this specific example, the similarity information is information (similarity) indicating similarity.

  When the coordinate assigning unit 33 detects that the data extraction described in the specific example of the first embodiment has been completed, the coordinate assigning unit 33 determines to add the coordinate (step S201), and calculates the coordinates to the coordinate calculating unit 32. Give instructions to that effect. Then, the coordinate calculation unit 32 refers to the information shown in FIG. 15, and obtains sound data such as time codes 01: 18: 40.15-01: 18: 40.20 to which a coordinate calculation label is attached. Specify (step S202). Since there is sound data to which the specified coordinates are to be assigned (step S203), the coordinate calculation unit 32 calculates the coordinates for each specified record in FIG. For example, for sound data corresponding to the time code 01: 18: 40.15-01: 18: 40.20, the similar information corresponding to the sound source 101 is 68%, and the similar information corresponding to the sound source 102 is 43%. Since the similarity information corresponding to the sound source 103 is 43%, the coordinates (3, 3) are calculated as described above according to the similarity information and passed to the coordinate assigning unit 33 (step S204). The coordinate assigning unit 33 assigns the coordinates to the corresponding sound data (step S205). As a result, as shown in FIG. 16, coordinates corresponding to the time code 01: 18: 40.15 to 01: 18: 40.20 are given.

  Note that the coordinate calculation unit 32 calculates coordinates for all time codes to be calculated, and the coordinate assigning unit 33 assigns the calculated coordinates. However, when the same label and similar information as the previous time code are given as in time code 01: 18: 40.20 to 01: 18: 40.25 in FIG. 15, coordinates are calculated. Instead, the same coordinates as those already given may be given.

  As described above, according to the sound data label assigning device 2 according to the present embodiment, the coordinates indicating the position of the subject (strictly, the position of the microphone worn by the subject) by using the position correspondence information and the like. Can know. Therefore, it is possible to give more detailed information about the position to the sound data than when a label indicating the position is given.

  In the present embodiment, a case has been described in which three labels are assigned when a coordinate calculation label and similar information are assigned to sound data, but this need not be the case. For example, the label assigning unit 18 may assign each sound data with three or more labels and similar information respectively corresponding to the three or more labels. Further, the coordinate calculation unit 32 may calculate the coordinates corresponding to the sound data by using the three or more labels and similar information given to the sound data and the coordinates corresponding to the three or more labels. Good. For example, when the coordinate calculation unit 32 calculates coordinates in a three-dimensional space coordinate system, each sound data is provided with four labels and similar information corresponding to each of the four labels. The coordinate calculation unit 32 may calculate the coordinates in the three-dimensional space coordinate system using these labels, similar information, and position correspondence information. In this case, in the position correspondence information, the coordinates associated with the label are not the coordinates of the two-dimensional plane coordinate system described in the specific example of the present embodiment, but the coordinates of the three-dimensional space coordinate system. Even when calculating the coordinates of the plane, four or more labels and similar information corresponding to the four or more labels are associated with each sound data, and the four or more labels are used. Thus, the calculation of coordinates may be performed. Also, when calculating the coordinates of the space, each sound data is assigned with five or more labels and similar information corresponding to each of the five or more labels, and the five or more labels are used. Thus, the calculation of coordinates may be performed.

  Further, in each of the above embodiments, the sound data stored in the sound data storage unit 14 may be retained for a long period of time that is subject to labeling, erasing, extraction, or the like. The image may be temporarily stored for sound image conversion by the image conversion unit 15. In the latter case, sound data to be subjected to labeling, erasure, extraction, etc. may be stored in a sound data storage unit different from the sound data storage unit 14. For example, when the sound data receiving unit 11 receives sound data from a recording medium, processing such as labeling, erasing, and extraction may be performed on the sound data stored in the recording medium. .

  Further, in each of the above embodiments, the sound image associated with the label in the correspondence information stored in the correspondence information storage unit 16 has been described as being stored in advance. May be. For example, when the comparison unit 17 performs the comparison, if there is no label sound image having a similarity higher than a preset threshold value, a new label is added to the sound image to be compared. The correspondence information having the label and the label sound image as the comparison target sound image associated with each other may be stored in the correspondence information storage unit 16. In this case, the newly issued label is assigned to the sound data. In this way, when a new sound (that is, a sound that has no similarity higher than the threshold value with any sound image included in the correspondence information) is received, A new label is issued, the label is added to the sound data, and correspondence information about the label (that is, correspondence information that associates the issued label with a sound image corresponding to the new sound) is accumulated, When a sound similar to a sound received in the past is received, as described in the above embodiments, a label issued to the sound received in the past is assigned using the correspondence information. Will be. The label to be issued may be any label as long as each label can be distinguished. For example, it may be a number that increments in order, such as “1”, “2”, “3”, etc., or may be an alphabet or other symbol. In addition, after the fact, by listening to the sound corresponding to the label, the label may be replaced with a label indicating the position or the content of the sound, such as “office”, “crossing”, “break room”, etc. . The issuance of new labels, the accumulation of correspondence information, and the like may be performed by the comparison unit 17 or may be performed by other components (for example, a label issuance unit (not shown)). Moreover, when issuing and giving a label in this way, the same repetition of the label may occur. For example, when the sequence of labels “10”, “7”, “15”, and “16” repeatedly appears, a new label (for example, label “123”) is issued for the label sequence. The label sequence “10” “7” “15” “16” may be replaced with the newly issued label “123”.

  Moreover, although each said embodiment demonstrated the case where the sound data label provision apparatuses 1 and 2 were provided with the separation part 12, it does not need to be so. When the sound data label assigning devices 1 and 2 do not include the separation unit 12, the accumulation unit 13 accumulates the sound data received by the sound data reception unit 11 in the sound data storage unit 14. As a result, the sound data converted into the sound image by the sound image converting unit 15 is the same as the sound data to which the label attaching unit 18 assigns a label.

  In each of the above-described embodiments, the case where the erasure target label storage unit 19 and the erasure unit 20 are provided has been described. However, if it is not necessary to automatically erase the sound data according to the label, The data label assignment devices 1 and 2 do not have to include the erasure target label storage unit 19 and the erasure unit 20.

  Moreover, although each said embodiment demonstrated the case where the extraction object label memory | storage part 21 and the extraction part 22 were provided, if it is not necessary to perform the automatic extraction of the sound data according to a label, sound The data label assigning devices 1 and 2 may not include the extraction target label storage unit 21 and the extraction unit 22.

  In each of the above embodiments, label interpolation may be performed. For example, the label “corridor” is given from time code 00.00 seconds to 10.00 seconds, the label “workplace seat” is given from time codes 10.00 seconds to 10.10 seconds, and time code 10. When the label “corridor” is given from 10 seconds to 50.00 seconds, it is inappropriate that a label different by 0.1 seconds is given. It may be changed to the label “hallway” for up to 10 seconds. Generally speaking, the sound data of interest is less than or equal to a predetermined length (for example, 1 second or less), and is greater than or equal to a predetermined length before and after the sound data of interest. The same label (for example, “corridor” in the above example) is given to the sound data (for example, 5 seconds or more), and the sound data of interest has a different label (upper In this example, “workplace seat” is assigned), the label attached to the sound data of interest is changed to the label attached to the sound data before and after the sound data. It may be changed. In addition, the same label is assigned to sound data of a predetermined length before and after the section erased by separation by the separation unit 12 and the section in which the label could not be attached, as in the above description. If so, the same label as that of the preceding and following sound data may be given to that section.

  Further, in each of the above-described embodiments, an output (not shown) for outputting the label assignment result given by the label assigning unit 18, the sound data obtained by erasing the sound data by the erasing unit 20, and the sound data extracted by the extraction unit 22 May be provided. When the output target is sound data, the output may be, for example, transmission via a communication line to a predetermined device, sound output by a speaker, or accumulation in a recording medium. The output may be displayed on a display device (for example, a CRT or a liquid crystal display) when the output target is a labeling result, or may be transmitted via a communication line to a predetermined device. Alternatively, printing by a printer, sound output by a speaker, or accumulation in a recording medium may be performed. The output unit (not shown) may or may not include an output device (for example, a display device or a printer). The output unit (not shown) may be realized by hardware, or may be realized by software such as a driver that drives these devices.

  In each of the above embodiments, the case where the sound data label assigning devices 1 and 2 are stand-alone has been described. However, the sound data label assigning device may be a stand-alone device or a server device in a server / client system. It may be. In the latter case, the reception room output unit may receive information or output information via a communication line.

  In each of the above embodiments, each processing or each function may be realized by centralized processing by a single device or a single system, or distributed processing by a plurality of devices or a plurality of systems. May be realized.

  Also, in each of the above embodiments, information related to processing executed by each component, for example, each component received, acquired, selected, generated, transmitted, or received Information and information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing are retained temporarily or over a long period of time on a recording medium (not shown) even if not explicitly stated in the above description. May be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In each of the above embodiments, when information used by each component, for example, information such as a threshold value, an address, and various setting values used by each component may be changed by the user Even if it is not specified in the above description, the user may be able to change the information as appropriate, or it may not be. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  In each of the above embodiments, when two or more components included in the sound data label assigning apparatuses 1 and 2 include a communication device, an input device, and the like, the two or more components are physically single devices. Or may have separate devices.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the sound data label provision apparatus 1 in the said embodiment is the following programs. That is, this program converts the sound data received by the sound data receiving unit and the sound data receiving unit that receives sound data collected by the microphone at the position of the subject into a sound image indicating the intensity for each frequency. Correspondence information that is information having a sound image conversion unit, a sound image converted by the sound image conversion unit, and a label sound image that is a sound image indicating the intensity for each frequency of the sound corresponding to the label is 2 Compared with the label sound image stored in the correspondence information storage unit stored above, the comparison result by the comparison unit is used to produce a label sound image that is highly similar to the sound image converted by the sound image conversion unit. This is a program for specifying a corresponding label and causing the specified label to function as a label attaching unit that assigns sound data corresponding to a sound image.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in a reception unit that receives information is not included in at least the function realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disc such as a CD-ROM, a magnetic disc, a semiconductor memory, etc.) is read out. May be executed by Further, this program may be used as a program constituting a program product.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 17 is a schematic diagram showing an example of the appearance of a computer that executes the program and realizes the sound data label assigning apparatuses 1 and 2 according to the above embodiments. Each of the above embodiments can be realized by computer hardware and a computer program executed thereon.

  In FIG. 17, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 18 is a diagram showing an internal configuration of the computer system 900. In FIG. 18, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program for causing the computer system 900 to execute the functions of the sound data label assigning apparatuses 1 and 2 according to the above embodiments is stored in the CD-ROM 921 or FD 922 and inserted into the CD-ROM drive 905 or FD drive 906. May be transferred to the hard disk 914. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the sound data label assigning apparatuses 1 and 2 according to the above embodiments. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the sound data label assigning device and the like according to the present invention, a label can be assigned to the sound data collected at the position of the subject, which is useful as a device for giving a label to the sound data. is there.

DESCRIPTION OF SYMBOLS 1, 2 Sound data label provision apparatus 11 Sound data reception part 12 Separation part 13 Accumulation part 14 Sound data storage part 15 Sound image conversion part 16 Corresponding information storage part 17 Comparison part 18 Label provision part 19 Erase object label storage part 20 Erase part DESCRIPTION OF SYMBOLS 21 Extraction object label memory | storage part 22 Extraction part 31 Position corresponding | compatible information storage part 32 Coordinate calculation part 33 Coordinate provision part

Claims (10)

A sound data receiving unit for receiving sound data collected by the microphone at the position of the subject;
A correspondence information storage unit that stores two or more correspondence information, which is information having a label and a label sound image that is a sound image indicating the intensity for each frequency of sound corresponding to the label, and
A sound image conversion unit that converts the sound data received by the sound data reception unit into a sound image indicating an intensity for each frequency;
A comparison unit that compares the sound image converted by the sound image conversion unit with the label sound image stored in the correspondence information storage unit;
Using the comparison result by the comparison unit, a label corresponding to a label sound image having a high similarity to the sound image converted by the sound image conversion unit is specified, and the specified label is used as sound data corresponding to the sound image. A sound data label applying apparatus comprising: a label attaching unit for attaching to the sound data. The sound data received by the sound data receiving unit includes environmental sound data that is sound data of the subject's environment,
A separation unit that separates environmental sound data from the sound data received by the sound data reception unit;
The sound data label assigning apparatus according to claim 1, wherein the sound image conversion unit converts the environmental sound data separated by the separation unit into a sound image. The sound data label assigning device according to claim 1, wherein the sound image conversion unit converts sound data of a period having a frequency peak continuously into one sound image. The comparison unit calculates similarity information that is information on the similarity between the sound image and the label sound image,
The said label provision part matches and associates the specified 1 or 2 or more label and the similar information respectively corresponding to the said specified 1 or 2 or more label, and gives to sound data. The sound data label assigning device according to any one of the above. The label assigning unit associates and gives each sound data three or more labels and similar information respectively corresponding to the three or more labels,
A position correspondence information storage unit that stores position correspondence information that is information for associating a label with the coordinates of a sound output device that outputs a sound corresponding to the label;
A coordinate calculation unit that calculates coordinates corresponding to the sound data using three or more labels and similar information given to the sound data, and coordinates corresponding to the three or more labels;
The sound data label assigning device according to claim 4, further comprising: a coordinate assigning unit that assigns the coordinates calculated by the coordinate calculating unit to sound data corresponding to the coordinates. The sound data label assigning device according to claim 5, wherein the sound output from the sound output device is a sound in a non-audible range. A sound data storage unit for storing sound data received by the sound data receiving unit;
An erasure target label storage unit for storing label identification information for identifying a label corresponding to the erasure target sound data;
An erasure unit that erases, from the sound data storage unit, sound data related to the label identified by the label identification information stored in the erasure target label storage unit among the sound data to which the label is attached. The sound data label assigning device according to any one of claims 1 to 6. A sound data storage unit for storing sound data received by the sound data receiving unit;
An extraction target label storage unit for storing label identification information for identifying a label corresponding to the sound data to be extracted;
An extraction unit that extracts and accumulates sound data related to a label identified by label identification information stored in the extraction target label storage unit from among the sound data to which the label is attached. The sound data label providing apparatus according to any one of claims 1 to 6. A sound data reception unit, a correspondence information storage unit that stores two or more correspondence information, which is information including a label and a label sound image that is a sound image indicating the intensity of each sound frequency corresponding to the label, and A sound data labeling method processed using a sound image conversion unit, a comparison unit, and a labeling unit,
The sound data receiving unit receives sound data collected by a microphone at the position of the subject;
A sound image conversion step in which the sound image conversion unit converts the sound data received in the sound data reception step into a sound image indicating an intensity for each frequency; and
A comparison step in which the comparison unit compares the sound image converted in the sound image conversion step with the label sound image stored in the correspondence information storage unit;
The label assigning unit identifies a label corresponding to a label sound image having high similarity to the sound image converted in the sound image conversion step using the comparison result in the comparison step, and the identified label is A labeling step for labeling the sound data corresponding to the sound image. Computer
A sound data receiving unit for receiving sound data collected by the microphone at the position of the subject;
A sound image conversion unit that converts the sound data received by the sound data reception unit into a sound image indicating an intensity for each frequency;
Two or more correspondence information, which is information including the sound image converted by the sound image conversion unit and the label sound image that is a sound image indicating the intensity for each frequency of the sound corresponding to the label, is stored. A comparison unit that compares the label sound image stored in the correspondence information storage unit
Using the comparison result by the comparison unit, a label corresponding to a label sound image having a high similarity to the sound image converted by the sound image conversion unit is specified, and the specified label is used as sound data corresponding to the sound image. A program for functioning as a label assigning unit to be provided to the user.

Images