JP2005099418A - Object display device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To permit the display of an object having a plurality of meanings by diversely analyzing input sound, when the object is displayed in response to the input sound. <P>SOLUTION: A CPU 1 calculates an attribute value corresponding to each of a plurality of attribute kinds, by analyzing the sound inputted from an acoustic sensor 7, determines the display form corresponding to each of the plurality of attribute kinds regarding the display of the object, referring to an acoustic kind management table and an attribute display management table, on the basis of the values of the plurality of attribute kinds, and performs the display of the object for each attribute. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an object display device and a program for displaying an object in response to an input sound.

Conventionally, as a device that displays an image in response to an input sound, for example, a telephone that changes the color of an image based on the type and scale of a ringtone is known (see Patent Document 1).
In addition, an information processing apparatus is known that assists in recognizing a user's sound by displaying a corresponding image mark by recognizing a predetermined input sound (see Patent Document 2).
JP 2001-237922 A JP-A-10-171626

However, in the above-described telephone set of Patent Document 1, for example, if the type of the ringtone and the scale are set to be different for each destination, the destination can be determined depending on the color of the image displayed at the time of the incoming call. Someone can be easily confirmed, but the information that can be transmitted to the user using the display as a medium is limited to the confirmation of the other party, and moreover, only the color change of the image can be transmitted to the user. (Warning-awareness) was weak and there was a risk of misunderstanding.
In addition, even in the information processing apparatus of Patent Document 2 described above, as in the case of Patent Document 1 described above, the information that can be transmitted to the user using the display as a medium is limited and assists the hearing impaired person. The function was insufficient.

  An object of the present invention is to display an object having a plurality of meanings by analyzing the input sound from various angles when performing object display in response to the input sound.

According to the first aspect of the present invention, in an object display device that displays an object in response to an input sound, attribute correspondence information that stores and manages attribute correspondence information that associates multiple types of attributes related to sound with multiple types of attributes related to objects Based on information storage means, calculation means for calculating an attribute value corresponding to each of a plurality of types of attributes by analyzing an input sound, and a plurality of types of attribute values calculated by the calculation means and the attribute correspondence information And determining means for determining a display form corresponding to each of a plurality of types of attributes relating to object display, and display control means for displaying an object in a display form for each attribute determined by the determining means, It is an object display device.
Furthermore, a program for realizing the main functions shown in the invention described in claim 1 is provided to the computer (the invention described in claim 11).

The invention described in claim 1 may be as follows.
The plurality of types of attributes relating to the sound is information including at least one of the type of sound, the strength of the sound, the height, the length, the length of the sound, the direction of the sound source, the distance from the sound source, and the urgency level. Invention).

  The plurality of types of attributes related to the object is information including at least one of the type, shape, display position, size, hue, color density, transparency, blinking speed, and operation change of the object (claim). Item 3).

  The attribute correspondence information is information that associates at least the type of sound with the type of object or the shape of the object (the invention according to claim 4).

  The attribute correspondence information is information in which at least sound intensity / high / low / long / short is associated with object size / color / transparency / flashing speed / motion change (invention according to claim 5).

 The plurality of attributes relating to the sound includes information on a distance to the sound source or a direction of the sound source, and the calculation means analyzes a plurality of types including the distance to the sound source or the direction of the sound source by analyzing the input sound from the plurality of microphones. The attribute value is calculated (the invention according to claim 6).

  The attribute correspondence information is information in which at least the direction of the sound source or the distance to the sound source is made to correspond to the position of the object or its size.

  There is provided registration means for arbitrarily registering attribute correspondence information in which a plurality of kinds of attributes relating to the sound and a plurality of kinds of attributes relating to the object are associated, and the acquisition means acquires the attribute correspondence information registered by the registration means (Invention of Claim 8).

It is obtained by analyzing the input sound by providing registration means for arbitrarily registering the characteristic data obtained by sampling the input sound waveform and extracting the characteristic and the attribute indicating the type. The type of the input sound is determined by comparing the registered feature data with the registered feature data for each type (the invention according to claim 9).
In this case, when a sound that matches the feature data is detected when the input sound is analyzed, a notification vibration function may be activated (the invention according to claim 10).
Further, when the vibration function is activated, the vibration notification may be performed with the strength or the frequency according to the degree of urgency (the invention according to claim 11).

  According to the first aspect of the present invention, in a state where attribute correspondence information in which a plurality of types of attributes related to sound and a plurality of types of attributes related to objects are associated and stored is managed, An attribute value corresponding to each of the types of attributes is calculated, a display form corresponding to each of the plurality of types of attributes relating to the object display is determined based on the plurality of types of attribute values and the attribute correspondence information. Since the object is displayed in the display form, it is possible to display objects with multiple meanings by analyzing the input sound from multiple perspectives. For example, when used as an auxiliary function for the hearing impaired It is possible to display various surrounding situations from a single input sound in an ambiguous manner, which is extremely effective in avoiding danger.

  According to the invention described in claim 2, in addition to having the same effect as that of the invention described in claim 2 above, the plurality of attributes relating to the sound are the type of sound, the strength of the sound, the height, the length, the length, the direction of the sound source, Since the information includes at least one of the distance to the sound source and the degree of urgency, the input sound can be analyzed in a multifaceted manner for each attribute based on the plurality of types of attributes.

  According to the invention described in claim 3, in addition to the same effect as that of the invention described in claim 1 above, the plurality of types of attributes relating to the object include the object type, shape, display position, size, hue, and color density. Since the information includes at least one of the transparency, the blinking speed, and the operation change, an ambiguous object can be displayed based on the plurality of types of attributes.

  According to the invention described in claim 4, in addition to having the same effect as the invention described in claim 1, the attribute correspondence information is information in which at least the type of sound corresponds to the type of object or its shape. Thus, the user can intuitively determine the correspondence between the sound and the object.

  According to the fifth aspect of the present invention, in addition to the same effect as the first aspect of the present invention, the attribute correspondence information includes at least the sound intensity / high / low / long / short as the object size / color / transparency / Since the information corresponds to the blinking speed / motion change, the user can intuitively determine the correspondence between the sound and the object.

  According to the sixth aspect of the present invention, in addition to the same effect as the first aspect of the present invention, a plurality of types including the distance to the sound source or the direction of the sound source by analyzing the input sound from the plurality of microphones. Since the attribute value is calculated, it is possible to display an object indicating the distance to the sound source or the direction of the sound source. The distance to the sound source or the direction of the sound source can be obtained by analyzing the phase difference and volume difference of input sounds from a plurality of microphones.

  According to the seventh aspect of the invention, in addition to the same effects as the first and seventh aspects of the invention, the attribute correspondence information includes at least the direction of the sound source or the distance from the sound source as the position or size of the object. Since the information is associated, the user can intuitively determine the direction of the sound source or the distance from the sound source by object display.

  According to the eighth aspect of the invention, in addition to the same effect as the first aspect of the invention described above, the attribute correspondence information in which a plurality of types of attributes related to sound and a plurality of types of attributes related to objects are associated is arbitrarily set. Since registration is performed, the correspondence between the input sound and the object display can be arbitrarily changed according to the preference and necessity of the user.

  According to the ninth aspect of the invention, in addition to having the same effect as that of the first aspect of the invention described above, the feature data of the sound is arbitrarily registered corresponding to the type of sound. The type of sound, such as a voice that calls you, can be arbitrarily registered according to the user's preferences and needs. In this case, when the input sound is analyzed, if a sound that matches the feature data is detected, the vibration function for notification is activated (the invention according to claim 10). Notification is possible, and reliable notification can be realized. Further, when the vibration function is activated, if the vibration notification is performed with the intensity or the frequency according to the urgency level (the invention according to claim 11), the urgency level can be easily known by the vibration notification. it can.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing an outline of the overall configuration of the portable device for the hearing impaired in this embodiment.
This portable device has an object display function that displays objects in response to input sound as an auxiliary function for the hearing impaired, and displays objects with multiple meanings by analyzing the input sound from multiple perspectives. By doing so, various surrounding situations can be guided in an ambiguous manner from a single input sound, and the function is incorporated into a mobile phone, a PDA, or the like. That is, the portable device for the hearing impaired person calculates an attribute value corresponding to each of a plurality of types of attributes related to sound by analyzing the input sound, and relates to object display based on the plurality of types of attribute values. A display form corresponding to each of a plurality of types of attributes is determined, and object display is performed in a display form for each attribute.
Before detailed description of the features of this embodiment, the hardware configuration of this embodiment will be described below.

FIG. 1 is a block diagram showing basic components of a portable device for the hearing impaired.
The CPU 1 is a central processing unit that controls the overall operation of the portable device for the hearing impaired in accordance with an operating system and various application software in the storage device 2. The storage device 2 has a program storage area 21 and a data storage area 22, and has a drive system in addition to a magnetic / optical memory and a semiconductor memory. The program storage area 21 stores application programs (various management table registration processing and object display processing) for realizing the present embodiment in accordance with the operation procedure shown in FIGS. The storage area stores data (various management tables) shown in FIGS. 3 and 4 to be described later. The recording device 2 may be a detachable storage medium such as a CD-ROM or DVD in addition to a fixed memory such as a hard disk. Programs and data in the storage device 2 are loaded into a RAM (for example, a static RAM) 3 as needed, or data in the RAM 3 is saved in the storage device 2. The RAM 3 has a program execution area 31 and a work area 32.

  Further, the CPU 1 can directly access a program / data on the other electronic device side via the communication device 4 or can download and receive the program / data via the communication device 4. On the other hand, the communication device 4, the input device 5, the display device 6, the acoustic sensor 7, and the vibration motor 8, which are input / output peripheral devices, are connected to the CPU 1 via a bus line. The operation control of these input / output devices is performed.

  As shown in FIG. 2, the acoustic sensor 7 is composed of a plurality of (three) microphones 71 arranged on the back side of the portable device for the hearing impaired person, and detects the distance to the sound source and the direction of the sound source. In order to do so, the three microphones 71 are installed at a distance. In this case, the distance from the sound source and the direction of the sound source are detected in consideration of the phase difference and volume difference of the input sound. The vibration motor 8 constitutes a notification vibration function, and its strength and frequency can be varied.

FIG. 3 is a diagram showing the contents of the acoustic type management table 11.
The acoustic type management table 11 stores and manages attribute correspondence information in which a sound type and an object shape are associated with each other, and stores and manages an urgency level (importance level) corresponding to a sound type. That is, the acoustic type management table 11 has items of “identification name”, “sound feature data”, “object shape”, and “emergency (importance)” for each type of sound, which are necessary. Depending on the user, the user arbitrarily registers it. In other words, a user who is a hearing-impaired person can arbitrarily register attribute correspondence information indicating a correspondence relationship between a sound type and an object shape so that an object display suitable for each type of sound is performed. In addition, the urgency level (importance level) can be arbitrarily registered so that the urgency level (importance level) can be visually recognized for each type of sound.

“Identification name” is a name for identifying the type of sound. In the example shown in the figure, “car”, “train”, “crossing barrier”, “warf”, “siren”, “bottle of kettle” , “Danger Voice”, “Greeting Voice”, “Voice Calling Me”,... Are registered as sound types.
“Sound feature data” is a collection of feature data obtained by sampling a sound waveform input in advance and extracting the feature. That is, in this embodiment, feature data is extracted from the spatio-temporal distribution information obtained based on the acoustic spectrum method, and only the feature portion of the kind of sound that is required is sampled and acquired. . Here, in the example shown in the figure, when “identification name” is “car”, “feature data of driving sound of car” is registered, and when “train”, “feature data of driving sound of train” is registered. "In case of" crossing breaker "," characteristic data of sound of circuit breaker ", ..., in case of" notice of danger "," characteristic data of utterance of "absurd""," greeting in the case of voice "is,""Hello" characteristic data of the utterance of ", in the case of" voice calling me "shows a case where" Yamada ○○ characteristic data of the utterance of ", ‥‥ are registered ing.

  “Object shape” indicates the type (shape) of the object. In the example shown in the figure, “car image data” and “train image” correspond to each “identification name” indicating the type of sound described above. data ", image data", and the image data indicating the "horn of the" circuit breaker "," image data representing the siren ", character data of the" kettle of image data "," character data of the "dangerous" "," "Hello" ”,“ Character data of “Yamada XX” ”,...

  “Emergency (importance)” is data for determining the degree of urgency (importance) of sound. In the example shown in the figure, when the type of sound is “car”, “moving in the approach direction and distance” Is about 50m and is moderately dangerous, and when the type of sound is "Whistle", "Distance is about 50m and volume is about 54db, and it is moderately dangerous", and the type of sound is "greeting voice" and "self "Voice calling" is "No urgency". Although not shown, a plurality of “emergency levels (importance levels)” can be registered for each sound type, and the sound types are “car”, “train”, “breaker”, “warning”, “ In the case of “Siren”, “Kettle”, “Danger notification”, etc., for example, “Altitude” can be registered in addition to “Medium” described above.

FIG. 4 is a diagram showing the contents of the attribute display management table 12.
The attribute display management table 12 stores and manages attribute correspondence information in which a plurality of attributes related to sound and a plurality of attributes related to objects are associated with each other, and stores and manages the display form (state) for each object attribute. Each item of “attribute relating to sound”, “attribute relating to object”, and “object display form (state)” is associated with each other.
The contents of the attribute display management table 12 are preset by default, but the user can arbitrarily change the contents. That is, it is possible to arbitrarily change the object attribute associated with the attributes relating to the plurality of types of sounds set as default, and to change the display form.

  Here, as the multiple types of “sound attributes”, in the illustrated example, “sound type”, “volume”, “pitch (pitch)”, “distance from the sound source”, “sound source direction” , “Emergency”, and so on are set by default. In addition, as the “attribute related to the object”, in the illustrated example, when the sound attribute is “sound type”, the “shape” of the object is set by default, and when the sound attribute is “volume”, the “color intensity” is set. ”Is set as the default,“ Color ”for“ Pitch ”,“ Size ”for“ Distance from sound source ”,“ Display position ”for“ Direction of sound source ” In the case of “Emergency”, “flashing speed”,... Are set as defaults.

  As the “object display form (state)”, in the illustrated example, when the sound attribute is “type” and the object attribute is “shape”, the “designated shape” indicated by the contents of the acoustic type management table 11 described above. Is the default setting. When the sound attribute is “volume” and the object attribute is “color density”, “50% density at 40 db” is set, and “sound pitch” is set as the default sound attribute. When the attribute is “color tone”, “orange at 1 KHz” is set as a default. Furthermore, when the sound attribute is “distance to the sound source” and the object attribute is “size”, “2 cm at 5 m” is set, the sound attribute is “direction of the sound source”, and the object attribute is “display position”. "If the center position of the display screen in the center direction" is set, the sound attribute is "Urgent", and the object attribute is "Flashing speed", it is "Moderately dangerous twice per second Blink "is set. Although not shown, a plurality of “object display forms (states)” can be set according to the attribute value of the sound, and the default setting values can be arbitrarily changed.

FIG. 5 is a diagram illustrating a case where an automobile is approaching a user who possesses this portable device for the hearing impaired. FIG. 6 is a diagram showing a specific example of object display in this case.
In this case, since the object image of the automobile is displayed, it can be known that the automobile is approaching, and the automobile is directed from the display position of the object image (left side of the screen) to the direction in which the portable device is directed. Can be seen approaching from the left. Furthermore, it is possible to know that the distance is about 50 m from the display size, and it is also possible to estimate the vehicle type and speed from the display color and density.

  Next, the operation concept of the portable device for the hearing impaired in this embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. It is also possible to sequentially execute operations according to the program code transmitted via the transmission medium. In other words, in addition to the recording medium, the program / data supplied externally via the transmission medium can be used to execute the operation specific to this embodiment.

FIG. 7 is a flowchart showing registration processing in the acoustic type management table 11. When the CPU 1 is instructed to register in the management table, the CPU 1 displays a guidance display that prompts the user to input data in an interactive manner. The table registration process is executed.
First, when an identification name indicating the type of sound is input, the CPU 1 takes it and registers it in the acoustic type management table 11 (step A1). Then, a guidance display for prompting input of this type of sample sound is performed. Here, for example, when a sample sound such as a car, a train, a railroad crossing breaker, a horn, and one's name is input from the acoustic sensor 7, the CPU 1 acquires time-space distribution information of the acoustic spectrum (step A2). The spatio-temporal distribution information is edited and processed as necessary, and a feature portion related to the tone type to be registered is extracted from the information, and the feature data is associated with the identification name in the acoustic type management table 11. (Step A3).

 Then, CPU1 is, a variety of objects that are prepared in advance, for example, "car, a train, a railroad crossing breaker, horn, siren, kettle" object images such as and "dangerous, Hello" list is read out a string of such In addition, a guidance display that prompts the user to select and specify an arbitrary object from the object list is displayed. Here, if there is a corresponding object in this object list, the user selects and designates an arbitrary object from among the objects, and if there is no corresponding object such as his own name, the key operation is performed. Input data with. Then, the CPU 1 takes in an object selected from the list screen or an object input by a key operation, and registers it in the acoustic type management table 11 as an object type (shape) in association with the identification name. (Step A4).

Furthermore, a guidance display that prompts the user to input urgency data is displayed. When data for determining the urgency level is input, the input data is fetched, and the sound type management is performed in association with the identification name. Register in the table 11 (step A5). In this case, when a plurality of data such as “medium” and “altitude” are registered, the data input for each “medium” and “altitude” is managed in accordance with the identification name. Registered in the table 11.
In this case, as the guidance information related to the identification name registered this time, for example, a guidance display such as “distance is XXm and volume is XXdb and moderate danger” is a blank part. By providing input support for prompting numerical input to ○○, it becomes possible to input data reliably and easily.

FIG. 8 is a flowchart showing registration processing (edit registration) in the attribute display management table 12. Even in this case, when the registration editing to the management table is instructed, the CPU 1 executes the table registration process while performing guidance display for prompting the user to input data in an interactive manner.
First, the CPU 1 reads out the default value set in the attribute display management table 12 and displays the guidance (step B1). In this state, the user arbitrarily selects and designates an editing target (sound attribute) from this display screen (step B2). Here, when a sound attribute that is considered unnecessary for the user is selected from a plurality of types of sound attributes that are set as default, the sound attribute in the attribute display management table 12 is selected and specified. The sound attribute is invalid, and all other attributes can be made valid. For example, when “pitch” is selected as an unnecessary attribute, only “pitch” can be invalidated.

In addition, in order to edit the default setting of the object attribute, when the corresponding sound attribute is selected and specified, the default setting of the object attribute is specified as an editing target. In this case, when the object attribute is arbitrarily input and specified in order to change the default setting, the object attribute is set in the attribute display management table 12 in place of the default value (step B). Here, in the example of FIG. 4, the default setting of the object attribute corresponding to “volume” is “color density”, but when this is changed to “size”, it corresponds to “volume”. “Size” may be specified as the object attribute.
As input support in this case, a list of object attributes may be displayed, and an arbitrary attribute may be selected and designated from the list. In addition, for example, when the object attribute default “color density” corresponding to “volume” and the object attribute default “color shade” corresponding to “sound pitch” are interchanged, the attribute replacement is performed. May be instructed.

 When the object attribute corresponding to the sound attribute is changed in this way, a guidance display that prompts the user to change the corresponding display form is performed. In this case, as described above, when the default mutual replacement of the object attribute is instructed, the default value of the display form is interchanged accordingly, but the object attribute corresponding to the sound attribute is not changed, and Only the default value of the corresponding display form can be changed. In this case, when an arbitrary display form is input, this display form is set in the attribute display management table 12 in place of the default value (step B4). Here, when a plurality of display form default values that change in accordance with the sound attribute value are changed, the default values are changed by inputting the display forms for that number. Hereinafter, the above-described operation is repeated until the end of editing is instructed (steps B1 to B5).

FIG. 9 is a flowchart showing object display processing executed in response to an input sound.
First, the CPU 1 takes in the acoustic data input from the acoustic sensor 7 (three microphones 71) and stores it in time series (step C1), and analyzes the spatiotemporal distribution according to the acoustic spectrum (step C2). The analysis result and each “feature data” in the acoustic type management table 11 are compared (step C3), and it is checked whether or not a matching one is registered (step C4). Here, if all do not match, the process returns to step C1 as being not subject to object display, but if it matches any “feature data” (YES in step C3), the “feature” is selected from the acoustic type management table 11. “Object image data” corresponding to “data” is acquired (step C4). For example, in the case of coincidence with “feature data of car driving sound”, “car image data” as shown in FIG. 6 is read as an object image corresponding to the “feature data”.

 Next, the volume of the matching portion is determined, the attribute display management table 12 is accessed based on this volume, the corresponding “object attribute” is determined, and the “display mode” of the object is determined according to the volume level. Is determined (step C5). That is, “color density” is determined as the object attribute corresponding to the volume, and “50% density” is determined from the attribute display management table 12 when the loudness of the driving sound of the car is “40 db”, for example. Read out.

 Further, the pitch of the matching portion is determined, the attribute display management table 12 is accessed based on this pitch, the corresponding “object attribute” is determined, and the pitch is increased. Accordingly, the “display form” of the object is determined (step C6). That is, “color tone” is determined as an object attribute corresponding to the pitch of the sound, and “orange” is read from the attribute display management table 12 if the driving sound level of the car is “1 KHz”, for example. .

 Then, the sound source direction of the matching portion is determined based on the phase difference and volume difference of the sound corresponding to each microphone, and the attribute display management table 12 is accessed based on the sound source direction, and the corresponding “object attribute” is set. At the same time, the “display form” of the object is determined according to the sound source direction (step C7). That is, the “display position” is determined as the object attribute corresponding to the sound source direction. For example, when the automobile approaches from the left side in the direction in which the mobile device is directed, the object display position is “ “Left side” is read from the attribute display management table 12.

 Further, the distance of the sound source of the coincident portion is determined based on the phase difference and volume difference of the sound corresponding to each microphone, and the attribute display management table 12 is accessed based on the distance to the sound source, and the corresponding “object” The “attribute” is determined, and the “display form” of the object is determined according to the distance from the sound source (step C8). That is, “size (size)” is determined as the object attribute corresponding to the distance to the sound source. For example, if the automobile is located at 50 m, “5 cm” is read from the attribute display management table 12 as the object size. It is.

 Next, the sound type management table 11 is accessed, the urgency level is determined by referring to the determination result of the volume, the direction of the sound source, the distance to the sound source, and the like, and the attribute display management table 12 is set based on the urgency level. Access is made and the “object attribute” corresponding to the access is determined, and the “display form” of the object is determined according to the degree of urgency (step C9). That is, “flashing speed” is determined as the object attribute corresponding to the urgency level, and, for example, if the vehicle is moving in the approaching direction of the vehicle and the distance is 50 m, it is a moderate danger. “Two blinks per second” is read from the attribute display management table 12. Then, it is checked whether or not the urgency level is high (step C10). If the urgency level is high, the vibration motor 8 is driven with a strength or a frequency corresponding to the urgency level to generate vibration (step C12).

 When the object attribute and the display form are determined for each sound attribute in this way, the CPU 1 displays the image data acquired in the above steps in the display form for each sound attribute (step C11). In this case, as shown in FIG. 6, since the object image of the car is displayed on the display screen, it can be known that the car is approaching, and the car approaches from the left side from the display position. It is possible to know that the distance is about 50 m from the display size, and it is possible to estimate the vehicle type and speed from the display color and density.

  As described above, the portable device for the hearing impaired in this embodiment calculates an attribute value corresponding to each of a plurality of types of sound by analyzing the input sound, and sets the plurality of types of attribute values. Based on the sound type management table 11 and the attribute display management table 12, a display form corresponding to each of a plurality of types of attributes related to object display is determined, and object display is performed in the display form for each attribute. Therefore, it is possible to display objects with multiple meanings by analyzing the input sound from various perspectives, and it is possible to display various surrounding situations from a single input sound, thereby avoiding danger. And so on.

In this case, since the type of sound is associated with the object shape, the correspondence between the sound and the object can be determined intuitively. In addition, the user can change the correspondence between the sound and the object because the volume is the color depth, the pitch is the tone, the distance from the sound source is the size, the direction of the sound source is the display position, and the urgent level is the flashing speed. The relationship can be judged intuitively.
In addition, since the contents set as default in the attribute display management table 12 can be arbitrarily changed, the correspondence between the input sound and the object display according to the user's preference and necessity can be arbitrarily changed. .

  Further, in the acoustic type management table 11, the characteristic data of the sound is arbitrarily registered corresponding to the type of sound. It can be registered arbitrarily according to. Further, since the vibration notification is performed in addition to the object display, reliable notification can be realized. In this case, when the vibration function is activated, the vibration notification is performed with the intensity or the frequency according to the urgency level, and therefore the urgency level can be easily known by the vibration notification.

In the above-described embodiment, the attributes relating to a plurality of types of sound include “sound type”, “volume”, “pitch (pitch)”, “distance to sound source”, “direction of sound source”, “Urgent level” is exemplified, but “sound length”, “rhythm”, “melody”, and the like may also be used.
In the above-described embodiment, the attributes related to a plurality of types of objects include “shape”, “color density”, “color shade”, “size (size)”, “display position”, “flashing speed” of the object. ”Is illustrated, but“ transparency ”,“ operation change ”, and the like may be used.
In the above-described embodiment, a plurality of “emergency levels” such as medium and altitude are registered for each type of sound, and the urgency level is determined from the volume, the direction of the sound source, the distance from the sound source, and the like. However, it is also possible to register only the standard “reference data regarding urgency” for each type of sound and automatically determine the “urgency” steplessly based on this.
In addition, the present invention is not limited to the application to the portable device for the hearing impaired, but can be widely used for general use.

  On the other hand, a recording medium (for example, a CD-ROM, a flexible desk, a RAM card, or the like) on which a program code for causing each of the above-described units to be executed is recorded may be provided to the computer. That is, a recording medium having a computer-readable program code, a function of acquiring attribute correspondence information in which a plurality of attributes related to sound and a plurality of attributes related to an object are associated, and analysis of input sound A function for calculating attribute values corresponding to each of a plurality of types of attributes, and a display form corresponding to each of the plurality of types of attributes related to object display based on the calculated plurality of types of attribute values and the attribute correspondence information You may make it provide the computer-readable recording medium which recorded the program for implement | achieving the function to determine and the function to display an object by the display form for every determined attribute.

The block diagram which showed the outline | summary of the whole structure of the portable apparatus for hearing impaired persons. The figure which showed the several (three) microphone 71 arrange | positioned at the back surface side of the portable apparatus for hearing impaired persons. The figure which showed the content of the acoustic classification management table. The figure which showed the content of the attribute display management table 12. FIG. The figure which illustrated the case where a car is approaching towards the user who has the portable device for hearing impaired people. It is the figure which showed the specific example of the object display in the case of FIG. The flowchart which showed the registration process to the acoustic classification management table 11. FIG. The flowchart which showed the registration process (edit registration) to the attribute display management table 12. FIG. The flowchart which showed the object display process performed in response to an input sound.

Explanation of symbols

1 CPU
2 storage device 4 communication device 5 input device 6 display device 7 acoustic sensor 8 vibration motor 11 acoustic type management table 12 attribute display management table

Claims (12)

In an object display device that displays an object in response to an input sound,
Attribute correspondence information storage means for storing and managing attribute correspondence information in which a plurality of types of attributes related to sound and a plurality of types of attributes related to objects are associated;
Calculating means for calculating an attribute value corresponding to each of the plurality of types of attributes by analyzing the input sound;
Determining means for determining a display form corresponding to each of a plurality of types of attributes related to object display based on the plurality of types of attribute values calculated by the calculation unit and the attribute correspondence information;
Display control means for displaying an object in a display form for each attribute determined by the determination means;
An object display device comprising: The plurality of types of attributes related to the sound is information including at least one of the type of sound, the strength of the sound, the height, the length, the direction of the sound source, the distance to the sound source, and the urgency level.
2. The object display device according to claim 1, wherein: The plurality of types of attributes related to the object is information including at least one of the type, shape, display position, size, hue, color density, transparency, blinking speed, and operation change of the object.
2. The object display device according to claim 1, wherein: The attribute correspondence information is information that associates at least the type of sound with the type of object or its shape,
2. The object display device according to claim 1, wherein: The attribute correspondence information is information in which at least sound intensity / high / low / long / short is associated with object size / color / transparency / flashing speed / motion change.
2. The object display device according to claim 1, wherein: The plurality of attributes relating to the sound includes information on a distance to the sound source or a direction of the sound source,
The calculation means calculates a plurality of types of attribute values including a distance to a sound source or a direction of a sound source by analyzing input sounds from a plurality of microphones.
2. The object display device according to claim 1, wherein the object display device is configured as described above. The attribute correspondence information is information that associates at least the direction of the sound source or the distance to the sound source with the position of the object or its size.
The object display device according to claim 6. Providing a registration means for arbitrarily registering attribute correspondence information in which a plurality of attributes related to the sound and a plurality of attributes related to the object are associated;
The acquisition means acquires attribute correspondence information registered by the registration means;
2. The object display device according to claim 1, wherein the object display device is configured as described above. A registration means for arbitrarily registering the feature data obtained by sampling the input sound waveform and extracting the feature and the attribute indicating the type is provided,
Determining the type of the input sound by comparing the feature data obtained by analyzing the input sound and the feature data for each registered type,
The object display device according to claim 1, which is configured as described above. When a sound that matches the feature data is detected when the input sound is analyzed, a notification vibration function is activated.
The object display device according to claim 9, which is configured as described above. When activating the vibration function, vibration notification is performed at a strength or frequency according to the degree of urgency.
The object display device according to claim 10, which is configured as described above. Against the computer,
A function for storing and managing attribute correspondence information in which multiple types of attributes related to sound and multiple types of attributes related to objects are associated;
A function of calculating attribute values corresponding to each of a plurality of types of attributes by analyzing the input sound;
A function of determining a display form corresponding to each of a plurality of types of attributes related to object display based on the calculated plurality of types of attribute values and the attribute correspondence information;
A function for displaying an object in a display mode for each determined attribute;
A program to realize

Images