JP2007272534A - Apparatus, method and program for complementing ellipsis of word - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for complementing ellipsis of words or the like that is capable of more appropriately complementing ellipsis of words in a situation where two or persons converse. <P>SOLUTION: The apparatus includes: a voice information accepting part 11 for accepting voice information indicating voice issued by at least one user in a situation where two or more users converse; a character information conversion part 13 for converting the voice information into corresponding character information; a zero pronoun specifying part 14 for specifying a zero pronoun as a case element omitted in a document shown by the character information; an action information accepting part 15 for accepting action information as information including at least user position information, i.e., information about the position of the two or more users, and as information about the action of the two or more users; an ellipsis of words complementing part 21 for creating complementing character information such that the omitted word is complemented in the position of the zero pronoun using the action information; and a complementing character information output part 22 for outputting complementing character information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an abbreviation complement device that complements abbreviations.

In natural language, in order to avoid redundant expressions, the subject, object, etc. may be omitted. In particular, in Japanese, it is known that words that can be easily inferred from the context and situation tend to be omitted. When such abbreviations exist, there is a problem that appropriate processing cannot be executed in context analysis, speech dialogue, machine translation, and the like. Therefore, research on techniques for complementing such abbreviations has been conducted (for example, see Non-Patent Document 1).
Kazuhide Yamamoto, Eiichiro Sumida, “Completion of case element omission in Japanese dialogue sentence by decision tree learning”, Natural language processing, Vol. 6, no. 1, p3-28, 1999

  However, in dialogue situations, communication may be attempted using gestures or eye contact. Accordingly, there is a tendency for the listener to omit the natural matter from the situation of the dialogue, and in such a case, there is a limit to the method of complementing the abbreviation by analyzing only the word.

  The present invention has been made to solve the above problem, and provides an abbreviation complement device that can more appropriately complement abbreviations in a situation where two or more users are conversing. With the goal.

  In order to achieve the above object, the abbreviation completion apparatus according to the present invention receives voice information that receives voice information, which is information indicating voice uttered by at least one user, in a situation where two or more users are talking. Is omitted in the document indicated by the character information, the character information converting unit that converts the voice information received by the voice information receiving unit into character information that is information indicating the character corresponding to the voice indicated by the voice information, and the character information. Information including at least a zero pronoun specifying unit that specifies a zero pronoun that is a case element and user location information that is information relating to the location of the two or more users, and information relating to the behavior of the two or more users. Using the behavior information receiving unit that receives the behavior information and the behavior information received by the behavior information receiving unit, the zero cost in the document indicated by the character information An abbreviation completion unit that generates supplementary character information that is information obtained by supplementing an abbreviation that is an abbreviated word at the position of the zero pronoun specified by the verb specification unit, and supplementary character information that outputs the supplementary character information And an output unit.

  With such a configuration, abbreviations can be complemented more appropriately by using behavior information in a situation where two or more users are talking. For example, abbreviations that cannot be known from the user's conversation context can be supplemented.

In the abbreviation complement device according to the present invention, the behavior information may also include motion information that is information indicating a user's movement.
With such a configuration, the user's pointing action, gaze direction, nodding action, etc. can be detected from the action information, and abbreviations can be complemented more appropriately by using these actions, etc. .

In the abbreviation completion device according to the present invention, the abbreviation completion unit indicates a user specifying means for specifying a user corresponding to a human zero pronoun and a user corresponding to the zero pronoun using the behavior information. User character information determining means for determining user character information, which is character information, and information indicating abbreviations complemented at the position of the zero pronoun, and information including user character information determined by the user character information determining means And user abbreviation information insertion means for inserting the user abbreviation information at the position of the corresponding zero pronoun.
With such a configuration, user abbreviation information corresponding to a person's zero pronoun can be inserted at the position of the zero pronoun.

The abbreviation completion device according to the present invention further includes a user attribute information receiving unit that receives user attribute information that is information relating to the user's attributes, and the user character information determining unit is received by the user attribute information receiving unit. The user character information may be determined using user attribute information.
With such a configuration, user character information including a user attribute indicated by the user attribute information can be determined.

  In the abbreviation complementing apparatus according to the present invention, the abbreviation complementing unit further includes user instruction information storage means for storing user instruction information which is information for associating a user's positional relationship with a user's instruction word. The user character information determination means selects, using the user instruction information, an instruction word corresponding to the positional relationship of the user, the speaker user, and the listener user specified by the user specifying means indicated by the behavior information. Then, user character information that is character information including the selected instruction word may be determined.

  With such a configuration, it is possible to select an appropriate instruction word according to the positional relationship of the user, and it is possible to determine appropriate user character information using the selected instruction word.

The apparatus further includes an object information receiving unit that receives object information that is information related to an object in a situation where two or more users are talking and includes at least object position information that is information related to the position of the object. The word completion unit uses the object information to identify an object identifying unit that identifies an object corresponding to the zero pronoun of the object, and an object character that determines the object character information that is character information indicating the object corresponding to the zero pronoun Information determining means and information indicating abbreviations supplemented at the position of the zero pronoun, and the abbreviation information which is information including the object character information determined by the object character information determining means, the corresponding abbreviation of the corresponding zero pronoun There may be provided object abbreviation information insertion means for insertion at the position.
With such a configuration, object abbreviation information corresponding to the zero pronoun of the object can be inserted at the position of the zero pronoun.

In the abbreviation complementing apparatus according to the present invention, the object information also includes object attribute information which is information related to an attribute of the object, and the object character information determining means determines the object character information using the object attribute information. May be.
With such a configuration, the object character information including the object attribute indicated by the object attribute information can be determined.

  Moreover, in the abbreviation complement apparatus according to the present invention, the abbreviation complement unit further includes an object instruction information storage means for storing object instruction information, which is information associating the positional relationship between the user and the object, and the object instruction word. The object character information determining means includes an instruction word indicated by the action information and the object information according to the positional relationship between the object specified by the object specifying means, the user of the speaker, and the user of the listener. Selection may be made using the instruction information, and the object character information which is information of the character including the selected instruction word may be determined.

  With such a configuration, it is possible to select an appropriate instruction word according to the positional relationship between the user and the object, and it is possible to determine appropriate object character information using the selected instruction word.

Further, in the abbreviation complementing apparatus according to the present invention, the zero pronoun specifying unit has a verb extracting means for extracting a verb and a case element of the verb from a document indicated by the character information, and a relationship between the verb and the case element. Case frame information storage means for storing case frame information, which is information to be shown, case element acquisition means for acquiring a case element corresponding to the verb extracted by the verb extraction means, using the case frame information, and the case A case element specifying means for specifying a case element not included in the document indicated by the character information by comparing the case element acquired by the element acquisition means and the case element extracted by the verb extraction means; May be.
With this configuration, the position of the zero pronoun can be specified.

  According to the abbreviation complementing device and the like according to the present invention, in a situation where two or more users are conversing with each other, the abbreviations are complemented by using behavior information including at least the user's position information. It is possible to complement abbreviations that take into account the fact that two or more users are present. As a result, abbreviations that are more appropriate than abbreviations can be complemented by analyzing only the conversation words. In addition, machine translation and the like can be appropriately performed by using a document in which abbreviations are complemented.

  Hereinafter, an abbreviation complement apparatus according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
An abbreviation complement apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an abbreviation complementing apparatus 1 according to the present embodiment. In FIG. 1, the abbreviation completion device 1 according to the present embodiment includes a voice information reception unit 11, a voice information storage unit 12, a character information conversion unit 13, a zero pronoun specification unit 14, and a behavior information reception unit 15. , Behavior information storage unit 16, user attribute information reception unit 17, user attribute information storage unit 18, object information reception unit 19, object information storage unit 20, abbreviation completion unit 21, and supplementary character information output unit 22.

  The voice information receiving unit 11 receives voice information. This voice information is information indicating a voice uttered by at least one user in a situation where two or more users are talking. Here, “two or more users having a conversation” includes users who are not directly participating in the conversation. For example, when two users are having a conversation and one of the two users is given a third user, the third user is also listening to the conversation. It is assumed that the third user is also included in “two or more users having a conversation”. Since the abbreviation word completion processing described later is performed on the character information corresponding to the voice information received by the voice information reception unit 11, at least the voice information reception unit 11 receives voice information for which abbreviation completion is to be performed. It will be good. Needless to say, voice information indicating voices uttered by each user in a situation where two or more users are talking may be accepted. Since this voice information is converted into character information by the character information converter 13 described later, it is preferable that the words uttered by the user are clear.

  The voice information may be acquired by, for example, one or a plurality of microphones installed on the environment side, or may be acquired by a microphone worn by the user. When a plurality of microphones are installed on the environment side, it is also possible to specify which user is speaking using the position of the microphone, the position of the user, and the signal level of the voice information. The microphone worn by the user may be, for example, a normal microphone, a bone conduction microphone, or a throat microphone. In the case of acquiring sound information using a bone conduction microphone or a throat microphone, it is possible to acquire sound information that does not include environmental sound. In addition, it is preferable to acquire audio | voice information so that it can be understood which user uttered which audio | voice for the subsequent process.

  For example, the audio information receiving unit 11 may receive information input from an audio device (for example, a microphone or the like), may receive information transmitted via a wired or wireless communication line, Information read from a recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be accepted. Note that the voice information receiving unit 11 may or may not include a device (for example, a modem or a network card) for receiving. The audio information receiving unit 11 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device. In the present embodiment, as shown in FIG. 1, the voice information receiving unit 11 receives voice information indicating each user's voice acquired by a microphone.

  The audio information storage unit 12 stores the audio information received by the audio information reception unit 11 in a predetermined recording medium. This recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the audio information storage unit 12 or may exist outside the audio information storage unit 12. In the present embodiment, it is assumed that the audio information storage unit 12 has the recording medium. Further, this recording medium may or may not temporarily store audio information.

  The character information conversion unit 13 converts voice information into character information. The character information is information indicating characters corresponding to the voice indicated by the voice information received by the voice information receiving unit 11. The processing by the character information conversion unit 13 is known as a so-called voice recognition technique, and since voice recognition software and the like are already on the market, detailed description thereof is omitted.

  The zero pronoun specifying unit 14 specifies a zero pronoun in the document indicated by the character information generated by the character information converting unit 13. Zero pronoun is an abbreviated case element. A case element is a pair of a semantic element and a case particle. For example, “you are”, “train is”, “to me”, “to that person”, “this book”, “that flower”, and so on.

  FIG. 2 is a block diagram showing a configuration of the zero pronoun specifying unit 14 according to the present embodiment. In FIG. 2, the zero pronoun specifying unit 14 according to the present embodiment includes a verb extracting unit 31, a case frame information storing unit 32, a case element acquiring unit 33, and a case element specifying unit 34.

  The verb extracting means 31 extracts a verb and a case element of the verb from the document indicated by the character information. For example, the verb extraction unit 31 extracts a verb by performing morphological analysis on a document indicated by character information, and sets the extracted case elements such as “O”, “GA”, “NI”, and the like. Extracted by searching using particles as clue phrases. Here, the case element corresponding to the extracted verb is generally present in the position before the verb and is included in the sentence in which the verb exists. The search is performed using as a clue phrase. When the extracted verb case element is not included in the document indicated by the character information, the verb extraction unit 31 does not have to extract the case element. A method for extracting a verb and a case element from a document is conventionally known, and a detailed description thereof will be omitted.

  Case frame information storage means 32 stores case frame information, which is information indicating the relationship between verbs and case elements. With this case frame information, it is possible to know what case element a certain verb takes. A specific example of the case frame information will be described later. The case frame information storage means 32 is realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.). The process in which the case frame information is stored in the case frame information storage means 32 does not matter. For example, the case frame information may be stored in the case frame information storage means 32 via a recording medium, and the case frame information transmitted via a communication line or the like is stored in the case frame information storage means 32. The case frame information input via the input device may be stored in the case frame information storage means 32.

  The case element acquisition unit 33 uses the case frame information stored in the case frame information storage unit 32 to acquire a case element corresponding to the verb extracted by the verb extraction unit 31. That is, the case element acquisition unit 33 searches the case frame information for the verb extracted by the verb extraction unit 31, and reads the case element corresponding to the searched verb from the case frame information.

  The case element specifying unit 34 specifies a case element that is not included in the document indicated by the character information by comparing the case element acquired by the case element acquiring unit 33 and the case element extracted by the verb extracting unit 31. That is, the case element specifying unit 34 converts the case elements included in the one or more case elements acquired by the case element acquisition unit 33 but not included in the zero or more case elements extracted by the verb extraction unit 31 into the character information. Identified as a case element not included in the document shown. The case element specified in this way is a zero pronoun. Specifying a case element that is not included in the document indicated by the character information may be, for example, inserting information indicating that the case element is missing at the position of the document in which the case element should originally be included, Information that associates the position where the case element should originally be included with the information about the missing case element may be stored in a recording medium (not shown), and it is possible to know what kind of case element should be located. In this case, other specific methods may be used. The missing case element information is, for example, information indicating the case particle used in the case element and the type of semantic element. The types of semantic elements include, for example, “person”, “thing”, “location”, and the like.

  The behavior information receiving unit 15 receives behavior information. Here, the action information is information regarding the actions of the two or more users in a situation where two or more users are having a conversation. The action information is information including at least user position information that is information regarding the positions of the two or more users. The behavior information may include motion information that is information indicating the movement of the user. The motion information may include, for example, information related to the movement of the user's body (hereinafter also referred to as “body motion information”), and information related to the movement of the user's line of sight (hereinafter referred to as “line of sight information”). May be included).

  The user position information may be acquired by, for example, a motion capture system, may be acquired by analyzing an image obtained by photographing the user with a camera installed on the environment side, or may be acquired by other methods. . When the motion capture system is used, the position of the marker attached by the user can be detected, so that the user position information can be easily acquired. The motion capture system will be described later. Analyzing an image of a user photographed by a camera means photographing the user image, identifying the user image area from the photographed image using techniques such as skin color detection, difference extraction, or pattern matching. The position corresponding to the image area is extracted. When the position of the user is extracted from the image, the position in the image may be converted to a position in a coordinate system where the user exists (hereinafter also referred to as “world coordinate system”). Further, the position of the user may be acquired by, for example, GPS (Global Positioning System), may be acquired by LPS (Local Positioning System), or may be acquired by using a non-contact type thermo sensor, You may acquire using a laser range finder. The LPS is a device that can calculate a coordinate value from a reference point by using, for example, an active RFID or an autonomous navigation type position measurement sensor having a geomagnetic sensor and an acceleration sensor. A non-contact type thermosensor is a device that can detect the temperature of an object by using infrared rays or the like. Therefore, the presence of a person can be detected by the device, and the position of a person can be detected with high accuracy by using two or more thermosensors. A laser range finder is a device that detects the distance from the reflected light of emitted laser light using the principle of triangulation. A method for acquiring user position information has been conventionally known, and a detailed description thereof will be omitted.

  The body motion information may include information indicating the movement of the user's hand, may include information indicating the movement of the user's head, may include information indicating the movement of the user's torso, It may include information indicating the movement of the other part of the user. The body motion information may be acquired by, for example, a motion capture system, may be acquired by analyzing a video captured by a user with a camera installed on the environment side, or may be acquired by other methods. . The motion capture system attaches a predetermined marker that reflects light to the subject's body, images the subject by using one or more cameras, and detects the movement of the subject's body by calculating the position of the marker. Is. Instead of the reflective marker, an active marker that emits light itself may be used. Moreover, you may use a magnetic marker instead of an optical marker. Thus, the motion capture system may be of any type as long as it can detect the movement of the subject. When the behavior information is acquired by the motion capture system, the body motion information is, for example, time-series data indicating the position of each marker in the three-dimensional coordinate system. To analyze a video by shooting a user, the user's video is shot, the image area of the user is identified from the shot video using a technique such as skin color detection, difference extraction, or pattern matching. Extracting the user's hand movement, head movement, and the like from the image area. Other methods for acquiring body motion information include, for example, a method in which an acceleration sensor is attached to one or more parts of the subject's body, and the motion of the subject's body is detected by the acceleration sensor. Also, the above two or more methods for acquiring body motion information may be combined. A method for acquiring body motion information has been conventionally known, and a detailed description thereof will be omitted.

  The line-of-sight information may include information indicating the direction of the user's line of sight, and may include information indicating the direction of the user's head when approximating the direction of the user's line of sight with the direction of the user's head. Of course, it may contain information about the user's other line of sight. Here, when the subject is standing, it is generally known that the line of sight is within the range of 90 degrees of the front direction of the face, so the direction of the user's line of sight is approximated by the direction of the user's head. It is also possible to do. The line-of-sight information may be acquired by, for example, an eye camera, may be acquired by analyzing a moving image shot by a camera installed on the environment side of the user, or may be acquired by other methods. An eye camera is a device that measures the eye movement of a subject. The eye camera measures the eye movement of the subject using, for example, a Limbus tracking method (scleral reflection method), a corneal reflection method, an image processing method, or the like. The image processing method is a method of detecting the center of the pupil and extracting the line-of-sight direction, and there is almost no burden on the subject. Analyzing a video shot of a user is to identify the position of the user's eyes using a technique such as pattern matching, and use the same method as the above image processing method or a method of comparing the white and black eye areas. It is to extract the direction. In addition, as another method of acquiring line-of-sight information, a method of detecting the orientation of the subject's head using a motion capture system, an acceleration sensor is attached to the subject's head, and the motion of the subject's head is detected by the acceleration sensor. A method of detecting the movement of the subject's head by attaching a light-emitting unit that outputs light having directivity and convergence to the subject's head, and detecting the light from the light-emitting unit by a detection unit installed on the environment side And a method of detecting the movement of the subject's head by taking an image of the subject's line of sight and the like by attaching a head-mounted camera to the subject's head and analyzing the image. In addition, the above two or more methods for acquiring line-of-sight information may be combined. In addition, when the line-of-sight information is information indicating the direction of the subject's line of sight, information in the local coordinate system (the visual field coordinate system set on the subject's head) may be used as the line-of-sight information, or the local coordinate system Information obtained by converting the value of into the world coordinate system that is the same coordinate system as the data of the motion capture system or the like may be used as line-of-sight information. A method for obtaining the line-of-sight information has been conventionally known, and a detailed description thereof will be omitted.

  For example, the behavior information receiving unit 15 may receive information input from an input device (for example, a keyboard, a mouse, a touch panel, etc.), or may receive information transmitted via a wired or wireless communication line. In many cases, information read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be accepted. Note that the behavior information receiving unit 15 may or may not include a device (for example, a modem or a network card) for receiving. The accepting unit may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device. In the present embodiment, as shown in FIG. 1, the behavior information receiving unit 15 receives line-of-sight information acquired by an eye camera and behavior information acquired by a motion capture system.

  The behavior information storage unit 16 stores the behavior information received by the behavior information reception unit 15 in a predetermined recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the behavior information storage unit 16 or may exist outside the behavior information storage unit 16. In the present embodiment, it is assumed that the behavior information storage unit 16 has the recording medium. Moreover, this recording medium may memorize | store behavior information temporarily, and may not be so.

  The user attribute information receiving unit 17 receives user attribute information. Here, the user attribute information is information relating to user attributes. The user attribute information receiving unit 17 may receive user attribute information corresponding to at least one user in a situation where two or more users are talking. User attributes include, for example, the user's gender, the user's age, the user's name, the user's occupation, and the like. The user attribute information may be information indicating the user attribute as long as the user attribute information can be finally specified, or may be information associated with the user attribute. Good. The information associated with the user attribute is, for example, a user ID. In that case, the user ID only needs to be associated with the user attribute in the predetermined database information. This is because user attributes can be acquired using the user ID. The user attribute information may be acquired from, for example, an RFID attached to the user's clothes and the like, and identification information such as a barcode attached to the user's clothes and the like is specified in an image taken of the user Or may be obtained by other methods. Moreover, the user attribute information reception part 17 may receive user attribute information with the information which shows a position. For example, user attribute information acquired from an RFID and information indicating the position of the RFID may be received by the user attribute information receiving unit 17. In the present embodiment, as shown in FIG. 1, user attribute information and information indicating a position are acquired by an RFID attached to a user's clothes and received by the user attribute information receiving unit 17. .

  For example, the user attribute information receiving unit 17 may receive information input from an input device (for example, an RFID receiving device) or may receive information transmitted via a wired or wireless communication line. In many cases, information read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be accepted. Note that the user attribute information reception unit 17 may or may not include a device (for example, a modem or a network card) for reception. Further, the user attribute information receiving unit 17 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The user attribute information storage unit 18 stores the user attribute information received by the user attribute information reception unit 17 in a predetermined recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the user attribute information storage unit 18 or may exist outside the user attribute information storage unit 18. In the present embodiment, it is assumed that the user attribute information storage unit 18 has the recording medium. In addition, this recording medium may temporarily store user attribute information or not.

  The object information receiving unit 19 receives object information. Here, the item information is information about the item in a situation where two or more users are talking. The thing exists in the place where two or more users are talking, for example, any user may hold | maintain or may exist in the vicinity of either user. The object information is information including at least object position information that is information regarding the position of the object. The object information may include object attribute information that is information related to the attribute of the object. The attribute of an object is, for example, the name of the object, the type of the object, the size of the object, the color of the object, and the like. The object attribute information may be information indicating the attribute of the object as long as it can finally specify the attribute of the object, or may be information associated with the attribute of the object. Good. The information associated with the attribute of the object is, for example, an object ID. In that case, the object ID only needs to be associated with the attribute of the object in the predetermined database information. This is because the attribute of the object can be acquired using the object ID.

  The object position information may be acquired, for example, by analyzing an image obtained by photographing an object with a camera installed on the environment side, or may be acquired by other methods. Analyzing an image of an object taken with a camera means taking an image of the object, identifying an image area of the object using a technique such as pattern matching from the taken image, and a position corresponding to the identified image area Is to extract. When the position of the object is extracted from the image, the position in the image may be converted to a position in the world coordinate system where the object exists. In addition, the position of the object may be acquired by GPS or may be acquired by LPS, for example.

  The object attribute information may be acquired from, for example, an RFID attached to the object, or may be acquired by specifying identification information such as a barcode attached to the object in an image obtained by photographing the object. It may be obtained by other methods. When the object information is acquired from the RFID attached to the object, the object attribute information and the object position information can be acquired together by the RFID. In the present embodiment, as shown in FIG. 1, the object attribute information and the object position information are acquired by the RFID attached to the object, and the object information including the object attribute information and the object position information. Is received by the object information receiving unit 19.

  For example, the object information receiving unit 19 may receive information input from an input device (for example, an RFID receiving device) or may receive information transmitted via a wired or wireless communication line. Information read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be accepted. Note that the object information reception unit 19 may or may not include a device (for example, a modem or a network card) for reception. In addition, the object information receiving unit 19 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The product information storage unit 20 stores the product information received by the product information reception unit 19 in a predetermined recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the physical information storage unit 20 or may exist outside the physical information storage unit 20. In the present embodiment, it is assumed that the object information storage unit 20 has the recording medium. In addition, this recording medium may or may not store physical information temporarily.

  The abbreviation complementing unit 21 generates supplementary character information using the behavior information received by the behavior information receiving unit 15, the user attribute information received by the user attribute information receiving unit 17, and the object information received by the object information receiving unit 19. To do. Here, the supplementary character information is information obtained by supplementing an abbreviation that is an abbreviated word at the position of the zero pronoun specified by the zero pronoun specifying unit 14 in the document indicated by the character information. The abbreviation complement unit 21 may supplement abbreviations using voice information.

  FIG. 3 is a block diagram showing a configuration of the abbreviation complement unit 21 according to the present embodiment. In FIG. 3, the abbreviation complementing unit 21 according to the present embodiment includes a user specifying means 41, a user instruction information storage means 42, a user character information determining means 43, a user abbreviation information inserting means 44, and an object specifying means. 45, an object instruction information storage means 46, an object character information determination means 47, and an object abbreviation information insertion means 48.

  The user specifying means 41 specifies a user corresponding to a person's zero pronoun using the behavior information. This user identification may be performed only when a person's zero pronoun is identified by the zero pronoun identifying unit 14. A person's zero pronoun is a zero pronoun that is a combination of a semantic element indicating a person and a case particle. Specifying the user may be, for example, accumulating identification information for identifying the user in a predetermined recording medium, or setting a predetermined flag on the identification information stored in advance corresponding to the user. Other specific methods may be used as long as it is known which user corresponds to the zero pronoun. The user specifying unit 41 may specify a user using information other than behavior information. For example, the user specifying unit 41 may specify the user using the voice information received by the voice information receiving unit 11.

Next, a method in which the user specifying unit 41 specifies a user will be described.
[Method based on speaker's user instruction]
A user who corresponds to a person's zero pronoun may be able to be identified by an instruction act of a speaker's user. Here, the user's instruction act is an act of pointing the user in a certain direction, for example, an act of pointing in a certain direction, an act of directing a face in a certain direction, an act of directing a line of sight in a certain direction, or the like. For example, the user of the speaker points to another user (hereinafter sometimes referred to as a “third-party user”) while saying “Can you help?” By moving the line of sight in the direction of the user, the listener's user can know that the third-party user is requested to help. By detecting the pointing action, it may be possible to identify a user corresponding to a person's zero pronoun.

  The user specifying means 41 holds, for example, condition information, which is information indicating the condition of occurrence of the user's instruction action, and refers to the condition information so that the user can respond to the voice (for example, “help When the occurrence of the pointing action is detected, the direction indicated by the pointing action is specified. And the user specific | specification means 41 specifies the user located in the direction as a user corresponding to a person's zero pronoun using action information. If there are two or more users in that direction, for example, the user closest to the speaker's user may be specified.

  Note that the condition information related to pointing is, for example, that at least from the elbow to the fingertip extends in a direction that is almost horizontal and is stationary for a predetermined time (for example, 1 second). Moreover, the condition information regarding the action of turning the line of sight is, for example, that the line of sight is fixed in a certain direction for a predetermined time (for example, 1 second).

[How to judge from the situation]
In a situation where two or more users are having a conversation, it may be possible to identify a user corresponding to a person's zero pronoun from the situation. For example, in a situation where the speaker user asks the listener user, "Can you help?" If there are only two users, the speaker user and the listener user, the listener user is the speaker user. You can know that you are asked to help. Thus, the user corresponding to a person's zero pronoun may be able to be specified according to the situation of the place.

  For example, as the person corresponding to the purpose case of the verb, the user specifying means 41 includes personal purpose case information which is information indicating which user is included among the user of the speaker, the user of the listener, and the user of the third party. It retains, and it specifies as a user corresponding to a person's zero pronoun by using the personal purpose personality information and the number of users shown by action information. For example, the user corresponding to the purpose of the verb “help” corresponds to the purpose of the verb “help” in a situation where there is no third-party user because the user is a speaker user and a third-party user. The user can be identified as the speaker's user, and the user corresponding to the person's zero pronoun can be identified.

[Method of judging from user action results]
In some cases, a user corresponding to a person's zero pronoun can be identified from the user's action result made in response to the voice uttered by the user of the speaker. For example, if a speaker's user approaches a third-party user after the speaker's user says, "Can you help me?" Was asked to help the third-party user. Thus, the user corresponding to a person's zero pronoun may be able to be specified by a user's action result.

  The user specifying unit 41 may specify, for example, a user who has approached the listener as a user corresponding to a person's zero pronoun corresponding to the voice uttered by the user of the speaker. In this case, the user specifying unit 41 can determine from the voice information, the character information, and the behavior information that the user's response corresponding to the speaker's user is positive, that is, “Yes”, This determination method may be used only when the listener's user utters a positive sound such as “OK” or when the listener's user makes a nodding action.

  Needless to say, if the user can be specified by a method other than the above, that method may be used. Further, a user who exists outside a certain range (for example, a range within a radius of 10 m) from the position of the speaker's user may not be included in the specified user. This is because it is difficult for the speaker user to assume such a distant user as a user corresponding to zero pronouns.

  Next, a method for identifying the speaker user and the listener user will be described. Here, the user of the speaker is a user who made an utterance corresponding to character information that complements the abbreviation. Therefore, the first user and the second user have a conversation, the first user speaks, and the second user responds to the utterance. When the abbreviations are complemented for the character information corresponding to, the first user becomes the speaker user and the second user becomes the listener user.

[Speaker user]
Which user is the speaker's user may be determined from text information or from voice information. For example, it is determined that user identification information is associated with character information, and the user identified by the user identification information is determined to be a user who uttered a voice corresponding to the character information, that is, a speaker user. Also good. Further, for example, even if the identification information of the user is associated with the voice information, and the user identified by the identification information of the user is determined to be the user of the speaker who uttered the voice corresponding to the voice information. Good.

[Interviewer user]
When the user's line-of-sight direction and body direction can be known from the behavior information, it may be determined that the user whose face is facing the user of the speaker is the user of the listener. In addition, when the voice information, text information, or behavior information indicates that the listener user is responding to the voice of the speaker user, it is determined that the responding user is the listener user. May be. For example, a user who is responding to a speaker's user's utterance, a user who is nodding, or a user who is gesturing an answer to the user of the speaker may be determined as the listener's user. Whether or not a reply is made to the speaker's user's utterance may be determined from, for example, audio information, and is determined based on whether or not the user's mouth is moving using image data of a moving image of the user. May be.
Needless to say, if the user of the speaker and the user of the listener can be specified by a method other than the above, that method may be used.

  The user instruction information storage means 42 stores user instruction information. Here, the user instruction information is information that associates the positional relationship of the user with the instruction word of the user. The user's positional relationship is a positional relationship among a speaker user, a listener user, and a third party user. The user instruction word is, for example, “over there” or “here”. A specific example of the user instruction information will be described later.

  The user instruction information storage means 42 is realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.). The process in which the user instruction information is stored in the user instruction information storage means 42 does not matter. For example, user instruction information may be stored in the user instruction information storage means 42 via a recording medium, and user instruction information transmitted via a communication line or the like is stored in the user instruction information storage means 42. Alternatively, the user instruction information input via the input device may be stored in the user instruction information storage unit 42.

  The user character information determination unit 43 determines user character information that is character information indicating a user corresponding to a zero pronoun. A case element which is a combination of the user character information and a case particle is inserted at the position of the specified zero pronoun. When the user attribute information corresponding to the user specified by the user specifying unit 41 is received by the user attribute information receiving unit 17, the user character information determining unit 43 uses the user attribute information to May be determined. In this case, for example, the user character information “A woman there” is determined. Further, the user character information determination unit 43 uses the user instruction information stored in the user instruction information storage unit 42 to select an instruction word corresponding to the positional relationship indicated by the behavior information, and selects the selected instruction word. User character information, which is information of characters to be included, may be determined. The positional relationship indicated by the behavior information is the positional relationship of the speaker user, the listener user, and the user specified by the user specifying means 41. In addition, when any two of the three users are the same user, the positional relationship of the two users is obtained. The character information including the selected instruction word may be the selected instruction word itself, or may include the word indicated by the user attribute information in addition to the selected instruction word. Determining the user character information may be, for example, generating user character information, which is user character information among user instruction words in the user instruction information stored in the user instruction information storage unit 42. A predetermined flag may be set on the object, and the user character information determination method is not limited as long as the user character information can be specified.

  User abbreviation information insertion means 44 inserts user abbreviation information at the position of the corresponding zero pronoun. Here, the user abbreviation information is information indicating an abbreviation supplemented at the position of the zero pronoun, and is information including the user character information determined by the user character information determination unit 43. The user abbreviation information is information indicating case elements including user character information and case particles. The case particles included in the abbreviations indicated by the user abbreviation information may be determined by the user abbreviation information insertion means 44, or may be determined by the zero pronoun specifying unit 14. . When the user abbreviation information insertion means 44 inserts user abbreviation information, abbreviations relating to the user are complemented. The position of the zero pronoun corresponding to the user abbreviation information is the position of the person's zero pronoun corresponding to the user specified by the user specifying means 41 in order to determine the user character information included in the user abbreviation information.

  The object specifying means 45 specifies an object corresponding to the zero pronoun of the object using at least the object information. The object specifying means 45 may use action information in addition to the object information, and may also use voice information in specifying the object corresponding to the zero pronoun of the object. The identification of the object may be performed only when the zero pronoun of the object is specified by the zero pronoun specifying unit 14. The zero pronoun of a thing is a zero pronoun that is a combination of a semantic element indicating a thing and a case particle. Specifying an object may be, for example, accumulating identification information for identifying the object on a predetermined recording medium, or setting a predetermined flag on identification information stored in advance corresponding to the object. It may be set, and other specific methods may be used as long as it can be understood which object corresponds to the zero pronoun.

Next, a method in which the object specifying unit 45 specifies an object will be described.
[Method based on speaker's user instruction]
In some cases, the thing corresponding to the zero pronoun of the thing can be specified by the instruction action of the speaker's user. For example, when the speaker user says to the listener user, "Can you take it?", Pointing the object or moving the line of sight toward the object, You can know that you have been requested to acquire. By detecting the pointing action, it may be possible to identify the user corresponding to the zero pronoun of the object.

  The object specifying unit 45 holds, for example, condition information that is information indicating a condition of occurrence of the user's instruction act, and refers to the condition information, thereby allowing the user to respond to a voice (for example, “ When the occurrence of the pointing action is detected, the direction indicated by the pointing action is specified. And the object specific | specification means 45 specifies the thing located in the direction as an object corresponding to the zero pronoun of an object using action information. If there are two or more objects in that direction, for example, the object closest to the user of the speaker may be specified.

[How to judge from the situation]
In a situation where two or more users are having a conversation, it may be possible to identify the user corresponding to the zero pronoun of the object from the situation. For example, in a situation where the speaker user asks the listener user to “take it?”, If there is only one item on the spot, the listener user is requested to acquire the item. Can know. Thus, the thing corresponding to the zero pronoun of a thing may be able to be specified according to the situation of the place. In a situation where two or more users are having a conversation, even if there are actually two or more objects, this determination is made when only one object can be recognized by the abbreviation complementing apparatus 1. You may identify a thing using a method.

[Method of judging from user action results]
In some cases, the user corresponding to the zero pronoun of an object can be identified from the user's action result made in response to the voice uttered by the user of the speaker. For example, if a speaker's user approaches a certain object after the speaker's user has asked the listener's user "Can you take it?" You can see that the acquisition was requested. Thus, the thing corresponding to the zero pronoun of a thing may be able to be specified by a user's action result.

  For example, the object specifying unit 45 may specify an object that the listener's user has approached in response to a voice uttered by the user of the speaker as an object corresponding to the zero pronoun of the object. In this case, the object specifying unit 45 can determine from the voice information, the character information, and the behavior information that the user's response corresponding to the speaker user is affirmative, that is, “Yes”, This determination method may be used only when the listener's user utters a positive sound such as “OK” or when the listener's user makes a nodding action.

  Needless to say, if an object can be specified by a method other than the above, that method may be used. In addition, an object that exists outside a certain range (for example, a range within a radius of 10 m) from the position of the speaker's user may not be included in the specified object. This is because it is unlikely that the speaker's user assumes such a distant object as an object corresponding to the zero pronoun. The method for specifying the user of the speaker and the user of the listener is as described above.

  The object instruction information storage means 46 stores object instruction information. Here, the object instruction information is information that associates the positional relationship between the user and the object and the instruction word of the object. The positional relationship between the user and the object is a positional relationship between the user of the speaker, the user of the listener, and the object. The instruction word for an object is, for example, “that”, “this”, “it”, or the like. A specific example of the object instruction information will be described later.

  The object instruction information storage means 46 is realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.). The process in which the object instruction information is stored in the object instruction information storage means 46 does not matter. For example, the object instruction information may be stored in the object instruction information storage means 46 via a recording medium, and the object instruction information transmitted via a communication line or the like is stored in the object instruction information storage means 46. Alternatively, the object instruction information input via the input device may be stored in the object instruction information storage means 46.

  The object character information determination unit 47 determines object character information which is information of a character indicating an object corresponding to the zero pronoun. The case element, which is a combination of the object character information and the case particle, is inserted at the position of the specified zero pronoun. When the object attribute information corresponding to the object specified by the object specifying unit 45 is received by the object information receiving unit 19, the object character information determining unit 47 uses the object attribute information to obtain the object character information. You may decide. In this case, for example, the user character information “that book” is determined. Further, the object character information determination unit 47 selects an instruction word corresponding to the positional relationship indicated by the action information and the object information by using the object instruction information stored in the object instruction information storage unit 46, and selects the selected item. Object character information that is character information including the instruction word may be determined. The positional relationship indicated by the behavior information and the object information is the positional relationship between the speaker user, the listener user, and the object specified by the object specifying means 45. The character information including the selected instruction word may be the selected instruction word itself, or may include a word indicated by the object attribute information in addition to the selected instruction word. Determining the object character information may be, for example, generating object character information, which is the object character information among the object instruction words in the object instruction information stored in the object instruction information storage unit 46. A predetermined flag may be set on the object, and the method for determining the object character information is not limited as long as the object character information can be specified.

  The thing abbreviation information insertion means 48 inserts the thing abbreviation information at the position of the corresponding zero pronoun. Here, the thing abbreviation information is information indicating an abbreviation supplemented at the position of the zero pronoun, and is information including the object character information determined by the object character information determination unit 47. The object abbreviation information is information indicating case elements including object character information and case particles. The case particles included in the abbreviations indicated by the thing abbreviation information may be determined by the object abbreviation information insertion means 48, or may be determined by the zero pronoun specifying unit 14. . When the abbreviation information insertion means 48 inserts the abbreviation information, the abbreviations relating to the thing are complemented. The position of the zero pronoun corresponding to the object abbreviation information is the position of the zero pronoun of the object corresponding to the object specified by the object specifying means 45 in order to determine the object character information included in the object abbreviation information.

  The supplementary character information output unit 22 outputs the supplementary character information generated by the abbreviation complementation unit 21. Here, the output may be, for example, display on a display device (for example, a CRT or a liquid crystal display), transmission via a communication line to a predetermined device, printing by a printer, or audio output by a speaker. However, it may be stored on a recording medium. The supplementary character information output unit 22 may or may not include a device that performs output (for example, a display device or a printer). The complementary character information output unit 22 may be realized by hardware, or may be realized by software such as a driver that drives these devices. In the present embodiment, the supplementary character information output unit 22 displays the supplementary character information on the display.

  The recording medium included in the audio information storage unit 12, the recording medium included in the behavior information storage unit 16, the recording medium included in the user attribute information storage unit 18, the recording medium included in the object information storage unit 20, and the case frame information. Of the storage unit 32, the user instruction information storage unit 42, and the object instruction information storage unit 46, any two or more recording media may be realized by the same recording medium or by different recording media. Also good. In the former case, for example, an area in which the case frame information is stored in the recording medium becomes the case frame information storage means 32.

  Next, the operation of the abbreviation complement device 1 according to the present embodiment will be described using a flowchart. FIG. 4 is a flowchart showing the overall operation of the abbreviation complementing apparatus 1 according to this embodiment.

  (Step S101) The abbreviation complement device 1 determines whether to abbreviate abbreviations. If the abbreviation is complemented, the process proceeds to step S102. If not, the process proceeds to step S103. Here, the abbreviation complementing device 1 has received a predetermined event (for example, completion of a series of accumulations of voice information, behavior information, user attribute information, and object information, and an instruction to supplement abbreviations). Or the like) as a trigger, it may be determined that the abbreviation is supplemented, it may be determined that the abbreviation is supplemented every predetermined period, or it may be determined that the abbreviation is supplemented at other timings.

  (Step S102) The abbreviation complement unit 21 and the like perform processing for complementing abbreviations. Then, the process returns to step S101. Details of the process of complementing this abbreviation will be described later using the flowchart of FIG.

(Step S103) The voice information receiving unit 11 determines whether voice information has been received. If the audio information is received, the process proceeds to step S104. If not, the process proceeds to step S105.
(Step S104) The audio information storage unit 12 stores the audio information received by the audio information reception unit 11 in a predetermined recording medium. Then, the process returns to step S101.

  (Step S105) The behavior information reception unit 15 determines whether behavior information has been received. When the behavior information includes body motion information, line-of-sight information, user position information, or the like, it may be determined that the behavior information has been received when at least one of the information is received. If the behavior information is accepted, the process proceeds to step S106. If not, the process proceeds to step S107.

(Step S106) The behavior information storage unit 16 stores the behavior information received by the behavior information reception unit 15 in a predetermined recording medium. Then, the process returns to step S101.
(Step S107) The user attribute information receiving unit 17 determines whether user attribute information has been received. If user attribute information is accepted, the process proceeds to step S108, and if not, the process proceeds to step S109.

  (Step S108) The user attribute information storage unit 18 stores the user attribute information received by the user attribute information reception unit 17 in a predetermined recording medium. Then, the process returns to step S101.

  (Step S109) The object information reception part 19 judges whether the object information was received. When the object information includes the object position information and the object attribute information, it may be determined that the object information is received when at least one of them is received. If the object information is accepted, the process proceeds to step S110. If not, the process returns to step S101.

(Step S110) The product information storage unit 20 stores the product information received by the product information reception unit 19 in a predetermined recording medium. Then, the process returns to step S101.
In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

FIG. 5 is a flowchart showing details of the abbreviation complementing process (step S102) in the flowchart of FIG.
(Step S201) The character information conversion unit 13 reads the voice information from the voice information storage unit 12. The character information conversion unit 13 may read all the voice information stored by the voice information storage unit 12, or may read a part thereof.

(Step S202) The character information conversion unit 13 converts the voice information read from the voice information storage unit 12 into character information.
(Step S203) The zero pronoun specifying unit 14 specifies a zero pronoun in the document indicated by the character information. Details of the process for specifying the zero pronoun will be described later with reference to the flowchart of FIG.

  (Step S204) The abbreviation complementing unit 21 generates supplementary character information that is information obtained by complementing the abbreviation at the position of the zero pronoun using the behavior information, the user attribute information, and the object information. Details of the process of generating the supplementary character information will be described later with reference to the flowchart of FIG.

  (Step S205) The supplementary character information output unit 22 outputs the supplementary character information generated by the abbreviation complementation unit 21. Then, the series of processes for complementing the abbreviations ends, and the process returns to step S101 in FIG.

  In this embodiment, a case where speech information is converted into character information when executing abbreviation complement processing will be described. However, conversion from speech information to character information is performed after speech information is accumulated. What is necessary is just to be performed at the arbitrary time until abbreviation completion processing is performed. For example, when the processing load of the abbreviation complement device 1 is small, the conversion process from voice information to character information may be executed.

FIG. 6 is a flowchart showing details of the zero pronoun specification process (step S203) in the flowchart of FIG.
(Step S301) The verb extracting means 31 extracts a verb and a case element of the verb from the document indicated by the character information.

(Step S302) The case element acquisition means 33 acquires the case element corresponding to the verb extracted in step S301 from the case frame information stored in the case frame information storage means 32.
(Step S303) The case element specifying unit 34 compares the case element extracted in Step S301 with the case element acquired in Step S302.

  (Step S304) The case element specifying means 34 includes a case element included in one or more case elements acquired in step S302 but not included in zero or more case elements extracted in step S301 in the document indicated by the character information. It is specified as a case element. This identified case element is a zero pronoun. Then, the series of processes for specifying the zero pronouns ends, and the process proceeds to step S204 in FIG.

  When two or more verbs are extracted in step S301 in the flowchart of FIG. 6, the processing from step S302 to step S304 is executed for each verb.

  FIG. 7 is a flowchart showing details of the complementary character information generation processing (step S204) in the flowchart of FIG.

  (Step S401) The object specifying unit 45 determines whether there is a zero pronoun of the object, that is, whether the zero pronoun of the object is specified by the zero pronoun specifying unit 14. If there is a zero pronoun of the object, the process proceeds to step S402, and if not, the process proceeds to step S406.

  (Step S402) The object specifying means 45 determines whether an object corresponding to the zero pronoun of the object can be specified. If the object can be specified, the process proceeds to step S403, and if not, the process proceeds to step S406.

(Step S403) The object specifying means 45 specifies an object corresponding to the zero pronoun of the object.
(Step S404) The physical character information determination means 47 determines physical character information indicating an object corresponding to the zero pronoun. The details of the object character information determination process will be described later with reference to the flowchart of FIG.

  (Step S405) The object abbreviation information insertion means 48 inserts the object abbreviation information including the object character information determined in step S404 at the position of the corresponding zero pronoun in the character information. In this way, the completion of the abbreviation of the object is completed.

  (Step S406) The user specifying unit 41 determines whether there is a user's zero pronoun, that is, whether the user's zero pronoun is specified by the zero pronoun specifying unit 14. If the user's zero pronoun is present, the process proceeds to step S407. If not, the series of processes for complementing the abbreviation is terminated, and the process proceeds to step S205 in FIG.

  (Step S407) The user specifying means 41 determines whether a user corresponding to the user's zero pronoun can be specified. If the user can be specified, the process proceeds to step S408. If not, the series of processes for complementing the abbreviation is terminated, and the process proceeds to step S205 in FIG.

(Step S408) The user specifying means 41 specifies a user corresponding to the user's zero pronoun.
(Step S409) The user character information determination means 43 determines the user character information which shows the user corresponding to a zero pronoun. Details of the user character information determination process will be described later with reference to the flowchart of FIG.

  (Step S410) The user abbreviation information insertion means 44 inserts the user abbreviation information including the user character information determined in step S409 at the position of the corresponding zero pronoun in the character information. In this way, completion of the user abbreviations is completed, and the process proceeds to step S205 in FIG.

  The flowchart of FIG. 7 shows the case where the user abbreviation information insertion process is performed after the object abbreviation information insertion process is performed. However, after the user abbreviation information insertion process is performed, the object abbreviation information is omitted. Word information insertion processing may be performed.

FIG. 8 is a flowchart showing details of the object character information determination process (step S404) in the flowchart of FIG.
(Step S501) The object character information determining means 47 determines the positional relationship between the speaker user, the listener user, and the object specified by the object specifying means 45. The method for identifying the user of the speaker and the user of the listener is as described above. By specifying the user of the speaker and the user of the listener and using the action information, the object character information determination unit 47 can know the positions of the user of the speaker and the user of the listener. The user of the speaker and the user of the listener may be specified by other components, for example, the user specifying means 41. Further, the object character information determination unit 47 can know the position of the object specified by the object specifying unit 45 by using the object information.

  In the present embodiment, it is assumed that the positional relationship between the speaker user, the listener user, and the object specified by the object specifying means 45 includes a near name, a middle name, and a far name. The names of these positional relations “near name”, “middle name”, and “distant name” are derived from the corresponding pronoun names “near name pronoun”, “middle name pronoun”, and “far name pronoun”. .

The near name is when the position of the object specified by the object specifying means 45 is closer to the speaker than the listener, and the distance between the speaker and the specified object is smaller than the distance between the speaker and the listener. In other words, the following two conditions are satisfied:
Condition 1: Distance between the speaker and the identified object ≦ Distance between the listener and the identified object Condition 2: Distance between the speaker and the identified object ≦ Distance between the speaker and the listener

The middle name is a case where the position of the object specified by the object specifying unit 45 is closer to the listener than the speaker, and the distance between the listener and the specified object is smaller than the distance between the speaker and the listener. In other words, if the following two conditions are satisfied, it is an intermediate name.
Condition 1: Distance between the listener and the identified object <Distance between the speaker and the identified object Condition 2: Distance between the listener and the identified object ≦ Distance between the speaker and the listener

A distant name is a case where the position of the object specified by the object specifying means 45 is farther from the speaker and the listener than the distance between the speaker and the listener. That is, when the following two conditions are satisfied, it becomes a distant name.
Condition 1: Distance between the speaker and the identified object> Distance between the speaker and the listener Condition 2: Distance between the listener and the identified object> Distance between the speaker and the listener

  (Step S502) The object character information determining unit 47 determines whether or not the object information including the object attribute information corresponding to the object specified by the object specifying unit 45 is received by the object information receiving unit 19. For example, the object character information determining unit 47 may identify the object specified by the object specifying unit 45 when the object attribute information corresponding to the object specified by the object specifying unit 45 is stored in the recording medium of the object information storage unit 20. It may be determined that the item information including the item attribute information corresponding to is received by the item information receiving unit 19. If accepted, the process proceeds to step S503, and if not, the process proceeds to step S504.

  (Step S503) The object character information determination unit 47 selects an object instruction word corresponding to the positional relationship determined in Step S501 from the object instruction information storage unit 46. In this case, the object character information determination unit 47 selects an instruction word when the object character information includes object attribute information. Then, the object character information determining means 47 determines object character information using the selected instruction word and object attribute information. In this way, a series of processes for determining physical character information is completed, and the process proceeds to step S405 in FIG.

  (Step S504) The object character information determination unit 47 selects an object instruction word corresponding to the positional relationship determined in step S501 from the object instruction information storage unit 46. In this case, the object character information determination unit 47 selects an instruction word when the object character information does not include the object attribute information. Then, the object character information determining means 47 determines the selected instruction word as object character information. In this way, a series of processes for determining physical character information is completed, and the process proceeds to step S405 in FIG.

FIG. 9 is a flowchart showing details of the object character information determination process (step S409) in the flowchart of FIG.
(Step S601) The user character information determination means 43 determines the positional relationship among the speaker user, the listener user, and the user specified by the user specifying means 41. The method for identifying the user of the speaker and the user of the listener is as described above. By identifying the user of the speaker and the user of the listener and using the behavior information, the user character information determination unit 43 can know the position of the user specified by the user of the speaker, the user of the listener, and the user specifying unit 41. . The user of the speaker and the user of the listener may be specified by other components, for example, the user specifying means 41. In this embodiment, it is assumed that the positional relationship includes a near name, a middle name, and a far name. The near name, middle name, and far name are the same as those described in step S501 except that the item specified by the item specifying unit 45 is the user specified by the user specifying unit 41, and detailed description thereof is omitted. To do. However, when the user specified by the user specifying means 41 is the same as the speaker or the listener, the instruction word corresponding to the user specified by the user specifying means 41 is “I” or “you”. Become.

  (Step S <b> 602) The user character information determination unit 43 determines whether user attribute information corresponding to the user specified by the user specifying unit 41 has been received by the user attribute information receiving unit 17. For example, the user character information determination means 43 is specified by the user specification means 41 when the user attribute information corresponding to the user specified by the user specification means 41 is stored in the recording medium of the user attribute information storage unit 18. It may be determined that the user attribute information corresponding to the user has been received by the user attribute information receiving unit 17. If accepted, the process proceeds to step S603; otherwise, the process proceeds to step S604.

  (Step S603) The user character information determination unit 43 selects a user instruction word corresponding to the positional relationship determined in step S601 from the user instruction information storage unit 42. In this case, the user character information determination unit 43 selects an instruction word when the user character information includes user attribute information. And the user character information determination means 43 determines user character information using the selected instruction word and user attribute information. In this way, a series of processes for determining the user character information ends, and the process proceeds to step S410 in FIG.

  (Step S604) The user character information determination unit 43 selects a user instruction word corresponding to the positional relationship determined in step S601 from the user instruction information storage unit 42. In this case, the user character information determination unit 43 selects an instruction word when the user character information does not include user attribute information. Then, the user character information determination unit 43 determines the selected instruction word as user character information. In this way, a series of processes for determining the user character information ends, and the process proceeds to step S410 in FIG.

Next, operation | movement of the abbreviation complement apparatus 1 by this Embodiment is demonstrated using a specific example.
In this specific example, body motion information of each user is acquired by a motion capture system, user's line-of-sight information is acquired by an eye camera, and motion information including body motion information and line-of-sight information is received by the behavior information receiving unit 15. Shall. In addition, it is assumed that the voice uttered by each user is acquired by a microphone and the voice information is received by the voice information receiving unit 11. It is assumed that these behavior information and audio information are received in association with user identification information for identifying each user, and are stored in association with the user identification information.

  In this specific example, it is assumed that user attribute information is stored in the RFID attached to each user's clothes. Further, it is assumed that the object attribute information is stored in the RFID attached to the object. The RFID reader acquires user attribute information and the position of the RFID from the RFID attached to the user's clothes. Further, the RFID reader acquires the object attribute information and the position of the RFID from the RFID attached to the object.

  FIG. 10 is a diagram for explaining devices and the like worn by each user in this specific example. As shown in FIG. 10, each user wears a microphone, and the voice uttered by the user is acquired by the microphone. The user wears an eye camera, and the user's line-of-sight direction is acquired by the eye camera. In addition, the user is provided with markers for detecting positions with a motion capture system on a plurality of parts of the body. In addition, the user is provided with an RFID in which user attribute information corresponding to the user is stored.

  The voice information acquired by the microphone, the line-of-sight information acquired by the eye camera, and the body motion information acquired by the motion capture system are respectively passed to the abbreviation complementing device 1 by wired or wireless communication. The voice information is received by the voice information receiving unit 11 (step S103) and stored by the voice information storage unit 12 (step S104). The line-of-sight information and the body motion information are received by the behavior information receiving unit 15 (step S105) and accumulated by the behavior information accumulating unit 16 (step S106). The user attribute information and the RFID position acquired by the RFID reader are received by the user attribute information receiving unit 17 (step S107) and stored by the user attribute information storage unit 18 (step S108). Also, the object information including the object attribute information and the object position information that is information indicating the position of the RFID is received by the object information receiving unit 19 (step S109) and stored by the object information storing unit 20 (step S110). .

  FIG. 11 is a diagram illustrating an example of audio information accumulated by the audio information accumulation unit 12. As shown in FIG. 11, time-series audio information is stored in a predetermined recording medium (not shown) in association with the user identification information U001 of each user.

  FIG. 12 is a diagram illustrating an example of behavior information accumulated by the behavior information accumulation unit 16. As shown in FIG. 12, time-series action information is stored in a predetermined recording medium (not shown) in association with user identification information U001 of each user. The motion information included in the behavior information includes line-of-sight information and body motion information. The line-of-sight information is information indicating a pixel position in the user's visual field. The body motion information is information indicating the position of each marker in the three-dimensional orthogonal coordinate system acquired by the motion capture system. Since the position of the user can be known from the body motion information, in this specific example, the position information is included in the body motion information. For example, the position of the user can be represented by the position of one marker included in the body motion information.

  FIG. 13 is a diagram illustrating an example of user attribute information accumulated by the user attribute information accumulation unit 18 and object information accumulated by the object information accumulation unit 20. As shown in FIG. 13, the user attribute information includes information indicating the attribute of each user (in this specific example, the gender of the user) and time-series information indicating the position of the RFID of each user. The object information also includes information indicating the attribute of the object (in this specific example, the name of the object) and time-series information indicating the position of the RFID of the object.

  It is assumed that after the voice information, behavior information, user attribute information, and object information are accumulated, the abbreviation complementing apparatus 1 accepts an instruction to complement abbreviations by operating the input device (not shown). Then, the abbreviation complement device 1 determines that it is time to complement the abbreviation (step S101), and starts a process of complementing the abbreviation. Hereinafter, the process of complementing abbreviations will be described in detail.

  First, the character information conversion unit 13 reads audio information from a recording medium (not shown) included in the audio information storage unit 12 (step S201). In this specific example, a voice “sorry, but uttered by a user identified by the user identification information U001 (hereinafter, this user may be referred to as“ user U001 ”; the same applies to other users). , Can you give me? ", The case of executing the process of complementing abbreviations will be described. Therefore, it is assumed that the character information conversion unit 13 reads only the voice information indicating the voice uttered by the user U001 from the recording medium included in the voice information storage unit 12.

  The character information conversion unit 13 performs voice recognition processing on the voice indicated by the read voice information, and generates character information corresponding to the voice information (step S202). FIG. 14 is a diagram showing an example of character information generated in this way. In the character information shown in FIG. 14, kanji conversion is performed for hiragana recognized from the speech information, but the hiragana may not be converted to kanji.

  The zero pronoun specifying unit 14 performs a process of specifying a zero pronoun in the character information “I'm sorry, can you give it?” Shown in FIG. 14 (step S203). It is assumed that the case frame information storage means 32 of the zero pronoun specifying unit 14 stores the case frame information shown in FIG. As shown in FIG. 15, in the case frame information, a verb is associated with a case element taken by the verb. For example, for the verb “pass”, it is understood that a case element corresponding to an object corresponds to a case element corresponding to a person. Further, it is understood that the case particle “O” is used in the case element corresponding to the object. It can also be seen that the case particle “ni” is used in the case element corresponding to the person. Here, in the case frame information shown in FIG. 15, the case element corresponding to the subject is omitted, but the case element corresponding to the subject may also be included in the case frame information. In this case, for example, the information indicating the case element corresponding to the verb “pass” is “(person) passes (thing) (person)”.

  In the process of specifying the zero pronoun, first, the verb extracting means 31 of the zero pronoun specifying unit 14 performs a morphological analysis on the character information “Can I pass it?” And extracts the verb “pass”. The verb extracting means 31 extracts a case element corresponding to the verb “pass”. In this case, no case element is extracted because there is no corresponding case element (step S301). The case element acquisition unit 33 acquires the case element corresponding to the verb “pass” extracted by the verb extraction unit 31 from the case frame information stored in the case frame information storage unit 32 (step S302). Here, the case elements “thing” and “to person” corresponding to the verb “pass” are acquired.

  Next, the case element identification unit 34 compares the case element extracted by the verb extraction unit 31 with the case element acquired by the case element acquisition unit 33 (step S303). Here, since the verb extracting means 31 does not extract case elements, the case element specifying means 34 does not include all case elements acquired by the case element acquiring means 33 as a result of comparison. It is determined that it is an element, and those case elements are specified as case elements not included in the character information (step S304). Specifically, as shown in FIG. 16, the case element specifying unit 34 inserts a zero pronoun corresponding to an object and a zero pronoun corresponding to a person at a corresponding position of the character information. Identify case elements not included in.

  The abbreviation complement unit 21 performs a process of complementing abbreviations on the character information for which the zero pronoun is identified as shown in FIG. 16 (step S204). Note that the user instruction information storage means 42 stores the user instruction information shown in FIG. Further, it is assumed that the object instruction information storage means 46 stores the object instruction information shown in FIG. In the user instruction information shown in FIG. 17 and the object instruction information shown in FIG. 8, the position information “near name”, “middle name”, “distant name” corresponds to the user instruction word or the object instruction word. It is attached. In the user instruction word and the object instruction word, each instruction word may or may not have an attribute. When there is user attribute information or object attribute information, an instruction word when there is an attribute is used, and when there is no user attribute information or object attribute information, an instruction word when there is no attribute is used. In the instruction word when there is an attribute, user attribute information or object attribute information may be inserted at the (attribute) position.

  Further, it is assumed that the user U001 can grasp the positional relationship between each user and the object as shown in FIG. 19 at the time point when the user U001 speaks "I'm sorry, can you give me" from the behavior information and the object information. Here, it is assumed that the instruction direction of the user U001 is a direction in which the user points.

  In such a situation, the object specifying means 45 first determines that there is a zero pronoun of the object by referring to the character information specified by the zero pronoun shown in FIG. 16 (step S401). And the thing specific | specification means 45 judges whether the thing corresponding to the zero pronoun of the thing can be specified (step S402). Here, as shown in FIG. 19, since only the object information corresponding to one object (book) is accepted, the object specifying means 45 determines that the object can be specified, and The “book” existing near the user U001 is identified (step S403).

  Next, the object character information determination unit 47 performs a process of determining object character information corresponding to the object “book” (step S404). Specifically, as shown in FIG. 19, the object character information determination unit 47 is configured such that the user U001 who is the speaker is facing the user U002 and the user U002 is also facing the user U001. Is user U001, and the listener is user U002. As shown in FIG. 19, the object “book” exists near the user U001, and therefore the positional relationship between the speaker user, the listener user, and the object specified by the object specifying unit 45 is “near”. It is determined that it is “name” (step S501). Further, as shown in FIG. 13, since the object attribute information “book” is received by the object information receiving unit 19 (step S <b> 502), the object character information determining unit 47 has an attribute corresponding to the near name. The instruction word “this (attribute)” is acquired from the object instruction information shown in FIG. 18, and the object character information “book” is determined by inserting the object attribute information “book” at the position of the attribute. (Step S503).

  The object abbreviation information insertion means 48 uses the object abbreviation “this book”, which is a combination of the object character information “this book” determined by the object character information determination means 47 and the case particle “O”, as the zero of the object. It inserts into the position of a pronoun (step S405). As a result, the text information is "I'm sorry, can you give this book to someone?"

  Next, the user specifying unit 41 refers to the character information in which the zero pronoun of the object is inserted, and determines that there is a human zero pronoun (step S406). Then, the user specifying unit 41 determines whether a user corresponding to the person's zero pronoun can be specified (step S407). Here, as shown in FIG. 19, since the speaker user U001 indicates the direction of the user U003, the user specifying unit 41 determines that the user can be specified, and specifies the user as the user U003. (Step S408).

  Next, the user character information determination unit 43 performs a process of determining user character information corresponding to the user U003 (step S409). Specifically, the user character information determination unit 43 determines that the user U001 is a speaker and the user U002 is a listener, similarly to the object character information determination unit 47. As shown in FIG. 19, the user U003 is far away from the speaker user U001 and the listener user U002 by more than the distance between the speaker and the listener, so that the speaker user, the listener user, and the user It is determined that the positional relationship with the user specified by the specifying means 41 is “distant name” (step S601).

  Further, the user character information determination unit 43 compares the position indicated by the action information of the user U003 with the position included in the user attribute information, and the user attribute information in which both match within a predetermined error range, It is determined that the user attribute information is the user U003. Here, it is assumed that the third user attribute information “male” shown in FIG. 13 is user attribute information corresponding to the user U003. Then, the user character information determination unit 43 determines that there is user attribute information corresponding to the user U003 (step S602), and the user character information determination unit 43 determines that there is an instruction word “There "(Attribute)" is obtained from the user instruction information shown in FIG. 17, and the user attribute information "male" of the user U003 is inserted into the position of the attribute, whereby the user character information is determined to be "that male". (Step S603).

  The user abbreviation information insertion means 44 uses the user abbreviation “to that man” in combination with the user character information “that man” determined by the user character information decision means 43 and the case particle “ni”. Is inserted at the position of the zero pronoun (step S410). As a result, the text information is "I'm sorry, can you give this book to that man?" This character information is complementary character information.

  The supplementary character information output unit 22 displays the supplementary character information generated by the abbreviation complementation unit 21 “Excuse me, but can you give this book to that man” on the external display (step S205). As a result, the user who operates the abbreviation complement device 1 can see the supplementary character information supplemented with the abbreviations. Thereafter, the supplementary character information may be used for machine translation or the like.

  In this specific example, the case where the process of complementing the abbreviation is started after all of the voice information, the behavior information, the user attribute information, and the object information has been accumulated is described. However, as described above, the voice information, the behavior information, etc. In parallel with the acceptance of the abbreviation, a process of complementing the abbreviation may be executed.

  Moreover, in this specific example, the case where the thing corresponding to the zero pronoun of the thing is judged from the situation and the user corresponding to the person's zero pronoun is judged based on the user's instruction act of the speaker is explained. In addition, since there are various determination methods, it goes without saying that the determination may be made by methods other than those described in this specific example.

  Further, in this specific example, the case where the user and the user attribute information are associated with each other by comparing the position of the user indicated by the behavior information and the position of the RFID included in the user attribute information has been described. The user and the user attribute information may be associated with each other. For example, user identification information (for example, “U001” or the like) for identifying a user is included in the RFID, and the user and user attribute information may be associated with each other using the user identification information.

  In this specific example, the user instruction information may include information indicating an instruction word when the user specified by the user specifying unit 41 is the same as the speaker or the listener. In this case, the instruction word is, for example, “I” or “you”.

  As described above, the abbreviation complement apparatus 1 according to the present embodiment can complement abbreviations by using behavior information or the like. Therefore, as compared with the case where only the words are analyzed and the abbreviations are complemented, the appropriate abbreviations according to the situation of the dialogue are complemented. As a result, the supplementary character information supplemented with the abbreviation can be used for context analysis, speech dialogue, machine translation, and the like. For example, the machine translation device (not shown) may receive the supplementary character information output by the supplementary character information output unit 22 and perform machine translation processing on the supplementary character information. An abbreviation complement machine translation system may be configured by the abbreviation complement apparatus 1 and its machine translation apparatus.

  Also, by supplementing abbreviations using user attribute information, it is possible to know information that cannot be known from the context, such as the gender of the user, for example, when translating supplementary character information into English The personal pronoun can be determined as “him” or “her”.

  Also, by complementing abbreviations using the object attribute information included in the object information, it is possible to know the name of the object that cannot be known from the context, and the document indicated by the supplementary character information is more clearly defined. Can be a thing.

  Although the case has been described with the present embodiment where the behavior information also includes motion information, the behavior information may not include motion information as long as it includes at least user position information.

  Moreover, the structure of the zero pronoun specification part 14 is not limited to what is shown by FIG. If the zero pronoun can be appropriately specified, the configuration of the zero pronoun specifying unit 14 may be a configuration other than FIG.

  Moreover, although the abbreviation complement part 21 demonstrated the case where an abbreviation complement part 21 complemented abbreviations using action information, user attribute information, and thing information in this Embodiment, the abbreviation complement part 21 uses at least action information. Abbreviations need only be complemented, and user attribute information and object information need not be used. When the user attribute information is not used for abbreviation completion, the abbreviation completion apparatus 1 may not include the user attribute information reception unit 17 and the user attribute information storage unit 18. When the object information is not used for abbreviation completion, the abbreviation complement apparatus 1 does not have to include the object information reception unit 19 and the object information storage unit 20.

  In the present embodiment, the case has been described in which the user character information determination unit 43 determines user character information using the user instruction information. However, the user character information determination unit 43 does not use the user instruction information. Information may be determined. For example, the user character information determination means 43 may determine user character information using only user attribute information. When the user character information determining unit 43 does not use user instruction information, the abbreviation complementing unit 21 may not include the user instruction information storage unit 42.

  In the present embodiment, the case where the user character information determining unit 43 determines the user character information using the user attribute information has been described. However, the user character information determining unit 43 does not use the user attribute information, for example, The user character information may be determined using only the user instruction information. In this case, the user instruction information may not include an instruction word when there is an attribute.

  In the present embodiment, the case has been described in which the object character information determining unit 47 determines the object character information using the object instruction information. However, the object character information determining unit 47 does not use the object instruction information. Information may be determined. For example, the object character information determining unit 47 may determine object character information using only object attribute information. When the object character information determination unit 47 does not use the object instruction information, the abbreviation complement unit 21 may not include the object instruction information storage unit 46.

  In the present embodiment, the case has been described in which the object character information determining unit 47 determines the object character information using the object attribute information. However, the object character information determining unit 47 does not use the object attribute information, for example, The object character information may be determined using only the object instruction information. In that case, the object instruction information may not include an instruction word when there is an attribute.

  Further, the configuration of the abbreviation complement unit 21 is not limited to that shown in FIG. As long as the abbreviation can be appropriately complemented, the configuration of the abbreviation complement 21 may be other than that shown in FIG. For example, the abbreviation complement unit 21 may only supplement abbreviations corresponding to human zero pronouns, or may only complement abbreviations corresponding to the zero pronouns of things. Also good.

  Further, in this embodiment, the case where the positional relationship is divided into the near name, the middle name, and the far name has been described, but the positional relationship may be divided into two names, the near name and the far name. For example, in English and Chinese, there is no concept of a middle name pronoun, so the positional relationship must be divided into two names, near and far, according to the distance from the speaker.

  In the present embodiment, in the user instruction information and the object instruction information, the instruction word when there is a single user or object is shown. However, the user instruction information or the like indicates an instruction when there are a plurality of users or objects. You may have words. For example, if the positional relationship is an instruction word of a user with a short name and no attribute, “this way” is displayed. In this case, the user specifying means 41 and the object specifying means 45 may specify whether the number of users or objects is singular or plural.

  In this embodiment, the case of performing abbreviation completion processing for Japanese has been described, but the abbreviation completion processing described in this embodiment may be used for other languages.

  In each of the above embodiments, each processing or each function may be realized by centralized processing by a single device or a single system, or distributed processing by a plurality of devices or a plurality of systems. May be realized.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component is realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the abbreviation complement apparatus 1 in each said embodiment is the following programs. That is, the program includes a voice information receiving step for receiving voice information that is information indicating a voice uttered by at least one user in a situation where two or more users are talking to the computer; A character information conversion step for converting the speech information received in the step into character information that is information indicating characters corresponding to the speech indicated by the speech information; and a zero pronoun that is a case element omitted in the document indicated by the character information The action information reception which receives the action information which is the information which contains the zero pronoun specification step which specifies, and the user position information which is the information regarding the position of the two or more users at least, and is the information regarding the actions of the two or more users The character information is indicated using the step and the behavior information received in the behavior information reception step. An abbreviation completion step for generating supplementary character information that is information obtained by complementing an abbreviation that is an abbreviated word at the position of the zero pronoun specified in the zero pronoun specification step in the book; And a complementary character information output step to be output.

  Further, in this program, the abbreviation complementing step is a user specifying step for specifying a user corresponding to a person's zero pronoun using the behavior information, and character information indicating a user corresponding to the zero pronoun. User character information determination step for determining user character information, and information indicating abbreviations complemented to the position of the zero pronoun, and user abbreviations that are information including user character information determined in the user character information determination step A user abbreviation information insertion step of inserting information at a corresponding zero pronoun position.

  Further, in this program, the user attribute information receiving step for receiving the user attribute information that is information relating to the user attribute is further executed by the computer, and the user attribute received in the user attribute information receiving step in the user character information determination step The user character information may be determined using information.

  Further, in this program, in the user character information determination step, an instruction word according to the positional relationship among the user specified in the user specifying step, the user of the speaker, and the user of the listener, which is indicated by the behavior information, The user character information that is information of characters including the selected instruction word may be determined by selecting using user instruction information that is information that associates the relationship with the user instruction word.

  Further, in this program, an object information receiving step for receiving object information that is information related to an object in a situation where two or more users are talking and includes at least object position information that is information related to the position of the object. Is further executed by the computer, and the abbreviation complementing step includes an object specifying step for specifying an object corresponding to the zero pronoun of the object using the object information, and character information indicating the object corresponding to the zero pronoun. Object character information determining step for determining certain object character information, and information indicating an abbreviation supplemented to the position of the zero pronoun, and information including object character information determined in the object character information determining step An object abbreviation information insertion step of inserting word information at the position of the corresponding zero pronoun.

  In this program, the object information may also include object attribute information that is information related to an attribute of the object. In the object character information determining step, the object character information may be determined using the object attribute information.

  In this program, in the object character information determination step, an instruction word according to the positional relationship between the object specified in the object specifying step, the user of the speaker, and the user of the listener, which is indicated by the action information and the object information. Alternatively, the object character information that is information of characters including the selected instruction word may be determined by selecting using the object instruction information that is information that associates the positional relationship between the user and the object and the instruction word of the object.

  Further, in this program, the zero pronoun identification step is a verb extraction step for extracting a verb and a case element of the verb from a document indicated by the character information, and information indicating a relationship between the verb and the case element. Using frame information, a case element acquisition step for acquiring a case element corresponding to the verb extracted in the verb extraction step, a case element acquired in the case element acquisition step, and a case element extracted in the verb extraction step A case element specifying step for specifying case elements that are not included in the document indicated by the character information.

  In the above program, the output step for outputting information, the accepting step for receiving information, etc. do not include at least processing performed only by hardware, for example, processing performed by a modem or an interface card in the output step. .

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disc such as a CD-ROM, a magnetic disc, a semiconductor memory, etc.) is read out. May be executed by

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 20 is a schematic diagram showing an example of the appearance of a computer that executes the program and realizes the abbreviation complementing apparatus 1 according to the embodiment. The above-described embodiment is realized by computer hardware and a computer program executed on the computer hardware.

  20, a computer system 100 includes a computer 101 including a CD-ROM (Compact Disk Read Only Memory) drive 105 and an FD (Flexible Disk) drive 106, a keyboard 102, a mouse 103, and a monitor 104.

  FIG. 21 is a diagram illustrating a computer system. In FIG. 21, in addition to the CD-ROM drive 105 and the FD drive 106, a computer 101 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112 for storing a program such as a bootup program, A CPU (Random Access Memory) 113 that is connected to the CPU 111 and temporarily stores application program instructions and provides a temporary storage space, a hard disk 114 that stores application programs, system programs, and data, a CPU 111 and a ROM 112. Etc. to each other. The computer 101 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 100 to execute the functions of the abbreviation complementing apparatus 1 according to the above-described embodiment is stored in the CD-ROM 121 or the FD 122, inserted into the CD-ROM drive 105 or the FD drive 106, and the hard disk 114. May be forwarded to. Instead, the program may be transmitted to the computer 101 via a network (not shown) and stored in the hard disk 114. The program is loaded into the RAM 113 at the time of execution. The program may be loaded directly from the CD-ROM 121, the FD 122, or the network.

The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 101 to execute the functions of the abbreviation completion device 1 according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 100 operates is well known and will not be described in detail.
Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the abbreviation complementing device and the like according to the present invention, it is possible to appropriately supplement abbreviations by using user behavior information and the like, which is useful as a device for complementing abbreviations.

The block diagram which shows the structure of the abbreviation complement apparatus by Embodiment 1 of this invention. The block diagram which shows the structure of the zero pronoun specific part by the embodiment The block diagram which shows the structure of the abbreviation complement part by the embodiment The flowchart which shows the whole operation | movement of the abbreviation complement apparatus by the same embodiment The flowchart which shows the process which complements the abbreviation of the abbreviation completion apparatus by the embodiment The flowchart which shows the process which specifies the zero pronoun of the abbreviation completion apparatus by the embodiment The flowchart which shows the production | generation process of the supplementary character information of the abbreviation completion apparatus by the embodiment The flowchart which shows the determination process of the object character information of the abbreviation complement apparatus by the embodiment The flowchart which shows the determination processing of the user character information of the abbreviation complement apparatus by the embodiment The figure which shows an example of the user who mounted | wore with the eye camera etc. in the embodiment The figure which shows an example of the audio | voice information in the embodiment The figure which shows an example of the action information in the embodiment The figure which shows an example of the user attribute information and the object information in the embodiment The figure which shows an example of the character information in the embodiment The figure which shows an example of the case frame information in the embodiment The figure which shows an example of the character information by which the zero pronoun was identified in the embodiment The figure which shows an example of the user instruction information in the embodiment The figure which shows an example of the object instruction information in the embodiment The figure which shows an example of the positional relationship of the user and the thing in the embodiment Schematic diagram showing an example of the appearance of the computer system in the embodiment The figure which shows an example of a structure of the computer system in the embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Abbreviation completion apparatus 11 Voice information reception part 12 Voice information storage part 13 Character information conversion part 14 Zero pronoun specification part 15 Behavior information reception part 16 Behavior information storage part 17 User attribute information reception part 18 User attribute information storage part 19 Object information Reception unit 20 Object information storage unit 21 Abbreviation completion unit 22 Complementary character information output unit 31 Verb extraction means 32 Case frame information storage means 33 Case element acquisition means 34 Case element specification means 41 User specification means 42 User instruction information storage means 43 User Character information determination means 44 User abbreviation information insertion means 45 Object identification means 46 Object instruction information storage means 47 Object character information determination means 48 Object abbreviation information insertion means

Claims (10)

A voice information receiving unit that receives voice information that is information indicating a voice uttered by at least one user in a situation where two or more users are talking;
A character information conversion unit that converts the voice information received by the voice information reception unit into character information that is information indicating characters corresponding to the voice indicated by the voice information;
A zero pronoun specifying unit that specifies a zero pronoun that is a case element omitted in the document indicated by the character information;
An action information receiving unit that receives at least user position information that is information related to the positions of the two or more users and receives action information that is information related to the actions of the two or more users;
Information supplemented with an abbreviation that is an abbreviated word at the position of the zero pronoun specification unit specified by the zero pronoun specification unit in the document indicated by the character information, using the action information received by the action information reception unit An abbreviation completion unit for generating supplementary character information that is,
An abbreviation complementing device comprising: a supplementary character information output unit that outputs the supplementary character information. The abbreviation complement device according to claim 1, wherein the behavior information includes motion information that is information indicating a user's movement. The abbreviation completion unit
User identifying means for identifying a user corresponding to a person's zero pronoun using the behavior information;
User character information determining means for determining user character information which is character information indicating a user corresponding to the zero pronoun;
A user who inserts user abbreviation information, which is information indicating abbreviations complemented at the position of the zero pronoun, and includes user character information determined by the user character information determination means, at the position of the corresponding zero pronoun The abbreviation complementing device according to claim 1, further comprising abbreviation information insertion means. The abbreviation completion unit
Further comprising user instruction information storage means for storing user instruction information, which is information associating the positional relationship of the user with the instruction word of the user,
The user character information determining means selects, using the user instruction information, an instruction word corresponding to the positional relationship of the user specified by the user specifying means, the user of the speaker, and the user of the listener indicated by the behavior information. The abbreviation complementing device according to claim 3, wherein user character information that is character information including the selected instruction word is determined. In a situation where two or more users are talking, further comprising an object information receiving unit that receives object information that is information related to an object and is information including at least object position information that is information related to the position of the object,
The abbreviation completion unit
An object specifying means for specifying an object corresponding to the zero pronoun of the object using the object information;
An object character information determining means for determining object character information which is information of a character indicating an object corresponding to the zero pronoun;
Information indicating an abbreviation supplemented at the position of the zero pronoun, and an object abbreviation information which is information including the object character information determined by the object character information determining means is inserted at the position of the corresponding zero pronoun An abbreviation complementing apparatus according to any one of claims 1 to 4, further comprising abbreviation information insertion means. The product information includes product attribute information that is information related to product attributes,
6. The abbreviation complementing device according to claim 5, wherein the physical character information determining means determines the physical character information using the physical property information. The abbreviation completion unit
It further comprises object instruction information storage means for storing object instruction information, which is information associating the positional relationship between the user and the object, and the instruction word of the object,
The object character information determining unit is configured to display an instruction word according to a positional relationship between the object specified by the object specifying unit, the user of the speaker, and the user of the listener, which is indicated by the action information and the object information. 7. The abbreviation complementing device according to claim 5 or 6, wherein the abbreviation complementing device determines the object character information which is the information of the character including the selected instruction word. The zero pronoun specifying unit is
Verb extraction means for extracting a verb and a case element of the verb from a document indicated by the character information;
Case frame information storage means for storing case frame information which is information indicating the relationship between the verb and the case element;
Case element acquisition means for acquiring a case element corresponding to the verb extracted by the verb extraction means using the case frame information;
A case element specifying means for specifying a case element not included in the document indicated by the character information by comparing the case element acquired by the case element acquisition means with the case element extracted by the verb extraction means; The abbreviation complement apparatus according to any one of claims 1 to 7, which is provided. A voice information receiving step for receiving voice information, which is information indicating voice generated by at least one user in a situation where two or more users are talking;
A character information conversion step for converting the voice information received in the voice information reception step into character information that is information indicating characters corresponding to the voice indicated by the voice information;
A zero pronoun identifying step for identifying a zero pronoun that is a case element omitted in the document indicated by the character information;
An action information receiving step for receiving action information that is information including at least user position information that is information relating to positions of the two or more users, and is information relating to actions of the two or more users;
Information obtained by complementing an abbreviation which is an abbreviated word at the position of the zero pronoun specified in the zero pronoun specification step in the document indicated by the character information using the action information received in the action information reception step Abbreviation completion step for generating supplementary character information which is
An abbreviation complement method comprising: a supplement character information output step for outputting the supplement character information. On the computer,
A voice information receiving step for receiving voice information, which is information indicating voice generated by at least one user in a situation where two or more users are talking;
A character information conversion step for converting the voice information received in the voice information reception step into character information that is information indicating characters corresponding to the voice indicated by the voice information;
A zero pronoun identifying step for identifying a zero pronoun that is a case element omitted in the document indicated by the character information;
An action information receiving step for receiving action information that is information including at least user position information that is information relating to positions of the two or more users, and is information relating to actions of the two or more users;
Information obtained by complementing an abbreviation which is an abbreviated word at the position of the zero pronoun specified in the zero pronoun specification step in the document indicated by the character information using the action information received in the action information reception step Abbreviation completion step for generating supplementary character information which is
A supplementary character information output step for outputting the supplementary character information.

Images