JP3815723B2 - ENVIRONMENTAL CHANGE TRANSMISSION METHOD AND DEVICE, ENVIRONMENTAL CHANGE TRANSMISSION PROGRAM, AND RECORDING MEDIUM CONTAINING THE PROGRAM - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an environment change transmission method and a communication method using ubiquitous computing technology.
[0002]
The above ubiquitous computing technology is a concept that is contrary to mobile computing, while mobile carries a computer, whereas ubiquitous (ubiquitous) is a technology in which a computer is embedded in every environment. .
[0003]
[Prior art]
Our lives depend heavily on the living environment. The degree of dependence varies among individuals, but when the living environment changes, the change becomes a stimulus and affects human behavior. For example, if you replace a pillow that you normally use, you may not be able to sleep for a while. If the room gets dirty, you might clean it. This is an example directly connected to an action. Actually, a human recognizes daily fine changes, and a result of the recognition becomes a trigger to perform various actions. One of them is communication.
[0004]
By recognizing changes in the environment, humans incorporate it into communication and talk to people. For example, in Japan, the weather changes frequently, so we often talk about the weather in greetings. Also, if a veranda plant blooms, you might talk about it with your family.
[0005]
However, there are two problems in order for humans to recognize changes in the environment and to incorporate into the communication the topics born from the recognized changes in the environment.
[0006]
One problem is that it is difficult for humans to perceive and memorize all the various daily changes.
[0007]
Changes that happened (caused) are forgotten quickly, and changes that occur in situations where you are not often do not even notice that changes have occurred.
[0008]
This is because human physical ability and memory ability are limited, and overlooks a lot of information transmitted by the environment and does not realize the meaning of a lot of information. Therefore, I can't find the glasses and TV remotes I used before, and look for the weekly magazines and last week's newspapers that were thrown away by my family.
[0009]
The second problem is how to talk to whom and how to talk about topics that have arisen from environmental changes even if the first problem is cleared.
[0010]
To talk to people, you need someone, topics, and means of communication, but when you talk to people about topics that arise from changes in the environment, you often don't know who you want to talk to, so there are often no appropriate means of communication. .
[0011]
For example, when a father in a house gets up in the morning and goes out to the front, the garbage is thrown away in front of the house (a change that someone put garbage in front of the entrance where there was nothing when I saw the previous day). Then, from the situation that the father is throwing away the garbage, the topic of anger about the garbage thrown away in front of the house is born. You will want to communicate with the other party who has thrown away the trash, "Don't throw trash in front of your house!"
[0012]
Also, I was searching for a video camera that I wanted to use as soon as possible, and I happened to find it on the lab desk (on the desk where there was nothing when I came to the lab last time). The change that someone put a video camera on). In this case, you must want to communicate with the owner of the video camera, "Can I borrow a video camera?" However, in these cases, even if you know the topic of communication, you can't communicate because you don't know who you want to communicate with, such as the person who dumped the trash or the owner of the video camera.
[0013]
In order to solve the above two problems,
-A mechanism is needed to recognize and effectively communicate changes in the environment that humans have overlooked or have not noticed,
・ It is necessary to create a system that effectively incorporates topics born from changes in the environment recognized by humans into communication.
[0014]
First, research that complements human memory ability is being conducted as a research related to a mechanism that recognizes changes in the environment that humans overlooked or did not notice and communicated effectively to humans.
[0015]
For example, Forget-me-not (Reference 1: Mik Lamming and Mike Flynn: Forget-me-not: Intimate computing in support of human memory, In FRIEND21 '94 International Symposium on Next Generation Human Interface, 1994.) By recording the user's operation history in the computer, human memory is indirectly assisted.
[0016]
Digital Reminder (Reference 2: Masashi Yoshida, Ikuo Yoshitaka: Digital Reminder: Construction of Real World Oriented Database from User's Viewpoint and its Interface-Dynamic Acquisition and Presentation of Visual Information Using Eyes-Interactive System and Software VIII, pp111-116, 2000.) records only information of interest based on line-of-sight information. These studies have the intention of assisting the user's memory by recording the user's behavior and using it for searching for things.
[0017]
In these studies, since the change of the environment of the place where the user existed in the past is recorded, it is possible to recognize the change of the environment by omitting the recorded user action.
[0018]
In addition, many security systems are designed to recognize changes in the environment such as human intrusion and fire using various sensors and to inform the police, landlords, etc. (Reference 3: http: // www) .geocities.co.jp / Technopolis / 8917 /., Reference 4: http://www.secom.co.jp/.).
[0019]
There are several related studies in the Research Area of Augmented Reality as a mechanism for effectively incorporating topics born from changes in the environment recognized by humans into communication.
[0020]
For example, Augment-ab1e Reality (Reference 5: Junichi Kyokumoto, Yuji Ayatsuka, Kazuaki Hayashi: Augment-able Reality: Information exchange that fuses real space and information space, interactive system and software VI, pp115-124, 1998. ), Space Tag (Reference 6: Hiroyuki Tarumi, Ken Morishita, Megumi Nakao, Yahiko Kamibayashi: Space-Time Limited Object System: Space Tag, Interactive System and Software VI, pp1-10, 1998.), Air Pen (reference) Reference 7: Ichiro Shiio, Yoshinobu Yamamoto: Simple AR system for communication tools, interactive system and software VIII, pp117-124, 2000. It becomes possible to put a sticker or a book place for the place where the problem occurred.
[0021]
Also, in space-time mail (reference 8: Hirotaka Ueda, Masahiko Tsukamoto, Shojiro Nishio: character space mail: messages beyond space-time, interactive system and software VIII, pp27-32, 2000.), location, time, user Mail can be sent to users who satisfy the attribute.
[0022]
[Problems to be solved by the invention]
However, each of the above studies was born as a study to solve a mechanism that recognizes changes in the environment that humans overlooked or did not notice and effectively communicates them to people, or was born from changes in the environment recognized by humans. There are a number of problems when researching the mechanism for effectively incorporating topics into communication.
[0023]
[A mechanism that recognizes changes in the environment that humans overlooked or did not notice and effectively communicates them to people]
Since Forget-me-not and Digital Reminder use a portable computer, changes in the place where the person is located can be stored, but changes occurring in other places cannot be stored.
[0024]
For example, when you go to work, you won't be able to remember the changes that are happening at home, so you won't be able to tell the location of the TV remote that your child moved. In other words, you can respond to changes that you haven't noticed while at that location, but you can't respond to changes you missed because you were at a different location.
[0025]
Conversely, many security systems are not portable systems, but are systems that are installed in the environment. Therefore, it is possible to check changes that have been missed in another place later.
[0026]
However, these systems are specialized for specific purposes such as illegal intrusion and fire, and there is a problem that many changes cannot be effectively transmitted to the user.
[0027]
[Mechanism for effectively incorporating topics born from changes in the environment recognized by humans into communication]
Augment-able Reality, Space Tag, and air pen can write a message to the position.
[0028]
However, when this method is considered as a means of communication, there is a problem that it is not known when the other party will come to the place again, there is a problem that the other party does not know when to read the message, and there will be a reply There are problems such as not knowing.
[0029]
The method of writing a message for a position is not different from the act of putting a book in real life, so the above problem has not been solved.
[0030]
On the other hand, in spatio-temporal mail, mail can be sent to users who satisfy the location, time, and user attributes, so a message can be sent to a person who has been in a certain place in the past, There is no synchronous communication or identification of people who have changed.
[0031]
In addition, a system that can solve the above two problems simultaneously has not been realized.
[0032]
The present invention is capable of knowing changes in the environment that the user tends to miss or not notice, and that there is a possibility that unexpected discoveries may be obtained, and that messages can be exchanged via the environment. It is intended to provide.
[0033]
[Means for Solving the Problems]
The present invention extracts changes that occur in the environment based on information output by sensors installed in the environment, accumulates the extracted changes in a database, and when a user approaches the environment, The position sensor recognizes the identifier of the approaching user, the time when the recognized user last approached the environment is obtained from the database, and the change that has occurred from the obtained time to the current time Is obtained from the database and presented to the user through the output device.
[0034]
BEST MODE FOR CARRYING OUT THE INVENTION
According to the present invention, when a user visits a certain environment, the user presents changes in the environment since the last time the user visited the environment to the user.
[0035]
In other words, the present invention provides a mechanism for recognizing changes in the environment that are overlooked or not noticed by humans and effectively transmitting them to people, and effectively discussing topics born from changes in the environment recognized by humans. Provide a mechanism for incorporating into communication.
[0036]
[A mechanism that recognizes changes in the environment that humans overlooked or did not notice and effectively communicates them to people]
First, the basic configuration of a system for recognizing a change in the environment and effectively transmitting it to a person is composed of a sensor provided in the environment and an output device.
[0037]
Here, “environment” refers to everything on the spot such as books, pencils, and creatures including human beings living there. In this way, the environment in which humans live is changing day by day. “Environmental change” means that the environment changes mainly visually.
[0038]
The output device may be provided in the environment or a portable device.
[0039]
For example, by installing a camera on the environment side, images are periodically acquired, and changes occurring in the living environment are detected from the acquired images. Next, based on the detected change, a spotlight is projected to a place where the change has occurred, for example, using a projector. This tells you where changes have occurred in the past. By memorizing the time at the place in the system, the next time you come, you can receive only the changes that have occurred since the last time you came.
[0040]
[Mechanism for effectively incorporating topics born from changes in the environment recognized by humans into communication]
The present invention is a mechanism for performing communication using changes detected using the above mechanism.
[0041]
First, a change that is desired to be made a communication topic is selected by designating a portion where the presented change is occurring by some method (for example, a portion where a spotlight is applied by a projector is indicated by a pen).
[0042]
The system obtains from the database information about the person who will have changed. For example, when a predetermined change occurs, it is estimated that the person who is closest to the place where the change has occurred is the person who caused the change. When a user selects a change and performs an action such as sending an e-mail, making a call, posting a message, etc., the user can communicate with a person who is assumed by the system to have changed. .
[0043]
[First embodiment]
FIG. 1 is a block diagram showing an environment change transmission device 100 according to a first embodiment of the present invention.
[0044]
The environment change transmission device 100 includes an environment change detection sensor 10 that detects a change in the environment, a position sensor 20, a processing device 30, an information presentation device 40, an environment database 50, and a human database 60. .
[0045]
The environment change detection sensor 10 includes a camera, an ultrasonic sensor, and the like.
[0046]
The position sensor 20 includes an infrared sensor, a position measurement system that uses electric field strength such as RFID and PHS, a GPS, and the like.
[0047]
References on infrared sensors include “Hiroshi Aoki: Infrared tags read by cameras and their applications, Interactive System and Software VIII, pp131-136, 2000.”, “R. Want and A. Hopper: Active badges and personal interactive computing objects, IEEE Trans. on Consumer Electronics, 38 (1) pp. 10-20, 1992. "
[0048]
RFID references include `` R. Want, KP Fishkin, A. Gujar and BL Harrison: Bridging physical and virtual worlds with electronic tags, In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI'99), pp371- 377, 1999.) ”.
[0049]
The processing device 30 includes an environment information acquisition unit 31, an environment change extraction unit 32, an environment information / environment change registration unit 33, a position information acquisition unit 34, a position information registration unit 35, an environment database search unit 36, Human database search means 37 and change information output means 38 are provided.
[0050]
The environmental information acquisition means 31 acquires environmental information using an environmental change detection sensor. The environment change extraction means 32 cuts out information that the environment has changed from the acquired environment information, and periodically uses the environment information / environment change registration means 33 to store information on environmental information / environment changes in the environment database 50. Accumulate.
[0051]
When the person approaches the environment, the position information acquisition unit 34 acquires the person's ID and position information from the position sensor 20.
[0052]
The position information registration means 35 is a means for accumulating the acquired person ID and position information in the person database 60.
[0053]
The environment database search means 36 searches the environment database 50 for information when the user corresponding to the acquired person ID has approached the environment in the past, and based on the information that the user has finally approached the environment, From 50, search for changes in the environment that occurred since the user last approached the environment.
[0054]
The person database search means 37 is a means for searching for information stored in the person database 60.
[0055]
The change information output means 38 outputs an environmental change to the information presentation device 40 based on the searched environmental change.
[0056]
The information presentation device 40 is configured by a projector, a portable computer, or the like.
[0057]
The environment database 50 is a database that accumulates information about the environment in which the system is installed.
[0058]
The person database 60 is a database that accumulates information about users, and is a database that accumulates personal information including user identifiers, information when approaching the environment in the past, and contact information.
[0059]
Next, the operation of the first embodiment will be described.
[0060]
FIG. 2 is a perspective view showing a specific example of the first embodiment.
[0061]
The environment change transmission device 100 shown in FIG. 2 uses a camera as the environment change detection sensor 10, uses a projector as the information presentation device 40, uses an IR (infrared) sensor as the position sensor 20, and uses a bulletin board BD. It is an Example which shows a user the change of the posting thing in.
[0062]
On the bulletin board BD that informs you of university closures and supplementary courses, and on the bulletin board BD that sells and buys at supermarkets, various people post, move, and delete postings every day. It is difficult to determine whether it is a thing. Therefore, since it is necessary to look at all the postings to some extent, the same information is repeatedly viewed or new information is missed. In such an environment, the necessary notices can be displayed by presenting to the user the changes that have occurred since the last time they were viewed (new notices were posted, notice places moved, deleted etc.). You can search efficiently.
[0063]
<Operation of Example>
FIG. 3 is a flowchart showing the operation of a system in which a camera is used as the environment change detection sensor 10, a projector is used as the information presentation device 40, and an IR (infrared) sensor is used as the position sensor 20 in the first embodiment. It is.
[0064]
Periodically, environmental images are acquired from the camera (S1), and changes in the environment are extracted using image processing (S2) and stored in the environmental database 50 (S3). In addition, when the user in the environment is constantly monitored and the IR sensor is used to recognize that the user has come to the environment, the environment change transmission device 100 from the person database 60 has previously visited the environment. Date and time (S4), and the changes that have occurred from the previous date and time to the present are acquired from the accumulated environmental changes (S5, S6). A spotlight is applied to the portion using a projector (S7).
[0065]
<Extraction of changes>
Next, a method for acquiring an environmental change from an image will be described.
[0066]
The resolution of the required change varies depending on the situation where the user is placed. Therefore, it is necessary to present changes with the resolution desired by the user.
[0067]
The “change resolution” is the difference in the speed of change and the average span in which the change occurs. In other words, the environment in which we live changes day by day, but there are also rapid changes such as the movement of cars and people, and gradual changes such as the growth of plants. In addition, there are those that change many times in a short time so that humans act, and those that do not change for a long time once they change so that the position of the television set at home does not move much.
[0068]
For example, when looking for something like a TV remote control, I want to get a change in the movement of the TV remote control, but I don't want a change in human movement. Therefore, the resolution of change is determined by the following method.
[0069]
First, as a premise, things such as TV remote control, books, posters, etc. will stay in the same place to some extent, and conversely, people and animals will not stay in one place. In accordance with this premise, after moving, a thing that stays in that place for a certain period of time is cut out and changed to an “object”.
[0070]
FIG. 4 is an explanatory diagram for extracting changes in the above embodiment.
[0071]
Specifically, for example, the change can be extracted by using the background difference.
[0072]
1. Accumulate regularly captured images (original images)
2. Create a density histogram for each pixel in the past several frames (n frames)
3. Create an image (background image) with the mode value density of each histogram as the pixel value
4). Create a difference image between the past background image and the current background image (difference image)
For example, while capturing 1 frame per second, a background image is created using captured images at n = 1800, that is, in the past 30 minutes. When the k-th background image is created, the gray scale value histogram of 1800 data is created for each pixel using the k-1800th to k-th images.
[0073]
The kth background image is created by setting the density of the mode value of this histogram as the gray value of each pixel. This background image is an image composed of pixels that have been photographed for the longest time in the past 60 minutes. Finally, the change can be extracted by taking the difference between the past background image and the currently created background image. In this case, when a new posting is pasted, the change is extracted after 15 minutes.
[0074]
By reducing the number of frames used to create the density histogram (changing the resolution), it is possible to detect objects that change frequently. When the number of frames is small, the change can be detected by using the median instead of the mode value of the histogram.
[0075]
<Optical flow>
In the first embodiment, when a change is detected, a background difference is used. For example, by using the optical flow method for detecting the change, not only the location of the change but also the speed of the change, the direction of the change, and the like can be acquired.
[0076]
By using such information, a variety of change presentation methods are possible. For example, it is possible to present only a fast change speed or present only a constant change direction. It can also emphasize the slow changes. Since an existing method has already been proposed as an optical flow detection method, the description thereof is omitted here.
[0077]
<Presentation of change>
When searching for things efficiently, when presenting changes in the environment to the user, they must be presented concisely and effectively. As this method, for example, a method in which the time interval is compressed while maintaining the time series and a spotlight is applied to a changing place can be considered.
[0078]
When showing the change from the previous visit, if the previous visit came 3 days ago, the change from 3 days ago is compressed into a few seconds and presented as an animation. Since the changes extracted by the above method are the position before the movement of the object and the position after the movement, the time during movement or stationary can be deleted and compressed. By presenting information compressed by this method in order from the past, a change animation can be created.
[0079]
For example, if the user wants to see changes in the environment in an instant by shining a spotlight in order from the past to the part where the change has occurred, the place where the change has occurred since coming in the past, All can be illuminated with spotlights. In this case, however, if a lot of changes occur, the environment may be filled with spotlights, and it may be difficult to understand the changed part.
[0080]
<Database>
In this embodiment, two types of environment database 50 and person database 60 are defined. The environment database 50 is a database that accumulates environmental changes. When an image obtained from a camera is used when an environmental change is extracted, the captured image and data (image) from which the change is extracted are held.
[0081]
FIG. 5 is a diagram showing the user ID, address, resolution, and last access time in the above embodiment.
[0082]
On the other hand, as shown in FIG. 5, the person database 60 stores data of people who come to the environment, and includes user IDs, user attributes (e-mail addresses, etc.), environment setting information (recognized change time intervals and change speeds). Etc.), the last access date and time, and the position and time when the system was in the past.
[0083]
In addition to the user's e-mail address, other personal information such as the actual address and age may be registered as the user attribute.
[0084]
Environment setting information not described in the above embodiment will be described in the second embodiment, and user attributes, position and time when the user has been in the system in the past will be described in the third and subsequent embodiments. Describe usage.
[0085]
<Multiple devices>
The apparatus according to the present embodiment includes three types of devices, that is, the environment change detection sensor 10, the position sensor 20, and the information presentation device 40. By using a plurality of devices, the environment change can be performed with high accuracy. Extraction, change information output method according to the situation, and the like are conceivable.
[0086]
For example, by using a camera 11 and an ultrasonic sensor as the environment change detection sensor 10, a change is extracted from the image of the camera 11, and information obtained from the ultrasonic sensor is used to change the three-dimensional portion of the changed portion. Can be detected. In addition, by using two cameras 11, changes can be detected as three-dimensional information.
[0087]
On the other hand, by using the projector 41 and the portable computer, it is possible to project a rough change by the projector 41 and project a fine change by the portable computer. The position sensor 20 can also acquire global position information with high accuracy by using an IR sensor and GPS.
[0088]
<User authentication>
When the user ID is not registered in the person database 60 or when the user comes to this place for the first time, the environment change transmission device 100 cannot acquire the time when the user came to this place last time. In this case, several methods can be considered as a method for presenting the change. One method presents the user with a change in the environment that has occurred within a certain period. For example, when the user comes to a certain place as described above, changes that have occurred within the past week are presented.
[0089]
Also, a method of not presenting environmental changes to the user is also conceivable. In this case, it is necessary for the user to interact with the environment, specify a period for which browsing is desired, and request the presentation of changes that have occurred in the past. A method for interacting with the environment will be described in the second embodiment.
[0090]
Further, when the user turns off the position sensor 20 (in the case of the position transmission type position sensor 20) and does not wear the position sensor 20, the user's position and ID cannot be detected. In such a case, it is possible for the user to interact with the environment and request presentation of changes that occurred in the past, but based on information obtained from sensors attached to the environment, the approaching object is human. If it is found that, it is possible to automatically present changes within a certain period. In order to recognize that an approaching object (change occurring in the environment) is a person, a method used for a sensor light, an automatic door, or the like can be used.
[0091]
When a human is recognized in this way, a method using a pyroelectric infrared sensor, a method for recognizing and judging an image based on an image obtained from a camera, a method using a capacitance, and the like can be considered. Regarding these methods, since there are existing technologies, the description thereof is omitted here. By using human sensors, environmental changes can be transmitted to an unspecified number of people who do not have the position sensor 20.
[0092]
The environment database search means 36 periodically acquires environmental images from the camera when extracting changes occurring in the environment, accumulates them in the environment database, and stores the accumulated past frames for each pixel. This is an example of density histogram creation means for periodically creating a density histogram, and image storage that periodically creates an image with the mode value density of each histogram as a pixel value and stores it in the environmental database. It is an example of a control means, and furthermore, a change area that compares an image created in the past with a currently created image and recognizes a portion where a difference of a certain threshold or more is recognized as a change area as a result of this comparison It is an example of the authorization means.
[0093]
Further, the environment information / environment change registering means 33 and the position information registering means 35 are registered in a person database and a setting change related to transmission of a change occurring in the environment based on information input by the user from the input device. This is an example of change / write means for changing user information and writing a message to the environment.
[0094]
[Second Embodiment]
<Dialogue with the environment>
FIG. 6 is a block diagram showing an environment change transmission device 200 according to the second embodiment of the present invention.
[0095]
The environment change transmission device 200 is an embodiment in which an input device 70 is newly added to the environment change transmission device 100 and a processing device 30 a is provided instead of the processing device 30.
[0096]
The processing device 30 a is a device in which the setting management unit 39, the input acquisition unit 71, and the input management unit 72 are provided in the processing device 30.
[0097]
The environment change transmission device 200 can achieve change acquisition, interactive message writing, environment setting change, and the like using an interactive method.
[0098]
When the position information acquisition unit 34 acquires the user ID and position information, the position information acquisition unit 34 sends the ID and position information to the person database 60. Based on the ID, the person database 60 searches the person database 60 for user settings, records of accessing the environment in the past, and the like, and the setting management means 39 manages user information.
[0099]
On the other hand, the input acquisition unit 71 acquires information input by the user, and the input management unit 72 changes various settings, rewrites user data in the person database 60, and writes a written message based on the acquired information. Registration and the like performed in the environment database 50 are executed.
[0100]
The processing device 30a searches the environment database 50 according to the changed setting of the user stored in the setting management means 39 or re-extracts the change of the environment according to the user's request. The change information is output to the information presentation device 40.
[0101]
Further, a menu at the time of setting change is also output to the information presentation device 40 through the change information output means 38. Even when another user registers a message in the environment, the environment database search / extraction means 36 searches for the message, and the change information output means 38 outputs the message to the information presentation device 40. The rest of the processing is the same as in the first embodiment.
[0102]
The input device 70 is a portable computer, a pen-type interface, voice input, or the like. When a portable computer is used, the user connects to the processing device 30a provided in the environment, and interacts using the portable computer. For example, the mobile phone with built-in Bluetooth is used to connect to the environment change transmission device 100. As a result, various menus are displayed on the screen of the mobile phone, and the user interacts with the environment accordingly. For example, by moving the up / down button or wheel of a mobile phone, changing the resolution of the presented environment change, or by pressing the left / right button, rewinding or fast-forwarding the presented environment change. Can do. You can also paste messages to yourself and others in the environment, change the date and time when you came to this place in the past, and define the changes you want to see again next time.
[0103]
On the other hand, when a pen-type interface is used, it can be operated more intuitively than an operation performed using a portable computer.
[0104]
FIG. 7 is a perspective view showing a case where a pen-type interface is used in the second embodiment.
[0105]
A contact sensor 24 and an infrared LED 25 are built in the tip of the pen 23, and the infrared LED 25 is turned on when the pen tip contacts a wall surface or the like. Further, a button 26 is provided on the pen 23, and when the button 26 is pressed, the infrared LED 25 is turned on.
[0106]
When the infrared ray 25 is generated and the camera 11 recognizes the infrared ray reflected by the bulletin board BD, the current position of the pen tip can be known. Thus, a message can be written on the wall surface directly using the pen 23, and the resolution of the presented environmental change is changed by pressing the button 26 and shaking the infrared LED 25 in front of the camera 11. Can do.
[0107]
This method is an example in which the input device 70 is realized by the environment change detection sensor 10 (camera 11) and the infrared LED 25 in the environment change transmission device 200 shown in FIG.
[0108]
When voice input is performed, a voice obtained from a microphone attached to the environment or a microphone owned by the user is recognized and a dialogue with the environment is performed.
[0109]
For example, DUG-1 realizes a dialog with an agent using voice. DUG-1 is "Mikio Nakano, Koji Dosaka, Noboru Miyazaki, Junichi Hirasawa, Shinji Tamoto, Masahito Kawamori, Satoshi Sugiyama, Go Kawabata: Spoken Dialogue System DUG-1 for Flexible Speaker Change, Language Processing Society of Japan The 5th Annual Conference Proceedings, 1999 ”.
[0110]
It is also possible to communicate using sound instead of words. For example, the resolution can be changed depending on the volume of the voice.
[0111]
[Third embodiment]
<Communication using change>
FIG. 8 is a block diagram showing an environment change transmission device 300 according to the third embodiment of the present invention.
[0112]
The environment change transmission device 300 is an implementation in which a person who has received a change in the first embodiment (environment change transmission device 100) sends a message to a person who has made a change in the second embodiment (environment change transmission device 200). It is an example.
[0113]
The second embodiment (environment change transmission device 200) is an embodiment in which the user and the environment interact with each other. When the second embodiment (environment change transmission device 200) is used, communication between people can be realized through changes in the environment. For example, assume that a user sees a newly attached poster and wants the poster. In this case, if you can send a message to the person who posted the poster, “If you throw it away after this poster has finished its role, can you give me a message?” Become.
[0114]
Therefore, the person who has received the change using the first embodiment (environment change transmission device 100) sends a message to the person who has caused the change using the second embodiment (environment change transmission device 200). .
[0115]
First, the user specifies the position of the poster when the poster is in a spotlight or when the poster is in a blinking position on the screen of the portable computer. To do. For example, the position of the poster is pointed by a pen-type interface, or the portion where the poster is displayed is clicked on the portable computer. As a result, a specific change is selected.
[0116]
Next, with the change selected, the user writes a message and sends it to the environment change transmission device 300. For example, “Can I have this poster?” Is written on a portable computer and transmitted to the first embodiment (environmental change transmission device 100). In the first embodiment (environment change transmission device 100), when a message is received, the user who has caused the change selected by the user is searched. Then, a message is sent to the searched user.
[0117]
The environment change transmission device 300 is a device in which a message communication unit 82, a change performer assignment unit 81, and a communication device 83 are added to the environment change transmission device 200.
[0118]
The input management means 72 sends the user message obtained from the input acquisition means 71 and the change of the environment selected by the user to the change performer assignment means 81. The change performer assigning means 81 is closest to the environment change transmission device 300 when a change in the environment selected by the user occurs based on the position and time of the change in the environment selected by the user. The user is retrieved from the person database 60, the retrieved user is identified as a person who has changed, and the address of the authorized person is transmitted to the message communication means 82.
[0119]
The message communication means 82 uses the communication device 83 to transmit the message input by the user to the address of the user who is recognized as the person who has caused the change. Other operations are the same as those in the first and second embodiments.
[0120]
<Input device 70>
The method described in the second embodiment can be used for specifying environmental changes and inputting messages. Here, a new input method that allows direct input using a portable computer is used. suggest.
[0121]
Currently, mobile phones and PHS are often used as portable computers, but input that enables selection of environmental changes and input and transmission of messages using mobile phone antennas and mobile phones. The method will be described.
[0122]
FIG. 9 is an explanatory diagram when the environment change transmission device 300 performs selection of an environment change, message input, and transmission using a mobile phone antenna and a mobile phone.
[0123]
Currently, there is an accessory for mobile phones called a shining antenna. This glowing antenna is an antenna that responds to electromagnetic waves generated when talking on a mobile phone, and the LED built in the tip blinks or changes color.
[0124]
Therefore, an environmental change is selected using the LED of the shining antenna. When the message is written on the cellular phone, the point of the shining antenna is placed over the change part of the environment to be selected and transmitted to the first embodiment (environment change transmission device 100). Then, connection with the first embodiment (environment change transmission device 100) is performed. At this time, since electromagnetic waves fly between the environment change transmission device 300 and the mobile phone, the LED of the shining antenna is turned on.
[0125]
When the first example (environment change transmission device 100) receives a message, the camera 11 recognizes the position of the LED of the shining antenna. When the recognition is completed, a recognized position and an environmental change are assigned, a message is transmitted to the user who is assumed to have caused the environmental change, and the connection with the mobile phone is disconnected.
[0126]
Then, the LED of the shining antenna is turned off, and it is understood that the recognition and message transmission are completed. The shining antenna does not necessarily need to be an infrared LED, but if it emits infrared rays, position recognition is easy. If you don't like the tip of the antenna to shine, it is preferable to use an infrared LED. Conversely, when it is desired to see visible light that is lit, for example, a red LED often emits energy of an infrared wavelength. In addition, the shining antenna usually blinks or changes color, but when recognizing the position, it is easier to recognize if it does not blink in a single color. Regarding the method of recognizing the LED, since there is an existing technology, the description thereof is omitted here.
[0127]
Further, if the LED at the tip of the antenna can be turned on using a button on the mobile phone or can be turned on using a contact sensor attached to the tip of the LED, the same as the pen type interface of the second embodiment. It can be used.
[0128]
<Determining the user who caused the change>
When searching for a user who has caused a change, for example, detection is performed as follows. When the system of the first embodiment receives the message, based on the time when the change selected by the user occurred, the person who was closest to the place where the change occurred from the person database 60 at that time. Search for. If you can find the person who was closest to where the change occurred, you can send a message to the user's address.
[0129]
<Message exchange method>
As a method of exchanging messages, there is a method of making a call in addition to a method of transmitting a message as described above. When making a call, the basic operation is the same as the message transmission method. In the case of a phone call, when a change is selected and a call is made to the system, the system calls the user suspected of having changed. In this case, since synchronous communication can be realized, the user can quickly and interactively transmit information to be transmitted.
[0130]
However, since the method of making a call and the method of sending a message are one-to-one communication, it becomes a form close to direct mail and has a great psychological barrier. In particular, in this case, since the message recipient suddenly receives a message from a stranger, there is a possibility of feeling uncomfortable depending on the situation and contents.
[0131]
Therefore, I think that a method of exchanging messages gently is also necessary. Messages to the bulletin board BD, mailing lists, etc. have lower psychological barriers than one-to-one communication with strangers. When a change occurs, an email may be sent to multiple people who existed nearby. For example, when a user sends a message, by specifying the number of people, such as 100 people, the message is sent to 100 people who existed in time and space close to the place and time when the change occurred. You may make it send.
[0132]
A person who exists spatially close at the same time, or a person who exists within a certain period within a certain range from the place where the change occurred may be designated. A person who often comes to the environment may automatically create a mailing list and post to the created mailing list. This method will be described later.
[0133]
In this method, information on frequently visited places flows to you at any time, so there is an effect of activating the local community. A bulletin board BD may be created for each environment and a message may be written on the bulletin board BD.
[0134]
<Automatic mailing list>
In the first embodiment, the time and position of the user accessing in the past are stored in the person database 60, so it is possible to calculate how many minutes the user has stayed in this environment during a certain past period. Therefore, a list is created based on the staying time for a certain period in the past, and a mailing list that automatically collects people whose staying time is longer than a certain time is automatically created. Then, messages are exchanged using this automatically created mailing list. It is also possible to regularly transmit environmental change information to this mailing list.
[0135]
This mailing list is automatically deleted if someone stays in that environment for a certain period of time. When registering to the mailing list, when deleting from the mailing list, register people who have stayed for a certain period of time in the mailing list and delete people who have fallen below a certain period of time. A method of registering in order is conceivable.
[0136]
Further, if a plurality of apparatuses according to the first embodiment are installed in some environments, a mailing list may be created using cooperation between apparatuses. People often want to belong to some kind of community. Therefore, the user may be automatically registered in an environment where the user goes most frequently (stays for a long time) among a plurality of environments. In this method, the user can always maintain the state registered in the mailing list of some environment. It is not necessary to narrow down the most frequently visited environment to one, and a plurality of frequently visited environments may be registered in order of frequent visit.
[0137]
Although not automatic, the user may explicitly specify an environment to register or delete the environment.
[0138]
[Fourth embodiment]
<Security>
The fourth embodiment is an embodiment considering security.
[0139]
In the first embodiment, the projector 41 is used to apply a spotlight to a place where a change has occurred. However, the captured image can be stored in the environment database 50 as it is, and the captured image is presented. You can also
[0140]
However, distributing an image obtained from the camera 11 installed in a public place to an unspecified number of people may infringe privacy.
[0141]
Therefore, it is necessary to express in consideration of privacy protection, such as transmitting only the position when the change occurs. On the other hand, if you want to use it for security purposes at home, you can browse the captured image as it is.
[0142]
<Privacy>
As described in the third embodiment, many people feel psychological pressure when a message comes from a stranger suddenly. Therefore, it is necessary to protect user privacy. In the third embodiment, examples of mail, telephone, and mailing list are given. However, since the system manages the address of the other party, the other user can be kept secret. In other words, in the example of sending the message “Would you please give me a poster?” As an example in the third example, the person who sent the message is more than the person who the poster put (possibly) the poster. do not know.
[0143]
On the other hand, the side receiving the message does not know more than the information that the person is in front of the poster unless the other party is specified. However, this is just a setting problem, and depending on the environment, the address and information of the other party may be specified by the system. For example, when used in elementary and junior high schools, it is considered that both the sender and receiver of the message may be specified.
[0144]
Also, if the person who received the message looks at the contents of the message and decides that he / she does not want to receive further messages from that person, he / she will receive messages from that person in the future by communicating that fact to the system. I'm sorry.
[0145]
In this way, the user can set permission / rejection for receiving messages from a specific party. It is also possible to set permission / rejection of message reception for a user having a certain attribute as well as a single partner. For example, when personal information registered in the person database 60 is used and only reception of messages of the same age is permitted, or when the person who sent the message is a person who does not usually come to the place so much , Can refuse to receive the message. As a result, the user can select a message to be received to some extent.
[0146]
In addition, the system side installed in the environment can refuse to transfer the message. For example, by using the data of the person database 60, a restriction may be set such that a message cannot be transmitted from a person who first came to the environment to a person who frequently comes to this place. This prevents casual messages from tourists to the local residents. However, in order to permit / deny message transfer on the system side, it is necessary to fully consider its influence. For example, in the above example, the environment may be more closed.
[0147]
<Handling of human data>
In the first to third embodiments, data relating to users is stored in the person database 60. When data related to users is centrally managed in a database, it is necessary to consider privacy protection.
[0148]
Therefore, a part of the data related to the user may be held in the portable computer on the user side. For example, in the first embodiment, when the user approaches the environment where the system is installed, the date and time when the user existed in the environment in the past is acquired from the person database 60 and the change is presented. The date and time that existed in the environment was stored in the user's portable computer, and when the user approached the environment, the environment system acquired the date and time when the user approached the environment from the portable computer and changed It is also possible to present. In this way, part of user data can be stored in the portable computer, and security can be enhanced.
[0149]
[Fifth embodiment]
<Method not using projector 41 (information presentation device 40)>
In the fifth embodiment, the environment change transmission device 100 exchanges messages via the environment without using the information presentation device 40.
[0150]
In the environment change transmission device 100, a change in the environment is detected by the camera 11, and a spotlight is applied to the place where the change has occurred by the projector 41. Spotlight cannot be applied unless the projector 41 (information presentation device 40) is used. In this state, the method of the third embodiment is executed. In the third embodiment, the change of the environment is specified by specifying the position of the spotlight with a pointer (for example, a shining antenna), but in the fifth embodiment, the spotlight is not hit. The environment is specified by the pointer.
[0151]
In accordance with this designation, in the fifth embodiment, a change occurring at the position of the pointer is retrieved from the environment database 50 and transmitted to the user's portable computer (for example, a cellular phone). The user selects a change to which a message is to be sent from changes in the environment displayed on the portable computer (for example, the time when the change occurred is displayed in time series).
[0152]
The subsequent processing is in accordance with the third embodiment. As a result, messages can be exchanged without using the projector 41 (information presentation device 40). If the portable computer is considered as the information presentation device 40, the configuration is the same as that of the environment change transmission device 100. However, in the first to third embodiments, the user sees the change in the environment presented by the system. After this, the user specifies an environmental change to send a message, but in this method, after the user specifies an environment to send a message, the environmental change is presented to the user. There is.
[0153]
If this idea is followed, after the user designates an environment for sending a message, the change in the environment can be presented to the user by the projector 41. For example, by projecting the time of a change that occurred at the place with a spotlight, or by projecting another change that occurred at a surrounding time as a spotlight together, it helps to know what kind of change it is Become.
[0154]
It is also possible to make the most recent change the environment selected by the user. In this method, the message can be exchanged only with the person who has changed most recently at the place, but the message can be transmitted in the same operation steps as in the third embodiment, and the information presentation device 40 Is not required at all.
[0155]
[Sixth embodiment]
<Expression method>
The sixth embodiment is a method of expressing by a method other than projecting a spotlight in the above embodiment.
[0156]
In the first embodiment, as a method for transmitting an environmental change, a spotlight is projected onto a portion where the change has occurred using a projector 41. When the projector 41 is used to transmit environmental changes, not only the spotlight is projected, but also various expression methods can be considered. For example, according to the speed of change, the spotlight applied to the portion where the change occurs You can vibrate in small increments and change the shape and color of the spotlight. On the other hand, it is also conceivable to illuminate the entire surroundings in advance using the projector 41 and darken only the part where the change has occurred, or darken the part where the change has not occurred.
[0157]
Further, when an output device other than the projector 41 is used, a method using sound can be considered. For example, it is conceivable to use three-dimensional acoustic technology to emit a sound at a place where the change occurs, or to transmit the change according to the intensity or tone of the sound.
[0158]
That is, in the above-described embodiment, an environment designating stage in which a desired environment is designated by a pointer, a search stage in which an environmental change designated by the pointer in the environment designating stage is searched from an environment database, and a search in the search stage. An environment change transmission stage for transmitting the changed environment to the user's portable computer, a user identifier detection stage for detecting an identifier of the approaching user when the user approaches the environment, and the user identifier A time acquisition stage for acquiring the time when the user detected in the detection stage last time approached the environment from the environment database, and a change that occurred between the acquired time and the current time is acquired from the environment database. And an environment change transmission method having an information presentation stage to be presented to the user.
[0159]
Moreover, the said Example can be grasped | ascertained as invention of a program. That is, in the above embodiment, an environment specifying procedure for specifying a desired environment with a pointer, a search procedure for searching the environment database for changes in the environment specified by the pointer in the environment specifying procedure, and a search with the search procedure. An environment change transmission procedure for transmitting the changed environment to the user's portable computer, a user identifier detection procedure for detecting an identifier of the approaching user when the user approaches the environment, and the user identifier The time acquisition procedure for acquiring the time when the user detected in the detection procedure last time approached the environment from the environment database, and the change that occurred between the acquired time and the current time are acquired from the environment database. And an environment change transmission program for causing a computer to execute an information presentation procedure to be presented to a user.
[0160]
The environment designating step and the environment designating procedure are for designating a desired environment by using a pointer. However, in both cases where the spotlight is not applied and the spotlight is applied, the environment is specified by the pointer. Specifies the environment of
The environment change transmission program may be recorded on a recording medium such as a CD, DVD, HD, or semiconductor memory.
[0161]
【The invention's effect】
According to the present invention, a change in the environment is detected by using a sensor installed in the environment, and the part where the change has occurred and the characteristics of the change are presented to the user. Who know the changes in the environment and know the changes in the environment that have occurred in the past, not only can you search for things, but you may also get unexpected discoveries, and people who have changed the environment By performing the estimation, it is possible to exchange messages via the environment.
[Brief description of the drawings]
FIG. 1 is a block diagram of an environment change transmission device 100 according to a first embodiment of the present invention.
FIG. 2 is a perspective view showing a specific example of the first embodiment.
FIG. 3 is a flowchart showing the operation of a system in which a camera is used as the environmental change detection sensor 10, a projector is used as the information presentation device 40, and an IR (infrared) sensor is used as the position sensor 20 in the first embodiment. It is.
FIG. 4 is an explanatory diagram for extracting changes in the embodiment.
FIG. 5 is a diagram illustrating a user ID, an address, a resolution, and a last access time in the embodiment.
FIG. 6 is a block diagram of an environment change transmission device 200 according to a second embodiment of the present invention.
FIG. 7 is a perspective view showing a case where a pen-type interface is used in the second embodiment.
FIG. 8 is a block diagram of an environment change transmission device 300 according to a third embodiment of the present invention.
FIG. 9 is an explanatory diagram when the environment change transmission device 300 performs selection of environment change, message input, and transmission using a mobile phone antenna and a mobile phone.
[Explanation of symbols]
100, 200, 300 ... environmental change transmission device,
10 ... Environmental change detection sensor,
20 ... position sensor,
30, 30a, 30b ... processing device,
31 ... Environmental information acquisition means,
32 ... Environmental change extraction means,
33 ... Environmental information / environmental change registration means,
34 ... Position information acquisition means,
35. Location information registration means,
36 ... Environmental database search means,
37. Human database search means,
38 ... change information output means,
39. Setting management means,
40. Information presentation device,
50 ... Environmental database,
60 ... Human database,
70 ... input means,
71 ... input acquisition means,
72. Input management means,
81. Change implementer assigning means,
82 ... message communication means,
83: Communication means.

Claims (13)

An environmental change extraction means for extracting a change that has occurred in the environment based on information output from an environmental change detection sensor installed in the environment;
Environmental change registration means for registering the extracted changes in the environmental database;
User identifier detection means for detecting an identifier of the approaching user when the user approaches the environment;
Time acquisition means for acquiring the time when the user detected by the user identifier detection means last approached the environment from the environment database;
Information presenting means for obtaining changes presented from the obtained time to the current time from the environmental database and presenting them to the user;
An environmental change transmission device characterized by comprising: In claim 1,
An environment change transmission device comprising a person database for storing personal information including a user identifier, information when the environment has been approached in the past, and contact information. In claim 1 or claim 2,
The environmental change detection sensor is a camera,
The user identifier detection means is an infrared sensor,
An environment change transmission apparatus, wherein the information presenting means is a projector. In claim 3,
When presenting a change in the environment to the user through the projector, the change area is projected with a spotlight using the projector onto the place where the change has occurred in the order of the change. Transmission device. In claim 2,
An input device that the user can input;
Based on the information input by the user from the input device, the setting change regarding the transmission of the change occurring in the environment, the change of the user information registered in the person database, and the writing of the message to the environment are performed. An environmental change transmission device comprising a change / write means. In claim 5 ,
The environmental change transmission device according to claim 1, wherein the input device is a pen-type interface in which a touch sensor and an infrared LED are built in a pen tip. In claim 6 ,
When the contact sensor built in the pen tip comes into contact with the wall surface, the infrared LED provided on the pen tip is turned on, and the position of the lit infrared LED is read by the camera, thereby An environment change transmission device characterized by acquiring the position of the pen tip. In any one of Claims 5-7 ,
Based on the message input by the user from the input device and the information on the environment change specified by the user, the environment change position and time specified by the user are temporally or spatially related to the environment. A means for searching for a user who has been in the environment and for determining that the searched user is a person who has changed the environment;
Transmitting means for transmitting a message entered by the user to the authorized person;
An environmental change transmission device characterized by comprising: In claim 8 ,
The transmission means specifies the number of persons in advance when transmitting a message, acquires information on the number of persons specified in order from the time and place where the environmental change has occurred, to a plurality of persons. An environmental change transmission device characterized by being a means for sending messages. In claim 8 ,
Search the person who has stayed in the above environment for more than a certain time in the above environment, or who has approached the past more than a certain number of times based on the records accumulated in the database, and a mailing list composed of these searched persons, Create dynamically,
The environment change transmission device, wherein the transmission means is means for exchanging environmental changes or messages. An environmental change extraction stage for extracting changes that have occurred in the environment based on information output by an environmental change detection sensor installed in the environment;
An environmental change registration stage for registering the extracted changes in the environmental database;
A user identifier detection stage for detecting an identifier of the approaching user when the user approaches the environment;
A time acquisition step of acquiring the time when the user detected in the user identifier detection step last approached the environment from the environment database;
An information presentation stage of acquiring changes from the acquired time to the current time from the environmental database and presenting them to the user;
An environmental change transmission method characterized by comprising: The program which makes a computer perform the method of Claim 11. The computer-readable recording medium which recorded the program which makes a computer perform the method of Claim 11 .

Images