JP5554804B2 - A system that supports the discovery or recognition of objects, etc. in scenery - Google Patents

Description

  The present invention is provided in a landscape so that a user can easily find or recognize a real object (including an intangible object) that a user wants to find (including an intangible object) that exists in the actual scenery around the user. The present invention relates to a system that supports discovery or recognition of a user's object or the like.

  Conventionally, a laser pointer for a lecturer to point a portion of a projected image with a laser beam spot at a lecture using an overhead projector (OHP) or the like is known. This laser pointer is used in various situations such as lectures at universities and presentations at academic conferences.

Japanese Patent Laid-Open No. 9-192164

  However, the laser pointer described above is used only when the lecturer who is the user points “a point (point) that he / she already knows” to another person, and the user himself / herself targets the laser pointer. It is completely useless when searching for the location (the object or part to be searched for).

  Conventionally, a medical record search support system has been put into practical use for assisting in searching for a medical record on a hospital paper medium using light and sound. This system uses a barcode, color code, magnetic data, or IC chip (semiconductor element) that records the patient's name and identification number in advance on the spine of the medical record folder (container for storing medical records). In addition to recording, a light emitting diode (light emitting unit) and a buzzer (sound generator) are attached. For patients who need medical records, enter a search key such as their name and identification number. Based on that, search the folder where the corresponding name and identification number are recorded, and the light emitting unit attached to that folder Or a sound from the buzzer attached to the folder. Thereby, the person who searches for a medical chart can easily find out what he wants from a large number of medical charts.

  However, this chart search system displays and attaches in advance a bar code, color code, magnetic data, or semiconductor element that records a patient's name and identification number in a chart folder, and a light emitting diode (light emitting section). ) And buzzers (sound generators) must be installed, so it can be applied only to specific types of medical records, etc., and can not be arbitrarily applied to various types of loose items. There is a problem.

The present invention has been made inspired by the above-described problems of the prior art, and a plurality of objects (including intangible objects) that exist in the actual scenery around the user. Or, by the user, such as objects in the landscape, that makes it possible to easily find or recognize the object that they are seeking or looking for, the object that is the object, or the display contents from the display contents. The object is to provide a system that supports discovery or recognition.

(Definition)
In this specification, the “search object” is often a physical object (tangible object), but among a plurality of icons and buttons displayed on the display screen (graphical user interface screen). One or the like may be an “ intangible thing ( intangible )”. In this specification, these tangible and intangible objects may be collectively referred to as “ objects ”.
Further, in this specification, the “search key” is used for searching (checking, searching) such as characters (keywords, sentences, conversations, etc.), symbols, figures, images, predetermined sounds (acoustics, voices), etc. Any key can be used.
In the present invention, the identification method by “identification means for mutually identifying each object or part in the video including the search target” is, for example, detecting the color and shape of each object or part in the video. A method of identifying by comparing with a pattern of color or shape stored in advance, a method of reading and recognizing characters such as names displayed on each object or part, and for identification displayed on the surface of each object or part A method of identifying by reading a bar code, a two-dimensional code, an identification color code, or the like, and an identification code recorded on an IC chip (semiconductor element) attached to each object or part wirelessly with the IC chip Various methods such as a method of identifying by exchanging can be adopted.
In the present invention, for example, when the user desires “an operation to save the data on the screen of the personal computer to the external magneto-optical disk”, the pointing device according to the present invention first stores the “file” on the personal computer screen. Instruct the icon with a laser beam (this instructs the user to click this icon with a mouse, etc.), and then instruct the button part with the word “Save As” with the laser beam. (This tells the user to click the “Save As” button.) Then, the button with the word “Location to save” is indicated with a laser beam. Instruct the user to click the button part with the word “Location to save”, and then instruct the button part of “Execute save” with a laser beam ( In this way, the user is instructed to click the “execute save” button part), and the object or part necessary for “a series of operations necessary for data storage” is Point to the user sequentially in accordance with the operation.
Further, in the present invention, “a sound generating means for emitting a predetermined sound at or near the place where the laser beam hits” is detected to detect that the laser beam has been applied. And a diaphragm or a buzzer that is driven based on a signal from the detection means. As means for detecting that the laser beam has been irradiated, for example, an optical sensor such as a light receiving element, or a thermal sensor such as a thermoelectric element (the spot irradiated with the laser beam depends on the energy of the laser beam) Various things can be adopted.
In this specification, a laser beam is irradiated on a search object (thing that the user wants, a thing that the user wants to find) to indicate the search object to the user. And displaying the video showing the surrounding objects (including people, animals, plants), and further indicating the search object on the screen with, for example, an arrow mark, etc. " The present invention discloses an invention that indicates a search object to be searched for, for example, an elderly person whose vision has declined.
In the present invention, the “character” includes a number, and the “symbol” includes a symbol formed of a predetermined graphic or color pattern such as a bar code, a two-dimensional code, or a color code.

The contents of the invention described in the claims of the present application are as follows.
1. "Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels Video acquisition means for acquiring video (moving image or still image) of “including landscape”, search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols, Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, "an object, a picture, a figure, a character or a character corresponding to the search data" Based on the identification means for identifying or extracting “symbols” and the results identified or extracted by the identification means, “corresponding to the“ specified or extracted “data for search” The above-mentioned user of a thing, a picture, a figure, a character or a symbol "or a thing to which it is attached (including not only a tangible object but also an intangible object such as an icon or a button existing in a display screen) Display control means for performing “display indicating the position or presence” on the screen of the display device so that the user can easily find or recognize the “position or presence in the actual scenery around the A system for supporting discovery or recognition by a user of an object in a landscape, etc.
2. "Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels Video acquisition means for acquiring a video (moving image or still image) of “landscape included as”, search data acquisition means for inputting, receiving or acquiring search data composed of characters or symbols and having a predetermined meaning; Based on the search data input, received or acquired by the search data acquisition means, the video acquired by the video acquisition means “one or more having a meaning matching the meaning of the search data Based on the identification means for identifying or extracting the "character or symbol string" and the result identified or extracted by the identifying means, The “string of characters or symbols” or the object to which it is attached (including not only tangible objects but also intangible objects such as icons or buttons present in the display screen) Display control means for performing "display indicating the position or presence" on the screen of the display device so that the user can easily find or recognize "position or presence in the landscape" A system that supports the discovery or recognition of objects in the landscape.
3. "Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels Video acquisition means for acquiring video (moving image or still image) of “including landscape”, search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols, Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, "an object, a picture, a figure, a character or a character corresponding to the search data" Based on the identification means for identifying or extracting “symbols” and the results identified or extracted by the identification means, “corresponding to the“ specified or extracted “data for search” The above-mentioned user of a thing, a picture, a figure, a character or a symbol "or a thing to which it is attached (including not only a tangible object but also an intangible object such as an icon or a button existing in a display screen) By indicating the position or presence in the actual scenery around the user with light or the like so that the user can easily find or recognize "the position or presence in the actual scenery around the user" A system for supporting the discovery or recognition by a user of an object or the like in a landscape.
4). In the above item 3, the recognition assisting means indicates the position or presence in an actual landscape around the user with a laser beam emitted from the laser beam emitting means. A system that supports the discovery or recognition of objects in the landscape.

(1) According to the present invention, the user can select a plurality of objects existing in the actual scenery around the user (in the display screen when the display screen exists in the scenery around the user). Including intangible objects such as icons and buttons that are displayed) or their display contents, you can easily find or recognize the object that you are seeking or looking for, the object that is the object, or the display contents I can do it.
(2) According to some inventions of the present invention, a user can use a plurality of objects existing in an actual scenery around the user (when a display screen is present in the scenery around the user). (Including intangible objects such as icons and buttons displayed on the display screen) or the display contents of the display, the object that the person is seeking or looking for, the object that is the object, or the display contents For example , it can be easily found or recognized by a display object or display content displayed on the screen of the display device .
(3) According to some inventions of the present invention, the user can select a plurality of objects existing in the actual scenery around the user (when the display screen is present in the scenery around the user). (Including intangible objects such as icons and buttons displayed on the display screen) or the display contents of the display, the object that the person is seeking or looking for, the object that is the object, or the display contents For example, it can be easily found or recognized by a laser beam actually irradiated in the actual (real) world .

It is a block diagram which shows Embodiment 1 of this invention. It is a flowchart which shows the operation | movement of this Embodiment 1. It is a block diagram which shows Embodiment 2 of this invention. It is a figure for showing operation of this Embodiment 2.

Embodiment 1
FIG. 1 is a block diagram conceptually showing Embodiment 1 of the present invention. In FIG. 1, 1 is an imaging unit (for example, a CCD camera) for imaging an object 2 and 3 is for a user to input a search key (for example, a keyword, a key image, a key graphic, etc.). A search key input unit 4 is a search processing unit (by a microcomputer or the like) for searching for a target object corresponding to the search key based on the video from the imaging unit 1 and the search key from the search key input unit 3. 5) is a laser beam generator that generates a laser beam toward the target object based on the search result of the search processor 4.

  In the apparatus of FIG. 1, the imaging unit 1 and the laser light generation unit 5 are housed in one housing 6 (see reference numeral 6 in FIG. 1) and attached to a position such as a user's head, neck, or chest. It is desirable that That is, this housing | casing 6 may be formed as body-worn type members, such as a necklace, a brooch, and a head holding | maintenance part, for example. Further, the housing 6 may be used by a user holding it like a normal personal digital assistant (PDA) or notebook computer. The reason why the imaging unit 1 and the laser light generation unit 5 are preferably housed in the single housing 6 is described above because the laser light generation unit 5 accurately directs the laser beam toward the target object. This is because the positional relationship between the imaging unit 1 and the laser light generation unit 5 needs to be specified in advance in order to irradiate. For this reason, if the positional relationship between the imaging unit 1 and the laser light generation unit 5 can be specified (for example, the spatial position of the imaging unit 1 or the laser light generation unit 5 is determined by a gyro sensor or a magnetic field). The information on the spatial position can be detected by a sensor or the like so that the information on the spatial position can be communicated between the image capturing unit 1 and the laser light generating unit 5 by radio such as infrared rays at any time). The generator 5 may be housed in separate housings that are separated from each other.

  In FIG. 1, it is desirable that the search processing unit 4 is also housed in one housing 6 (reference numeral 6 in FIG. 1) together with the imaging unit 1 and the laser light generation unit 5. This is because the search processing unit 4 receives the video data from the imaging unit 1 and needs to grasp the spatial position data of the imaging unit 1 and the laser light generation unit 5. It is because it is more convenient to store in the case 6. However, if data can be communicated wirelessly between the imaging unit 1 and the laser light generation unit 5 and the search processing unit 4, the search processing unit 4 is separated from the imaging unit 1 or the laser light generation unit 5. You may make up your body. The same applies to the search key input unit 3. That is, the search key input unit 3 is preferably housed in one housing (reference numeral 6 in FIG. 1) together with the imaging unit 1, the laser light generation unit 5, and the search processing unit 4, but these devices If it is possible to wirelessly communicate with 1, 5, and 4, the search key input unit 3 may be included in a separate housing.

  Examples of objects (objects including search objects) to be imaged by the imaging unit 1 include, for example, an operation panel provided with various operation buttons of a keyboard portion of a personal computer or a video tape recorder that a user is trying to operate, and a large number of books. Bookshelves lined up, shelves with medical charts in the hospital, landscapes on street corners (including building signs such as buildings and road signs), operation panels that can be seen from the driver seats of cars and airplanes An operation unit), a map on a paper or a display, a newspaper, an affected part of a patient's body that is a target of surgery performed in a hospital, and the like.

  In FIG. 1, reference numeral 3 denotes a search key input unit for a user to input a search key. The search key input unit 3 includes, for example, a keyboard or pen input tablet for the user to input keywords and sentences, a microphone for the user to input keywords and sentences by voice, and a user from a scanner or a data recording device. And various devices such as an image capturing device for inputting the image.

  Further, the search processing unit 4 including a microcomputer or the like is based on the video (image) from the imaging unit 1 and the search key (character, voice, figure, or image) from the search key input unit 3. Of the objects in the video, the object corresponding to (applicable to) the search key (search object, which is often a physical object (tangible object), but a display screen (graphical user interface screen) It may be an “intangible thing (intangible object)” such as one of a plurality of icons and buttons displayed above, etc. In the following, these tangible object and intangible object will be referred to as “object”. Search).

  In the search process, the search processing unit 4 first identifies each object in the video from another object by a predetermined method (in this case, the identification method is, for example, the video A method of identifying by detecting the color and shape of each object inside and comparing it with a pre-stored color or shape pattern, an identification barcode or two-dimensional code or identification color displayed on the surface of each object Various methods, such as a method for identifying by reading a code, a method for identifying the identification code recorded on the IC chip (semiconductor element) attached to each object by wirelessly communicating with the IC chip, etc. Method can be adopted). Then, the feature of each identified object is compared and collated with the input search key, and the object corresponding to the search key is specified. Then, the search processing unit 4 determines the irradiation direction of the laser light for generating the laser light from the laser light generating unit 5 toward the specified object (the search processing unit 4 is as described above). Since the data of the mutual positional relationship between the laser light generation unit 5 and the imaging unit 1 is possessed, based on the video from the imaging unit 1, the laser beam for irradiating a certain object in the video The calculation of the irradiation direction is possible).

  Note that the search processing unit 4 not only receives the video from the imaging unit 1 and the search key from the search key input unit 3, but also receives data from the external device together with them, so that the target May search for objects or parts of For example, if the object 2 (refer to reference numeral 2 in FIG. 1) that the image capturing unit 1 captures is a personal computer, the user can tell me how to copy a certain data on the screen of the personal computer. A search key “” (the search key can also be input as “sentence” or “conversation”) will be described. At this time, the imaging unit 1 captures an image of the personal computer that is the object 2 and sends it to the search processing unit 4. The search processing unit 4 transmits a signal for inquiring about an operation method corresponding to the search key to a personal computer that is the object 2 by radio such as infrared rays. Then, for example, an operation method “first specify a range and click a copy icon” is returned from the personal computer (object 2). Therefore, the search processing unit 4 is prepared on the screen for copying, such as a target object or part to be instructed, based on the operation method data returned from the personal computer (target object 2). The icon is searched from the video and specified. And the search process part 4 transmits a control signal to the laser beam generation part 5 so that a laser beam may be irradiated toward this specified icon.

  Further, for example, when the object 2 to be imaged by the imaging unit 1 is an operation panel of an airplane cockpit, a case where a user inputs a search key “operation method when the airplane turns right” by voice input will be described. . At this time, the search processing unit 4 captures an image of the operation panel from the imaging unit 1 and also uses an external notebook personal computer 7 (see FIG. 1). The database of operation methods in the cockpit is recorded.) "The operation method when the airplane turns right" is inquired wirelessly. After confirming that the personal computer 7 (this personal computer 7 is not the object 2 in FIG. 1) is the “operation method” that is the answer to this inquiry, for example, “the wind meter is below a predetermined value, "Slowly turn to the right" to send the answer (a series of operation methods) wirelessly. Therefore, the search processing unit 4 that has received this answer (a series of operation methods) analyzes the video and identifies the target wind meter portion and the operation can. Then, in accordance with the series of operation methods, the search processing unit 4 first irradiates the wind meter with laser light in time series, and then irradiates the operation can with laser light. The laser beam generator 5 is controlled so that That is, the search processing unit 4 irradiates the wind meter with laser light in order to instruct the user that “first, confirm that the wind meter is equal to or less than a predetermined value”. And then, “Next, in order to instruct to turn the operating can slowly to the right, the operating can is irradiated with laser light.

  The laser light generation unit 5 irradiates the target target object with the laser light based on the control data indicating the laser light irradiation direction from the search processing unit 4. When the irradiated laser light is visible light, a laser light spot is displayed on the target object, so that the user can easily know (view) the target object. Therefore, it is desirable that the laser beam from the laser beam generator 5 is a visible light laser at least in a room. On the other hand, in the case of the outdoors, if a visible light laser is frequently generated, it may cause trouble for others, and it is unavoidable to generate laser light that cannot be seen with the naked eye, such as an infrared laser. When the irradiated laser light is not visible light such as an infrared laser, the user can use a night vision glasses or a night vision camera (a CCD camera to view the surrounding scenery (the infrared laser light or the like is By using a small liquid crystal display in front of the eye) or a star eye (amplifying ambient light to display in front of the eye) The locus of the laser beam and the laser beam spot on the target object can be recognized.

  Next, the operation of the first embodiment will be described with reference to FIG. The user captures an image of a surrounding landscape (a landscape including the search target as a part thereof) by the imaging unit 1 such as a CCD camera (step S1). Then, the search processing unit 4 identifies each object in the captured video from each other. As described above, various identification methods are employed in this case. Further, the user inputs a search key from the search key input unit 3 by voice, text, image, or the like (step S3). Note that either step S1, step S2, or step S3 may be earlier or may be in a simultaneous progress relationship. Next, the search processing unit 4 identifies a target object corresponding to (corresponding to) the search key while comparing / collating the input search key with each identified object (step S5). . Next, the search processing unit 4 is based on the image from the imaging unit 1 and the spatial positional relationship data between the imaging unit 1 and the laser beam generation unit 5 from the laser beam generation unit 5. The irradiation direction for irradiating the target object with the laser beam is calculated, and the irradiation direction data is transmitted to the laser beam generator 5 (step S5). Based on the irradiation direction data, the laser light generator 5 generates laser light in the direction of the target object (step S6).

Next, various uses of the first embodiment will be described.
(1) When a user is learning to operate a computer in a classroom or the like, it is often difficult to know which key to press when operating a keyboard or the like while receiving instructions from a lecturer. At that time, the first embodiment (brooch type, necklace type, head-mounted type, etc.) is worn on the user's chest, neck, head (forehead), etc. The keyword or sentence “is input”. Then, since the laser beam is emitted from the apparatus of the first embodiment toward the “key button” for the “operation for planting out the screen” and the “icon (pictogram) on the screen”, the user can You can easily know the “key to be pressed next” (computer operation method).

  (2) When the user is learning how to drive a car or an airplane, press the “operation button” or “operator” to see which part of the meter to see while receiving instructions from the instructor. There are many things that do not know what to do. Even in this case, the user can say, for example, “operation when turning right”, so that a series of operation methods necessary for that (the position of the meter to be checked first, the operation button to be operated after the check, It is convenient if the position of the operation can) can be indicated with a laser beam.

  (3) When a user searches for a book whose title is a library, it is often difficult to find a book because there are a large number of books in the bookshelf. Even in this case, when the user inputs a voice to the search key input unit 3 of the first embodiment using the title and author name of the book being searched for (for example, “Natsume Soseki”) as a search key, the imaging unit 1 Video of the bookshelf is captured and sent to the search processing unit 4. Based on this video, the search processing unit 4 reads (recognizes characters) the author's name and title on the spine of the book, and searches for a book with the corresponding title. If there is an applicable book, laser light is irradiated toward the book. As a result, the user can easily find the book that he / she wants from a large number of books.

  (4) When a user wants to find a place name or building name while looking at a map (paper map or electronic map on the display of an information device), a lot of small characters are displayed on the map. In many cases, the name of the place or the name of the building cannot be found (although some electronic maps have a search function, paper maps do not). At that time, it is assumed that, for example, the user inputs a search key “Yaesu Building” by voice while looking at a map of Tokyo using the apparatus of the first embodiment. Then, the imaging unit 1 captures the map as a video and sends it to the search processing unit 4. The search processing unit 4 recognizes a large amount of characters displayed on the map based on the captured map image. Then, the characters “Yaesu Building” are searched while comparing with these recognized characters. When the search processing unit 4 finds the location of the characters “Yaesu Building”, laser light is irradiated toward the location. As a result, the user can easily find the location of the “Yaesu Building” on the map.

  (5) When a user reads a newspaper (paper newspaper or electronic newspaper on the display of an information device), for example, even if he wants to read articles on "patents" without missing, there are a lot of newspaper letters For this reason, it is often missed. Therefore, according to the first embodiment, the user inputs the keyword “patent” by voice, and the imaging unit 1 captures a video of a newspaper in front of the user and transmits it to the search processing unit 4. The search processing unit 4 performs character recognition on the received video to compare and collate the character recognized character with the keyword “patent” to find out if there is an article containing the keyword “patent”. If there is an article to do, the part of the article is irradiated with laser light. Thereby, the user can read an article including a desired keyword without missing the article.

  (6) When a user tries to take out a patient's chart from a shelf in which a large number of charts (paper charts) are arranged in the hospital, it is difficult to find out in a short time because there are many charts. Therefore, when the user inputs the name of a patient by voice using the first embodiment, the imaging unit 1 first images the shelf on which the charts are arranged and sends the imaging data to the search processing unit 4. Further, the search processing unit 4 extracts an identification number corresponding to the inputted name and transmits a signal indicating the identification number wirelessly. On the other hand, an IC chip is attached in advance to each chart, and each IC chip is formed with a recording unit that records the identification number of each patient and a wireless transmission unit that wirelessly transmits its position data. ing. Therefore, when an identification number of a patient is transmitted from the search processing unit 4, the IC chip corresponding to the identification number returns its own position data to the search processing unit 4 wirelessly. The search processing unit 4 calculates the irradiation direction of the laser beam based on the position data sent back wirelessly and the video, and irradiates the laser beam toward the place where the medical chart to be obtained is located.

  (7) When a user walks outdoors or drives by car, the building (characters on the building signboard) and road signs (characters on the road signs) that are sought in the city are overlooked, and it is difficult to find them. There are many. Therefore, it is assumed that the user inputs a keyword such as “Otemachi Building” or “National Route 246” using the first embodiment. Then, the imaging unit 1 captures the surrounding scenery in the traveling direction as a video and sends it to the search processing unit 4. The search processing unit 4 recognizes a predetermined character in the video (a character of a building sign or a character of a road sign), and compares and collates it with the input keyword. And if there exists a thing applicable to the said keyword, a laser beam is irradiated toward the applicable building or road sign.

  (8) Even when the user (doctor) is operating on the patient, for example, if the keyword “liver” is input, the internal part of the patient is imaged and directed to the “liver” portion of the video. Then, laser light can be irradiated.

  In the first embodiment, an operating device that is a “target object (see reference numeral 2 in FIG. 1)” including a search target object such as a diaphragm or a buzzer that detects a laser beam and generates sound. If each part is attached in advance, the target object or part as a search result can be indicated to the user not only by the laser beam but also by sound.

[Embodiment 2]
Next, Embodiment 2 of the present invention will be described with reference to FIG. In FIG. 3, the same reference numerals are given to portions common to FIG. 1. In FIG. 3, 15 is a display such as a liquid crystal display device. Reference numeral 14 denotes a search processing unit for searching for a target object or part based on the image of the target object from the imaging unit 1 and a search key, and pointing the target on the display 14.

  That is, in the second embodiment, the video from the imaging unit 1 is displayed almost as it is on the display 15. The search operation of the search processing unit 14 of the second embodiment is substantially the same as that of the search processing unit 4 (FIG. 1) of the first embodiment, but the method of pointing the target of the search result is different from that of the first embodiment. Yes. That is, in the second embodiment, as a method of pointing to the target object or portion as a search result, not the laser beam as in the first embodiment, but a display on the display 15 (for example, display of an arrow mark). I try to point. There are various ways of pointing and displaying such as an arrow on the screen, a red dot on the screen, and blinking of a certain point on the screen.

  For example, FIG. 4 shows an example of the display screen of the display 15. In this example, the user is going out and going to a certain building (for example, “Otemachi Building”). When the user has difficulty finding the building, the user voices “Otemachi Building” and a search key. Then, the imaging unit 1 captures the surrounding scenery as an image and sends it to the search processing unit 14. The search processing unit 14 compares and collates the search key “Otemachi Building” with the characters on the billboard of the building in the video. If there is a corresponding building, the search ends. Then, on the screen of the display 15, an image from the imaging unit 1 (scenery including the search object as a part thereof) is displayed, and a building of “Otemachi Building” corresponding to the search result (FIG. 4). An arrow (reference numeral 21 in FIG. 4) is displayed on the reference numeral 20) to indicate the building corresponding to the user. Since the screen of the display 15 displays a landscape that is almost the same as or similar to the landscape in front of the user, the user can easily find the desired building by looking at the screen of the display 15.

DESCRIPTION OF SYMBOLS 1 Image pick-up part 3 Search key input part 4 Search processing part 5 Laser beam generation part 6 Case 7 Personal computer 14,15 Display

Claims (12)

" Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. Display control means for performing "display indicating the position or presence" on the screen of the display device ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of characters or symbols and having a predetermined meaning;
Based on the search data input, received, or acquired by the search data acquisition means, one of the videos acquired by the video acquisition means, “one having a meaning that matches the meaning of the search data, or A means for specifying or extracting a plurality of character or symbol strings;
Based on the result specified or extracted by the specifying means, "the specified or extracted 'string of characters or symbols' or an object to which it is attached (not only a tangible object but also a display screen) (Including intangible objects such as icons or buttons) on the screen of the display device so that the user can easily find or recognize the “ location or presence in the actual scenery around the user ”. Display control means for performing `` display indicating position or presence '' ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. As described above, in the actual scenery around the user, by indicating the position or presence by light or the like , recognition support means for supporting the user's recognition,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: 4. The recognition support unit according to claim 3, wherein the recognition support unit is configured to indicate the position or presence in an actual scene around the user by a laser beam emitted from the laser beam irradiation unit. A system that supports the discovery or recognition of objects in the landscape. " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. Display control means for performing "display indicating the position or presence" on the screen of the display device ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of characters or symbols and having a predetermined meaning;
Based on the search data input, received, or acquired by the search data acquisition means, one of the videos acquired by the video acquisition means, “one having a meaning that matches the meaning of the search data, or A means for specifying or extracting a plurality of character or symbol strings;
Based on the result specified or extracted by the specifying means, "the specified or extracted 'string of characters or symbols' or an object to which it is attached (not only a tangible object but also a display screen) (Including intangible objects such as icons or buttons) on the screen of the display device so that the user can easily find or recognize the “ location or presence in the actual scenery around the user ”. Display control means for performing `` display indicating position or presence '' ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. As described above, in the actual scenery around the user, by indicating the position or presence by light or the like , recognition support means for supporting the user's recognition,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: 8. The recognition support unit according to claim 7, wherein the recognition support unit is configured to indicate the position or presence in an actual scenery around the user by a laser beam emitted from the laser beam irradiation unit. A system that supports the discovery or recognition of objects in the landscape. " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. Display control means for performing "display indicating the position or presence" on the screen of the display device ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of characters or symbols and having a predetermined meaning;
Based on the search data input, received, or acquired by the search data acquisition means, one of the videos acquired by the video acquisition means, “one having a meaning that matches the meaning of the search data, or A means for specifying or extracting a plurality of character or symbol strings;
Based on the result specified or extracted by the specifying means, "the specified or extracted 'string of characters or symbols' or an object to which it is attached (not only a tangible object but also a display screen) (Including intangible objects such as icons or buttons) on the screen of the display device so that the user can easily find or recognize the “ location or presence in the actual scenery around the user ”. Display control means for performing `` display indicating position or presence '' ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of images, figures, characters or symbols;
Based on the search data input, received or acquired by the search data acquisition means, from the video acquired by the video acquisition means, “objects, pictures, figures, characters corresponding to the search data” Or a specifying means for specifying or extracting a symbol,
Based on the result specified or extracted by the specifying means, “the specified or extracted“ object, picture, figure, character or symbol corresponding to the search data ”or an object attached to it (tangible object) The user can easily find or recognize the “ position or presence of an intangible object such as an icon or button existing in the display screen as well as an actual scene around the user”. Display control means for performing "display indicating the position or presence" on the screen of the display device ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising: " Actual scenery around the user (visible to the user), including at least some artifacts such as buildings, signs, roads, road signs, shelves, bookshelves, computers, video tape recorders, or control panels a video obtaining means for obtaining an image (moving image or still image) landscape "containing as,
Search data acquisition means for inputting, receiving or acquiring search data composed of characters or symbols and having a predetermined meaning;
Based on the search data input, received, or acquired by the search data acquisition means, one of the videos acquired by the video acquisition means, “one having a meaning that matches the meaning of the search data, or A means for specifying or extracting a plurality of character or symbol strings;
Based on the result specified or extracted by the specifying means, "the specified or extracted 'string of characters or symbols' or an object to which it is attached (not only a tangible object but also a display screen) (Including intangible objects such as icons or buttons) on the screen of the display device so that the user can easily find or recognize the “ location or presence in the actual scenery around the user ”. Display control means for performing `` display indicating position or presence '' ,
A system for supporting discovery or recognition by a user of an object or the like in a landscape, characterized by comprising:

Images