JP3311025B2 - Information retrieval device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval apparatus, and more particularly to an artificial reality in which a retriever grasps and selects a capsule image displayed in a stereoscopic image space by a stereoscopic device through a data input device. The present invention relates to an information search device capable of performing a collation search of a database under Artificial Reality (AR).

[0002]

2. Description of the Related Art Entering a period of flooding information, it is required to quickly and accurately obtain necessary information. For this reason, the importance of the task of easily retrieving a large amount of information has attracted attention. In an input / output interface of a conventional information search device, a keyboard, a touch panel, a mouse, and the like are used for inputting a keyword or the like. A collation search is performed using the input keyword, and the obtained information is extracted by an image or print output. I have. Therefore,
In an actual information search, only an expert who is used to the keyboard and who knows the database system operates the terminal keyboard or the like to perform the information search.

On the other hand, in the field of recent computer simulations, computer interfaces utilizing human movements and the five senses such as sight and hearing have been developed and studied in order to be usable by many users. This is so-called artificial reality (AR).

[0004] This artificial reality is currently roughly divided into two development fields. One area is called virtual reality. In this world of virtual reality, a "user agent", which is an alternation of the operator of the interface, appears in a computer graphics image, and this "user agent" Allows the operator to experience the world unfolded in computer graphics. Another area is Tele-Existence
ence), which allows the operator to operate a remote robot like his / her own through a remotely linked manipulator. In both cases, telexistence focuses on the work surface corresponding to the real environment, while virtual reality uses the computer itself as an interface.

[0005] As an example of a computer simulation tool for realizing the latter virtual reality, a stereoscopic device and a data glove have been developed. Among them, the stereoscopic device provides the user with the world of three-dimensional computer graphics, and is disclosed in U.S. Pat. Nos. 4,881,068, 4,853,769, 4,834,476, and 4,104. 160,973 and many others have been proposed. Among them, in virtual reality, a device that does not obstruct the image due to the operation of the hand of the operator is adopted in order to make the user feel as if the actually formed stereoscopic image can be handled directly by hand. For example, developed by NASA, VPLResearch
The "eyephone" commercialized by Company h is a stereoscopic device in which a liquid crystal display is attached to the inner surface of an eyeglass-like device similar to ski goggles. On the other hand, a data glove is an input interface for inputting an operation of an operator's hand to a computer, and a magnetic sensor and an optical fiber are mounted on the surface of each part of the glove. As a result, the position of the hand of the operator and the movement pattern of the hand can be detected and recognized, and predetermined input information can be input to the computer.

[0006]

However, as described above, the information retrieval apparatus using a conventional keyboard or the like has a problem in that it is difficult for ordinary people to use it and it is not efficient to use it. . The information display method is CR
Since a flat display such as T (cathode ray tube) is displayed, the searcher could not directly recognize the weight of the search index. In addition, in this flat display display, a display method such as a multi-window that simultaneously displays a plurality of screens is possible. However, when the amount of information increases, a plurality of display screens are developed in an overlapping manner. The connection between them could not be displayed effectively, often leading to confusion.

[0007] On the other hand, virtual reality has been used as a simulation tool for computer graphics. However, a medium for directly accessing a database in a computer using a data globe is set, and a set logical operation is performed via this medium. Had not been done.

Accordingly, an object of the present invention is to solve the above-mentioned problems of the prior art, and to allow a searcher to directly select his / her hand to select a medium as an index arranged as a video in an image space, It is an object of the present invention to provide an information search device which can perform a collation search of a database by performing a command operation.

[0009]

According to the present invention, there is provided a stereoscopic apparatus for displaying a set of indexes for managing the attributes of a database in a stereoscopic image space within a field of view of a searcher using an index display body image and visually recognizing the set. An input device that detects a body motion pattern of a searcher with respect to an index display body image as input information and displays the body motion pattern in the stereoscopic video space, and a body motion pattern input via the input device. An information search device comprising: an operation unit for recognizing input information and performing a set operation of indices displayed by a predetermined index display image to perform collation search of the database; and attribute management of the database. Index sets in a two-dimensional video space within the searcher's field of view A two-dimensional visual device for showing and visually recognizing, an input device for detecting a motion pattern of a searcher's body with respect to an index display body image as input information and displaying the motion pattern in the two-dimensional video space; An operation unit for recognizing input information based on a body movement pattern input through the apparatus and performing a set operation of an index displayed by a predetermined index display body image to perform a collation search of the database. It is an information retrieval device that is a feature.

[0010]

As described above, in the present invention, the searcher directly operates the index display body image in the stereoscopic image space or the two-dimensional image space instead of the keyboard, and recognizes the input information based on the body movement pattern at that time. In addition, since the calculation unit performs a set logical operation of the index displayed by the predetermined index display body image to perform the collation search of the database, even a general person who is not used to operating the keyboard can perform the collation search of the information medium such as the database. Easy to do. In addition, since a set of indexes for collation search is displayed as an index display image in the stereoscopic video space or the two-dimensional video space, the index to be searched can be visually and clearly recognized, and the collation search operation can be performed. Can be facilitated. Further, since various operation patterns of the searcher's body are defined as unique input information with respect to the index display body image, the search device can perform a predetermined operation based on the various operation patterns. . At this time, selection of the search index can be easily performed by the input information. For example, the corresponding search index can be deleted by grabbing and playing a certain index display image, and the search index can be captured by pulling the index display image toward the user.

[0011] Also, while displaying an index display image on the display unit, input information for causing the index display image to perform a predetermined operation pattern is input on the display unit,
By performing the set logical operation of the index in the operation unit, the collation search of the database can be easily performed.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the information retrieval apparatus according to the present invention will be described below based on a specific collation retrieval procedure.

FIG. 1 is a virtual view showing the relationship between a three-dimensional image space projected by the information search apparatus according to the present invention and a searcher from the viewpoint of a third party. Reference numeral 90 in the figure
Indicates a searcher. The searcher 90 wears a ski goggle-shaped stereoscopic device 91 on his / her face, and puts data gloves 92R and 92L as examples of an input device on both hands.
Is wearing. Signal lines 93R and 93L extend from the data gloves 92R and 92L, respectively, and these signal lines 93R and 93L are connected to an arithmetic unit 94. The operation unit 94 generates input information data or a processing command to be input to a predetermined search operation unit 96 from the operation pattern of the searcher 90 detected via the data glove 92, and detects the input information data via the data glove 92. A signal to feed back the movement of the searcher 90 and the movement of the capsule image to be displayed in the three-dimensional image space and to project the image is transmitted via signal lines 95R and 95L. Further, the search operation unit 96 is connected to a database 101 whose attributes are managed by an index. FIG.
As shown in the figure, in the stereoscopic device 91, the movement of the data gloves 92R, 92L is represented by the virtual arms 8R, 8L
And various capsule images 4 as if the virtual arms 8R and 8L were projected in the image space.
We are building a world of artificial reality that deals with 5 and 6. That is, a three-dimensional video space as shown in FIG. 2 is displayed in front of the searcher 90. In this three-dimensional video space, the searcher 90 has the right hand data glove 9
The capsule image 4g is grasped by the 2R, and the capsule image 4c is about to be played by the left hand data glove 92L. At this time, the capsule image 4g is grasped by the virtual arm right hand 8R in front of the searcher 90 and the capsule image 4C is grasped by the virtual arm left hand 8L.
The video is about to play. In FIG. 1, reference numeral 7 denotes a storage area. The storage area 7 has a box shape, and capsules can be accommodated in the box.

Next, the situation of the three-dimensional image space projected in front of the searcher will be described in detail.

In FIG. 2, a frame 2 is a box 3 on which information is displayed.
This is a virtual view of the outline of the three-dimensional video space. The three-dimensional image space is set within a range where the virtual arms 8R and 8L of the searcher 90 wearing the data gloves 92R and 92L can sufficiently reach. Also, in this three-dimensional image space, capsule images 3a, 3b,
3c, 3d, 3e; oval capsule images 4a, 4b, 4c, 4d, 4e, 4f, 4g; oval capsule images 5a, 5b, 5c, 5d, 6a, 6
b, 6c, 6d, 6e, 6f and spherical capsule image 7
a and 7b are displayed. The capsule images 3a, 3b, 3c, 3d, and 3e represented by rectangles indicate a set of search indexes of personal information of a small group such as an address book. In the depth direction, they are sorted in alphabetical order. Oval capsule images 4a, 4
b, 4c, 4d, 4e, 4f, and 4g, capsule images 4a, 4b, 4c, and 4g of the same classification level are positioned at a fixed depth, and capsule images 4d of the next classification level
4e is displayed behind them, and a capsule image 4f of a lower level is displayed behind the capsule images 4d and 4e. Oval capsule images 5a, 5b, 5c, 5
Similarly, d, 6a, 6b, 6c, 6d, 6e, and 6f are weighted according to the information, that is, the importance of the search index. The circular capsule images 7a and 7b displayed on the right end of the frame 2 can be set as processing capsules indicating a method of selecting information such as a logical operation or a storage area. Among the capsule images, the capsule images 3a, 3b... 6e, 6f for displaying a set of indexes are index display images, and the capsule images 7a, 7b for displaying calculation information are operator capsule images.

Here, the search index refers to an index (index) for collating and searching the database 101, and is generally understood by terms such as keywords and key items. Further, the search index is configured by attributes extracted from information stored in the database 101. Therefore, in a database search, a set of necessary information can be extracted by collating the search index with the index in the information.
At this time, a plurality of search indexes can be combined by a logical operation to obtain more accurate information. For this reason,
You can easily select the right search index,
It is important to perform these logical operations as desired.
In the present invention, this search is performed using the capsule image.

Next, an operation of selecting a search index based on the capsule images 3, 4, 5, 6,... Will be described with reference to FIG. In FIG. 2, reference numeral 8R indicates the right hand of the virtual arm of the searcher. This virtual arm 8 is based on input information from the data glove 92R actually worn on the hand of the searcher 90 as shown in FIG. It is a virtual image that can move in a stereoscopic image space and take the same free motion pattern as actual human motion such as closing and opening. Now, this screen shows a state in which the right hand 8R of the virtual arm 8 is holding the capsule image 4g with the thumb and the index finger. This means that the search index for the set logical operation indicated by the capsule image 4g is selected by the operation pattern for the capsule image 4g. Here, the pinching operation is specifically described as an example, but the input operation may be such that the capsule image is grasped by the entire right hand 8R of the virtual arm or lightly touched by a finger. In the present embodiment, for example, when the capsule image 4g is picked with the right hand 8R, the state of information selection is changed by a change in color of the color-coded capsule image 4g, a selection sound, or light (flash flashes at the moment of selection). The search index can be recognized by the searcher, and the search index can be easily classified, and an erroneous operation can be alerted or warned by sound or light.

Further, for example, when the capsule image 4g is picked up by the virtual right hand 8R, a sense of pressure can be transmitted from the data glove 92R to the actual right hand of the searcher 90. That is, as shown in FIG. 19, the pressure sensing mechanism 105 is built in the data glove 92R, and the pressure sensing mechanism 105 is operated by input information input to the data glove 92R from the actual right hand of the searcher 90. .

For example, when the arithmetic unit 94 recognizes that the capsule image 4g has been picked up by the virtual right hand 8R based on the input information from the data glove 92, the pressure sensing mechanism 105 of the data glove 92 operates to sense the pressure on the actual right hand. Is transmitted, and the searcher 90 can more surely have a “pick” feeling. As a specific example of the pressure sensing mechanism 105, a method of detecting a force applied to a finger with a strain gauge and converting the detected force into an air pressure applied to a balloon provided therein can be adopted. Further, in the present embodiment, by changing the size of the capsule according to the size of the information amount of the search index of the capsule image, the size of the information can be immediately visually determined. On the other hand, the left hand 8 of the virtual arm shown in FIG.
L indicates a state in which the user attempts to flick the capsule image 4c with the index finger. By this operation, capsule image 4
c can be deleted from the screen. Although the operation of flipping is specifically shown here, the operation of deletion may be performed by paying or grabbing the capsule image with the back of the finger and throwing the capsule image out of the frame 2.

Further, the contents of the above-mentioned search index may be directly displayed on the capsule video like a keyword, and display information indicated by the keyword may be searched. In this case, for example, by directly grasping, for example, two capsule images on which the keywords are displayed with a finger, the information related to the two keywords can be displayed immediately. This search method is particularly effective when searching for complex information related to a plurality of keywords.

Here, an example of a hand movement pattern that can be recognized by the data glove 92 will be described with reference to FIGS. Generally, when there is an object, the hand can perform operations such as grasping, picking, playing, selecting, and moving the object to a predetermined position. By using a capsule image as the object, a set logical operation of the search index displayed in the capsule image in each of the above operation patterns and an operation processing command for the search index can be input to the operation unit.

An example of the definition of the movement pattern of the hand is shown below. FIG. 3 schematically shows the “gripping” operation, in which the capsule image 4 is wrapped in the palm of the virtual arm 8R.
At this time, the position coordinates of the capsule image 4 are known in the computer, and the coordinates (A) of the capsule image 4, the coordinates (B) of the sensor attached to the tip of the index finger, and the sensor attached to the tip of the thumb. Coordinates (C) and (A)
The state where the capsule image 4 is grasped is recognized based on the states arranged in the order of-(B)-(C). At this time, in the present embodiment, the search index of the capsule video 4 in the palm is extracted for collation search, and is defined as a standby state for the next search process. FIG. 4 schematically shows the “pick” operation. When the distance between the sensors attached to the index finger and the fingertip of the thumb is a predetermined amount or less and the capsule image of the keyword exists between the sensors, the capsule image is displayed. It recognizes the picked state. Or,
It can also be recognized by performing image processing on a stereoscopic video. FIG. 5 schematically shows an operation of “playing”. When the moving speed or acceleration of the fingertip detected by the sensor attached to the fingertip of the forefinger is equal to or more than a predetermined value and the forefinger and the capsule image interfere with each other. , The search index of the capsule video is deleted. FIGS. 6A and 6B schematically show an operation of “selecting”. Fig. 6
In (a), when the sensors attached to the four fingertips are close to each other and the virtual line segments (A)-(B) interfere with the capsule image, the search index of the capsule image is excluded. It recognizes the state. In addition, as shown in FIG. 6B, capsule images within a range surrounded by both hands can be selected. FIG. 7 shows an example of the logical operation of the set. In a state where the capsule image 4 which is the right hand 8R is picked up and the other capsule image 5 is picked up with the left hand 8L, it is recognized that the logical AND operation of both search indexes is performed. It is supposed to. At this time, the capsule video 4 of the set of search indexes to be searched in the right hand 8R
(Or a plurality of capsule images) is pinched (gripped), and the operator's capsule image 7a of an operation processing command for executing a logical operation of a set of databases with the left hand 8L is pinched (gripped) to the search index in the right hand 8R. Thus, a predetermined logical operation can be executed.

For example, pick the capsule image 4 with the right hand 8R,
By gripping the capsule image 5 with the left hand 8L, an “OR” operation of the index displayed by the capsule image 4 and the index displayed by the capsule image 5 can be performed. Alternatively, the operator capsule image 7a is "AND"
Is assumed to be displayed, the capsule images 4 and 5 picked up by the right hand 8R are sequentially brought into contact with the operator capsule image 7a picked up by the left hand 8L, so that the image capsules 4 and 5 respectively display. "AND" operation of the index of. In addition, it is also possible to define a predetermined hand motion pattern to stop processing during the search execution, and in this case, specify a hand motion that is not used in normal work to prevent erroneous operation Can be. In addition, although the data glove has been described as an example in the above operation pattern example, it is clear that any structure can be used as long as the input device can grasp the operation pattern of the hand. It is of course possible to use a device other than the iPhone as the stereoscopic device.

Next, an effective search method when a time factor is included in the search index of the capsule video will be described. FIG. 8 shows a linear time coordinate axis T projected in the three-dimensional image space, and the capsule image 4 is projected so as to float along the time coordinate axis T. The capsule image 4 includes a time factor as a search index, and data corresponding to the time axis coordinate T where the capsule image 4 is located can be extracted from the database. That is, if a certain capsule image 4 is located at the position of June 1990 on the time axis coordinate T, only the data of June 1990 is extracted from the data hit by the search index indicated by the capsule image. Further, when it is desired to obtain data when a certain width is given to the time factor of the capsule image, as shown in FIG. 9, by blocking the time coordinate axis T with both hands 8R and 8L of the virtual arm, the range (T ₁₎ To T ₂ ). As a result, of the time factors of the corresponding capsule image, for example,
Data from June 1989 to December 1990 can be extracted.

By the way, since the linear time coordinate axis T has a linear relationship with the depth of the axis and the retrospective time, there is a problem that it is difficult to perform hand operations when searching for an old age. Therefore, as shown in FIG. 10A, the time coordinate axis T can be spirally used to effectively use the depth direction of the image space. At this time, the time step of the time coordinate axis T is made coarser as it goes back,
You can assign the ages effectively. In FIG. 10a, the capsule image 4 can be advanced along the time coordinate axis T by pinching with the left hand 8L. Alternatively, the capsule image 4 can be advanced by drawing a circle several times with the left hand 8L, for example. . Further, the time coordinate axis T may be zigzag as shown in FIG. 10 (b), and the time coordinate axis T may be snake-shaped as shown in FIG. 10 (c).

As described above, according to the present invention, necessary information can be directly selected from a plurality of capsule images displayed in the three-dimensional image space by a human hand. It can be applied to information retrieval of sensitive data. For example, encyclopedia search system, newspaper search system, baseball information search system, patent search system, history search system, map search system, medical information search system, telephone directory, in-house organization search system, personal information search system, etc. Absent.

Hereinafter, a specific search method will be described with reference to FIG. 11 using an encyclopedia search system as an example. FIG. 11 conceptually shows a search method in which the concept of a plurality of stages is sequentially narrowed down from the large classification to the small classification, and the desired item is reached. First, capsule images 11a, 11b, and 11c of level 1 are displayed as large classification information in the above-described three-dimensional image space. The searcher directly grabs the "history" capsule image 11b, which is the information he wants to know, from the displayed capsule image 11 by the method described above. Then, the screen is switched, and the capsule images 13a, 13b, 13c of level 2 are displayed. As in the case of level 1, the capsule image 13b of the "region" that the user wants to know is selected. The screen is switched in the same manner as the transition from level 1 to level 2 and the level 3 capsule images 15a, 15b, 1
5c is displayed and the capsule image 15b of "Europe"
Select The screen is switched in the same manner as the previous screen development, and the level 4 capsule images 16a, 16b, 16c
Is displayed. Here, when the capsule image 16b of "Germany" that the user wants to know is selected, a selection item of German history information, for example, a capsule image of the age of "Medieval" is displayed on the next screen, and these are sequentially selected by one's own hand. You can reach the information you want by going. The operation of selecting the search index can be executed in a very short time.

Although the search method described above has shown a method in which the screens of each level are developed one after another, the search can be performed by other methods. Level 1, which is a major classification, is a search criterion and needs to be selected first. However, the order in which capsule images of level 2 and lower are displayed is not particularly limited. For example, the same concept can be reached in the following three orders. 1 History → Region → Europe → Germany → Era → Medieval → Religion 2History → Era → Medieval → Region → Europe → Germany → Religion 3History → Religion → Region → Europe → Germany → Era → Medieval Therefore capsule images of level 2 or lower When displaying an image, a plurality of levels may be displayed at a time as shown in FIG. Further, appropriate grouping may be performed through a plurality of selection screens, and for example, only level 1 or only level 2 in FIG. 11 may be displayed.

Next, a method of operating the searcher will be described with reference to FIGS. In FIGS. 12 and 13, the search order is represented by STEP1 to STEP6, and the following description will be made accordingly. STEP 1: Capsule images indicating the concept to be searched are selected from capsule images of level 1 which is a large classification projected so as to float in the three-dimensional image space, and a hand is selected.
For example, it is grasped with the left hand 8L. The capsule image grasped by hand is moved to, for example, a storage area 7 provided on the right side of the space (see FIG. 12A). STEP2: Capsule image indicating the necessary concept is selected from the level 2 capsule images, grasped with left hand 8L, and moved to storage area 7 (see FIG. 12 (b)). STEP3: Capsule image indicating the concept that the user needs is selected from the capsule images of level 3 and grasped by left hand 8L, and moved to storage area 7 (see FIG. 12 (c)). STEP 4: Capsule image indicating the concept necessary for the user is selected from the capsule images of level 4 and grasped by left hand 8L, and moved to storage area 7 (see FIG. 13A). STEP 5: Capsule image indicating the concept necessary for the user is selected from the level 5 capsule images, grasped by the left hand 8L, and moved to the storage area 7 (see FIG. 13B). STEP 6: Next, an extra capsule image is skipped with the left hand 8L from the storage area 7 where the selected capsule images are gathered (see FIG. 13C), and the selected capsule image remaining in the storage area 7 is newly added. The displayed chronological table is brought to the position of the time (time) designated by the user on the time coordinate axis T (see FIG. 14A). FIG. 13 (c)
The reason why the extra capsule video is skipped from the storage area 7 is to make the subsequent search operation clear and simple.

Here, if information on the Middle Ages is required,
It is sufficient to bring the selected capsule image to the position of 000 years (see (a) in the figure). In addition, 500-1500 AD
If you need information on the range of years, use 8L on the left and 500 AD
The year, the right hand 8R may be used to grab the year 1500 and indicate the range (see FIG. 3B). Once the display time is determined, the historical fact sought there can be displayed in a predetermined output format. This ends a series of search operations. If there is a range to be displayed, the three-dimensional video space is further expanded to search in detail. As a signal of the end of the processing, an operation such as turning over the hand, pushing the capsule image, or grasping the capsule image indicating “end” may be performed.

The above is an example of searching for historical facts. For example, when searching for the evolutionary process of an organism, for example, if the time axis is set in the horizontal direction of the display area, the operation of moving the hand to the left and right is performed. You can search for past or future (expected) information. In addition, if the time axis is set in the depth direction of the display area, it is possible to directly and easily search for the time factor along the time coordinate axis as described above by manually calling (or sending out). If the information does not include a time-series factor, "importance" may be applied to the coordinate axis so that the information weighted by the coordinate position along the coordinate axis may be displayed. If there are several sets of related information, only the information related to the information grasped by hand can be displayed again. This can prevent confusion when handling a large amount of information.

Next, a search method for narrowing down by a set logical operation between capsule images at the same level will be described with reference to FIGS. FIG. 15A shows the state shown in FIG.
For example, the capsule image 50 of “USA” is taken out from the same state as in STEP 4 of FIG. 4 and moved to the storage area 51 in which the “history” capsule image 54 has already been selected. This shows a state where a capsule image 52 is selected and put in the storage area 51. Next, in this state, unnecessary capsule images are skipped in the same manner as in STEP 6 of FIG. 13C, and three capsule images of the “US” capsule image 50, the “Japan” capsule image 53, and the “Germany” capsule image 52 are obtained. Display the capsule video. Here, as shown in FIG. 3B, the capsule image 53 of "Japan" and the capsule image 52 of "Germany" are grasped with the right hand 8R, and the two capsule images 52, 53 are brought into contact. This operation represents the logical product (AND operation) of the two capsule images 52 and 53,
Information common to 2, 53 can be retrieved in the next step. Further, the two capsule images 52 and 53 brought into contact with each other are
Grab with R and 8L left hand capsule video of "USA"
By grabbing 0, the logical sum (O
R operation). That is, the state shown in FIG. 15B includes “information common to the capsule video 53 of“ Japan ”and the capsule video 52 of“ Germany ”” and “US”.
Of the capsule image 50 of the "capsule image 50". Of course, the operation of bringing the capsule images 52 and 53 into contact is referred to as the capsule image 52,
It is also possible to define the ethical sum of 53 and the operation of grasping the capsule image 50 as a logical product. As described above, the logical sum and the logical product can be easily displayed by the two kinds of operations of collectively capturing a plurality of capsule images, and the set of search targets can be easily specified or changed.

Next, the operation of searching for an exclusive OR will be described with reference to FIG. First, a capsule image is selected in the order of “living organisms” → “plants” → “flowers” by the above-described search method and entered into the storage area. Next, "trees" are selected from "plants" and put into the storage area. Left hand 8 from this storage area
With L, grab the capsule image 61 of “flower” and then right hand 8
Bring the capsule image 63 of "tree" with R, once touch the capsule image 61 of "flower", and release it again. This operation represents an exclusive OR, and means a "flower" that does not bloom in "tree" or a "tree" in which "flower" does not bloom. In addition to this, a negative action can be expressed by the action of grabbing the capsule image and turning it over. In this embodiment, information retrieval can be performed by a natural operation such as touching, releasing, or flipping the capsule image thus grasped.

FIG. 17 is a schematic diagram of a search method according to another embodiment of the present invention. In FIG. 17, a term dictionary 70 in alphabetical order or alphabetical order is displayed in a three-dimensional video space. The searcher trying to search the dictionary can touch the surface of the dictionary, which is an image, with his hand 8 and scroll the screen so as to flip the page. In this case, the screen can be scrolled so as to turn the page by tapping the left hand and the right hand. If there is a word or topic to be sought, the search target can be directly searched by grabbing an icon displaying the word from the dictionary. Note that an icon 72 is embedded in a portion corresponding to all words and phrases on each page of the dictionary, and this icon 72 can function as an information index like a capsule video.
In this way, if you prepare a term dictionary of various genres and perform a search by this method, it will be a search method that alleviates human fatigue because it is not necessary to carry several heavy dictionaries each time as in the past. .

FIG. 18 shows a means for newly inputting a keyword which is not displayed in the capsule video, in the search method according to another embodiment of the present invention. This input means 8
Numeral 0 is the shape of the pen 81 displayed in the space, and the operator can grip the pen 81 with the virtual arm 8R (8L). At this time, by narrowing the interval between the fingers held, ink is discharged, and characters can be drawn on the space. When a new keyword is created, the condition is that an existing corresponding index provided in advance for managing the attribute of the database exists.

The information retrieval apparatus according to the present invention can be used as an educational device for infants by utilizing the fact that characters or figures can be drawn on the space. in this case,
It can provide an interface to the world of information retrieval and computer graphics for people of all ages, including infants and the elderly.

Further, if a keyboard suitable for the user's preference is displayed in the space and input can be made with the device for detecting the movement of the hand (movement of the finger), various keyboards having different standards at present can be used. You can type using the keyboard that is best for you. In the present invention, a conventional keyboard operation can be used at the time of information search.

In FIG. 1, the searcher 9 is held by the data gloves 92R and 92L which are put on both hands of the searcher 90.
Although the example in which the motion pattern of the hand 0 is detected and this is used as the input information is shown, the motion pattern of the head 110 of the searcher is not limited to the motion pattern of the hand of the searcher 90. This can be detected by the band 112 and used as input information. Alternatively, a motion pattern of the feet of the searcher 90 may be submitted.

Further, the motion pattern of the hand or the head of the searcher 90 can be detected by the TV camera 100 and sent to the arithmetic section 94 as input information.

In FIG. 1, the searcher 9 is mounted on a ski goggle-shaped stereoscopic device 91 worn by the searcher 90.
An example of displaying a three-dimensional image space in front of 0 is shown,
The present invention is not limited to this. For example, as shown in FIG. 20, the searcher 90 wears a pair of private eyes 114 at a position slightly away from the head 110 and projects information to both eyes using the private eyes 114. A three-dimensional image space may be projected. When the private eye 114 is used, the feeling of pressure on the head 110 can be reduced. Further, the capsule video projected in the three-dimensional video space can be set to an arbitrary shape according to the meaning of the capsule and the like.

Also, in FIG. 1 and FIG.
0 shows an example in which the three-dimensional image space is projected by mounting the stereoscopic device 91, but the searcher 90 mounts the two-dimensional visual device 116 instead of the stereoscopic device 91 to project the two-dimensional image space. May be. When the two-dimensional viewing device 116 is mounted, the virtual left hand 8L and right hand 8R and various capsule images 3, 4, 5, 6, and 7 are projected in the two-dimensional image space.

Next, another embodiment of the information retrieval apparatus according to the present invention will be described with reference to FIG. In FIG. 21, C
A capsule image (index display image) 124,
125 and 126 are displayed. These capsules 12
4, 125 and 126 represent a set of indexes for managing the attributes of the database.

Further, the touch pen 141 held by the actual right hand 140 is brought into contact with the capsule images 124, 125 and 126 displayed on the screen 121 from the screen 121, and the capsule images 124, 125 and 126 are touched. Input information for performing a predetermined operation pattern. For example, by touching the capsule image 125 with the touch pen 141 and moving the touch pen 141 to the capsule image 124 side, the capsule image 125 can be brought to the capsule image 124 side and can be superimposed on each other.

That is, the flat display screen 121 is connected to the calculation unit 130, which recognizes input information input from the touch pen 141 and displays the movement of the capsule images 124, 125, 126 on the screen 121. At the same time, the computing unit 130
Capsule image 12 based on input information input from
The set operation of the index displayed by 4,125,126 is performed. The calculation result of the calculation unit 130 is a search calculation unit 131
Is input to the search operation unit 131 and the database 132
Is input with search data.

In FIG. 21, for example, the touch pen 141
When the capsule image 125 is brought to the capsule image 124 side by using
And 125, and the operation unit 130 can perform an AND operation on the capsule images 124 and 125. As a result, information common to the respective indexes displayed by the capsule images 125 and 125 can be extracted from the database 132.

Next, still another embodiment of the information retrieval apparatus will be described with reference to FIG. In FIG. 22, the same portions as those shown in FIG. 21 are denoted by the same reference numerals, and detailed description will be omitted.

In FIG. 22, the right hand 140
The user holds the right-hand mouse 144 having the mouse 4a and slides on an operation plate (not shown) to perform a predetermined operation pattern on the capsule image 148. In addition, the user waits for the left hand mouse 146 having the code 146a with the left hand 142, and slides on the operation panel to perform a predetermined operation pattern on the capsule image 150. Then, for example, the two capsule images 148 and 150 are superimposed on each other.

In this case, the code 1 of the right hand mouse 144
44a and the code 146a of the left hand mouse 146 are
The computing unit 130 recognizes input information input from the right-hand mouse 144 and the left-hand mouse 146, and displays the capsule image 1 on the screen 121.
48, 150 movements are transferred. At the same time, the arithmetic unit 130 calculates the capsule images 148 and 15 based on the input information input from the right mouse 144 and the left mouse 146.
The set operation of the index indicated by 0 is performed. The calculation result of the calculation unit 130 is input to the search calculation unit 131, and the search data is input from the search calculation unit 131 to the database.

[0049]

According to the present invention as described above,
The searcher directly operates the index display body image in the stereoscopic image space or the two-dimensional space instead of the keyboard, recognizes the input information by the body movement pattern at that time,
Since the calculation unit performs a set operation of the index displayed by the predetermined index display body image and performs a collation search of the database, even a general person who is not accustomed to the operation of the conventional input device such as a keyboard has a large amount of information such as a database. This makes it possible to easily and accurately retrieve the desired search target from the list.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram virtually showing an embodiment of a three-dimensional video space projected by an information search device according to the present invention and a relation with a searcher.

FIG. 2 is a conceptual diagram showing a three-dimensional video space projected by the information search device shown in FIG.

FIG. 3 is an explanatory diagram showing an example of a hand movement pattern.

FIG. 4 is an explanatory diagram showing an example of a hand movement pattern.

FIG. 5 is an explanatory diagram showing an example of a hand movement pattern.

FIG. 6 is an explanatory diagram showing an example of a hand movement pattern.

FIG. 7 is an explanatory diagram showing an example of a hand movement pattern.

FIG. 8 is an explanatory diagram showing a concept of a time coordinate axis T in a three-dimensional video space.

FIG. 9 is an explanatory diagram showing a concept of a time coordinate axis T in a three-dimensional video space.

FIG. 10 is an explanatory diagram showing a concept of a time coordinate axis T in a three-dimensional video space.

FIG. 11 is a hierarchical diagram showing an example of the concept of an encyclopedia search system according to an embodiment of the present invention.

FIG. 12 is an explanatory diagram showing a search procedure of the search system shown in FIG. 11;

FIG. 13 is an explanatory diagram showing a search procedure of the search system shown in FIG. 11;

FIG. 14 is an explanatory diagram showing a procedure for further searching the content of the search result shown in FIG.

FIG. 15 is an explanatory diagram showing an example of an operation of performing a set logical operation.

FIG. 16 is an explanatory diagram showing an example of an operation of performing a set logical operation.

FIG. 17 is an explanatory diagram showing an example of a phrase search as another embodiment according to the present invention.

FIG. 18 is an explanatory diagram showing an operation of writing a character in a three-dimensional video space as another embodiment according to the present invention.

FIG. 19 is a diagram showing a pressure sensing mechanism built in the data glove.

FIG. 20 is a diagram showing a private eye mounted on the head.

FIG. 21 is a view showing a flat display screen for displaying a capsule image.

FIG. 22 is a view showing another flat display screen for displaying a capsule image.

[Explanation of symbols]

 Reference Signs List 1 3D stereoscopic image space 3 to 6 Capsule image 8R Virtual arm right hand 8L Virtual arm left hand 90 Searcher 91 Stereoscopic device 92 Input device (data glove) 100 TV camera 101 Database 112 Headband 121 Flat display screen 141 Touch pen

Continuation of the front page (56) References JP-A-3-94376 (JP, A) Hiroo Iwata, Exploring Research Issues of Artificial Reality Showing Excitement, Nikkei CG, March 1, 1990, No. 42, p. . 154-164 Makoto Sato, Perception Cycle in Virtual Workspace, Technical Report of the Institute of Television Engineers of Japan, February 22, 1990, Vol. 14, No. 15, p. 31-36 S.C. S. Fisher, Virtual Interface Environment Workstation, Proceedings of the Human Factors Society 32nd Annual Meeting, 1988, Volume 1, p. 91-95 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06F 17/30 JICST file (JOIS)

Claims (9)

(57) [Claims] 1. A stereoscopic device for displaying a set of indexes for managing attributes of a database in a stereoscopic video space within a field of view of a searcher using an index display body image to visually recognize the set of indexes, and an index display body An input device for detecting a motion pattern of a searcher's body with respect to video as input information and displaying the motion pattern of the body in the stereoscopic video space; and recognizing input information based on the motion pattern of the body input through the input device. And an operation unit for selecting an index displayed together with the index display body image corresponding to the operation pattern, and performing a collation search of the database by performing a set operation of the selected index corresponding to the operation pattern, The index display body image is a 3D image space corresponding to the searcher's body movement pattern. In which movable, index index information information retrieval apparatus, characterized in that changes in accordance with the position of the index display body image of a stereoscopic image space of. 2. The information retrieval apparatus according to claim 1, wherein the input device detects a motion pattern of the hand of the searcher as input information, and displays the motion pattern of the hand in a stereoscopic video space of the stereoscopic device. apparatus. 3. The information retrieval apparatus according to claim 1, wherein the input device detects a head motion pattern of the searcher as input information, and displays the head motion pattern in a stereoscopic video space of the stereoscopic device. apparatus. 4. A stereoscopic apparatus displays an index display image on a time coordinate axis set in a stereoscopic image space, and a calculation unit performs a set operation of an index displayed by a predetermined index display image in a time series. The information retrieval device according to claim 1, wherein: 5. The information retrieval device according to claim 1, wherein the input device is a mounting device directly mounted on the body of the searcher, and the input device detects a movement pattern of the body as input information. apparatus. 6. The information retrieval device according to claim 5, wherein the attachment device is a pair of data gloves attached to a searcher's hand. 7. The information retrieval apparatus according to claim 6, wherein the data glove has a pressure sensing mechanism for detecting pressure on a searcher's hand corresponding to a predetermined operation pattern for the index display image. 8. The information retrieval apparatus according to claim 1, wherein the input device is a TV camera for detecting a motion pattern of a searcher. 9. A set of indices for managing the attributes of a database is displayed by an index capsule image in a stereoscopic image space within the field of view of a searcher, and a set of logical operation means is displayed by an operator capsule image to visually represent the set. A stereoscopic device for recognizing a set of indices and a set of logical operation means, a motion pattern of a searcher's body for the index capsule image and the operator capsule image is detected as input information, and the motion pattern of the body is detected in the stereoscopic image space. An input device to be displayed, and logical operation means for recognizing input information based on a body motion pattern input through the input device and displaying an index and an operator capsule displayed together with an index capsule image corresponding to the motion pattern Is selected and selected using the selected logical operation means. An operation unit that performs a collation search of the database by performing a set operation of the index, wherein the index capsule image is movable in the stereoscopic image space in accordance with the motion pattern of the searcher's body, and the index of the index An information retrieval apparatus characterized in that information changes according to the position of an index capsule image in a stereoscopic image space.