CN1823256A - Information presentation apparatus and information presentation system using the same - Google Patents
Information presentation apparatus and information presentation system using the same Download PDFInfo
- Publication number
- CN1823256A CN1823256A CN200480019973.4A CN200480019973A CN1823256A CN 1823256 A CN1823256 A CN 1823256A CN 200480019973 A CN200480019973 A CN 200480019973A CN 1823256 A CN1823256 A CN 1823256A
- Authority
- CN
- China
- Prior art keywords
- information
- destination object
- image
- unit
- constituted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
An information presentation apparatus comprises an image input unit configured to input an image of a target object in a real space, a three-dimensional position posture relation detection unit configured to detect a relation of relative three-dimensional position and posture between the image input unit and the target object using the image of the target object input from the image input unit, a figuration information storage unit configured to store figuration information of the target object, a difference comparison detection unit configured to compare the position and posture of the target object detected by the three-dimensional position posture relation detection unit and a value stored in the figuration information storage unit corresponding to a part of the target object input from the image input unit so as to detect a difference between an actual position and posture of the input target object and stored figuration information of the target object, a figuration information management unit configured to reflect a value of the difference from the difference comparison detection unit on figuration information stored in the figuration information storage unit, a superposed image generation unit configured to generate an image which is obtained by superposing the image of the target object input from the image input unit and figuration information of the target object stored in the figuration information storage unit, and a display unit configured to display the image generated by the superposed image generation unit.
Description
Technical field
The present invention relates to the information presentation system of a kind of information presentation device and this information presentation device of use, it is by showing the information overlap relevant with the image of catching simultaneously this image and this information on the image of being caught, thereby makes and can present highdensity information to the image viewing person.
Background technology
There is well-known barcode reader in information presentation device as being used to present to the predetermined related information of the destination object of real world and/or regulation mark.
Wherein, at USP 6,389,182 disclose a kind of space assigned address information of destination object and/or regulation mark of utilizing comes the device of presentation information.This device uses video camera to catch the two-dimension code that serves as a mark that is printed on as on the business card of destination object, and uses the program that is stored in the computing machine that two-dimensional encoded thus ID is analyzed.Then, this device is read the personnel's corresponding with ID through analyzing thus facial portrait image data from image data base, and the facial portrait images that will read thus is presented at the two-dimension code next door on the image that is shown in the business card of being caught by video camera on the graphoscope.As a result, this face portrait images is shown as the next door that is positioned at the two-dimension code of business card as this image.
On the other hand, disclose following system in TOHKEMY 2002-92044 communique: it will be compared by the design information that destination object in the real space of catching such as the image-input device of CCD acquisition equipment (for example image of actual buildings) and three dimensional CAD system have, and carry out checking mutually, thereby make design information consistent with the destination object in the real space with detection difference (deviation) therebetween.
The design information of the destination object in the real space (for example buildings) means generally the bulk information item of the detail shape, size, material, component names of this buildings that is managed by three-dimensional CAD etc. etc. usually, and this is record in TOHKEMY 2002-92044 communique.Yet this three-dimensional CAD etc. must be equipped with the mass storage device that is used to store the bulk information item, and the high-performance display device that is used for detailed display shape and size.Therefore, the mancarried device that in fact is difficult to the working-yard by can taking construction work or plant engineering out of is realized this system.
USP 6,389, disclosed device identification two-dimension code in 182, and calculate the position that shows facial portrait images according to the two-dimension code coordinate data, and the position of the facial portrait images of described demonstration is always fixing with respect to two-dimension code.Under situation such as the card of business card, two-dimensional encodedly be positioned at identical space with display position, this can not bring problem.On the other hand, under presenting, must mark be set exactly according to design information about situation such as the relevant information of the extensive structure of workshop or shop equipment.Yet, in fact, be difficult to accurately be provided with mark so far forth.Under the situation different with design information as the mark of benchmark itself, unless to as destination object intrinsic coordinate system real space with as the design information of destination object carry out related (registration) between the model space of intrinsic coordinate system, otherwise can not be with the image and the overlapping demonstration of destination object model of using design information to form of destination object.
Summary of the invention
Therefore, the information presentation system that an object of the present invention is to provide a kind of information presentation device and use this information presentation device, thus it can accurately be calculated destination object and difference between the design information in the real space and makes design information and realistic objective object consistent each other under it is configured to the situation of pocket.
In addition, the information presentation system that another object of the present invention provides a kind of information presentation device and uses this information presentation device, it can easily carry out registration process.
According to an aspect of the present invention, provide a kind of information presentation device, it comprises: the image input block is constituted as the image of importing the destination object in the real space; The three-dimensional position posture concerns detecting unit, is constituted as use and comes the relative three-dimensional position between detected image input block and the destination object and the relation of posture from the image of the destination object of image input block input; The shape information storage unit is constituted as the shape information of storing destination object; Diversity ratio is than detecting unit, be constituted as the value that the three-dimensional position posture is concerned the position and the posture of the detected destination object of detecting unit and be stored in the shape information storage unit and compare, with the difference between the shape information of this destination object of the physical location that detects the input destination object and posture and storage corresponding to from the part of the destination object of image input block input; The shape information administrative unit is constituted as the difference value from difference comparison detecting unit is reflected on the shape information that is stored in the shape information storage unit; The superimposed images generation unit is constituted as generation by will be from the image and the overlapping image that obtains of shape information that is stored in the destination object the shape information storage unit of the destination object of image input block input; And display unit, be constituted as the image that shows that the superimposed images generation unit generates.
Preferably, the three-dimensional position posture concerns that detecting unit comprises: the three-dimensional position posture concerns output unit, is constituted as use and from the image of the destination object of image input block input the relative three-dimensional position between image input block and the destination object and the relation of posture is detected and export; And three-dimensional position pose information administrative unit, be constituted as and the difference value from difference comparison detecting unit be reflected in relative three-dimensional position between image input block that the three-dimensional position posture concerns that output unit detects and the destination object and the pass of posture is fastened.
According to a second aspect of the invention, provide a kind of information presentation system, it comprises information presentation device and three-dimensional CAD, and described information presentation device comprises: the image input block is constituted as the image of importing the destination object in the real space; The three-dimensional position posture concerns detecting unit, is constituted as use and comes the relative three-dimensional position between detected image input block and the destination object and the relation of posture from the image of the destination object of image input block input; The shape information storage unit is constituted as the shape information of storing destination object; Diversity ratio is than detecting unit, be constituted as the value that the three-dimensional position posture is concerned the position and the posture of the detected destination object of detecting unit and be stored in the shape information storage unit and compare, with the difference between the shape information of this destination object of the physical location that detects the input destination object and posture and storage corresponding to from the part of the destination object of image input block input; The shape information administrative unit is constituted as the difference value from difference comparison detecting unit is reflected on the shape information that is stored in the shape information storage unit; The superimposed images generation unit is constituted as generation by will be from the image and the overlapping image that obtains of shape information that is stored in the destination object the shape information storage unit of the destination object of image input block input; And display unit, being constituted as the image that shows that the superimposed images generation unit generates, described three-dimensional CAD comprises: the three-dimensional design information memory cell is constituted as the three-dimensional design information of storing destination object; And shape information generation unit, be constituted as and use the three-dimensional design information that is stored in the three-dimensional design information memory cell to generate shape information, wherein, the shape information of the shape information generation unit of the shape information cell stores three-dimensional CAD of information presentation device generation.
Preferably, the three-dimensional position posture of information presentation device concerns that detecting unit comprises: the three-dimensional position posture concerns output unit, is constituted as use and from the image of the destination object of image input block input the relative three-dimensional position between image input block and the destination object and the relation of posture is detected and export; And three-dimensional position pose information administrative unit, be constituted as and the difference value from difference comparison detecting unit be reflected in relative three-dimensional position between image input block that the three-dimensional position posture concerns that output unit detects and the destination object and the pass of posture is fastened.
Description of drawings
Fig. 1 shows the block diagram according to the information presentation system of the information presentation device of first embodiment of the invention and this information presentation device of use;
Fig. 2 shows the process flow diagram of expression to the process of related (registration) processing of the real space and the model space;
Fig. 3 A shows the destination object before registration process is finished and the figure of show state thereof;
Fig. 3 B shows the destination object after registration process is finished and the figure of show state thereof;
Fig. 4 shows the process flow diagram of the consistent each other processing of destination object that expression is used for making real space and design information (shape information);
Fig. 5 A show destination object that is used for making real space and design information (shape information) consistent each other finish dealing with before destination object and the figure of the shape information of show state and this destination object;
Fig. 5 B show destination object that is used for making real space and design information (shape information) consistent each other finish dealing with after destination object and the figure of the shape information of show state and this destination object;
The registration process that Fig. 6 A shows among second embodiment is finished the destination object before and the figure of show state thereof;
Fig. 6 B shows the destination object after registration process is finished and the figure of show state thereof;
Fig. 7 shows before registration process is finished and the figure of destination object afterwards and show state thereof, and it is used to illustrate the modified example of second embodiment;
Fig. 8 A shows registration process among the 3rd embodiment destination object before finishing and the figure of the shape information of show state and this destination object thereof;
Fig. 8 B shows the figure of the shape information of destination object after registration process is finished and show state and this destination object;
Fig. 9 A shows the figure of the shape information of destination object before registration process is finished and show state and this destination object, and it is used to illustrate the modified example of the 3rd embodiment;
Fig. 9 B shows the figure of the shape information of destination object after registration process is finished and show state and this destination object, and it is used to illustrate the modified example of the 3rd embodiment;
Figure 10 shows the information presentation device of a fourth embodiment in accordance with the invention and uses the block diagram of the information presentation system of this information presentation device;
Figure 11 shows information presentation device according to a fifth embodiment of the invention and uses the block diagram of the information presentation system of this information presentation device;
Figure 12 show the 5th embodiment modified example information presentation device and use the block diagram of the information presentation system of this information presentation device;
Figure 13 A shows at according to a sixth embodiment of the invention information presentation device and uses in the information presentation system of this information presentation device figure in the shape information of destination object that is being used for making real space and design information (shape information) consistent each other finish dealing with destination object before and show state and this destination object;
Figure 13 B show destination object that is used for making real space and design information (shape information) consistent each other finish dealing with after destination object and the figure of the shape information of show state and this destination object;
Figure 14 show first embodiment modified example information presentation device and use the block diagram of the information presentation system of this information presentation device;
Figure 15 show the 4th embodiment modified example information presentation device and use the block diagram of the information presentation system of this information presentation device;
Figure 16 shows the block diagram to the information presentation system of the information presentation device of the further modified example of modified example shown in Figure 14 and this information presentation device of use; And
Figure 17 shows the block diagram to the information presentation system of the information presentation device of the further modified example of modified example shown in Figure 15 and this information presentation device of use.
Embodiment
[first embodiment]
As shown in Figure 1, information presentation system according to the first embodiment of the present invention comprises: small portable information presentation device 10, for example be furnished with the PDA of video camera or the personal computer of notebook-sized, this is the information presentation device according to the first embodiment of the present invention; And be arranged on three-dimensional CAD 12 in the office etc.
Diversity ratio possesses following function than detecting unit 20: the three-dimensional position posture is concerned the position and the posture of output unit 16 detected destination objects and the value corresponding to the part of the destination object that sends from image input block 14 that is stored in the shape information storage unit 18 compares, thereby detect the physical location of the destination object that receives and posture and the shape information of this destination object of being stored between difference.Shape information administrative unit 22 possesses the function that will be reflected in from the difference value that diversity ratio sends than detecting unit 20 on the shape information that is stored in the shape information storage unit 18.
Superimposed images generation unit 24 possesses the image and the function that be stored in the shape information overlapping image that obtain of this destination object shape information storage unit 18 of generation by the destination object that will send from image input block 14.Display unit 26 is LCD etc., is used to the image that shows that superimposed images generation unit 24 generates.
Three-dimensional position posture relation information administrative unit 28 possesses the function that fasten the pass of the relative three-dimensional position that will be reflected in from the difference value that diversity ratio sends than detecting unit 20 between image input block 14 that the three-dimensional position posture concern output unit 16 detections and the destination object and posture.The three-dimensional position posture concerns that output unit 16 and three-dimensional position posture relation information administrative unit 28 constitute the three-dimensional position posture and concern detecting unit 30.
Three-dimensional CAD 12 comprises three-dimensional design information memory cell 32 and shape information generation unit 34.Three-dimensional design information memory cell 32 is the mass storage devices such as hard disk, is used to store the three-dimensional design information of destination object.Shape information generation unit 34 possesses and uses the three-dimensional design information be stored in the three-dimensional design information memory cell 32 to generate the function of shape information.
Be stored in shape information in the shape information storage unit 18 of information presentation device and be the information that the shape information generation unit 34 by three-dimensional CAD 12 generates.Shape information generation unit 34 can send shape information to shape information storage unit 18 online by wireless or wire communication, perhaps sends shape information to shape information storage unit 18 by certain storage medium off-line ground.
" the three-dimensional design information of destination object " is to make and be provided with destination object information necessary, for example shape, color, material, component names etc.Shape information generation unit 34 only extracts the shape information in the three-dimensional design information of destination object.In addition, destination object and shape information have intrinsic coordinate system respectively, the coordinate system of destination object is called " real space ", and the coordinate system of shape information is called " model space ".
The shape information that the shape information administrative unit 22 of information presentation device 10 was constructed to and was stored in the said extracted in the shape information storage unit 18 similarly, the three-dimensional design information that will be reflected in the three-dimensional design information memory cell 32 that is stored in three-dimensional CAD 12 from the difference value that diversity ratio sends than detecting unit 20 is on the original-shape information.In this case, can be from shape information administrative unit 22 online or off-line ground information is sent to the three-dimensional design information memory cell 32 of three-dimensional CAD 12.
To the operation of this configuration be described below.
In this embodiment, at first,, the real space and the model space are carried out related (registration) as pre-service.Promptly, use destination object itself, the destination object of real space is overlapping to be presented on the display unit 26 with the destination object model on the model space of using shape information to form, and destination object model on the model space is moved so that they the two overlap each other, thereby carried out related to real space with the model space.The amount of movement of destination object model writes down to be used for related to real space and the model space during then, to association process.
After association process, owing to make the coordinate system of the real space and the model space correspond to each other, so under the situation about when overlapping and display process, there are differences, between destination object and destination object model, can't guarantee consistance (destination object and shape information are inconsistent).
Shown in the process flow diagram of Fig. 2, carry out real space is handled with related (registration) of the model space.
That is, image input block 14 is caught destination object (step S10).In registration, use known in advance its shape in real space and the consistent each other destination object of shape information in the model space.This destination object can be registered as registration with indicating (landmark).
Then, the three-dimensional position posture concerns that output unit 16 uses the image of image input blocks 14 captured object objects to come relative three-dimensional position and posture (step S12) between detected image input block 14 and the destination object.At this moment, because real space is not finished as yet with related (registration) of the model space, so there are differences between them.
Can carry out detection by adopting the disclosed method that is used for certification mark and estimated position and pose information among the USP 6,577,249, wherein destination object is considered as mark three-dimensional position and posture.
The three-dimensional position posture concerns that output unit 16 also based on the image of image input block 14 captured object objects, selects and destination object corresponding shape information (step S14) from shape information storage unit 18.In this case, as mentioned above, owing in registration, use known in advance its shape in real space and the consistent each other destination object of shape information in the model space, so selected shape information automatically.In the accompanying drawings, in order to simplify, the shape information of so selecting to be depicted as directly to send to diversity ratio than detecting unit 20.In fact, the three-dimensional position posture concerns that output unit 16 changes the shape information of selecting thus based on detected three-dimensional position and posture, and the shape information that will so change sends to diversity ratio than detecting unit 20.
Then, diversity ratio is selected and is detected (step S16) through the difference between the shape information that changes from the input picture of image input block 14 and from shape information storage unit 18 than 20 pairs of detecting units.In this case, owing to do not have correct execution related to real space and the model space, so difference appears.Promptly, when superimposed images generation unit 24 generates by will be from the input picture of image input block 14 and the overlapping image that obtains of shape information of selection and demonstration thus during the image of generation on display unit 26, as shown in Figure 3A, destination object model 40 positions of the shape information formation of the image 36 of destination object 38 and this destination object of use have on the picture 42 that is presented at display unit 26 with departing from.
Based on diversity ratio than detecting unit 20 detected differences, the information that relative three-dimensional position between 28 pairs of three-dimensional position posture relation information administrative units and image input block 14 and the destination object 38 is relevant with the relation of posture is proofreaied and correct, and makes this difference become minimum (step S18).
Then, three-dimensional position posture relation information administrative unit 28 will so correctedly concern that with three-dimensional position and posture relevant information offers the three-dimensional position posture and concerns output unit 16, finish the association process (step S20) to the coordinate system of the real space and the model space thus.Therefore, shown in Fig. 3 B, the image 36 of destination object 38 is presented on the picture 42 of display unit 26 with the consistent each other state in its position with destination object model 40.
After pre-service, accurately calculate destination object in the real space and the difference between the design information (perhaps, being shape information in the present embodiment), and carry out and be used to make design information and the consistent each other processing of realistic objective object.
That is, shown in the process flow diagram of Fig. 4, image input block 14 is caught destination object 38 (step S30).
Then, the three-dimensional position posture concerns that output unit 16 uses the image 36 of image input blocks 14 captured object objects 38, and image input block 14 is detected (step S32) with relative three-dimensional position and posture between the destination object 38.At this moment, owing to finished to related (registration) of real space, so there is not difference between them with the model space.
Next, the three-dimensional position posture concerns the retrieval data (step S34) corresponding with detected three-dimensional position and posture in the multinomial shape information of output unit 16 from be stored in shape information storage unit 18.The shape information that retrieves is thus sent to diversity ratio than detecting unit 20.
Then, the difference between the shape information that retrieves from the input picture of image input block 14 and from shape information storage unit 18 than 20 pairs of detecting units of diversity ratio detects (step S36).The difference that detects for example is the amount of movement of shape information.Described amount of movement can be parallelly to move, amplify, dwindle, rotate or their combination.
Shape information administrative unit 22 judge diversity ratio than detecting unit 20 detected differences whether in allowed band (step S38).If in allowed band, then needn't carry out processing subsequently.
On the other hand, if detected thus difference (step S38) not in allowed band, then shape information administrative unit 22 is further added detected difference to the shape information that retrieves from shape information storage unit 18 to change this shape information (step S40).That is, because the shape of the destination object 38 that formed can't change, so change shape information.
Then, shape information administrative unit 22 will be stored in through the shape information of change like this in the shape information storage unit 18, thereby reflect this change (step S42).
For example, suppose that parts 44,46 that image input block 14 catches main body 48 and invest it are as destination object 38, shown in Fig. 5 A.In this case, to retrieving with destination object 38 corresponding shape information 50, shape information 50 comprises the shape information 52 of main body 48, the shape information 54 of parts 44 and the shape information 56 of parts 46, and compares with shape information 50 by the image 36 of diversity ratio than 20 pairs of destination objects 38 of detecting unit.At this moment, superimposed images generation unit 24 generates by image 36 and the overlapping image that obtains of destination object model 40 (models 62 of the model 58 of main body 48, the model 60 of parts 44 and parts 46) that uses shape information 50 to form with destination object 38, and the image that will generate thus is presented on the display unit 26.
As shown in the drawing, under situation about there are differences between the shape information 56 of the bonding station of physical unit 46 and parts 46, the image 64 of parts 46 and model 62 positions thereof have on the picture 42 that is presented at display unit 26 with departing from.When the shape information 56 that makes parts 46 in the operation of step S40 and step S42 becomes shape information 66 shown in Fig. 5 B, eliminated this difference, so the image 64 of parts 46 and change after as one man overlapping each other being presented on the picture 42 in model 68 positions.
In addition, shown in Fig. 5 A, because between the shape information 54 of the bonding station of physical unit 44 and parts 44 and all do not have difference separately between main body 48 and its shape information 52, so the image 70 of parts 44 is presented on the picture 42 of display unit 26 with the consistent each other state in their position with image 72 and its model 58 of its model 60 and main body 48.Therefore, in this case, the not operation of execution in step S40 and step S42.
As mentioned above, the shape information after can will changing from shape information administrative unit 22 returns to the three-dimensional design information memory cell 32 of three-dimensional CAD 12.
[second embodiment]
In first embodiment, in related (registration) of real space and the model space handled, it was necessary that the destination object that known its shape in real space in advance and shape information in the model space is consistent each other is used as destination object 38.
On the other hand, may there be the situation that not necessarily can prepare this destination object 38.
Therefore, in a second embodiment, when registration, mark is set movably on destination object 38, thereby makes and to use shape and the not consistent each other destination object 38 of shape information in the model space in real space by adhesion process etc.
Promptly, under the situation that the three-dimensional position posture concerns 16 pairs of three-dimensional positions of output unit and posture detects and diversity ratio compares than the shape information 50 of the image 36 of 20 pairs of destination objects 38 of detecting unit and destination object 38, detect the difference between them.In this case, because mark is used as benchmark, so the image of mark is consistent each other with the shape information of mark.Therefore, in this case, even the model 78 of the image 74 of mark 76 and mark 76 is presented at the consistent each other state in their position on the picture 42 of display unit 26, the image 36 of destination object 38 and destination object model 40 also are shown as the position to be departed to some extent, as shown in Figure 6A.
Therefore, in the present embodiment, do not use the three-dimensional position posture relation information administrative unit of using among first embodiment 28, make destination object 38 consistent each other by the position of moving real marking 76 with respect to destination object 38, shown in Fig. 6 B with the shape information 50 of destination object 38.That is, the image viewing person observes the image 36 and the destination object model 40 thereof that are presented at the destination object of catching 38 on the display unit 26, and begins the position of movement indicias 76 before from the position 80 of moving mark, so that the two is consistent each other.
In addition, in this case, in order to make the position that is easy to movement indicia 76, except the model 78 of the image 74 of the image 36 of destination object 38 and destination object model 40 and mark 76 and mark 76, can also add scale (scale) 82 with on the picture 42 that is presented at display unit 26, as shown in Figure 7 to destination object model 40.By show rulers 82, the image viewing person can use the scale 82 on the picture 42 to confirm bias, thereby comes mark 76 on the moving target object 38 according to the bias of determining thus.
[the 3rd embodiment]
In the 3rd embodiment, with second embodiment similarly, when carrying out registration, use shape in real space and the not consistent each other destination object 38 of shape information in the model space.On the other hand, in the 3rd embodiment, mark 76 is arranged on the destination object 38 by adhere to or printing treatment etc., and can not movement indicia 76.
Promptly, under the situation that the three-dimensional position posture concerns 16 pairs of three-dimensional positions of output unit and posture detects and diversity ratio compares than the shape information 50 of the image 36 of 20 pairs of destination objects 38 of detecting unit and destination object 38, detect the difference between them.In this case, because mark 76 is used as benchmark, so the shape information of the image 74 of mark 76 and mark 76 is consistent each other.Therefore, in this case, even the image 74 of mark 76 is presented at the consistent each other state in their position on the picture 42 of display unit 26 with the model 78 of mark 76, the image 36 of destination object 38 and destination object model 40 also are shown as its position to be departed to some extent, shown in Fig. 8 A.
Therefore, three-dimensional position posture relation information administrative unit 28 must be reflected in relative three-dimensional position between image input block 14 that the three-dimensional position posture concern output unit 16 detections and the destination object 38 from the difference value that diversity ratio sends than detecting unit 20 and the pass of posture is fastened.In this case, in the present embodiment, the shape information 84 that makes mark 76 is moved with respect to the position and the posture of the shape information 50 of destination object 38, so that resultant information is shape information 86, shown in Fig. 8 B.As a result, shown in Fig. 8 B, the image 74 of mark 76 is presented at the consistent each other state in their position on the picture 42 of display unit 26 through the model 88 that moves with its mark 76, and the image 36 of destination object 38 is also consistent each other with destination object model 40.
Can adopt following configuration: image input block 14 is caught destination object 38 to generate rest image from a plurality of directions, when the shape information 84 of mark 76 in described a plurality of rest images any is moved with respect to the position of the shape information 50 of destination object 38 and posture, this is moved is reflected on the relative rest image.State before the shape information 84 that Fig. 9 A shows mark 76 is moved with respect to the shape information 50 of destination object 38, and the state of the shape information 86 that Fig. 9 B shows mark 76 after being moved with respect to the shape information 50 of destination object 38.In these figure, the part of left column is seen from a direction " a ", and the part of right row is seen from other direction " b ".
Serve as a mark and carry out under the situation of registration in the characteristic of destination object 38 is concavo-convex or pattern, can similarly realize consistent with present embodiment.
In the present embodiment, the shape information 84 of mark 76 automatically moves.On the other hand, can adopt the observer to use keyboard etc. to specify the configuration of moving direction or amount of movement.In this case, ideally, the scale described in second embodiment is affixed to destination object model 40 with on the picture 42 that is presented at display unit 26.
[the 4th embodiment]
In first embodiment, the shape information of storage destination object, and detect the destination object of actual acquisition and the difference between the described shape information to reflect this difference.
In addition, can adopt following configuration: store attribute information, and reflect the change of this attribute information such as the destination object of component names.Therefore, the information presentation system of the 4th embodiment constitutes as shown in figure 10.
Promptly, except being configured to of first embodiment, three-dimensional CAD 12 also comprises attribute information generation unit 90, and described attribute information generation unit 90 possesses and uses the three-dimensional design information be stored in the destination object in the three-dimensional design information memory cell 32 to generate the function of attribute information.
In addition, except being configured to of first embodiment, information presentation device 10 also comprises attribute information storage unit 92, keyboard 94 and attribute information management unit 96.Attribute information storage unit 92 is storeies of the attribute information that generates of the attribute information generation unit 90 of storage three-dimensional CAD 12 etc.Attribute information management unit 96 possesses the change by the attribute information imported as the keyboard 94 of input block is reflected in function on the attribute information that is stored in the attribute information storage unit 92.
The superimposed images generation unit 24 of information presentation device 10 be constituted as the destination object that will send from image input block 14 image, be stored in the shape information of the destination object the shape information storage unit 18, be stored in attribute information in the attribute information storage unit 92 in addition and carry out superimposed images overlapping and that will generate thus and be presented on the display unit 26.
The attribute information generation unit 90 of three-dimensional CAD 12 can perhaps transmit by certain storage medium off-line by wireless or wire communication with the online attribute information storage unit 92 that is sent to information presentation device 10 of attribute information.
In addition, the attribute information management unit 96 of information presentation device 10 possesses following function: be similar to change with attribute information and be reflected in situation on the attribute information that is stored in the attribute information storage unit 92, the change of attribute information that will be by keyboard 94 inputs is reflected in the three-dimensional design information (being primitive attribute information) in the three-dimensional design information memory cell 32 that is stored in three-dimensional CAD 12.In this case, can the dependency information management unit 96 online or off-line ground to the three-dimensional design information memory cell 32 transmission information of three-dimensional CAD 12.
[the 5th embodiment]
The information presentation system of the 5th embodiment comprises diversity ratio alternative shown in Figure 1 keyboard 94 than detecting unit 20, and the amount of movement of shape information is directly offered three-dimensional position posture relation information administrative unit 28 and shape information administrative unit 22 as value from keyboard 94, as shown in figure 11.
That is, in above-mentioned first to the 3rd embodiment, automatically overlapping departing from proofreaied and correct than detecting unit 20 by diversity ratio.On the other hand, in the 5th embodiment, the observer observes for example to provide indication from keyboard 94 to three-dimensional position posture relation information administrative unit 28 and shape information administrative unit 22 superimposed images, so that the image of destination object and use the shape information of this destination object and departing between the destination object model that forms becomes minimum departs from thereby proofread and correct.
Adopt this configuration, can obtain with first to the 3rd embodiment in the same effect of effect that obtained.
In addition, similarly, substitute diversity ratio shown in Figure 10 than detecting unit 20, keyboard 94 can provide information to three-dimensional position posture relation information administrative unit 28 and shape information administrative unit 22, as shown in figure 12.In this configuration, can obtain with the 4th embodiment in the same effect of effect that obtained.
[the 6th embodiment]
In the 5th embodiment, the observer observes for example to provide indication from keyboard 94 to three-dimensional position posture relation information administrative unit 28 and shape information administrative unit 22 superimposed images, so that the image of destination object and use the shape information of this destination object and departing between the destination object model that forms becomes minimum departs from thereby proofread and correct.
In the 6th embodiment, not only provide the indication of position and posture, and further provide the indication of alteration of form or parts swap to shape information management unit 22 to shape information management unit 22.
The configuration of the information presentation system of the 6th embodiment is identical with the configuration of the information presentation system of the 5th embodiment.
In this information presentation system, for example, suppose that parts 98,100 that image input block 14 catches main body 48 and invest it are as destination object 38, as shown in FIG. 13A.In this case, concern that by the three-dimensional position posture shape information 50 (comprise the shape information 52 of main body 48, the shape information 102 of parts 98, and the shape information 104 of parts 100) of 30 pairs of destination objects 38 of detecting unit retrieves.At this moment, superimposed images generation unit 24 generates by with the image 36 of destination object 38 and the destination object model 40 that uses shape information 50 to form (promptly, the model 108 of the model 58 of main body 48, the model 106 of parts 98 and parts 100) the overlapping and image that obtains, and the image that will generate thus is presented on the display unit 26.
As shown in the drawing, under situation about there are differences between the shape information 102 of the shape of physical unit 98 and parts 98, image 110 and its model 106 positions of parts 98 have on the picture 42 that is presented at display unit 26 with departing from.Similarly, under situation about there are differences between the shape information 104 of the shape of physical unit 100 and parts 100, image 112 and its model 108 positions of parts 100 have on the picture 42 that is presented at display unit 26 with departing from.
That is, in the example shown in Figure 13 A, show shape information 102 of (length) straight tube and the shape information 104 of (weak point) straight tube according to design information.On the other hand, according to the destination object in the real space, unsatisfactory ground, parts 98 are bend pipes and parts 100 are (length) straight tubes.Because this difference, in the superimposed images in the display frame 42 that is shown in display unit 26, the image 110,112 of parts and model 106,108 are shown as its position to be had and departs from.
Therefore, in this case, the observer observes superimposed images, and from the value of the alteration of form of 94 pairs of shape information administrative units of keyboard 22 input expression shape informations, perhaps represents the value of the parts swap of shape information.Therefore, the shape information 102 of the shape information of shape information administrative unit 22 usefulness bend pipes 114 exchange (length) straight tubes, and the length of the shape information 104 of change (weak point) straight tube (it is changed into shape information 116 after the change) are shown in Figure 13 B.Carried out this processing, real space is consistent each other with destination object information.That is, when the shape information 102 of parts 98 being changed into shape information 114 and the shape information 104 of parts 100 changed into shape information 116, eliminated and departed from.Therefore, the model 120 after the image 112 of the image 110 of parts 98 and model 118 after it changes and parts 100 and its change is presented on the picture 42 of display unit 26 with the consistent each other state coincidence in position.
Though certain preferred embodiment according to the present invention has been described in detail the present invention, but it should be understood by one skilled in the art that, the present invention is not limited to these embodiment, but can realize multiple modification, alternative arrangements or equivalent under situation about not departing from the scope of the present invention with spirit.
For example, as Figure 14 and shown in Figure 15, needn't carry out the reflection of the three-dimensional design information memory cell 32 of the three-dimensional CAD 12 that carries out in shape information administrative unit 22 in first embodiment and/or the 4th embodiment and/or attribute information management unit 96.
In this case, described in the 5th embodiment, substitute diversity ratio, can provide information to three-dimensional position posture relation information administrative unit 28 and shape information administrative unit 22 by keyboard 94, as Figure 16 and shown in Figure 17 than detecting unit 20.
Input block is not limited to keyboard 94, can be mouse, tracking ball, touch pad etc.
(remarks)
According to above-mentioned specific embodiment, in claim, put down in writing be configured to, can also adopt following configuration.
(1) a kind of information presentation device comprises:
The image input block is constituted as the image that input is arranged on the destination object in the real space;
The three-dimensional position posture concerns output unit, be constituted as use from the image of the destination object of image input block input detect and output image input block and destination object between relative three-dimensional position and the relation of posture;
The shape information storage unit is constituted as the shape information of storing destination object;
Input block;
Three-dimensional position pose information administrative unit is constituted as the relative three-dimensional position that will be reflected in from the value of input block input between image input block that the three-dimensional position posture concern output unit detection and the destination object and the pass of posture and fastens;
The superimposed images generation unit is constituted as generation by will be from the image and the overlapping image that obtains of shape information that is stored in the destination object the shape information storage unit of the destination object of image input block input; And
Display unit is constituted as the image that shows that the superimposed images generation unit generates.
(2), wherein, comprise the amount of movement of shape information from the value of input block input according to (1) described information presentation device.
(3) according to (1) or (2) described information presentation device, also comprise the shape information administrative unit, described shape information administrative unit is constituted as the value from the input block input is reflected on the shape information that is stored in the shape information storage unit.
(4) according to each the described information presentation device in (1) to (3), wherein, three-dimensional position pose information administrative unit is used based on by catching image that destination object obtains from first direction from first amount of movement of the shape information of input block input and based on by catch image that destination object obtains second amount of movement from the shape information of input block input from the second direction that is different from first direction.
(5) according to each the described information presentation device in (1) to (4), wherein, thus thereby display unit shows that the superimposed images generation unit is based on by catching first image that image that destination object obtains generates and superimposed images generation unit based on by catch second image that image that destination object obtains generates from the second direction that is different from first direction from first direction.
(6), wherein, comprise the value of the alteration of form of describing shape information from the value of input block input according to (1) described information presentation device.
(7), wherein, comprise the value of the parts change of describing shape information from the value of input block input according to (1) described information presentation device.
(8) basis (6) or (7) described information presentation device further comprises being constituted as the shape information administrative unit that will be reflected in from the value of input block input on the shape information that is stored in the shape information storage unit.
Claims (20)
1, a kind of information presentation device comprises:
The image input block is constituted as the image of importing the destination object in the real space;
The three-dimensional position posture concerns detecting unit, is constituted as use and comes the relative three-dimensional position between detected image input block and the destination object and the relation of posture from the image of the destination object of image input block input;
The shape information storage unit is constituted as the shape information of storing destination object;
Diversity ratio is than detecting unit, be constituted as the value that the three-dimensional position posture is concerned the position and the posture of the detected destination object of detecting unit and be stored in the shape information storage unit and compare, with the difference between the shape information of this destination object of the physical location that detects the input destination object and posture and storage corresponding to from the part of the destination object of image input block input;
The shape information administrative unit is constituted as the difference value from difference comparison detecting unit is reflected on the shape information that is stored in the shape information storage unit;
The superimposed images generation unit is constituted as generation by will be from the image and the overlapping image that obtains of shape information that is stored in the destination object the shape information storage unit of the destination object of image input block input; And
Display unit is constituted as the image that shows that the superimposed images generation unit generates.
2, information presentation device according to claim 1, wherein, the difference that diversity ratio detects than detecting unit comprises the amount of movement of shape information.
3, information presentation device according to claim 1 further comprises the attribute information storage unit that is constituted as the attribute information of storing destination object,
Wherein, the superimposed images generation unit is constituted as the attribute information that also will be stored in the attribute information storage unit and carries out overlapping.
4, information presentation device according to claim 3 further comprises:
Input block; And
The attribute information management unit, the change that is constituted as the attribute information that will import by input block is reflected on the attribute information that is stored in the attribute information storage unit.
5, information presentation device according to claim 1, wherein, the three-dimensional position posture concerns that detecting unit comprises:
The three-dimensional position posture concerns output unit, is constituted as the image of use from the destination object of image input block input, and the relative three-dimensional position between image input block and the destination object and the relation of posture are detected and export; And
Three-dimensional position pose information administrative unit is constituted as and the difference value from difference comparison detecting unit is reflected in relative three-dimensional position between image input block that the three-dimensional position posture concerns that output unit detects and the destination object and the pass of posture is fastened.
6, information presentation device according to claim 5, wherein, the difference that diversity ratio detects than detecting unit comprises the amount of movement of shape information.
7, information presentation device according to claim 5 further comprises being constituted as the difference value from difference comparison detecting unit is reflected in shape information administrative unit on the shape information that is stored in the shape information storage unit.
8, information presentation device according to claim 5, wherein, thereby thereby three-dimensional position pose information administrative unit be constituted as usage variance comparison detecting unit based on by catch from first direction detected first difference value of image that destination object obtains and diversity ratio than detecting unit based on by catch detected second difference value of image that destination object obtains from the second direction that is different from first direction.
9, information presentation device according to claim 5, wherein, thus thereby display unit is constituted as and shows that the superimposed images generation unit is based on by catching first image that image that destination object obtains generates and superimposed images generation unit based on by catch second image that image that destination object obtains generates from the second direction that is different from first direction from first direction.
10, a kind of information presentation system comprises information presentation device and three-dimensional CAD,
Described information presentation device comprises:
The image input block is constituted as the image of importing the destination object in the real space;
The three-dimensional position posture concerns detecting unit, is constituted as use and comes the relative three-dimensional position between detected image input block and the destination object and the relation of posture from the image of the destination object of image input block input;
The shape information storage unit is constituted as the shape information of storing destination object;
Diversity ratio is than detecting unit, be constituted as the value that the three-dimensional position posture is concerned the position and the posture of the detected destination object of detecting unit and be stored in the shape information storage unit and compare, with the difference between the shape information of this destination object of the physical location that detects the input destination object and posture and storage corresponding to from the part of the destination object of image input block input;
The shape information administrative unit is constituted as the difference value from difference comparison detecting unit is reflected on the shape information that is stored in the shape information storage unit;
The superimposed images generation unit is constituted as generation by will be from the image and the overlapping image that obtains of shape information that is stored in the destination object the shape information storage unit of the destination object of image input block input; With
Display unit is constituted as the image that shows that the superimposed images generation unit generates;
Described three-dimensional CAD comprises:
The three-dimensional design information memory cell is constituted as the three-dimensional design information of storing destination object; With
The shape information generation unit is constituted as and uses the three-dimensional design information that is stored in the three-dimensional design information memory cell to generate shape information,
Wherein, the shape information of the shape information generation unit of the shape information cell stores three-dimensional CAD of information presentation device generation.
11, information presentation system according to claim 10, wherein, the shape information administrative unit of information presentation device is constituted as on the three-dimensional design information that will be reflected in from the difference value that diversity ratio sends than detecting unit the three-dimensional design information memory cell that is stored in three-dimensional CAD.
12, information presentation system according to claim 10, wherein, the difference that the diversity ratio of information presentation device detects than detecting unit comprises the amount of movement of shape information.
13, information presentation system according to claim 10, wherein,
Three-dimensional CAD comprises that further the three-dimensional design information that is constituted as the use destination object generates the attribute information generation unit of attribute information,
Information presentation device further comprises the attribute information storage unit of the attribute information of the attribute information generation unit generation that is constituted as the storage three-dimensional CAD, and
The superimposed images generation unit of information presentation device is constituted as the attribute information that also will be stored in the attribute information storage unit and carries out overlapping.
14, information presentation system according to claim 13, information presentation device further comprises:
Input block; And
The attribute information management unit, the change that is constituted as the attribute information that will import by input block is reflected on the attribute information that is stored in the attribute information storage unit.
15, information presentation system according to claim 14, wherein, the change that is constituted as attribute information that will be by input block input of the attribute information management unit of information presentation device is reflected on the three-dimensional design information in the three-dimensional design information memory cell that is stored in three-dimensional CAD.
16, information presentation system according to claim 10, wherein, the three-dimensional position posture of information presentation device concerns that detecting unit comprises:
The three-dimensional position posture concerns output unit, is constituted as the image of use from the destination object of image input block input, and the relative three-dimensional position between image input block and the destination object and the relation of posture are detected and export; And
Three-dimensional position pose information administrative unit is constituted as and the difference value from difference comparison detecting unit is reflected in relative three-dimensional position between image input block that the three-dimensional position posture concerns that output unit detects and the destination object and the pass of posture is fastened.
17, information presentation system according to claim 16, wherein, the difference that the diversity ratio of information presentation device detects than detecting unit comprises the amount of movement of shape information.
18, information presentation system according to claim 16, wherein, information presentation device further comprises being constituted as the difference value from difference comparison detecting unit is reflected in shape information administrative unit on the shape information that is stored in the shape information storage unit.
19, information presentation system according to claim 16, wherein, thereby thereby the three-dimensional position pose information administrative unit of information presentation device be constituted as usage variance comparison detecting unit based on by catch from first direction detected first difference value of image that destination object obtains and diversity ratio than detecting unit based on by catch detected second difference value of image that destination object obtains from the second direction that is different from first direction.
20, information presentation system according to claim 16, wherein, thus thereby the display unit of information presentation device is constituted as and shows that the superimposed images generation unit is based on by catching first image that image that destination object obtains generates and superimposed images generation unit based on by catch second image that image that destination object obtains generates from the second direction that is different from first direction from first direction.
Applications Claiming Priority (9)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP273565/2003 | 2003-07-11 | ||
| JP2003273565A JP2005031045A (en) | 2003-07-11 | 2003-07-11 | Information presentation device and information presentation system |
| JP2003273566 | 2003-07-11 | ||
| JP273566/2003 | 2003-07-11 | ||
| JP355462/2003 | 2003-10-15 | ||
| JP2003355462 | 2003-10-15 | ||
| JP400768/2003 | 2003-11-28 | ||
| JP2003400768A JP4540329B2 (en) | 2003-07-11 | 2003-11-28 | Information presentation device |
| PCT/JP2004/009786 WO2005005924A1 (en) | 2003-07-11 | 2004-07-02 | Information presentation device and information presentation system using the same |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1823256A true CN1823256A (en) | 2006-08-23 |
| CN1823256B CN1823256B (en) | 2010-05-12 |
Family
ID=34210765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200480019973.4A Expired - Fee Related CN1823256B (en) | 2003-07-11 | 2004-07-02 | Information display apparatus |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP2005031045A (en) |
| CN (1) | CN1823256B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207819A (en) * | 2010-03-29 | 2011-10-05 | 索尼公司 | Information processor, information processing method and program |
| CN102970514A (en) * | 2011-08-30 | 2013-03-13 | 株式会社日立制作所 | Apparatus, method, and program for video surveillance system |
| CN103020952A (en) * | 2011-07-08 | 2013-04-03 | 佳能株式会社 | Information processing apparatus and information processing method |
| CN103562970A (en) * | 2011-03-31 | 2014-02-05 | 维森股份有限公司 | Automatic determination of compliance of a part with a reference drawing |
| CN104704497A (en) * | 2012-10-08 | 2015-06-10 | 赫克斯冈技术中心 | Method and system for virtual assembly of a structure |
| CN103503030B (en) * | 2012-03-23 | 2017-02-22 | 松下电器（美国）知识产权公司 | Image processing device for specifying depth of object present in real space by performing image processing, stereoscopic viewing device, and integrated circuit |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4728822B2 (en) * | 2006-02-02 | 2011-07-20 | 日立Ｇｅニュークリア・エナジー株式会社 | Image inspection method, image inspection program, and image inspection apparatus |
| JP5762913B2 (en) * | 2011-10-04 | 2015-08-12 | 株式会社東芝 | Three-dimensional data processing apparatus, method and program |
| JP2017068849A (en) * | 2015-10-01 | 2017-04-06 | 株式会社巴コーポレーション | Article assembling status information display device |
| FI129042B (en) * | 2017-12-15 | 2021-05-31 | Oy Mapvision Ltd | Machine vision system with a computer generated virtual reference object |
| JP2020101573A (en) * | 2020-04-03 | 2020-07-02 | 株式会社トプコン | Position specifying device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS6274155A (en) * | 1985-09-27 | 1987-04-04 | Olympus Optical Co Ltd | Medical image filing device |
| JP3212113B2 (en) * | 1990-11-07 | 2001-09-25 | 株式会社日立製作所 | Map information display method and device |
| US5999186A (en) * | 1997-05-23 | 1999-12-07 | 3-Design L.L.C. | Reference based parametric dimensioning method and system |
| JP2002092044A (en) * | 2000-09-19 | 2002-03-29 | Olympus Optical Co Ltd | Equipment management system and its method and recording medium with equipment management program recorded thereon |
| JP3825994B2 (en) * | 2001-02-20 | 2006-09-27 | キヤノン株式会社 | Information processing apparatus and method |
| JP2003022445A (en) * | 2001-07-09 | 2003-01-24 | Nippon Telegr & Teleph Corp <Ntt> | Device and method for information retrieval, computer program for implementing the same method, and medium where the same program is recorded |
-
2003
- 2003-07-11 JP JP2003273565A patent/JP2005031045A/en active Pending
-
2004
- 2004-07-02 CN CN200480019973.4A patent/CN1823256B/en not_active Expired - Fee Related
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207819A (en) * | 2010-03-29 | 2011-10-05 | 索尼公司 | Information processor, information processing method and program |
| CN103562970A (en) * | 2011-03-31 | 2014-02-05 | 维森股份有限公司 | Automatic determination of compliance of a part with a reference drawing |
| CN103562970B (en) * | 2011-03-31 | 2016-09-28 | 维森股份有限公司 | Automatically determine parts and meeting with reference to drawing |
| CN103020952A (en) * | 2011-07-08 | 2013-04-03 | 佳能株式会社 | Information processing apparatus and information processing method |
| CN103020952B (en) * | 2011-07-08 | 2015-11-18 | 佳能株式会社 | Messaging device and information processing method |
| US9279661B2 (en) | 2011-07-08 | 2016-03-08 | Canon Kabushiki Kaisha | Information processing apparatus and information processing method |
| CN102970514A (en) * | 2011-08-30 | 2013-03-13 | 株式会社日立制作所 | Apparatus, method, and program for video surveillance system |
| CN103503030B (en) * | 2012-03-23 | 2017-02-22 | 松下电器（美国）知识产权公司 | Image processing device for specifying depth of object present in real space by performing image processing, stereoscopic viewing device, and integrated circuit |
| CN104704497A (en) * | 2012-10-08 | 2015-06-10 | 赫克斯冈技术中心 | Method and system for virtual assembly of a structure |
| CN104704497B (en) * | 2012-10-08 | 2018-06-15 | 赫克斯冈技术中心 | For the method and system of the Virtual assemble of structure |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2005031045A (en) | 2005-02-03 |
| CN1823256B (en) | 2010-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107251101B (en) | Scene modification for augmented reality using markers with parameters | |
| US10380469B2 (en) | Method for tracking a device in a landmark-based reference system | |
| CN1252576C (en) | Pen type optical mouse device and method of controlling the same | |
| KR100953606B1 (en) | Image displaying apparatus, image displaying method, and command inputting method | |
| US9245193B2 (en) | Dynamic selection of surfaces in real world for projection of information thereon | |
| WO2018133130A1 (en) | 3d marker model construction and real-time tracking using monocular camera | |
| CN105144283B (en) | For the method and device of the manipulated image rotated based on equipment | |
| JP6372149B2 (en) | Display control apparatus, display control method, and display control program | |
| CN1823256A (en) | Information presentation apparatus and information presentation system using the same | |
| CN1834582A (en) | Image processing method, three-dimensional position measuring method and image processing apparatus | |
| CN1975652A (en) | Input device and scroll control method using the same | |
| TW201209730A (en) | Object detection metadata | |
| EP3257026B1 (en) | Generating three dimensional models | |
| JP2010267113A (en) | Component management method, device, program and recording medium | |
| JP2017146938A (en) | Book detection device, book detection method, and computer program for book detection | |
| CN106446888A (en) | Camera module multi-identifier identification method and camera module multi-identifier identification equipment | |
| CN107077739A (en) | Use the three dimensional indicia model construction and real-time tracking of monocular camera | |
| CN1922615A (en) | Composite marker and composite marker information acquisition apparatus | |
| Belo et al. | Digital assistance for quality assurance: Augmenting workspaces using deep learning for tracking near-symmetrical objects | |
| JP2011141807A (en) | Method and system for management of mill ends | |
| JP2021170272A (en) | Positioning system, positioning device, positioning method, and positioning program | |
| CN1922464A (en) | Surface metering device | |
| Metzner et al. | Automatic camera calibration and sensor registration of a multi-sensor fringe measurement system using hexapod positioning | |
| CN105809690A (en) | Data processing method, device and electronic device | |
| JP2004227230A (en) | Light indicating system, questionnaire collecting system, game system, light indicator, imaging device, projector, light indicating method, position specifying method, questionnaire collecting method, and game executing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1090414 Country of ref document: HK |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1090414 Country of ref document: HK |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100512 Termination date: 20150702 |
|
| EXPY | Termination of patent right or utility model |