JP2007206618A - Information presentation apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To present relevant information so that many users can check the information simultaneously and favorably. <P>SOLUTION: The information presentation apparatus comprises: an image input part 10 including a camera which has a predetermined photographing range and photographs a marker 26 which is fixed at an object within the photographing range and one of optically characterized features; a projection part 24 including a projector which is located in a predetermined position/attitude to the above camera, and projects the information relevant to the above marker 26; a marker detection part 12 and a position/attitude detection part 14 for recognizing the position/attitude of the above marker and the above camera; an interference prevention control part 22 for preventing the interferences between the marker 26 in an image photographed by the above camera and the information projected by the above projector. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information presentation device that presents information.

  Conventionally, various information presentation apparatuses that recognize a photographed target image (marker image) and thereby present information related to the target image have been proposed.

  For example, Patent Document 1 discloses an optical information unit that obtains an angle of view from a focal length and an image cutout size at the time of shooting from information of an image input unit, a calculation unit that calculates a photographer position and a shooting direction based on the angle of view, An image information adding device comprising: a sign means having a pair of symbols; and an information storage means for storing a background image in the same direction as the direction in which the sign means is imaged and accompanying information associated therewith. An image information addition system that calculates a photographer position and a shooting direction from a pair of captured images of a pattern and processes background image information obtained from an information storage unit based on the calculated position to obtain background image information of the same field of view. There is disclosed an image information addition system that acquires accompanying information as well as a background image of the same field of view and accompanying information.

Patent Document 2 discloses a system that projects a marker using image projection means such as a projector.
JP 2003-244488 A JP 2002-324239 A

  In a conventional information presentation device, support information necessary for maintenance / inspection or the like is presented to an operator in a superimposed manner on an image of an object actually photographed using a notebook PC, PDA, or the like. However, for use at an actual work site, a PDA or the like has a small screen, which is inconvenient for an operator to check support information.

  In addition, when there are several workers, there is no problem, but it has been difficult for many workers to work simultaneously using the support information presented on the screen of a notebook PC or the like.

  Thus, as a means for directly presenting information to an object, a method of projecting information by an image projection means such as a projector can be considered. However, when information related to the marker portion is projected by the projector, if the information is projected onto the marker portion, there is a problem that the marker cannot be recognized due to interference.

  The present invention has been made in view of the above points, and an object thereof is to provide an information presentation apparatus capable of presenting related information so that a large number of users such as a large number of workers can confirm simultaneously and satisfactorily. .

One aspect of the information presentation device of the present invention is:
An imaging means for imaging a feature point having a predetermined imaging range and having an optical feature fixed to an object within the imaging range;
An information projection unit that is arranged in a predetermined position and orientation with respect to the photographing unit and projects information related to the feature point;
Recognizing means for recognizing the feature point and the position and orientation of the photographing means;
An interference prevention control means for preventing interference between feature points in the image photographed by the photographing means and information projected by the information projecting means;
It is characterized by comprising.

  According to the present invention, it is possible to prevent the interference between the feature point in the photographed image and the projected information, so that the feature point and the position and orientation of the photographing means can be surely recognized, and the relationship based thereon It is possible to provide an information presentation device capable of presenting information so that a large number of users such as a large number of workers can confirm simultaneously and satisfactorily.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration of an information presentation apparatus according to the first embodiment of the present invention. The information presentation apparatus includes an image input unit 10, a marker detection unit 12, a position and orientation detection unit 14, and a marker information storage unit 16. , A related information generation unit 18, a related information storage unit 20, an interference prevention control unit 22, and a projection unit 24.

  Here, the image input unit 10 includes a camera having a predetermined photographing range, and photographs an object and a feature point having an optical feature fixed to the object by the camera. In the present embodiment, the feature point is the marker 26 attached to the object. The marker 26 includes a frame portion having a predetermined shape, and a symbol or design including characters written therein. Of course, the present invention is not limited to such a marker 26. For example, at least three vertices of a symbol obtained by encoding information such as a barcode, a straight line constituting an object, an intersection of line segments, and a square frame Any feature points may be used as long as they have optically distinguishable features such as.

  An image acquired by the camera of the image input unit 10 is input to the marker detection unit 12. The marker detection unit 12 detects the image of the marker 26 from the input image based on the information stored in the marker information storage unit 16 and identifies the corresponding marker 26. Then, the identification result is supplied to the position / orientation detection unit 14.

  The marker information storage unit 16 stores information related to the marker 26 such as a template image of the marker 26, area information to which the marker 26 belongs, and position and orientation information where the marker 26 is arranged.

  Therefore, the marker detection unit 12 detects a predetermined shape of the marker 26 from within the image, that is, a frame portion that is a square frame in the present embodiment. Is omitted). If the feature points can be detected, then the coordinates in the four corner images of the detected square frame are detected, the inside of the square frame is extracted and affine transformation is performed, and the image and marker after the affine transformation are performed. By performing pattern matching with the template image of the marker 26 registered in the information storage unit 16 in advance, the marker 26 is identified (in this case, the symbol or design written inside the frame portion of the marker 26 is “50”). Identification).

  The position / orientation detection unit 14 uses the identification result from the marker detection unit 12 and based on the information stored in the marker information storage unit 16, the position / orientation of the camera of the image input unit 10 with respect to the identified marker 26. Is detected and supplied to the related information generation unit 18.

  That is, when the marker 26 is identified by the marker detection unit 12, the spatial localization information of the corresponding marker (area information to which the marker 26 belongs, position and orientation information at which the marker 26 is arranged) is obtained from the marker information storage unit 16. Therefore, the position and orientation detection unit 14 detects the position and orientation of the camera of the image input unit 10 from the positions of the four corners of the marker 26 in the image.

  The related information generation unit 18 extracts information directly or indirectly associated with each marker 26 from the related information storage unit 20 according to the position and orientation of the camera of the image input unit 10 detected by the position and orientation detection unit 14. Related information is generated and supplied to the interference prevention control unit 22. Note that the related information storage unit 20 stores related information such as position and orientation information, shape information, attribute information, and color information of models arranged in the model space.

  For example, in the related information storage unit 20, for a marker with a symbol or design “50” written inside the frame of the marker 26, the corresponding three-dimensional model of the target object or the model space of the three-dimensional model is used. Annotation information (“area 1”, “marker 50”, etc.) arranged under the marker 26 is stored as related information. The related information generation unit 18 reads out the related information and transforms it according to the position and orientation of the camera of the image input unit 10 (changes the tilt or changes the size of the information), This is supplied to the interference prevention control unit 22.

  The interference prevention control unit 22 calculates the position of the marker 26 based on the position and orientation obtained from the position and orientation detection unit 14, and the marker portion in the image projected on the model space obtained from the related information storage unit 20. The range corresponding to is cut out.

  The projection unit 24 includes a projector arranged at a predetermined position and orientation with respect to the camera of the image input unit 10, and projects information obtained from the interference prevention control unit 22. Thereby, the image 28 of the related information is projected and presented on the surface of the object. The “projector disposed at a predetermined position and orientation with respect to the camera” may be an integrated configuration in which the positional relationship between the camera and the projector is fixed, It may be installed in a position and orientation. If the position and orientation of the projector with respect to the camera can be detected by some method even if the position and orientation relationship is not arranged at a predetermined position and orientation, the position and orientation are detected and the “position with a predetermined position and orientation” is determined. Can be considered.

  With reference to FIG. 2, the example of the information presentation in the information presentation apparatus which concerns on this embodiment is demonstrated.

  FIG. 2A is a diagram showing the object 30 in the real space, and the marker 26 photographed by the camera of the image input unit 10 is fixed. FIG. 2B is a diagram illustrating the related information stored in the related information storage unit 20 and generated from the related information generation unit 18. This includes the three-dimensional model 28A of the object 30 and annotation information 28B (“area 1”, “marker 50”, etc.) arranged below the marker 26 in the model space of the three-dimensional model.

  When such related information is projected onto the real space, the information is projected onto the marker 26 as shown in FIG. 2C, making it difficult for the marker detection unit 12 to detect the marker 26.

  Therefore, in the present embodiment, the interference prevention control unit 22 applies the marker 26 to the related information obtained from the related information generation unit 18 as shown in FIG. 2B as shown in FIG. Data obtained by cutting out the information data projected onto the part is generated, and is output to the projection unit 24 as related information. Therefore, as shown in FIG. 2E, the projection unit 24 projects the related information from which the marker 26 is cut out to the real space, so that only the marker 26 is related information such as text data and color information. Will not be projected. Thereby, even during the projection period of the related information, the marker 26 can be imaged by the camera of the image input unit 10 and the marker detection unit 12 can detect the marker.

  As described above, in the information presentation apparatus according to the first embodiment, the projector 28 of the projection unit 24 is used to directly project and present the related information image 28 on the object 30 having the marker 26 that is a feature point. At that time, by not projecting onto the portion of the marker 26, marker detection and position / orientation detection are possible without causing interference even when related information is projected. Therefore, related information can be presented so that a large number of users can confirm simultaneously and satisfactorily.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.

  FIG. 3 is a diagram illustrating a configuration of the information presentation apparatus according to the second embodiment. As in the first embodiment, the information presentation device includes an image input unit 10, a marker detection unit 12, a position and orientation detection unit 14, a marker information storage unit 16, a related information generation unit 18, a related information storage unit 20, and interference prevention. It comprises a control unit 22 and a projection unit 24.

  However, in the present embodiment, the related information generated by the related information generation unit 18 is directly input to the projection unit 24. Accordingly, the related information as shown in FIG. 2B is projected, and the related information is also projected onto the portion of the marker 26 in the real space as shown in FIG. 2C.

  Further, the interference prevention control unit 22 in the present embodiment generates a projection operation signal and a marker recognition operation signal as shown in FIG. 4, and the projection operation signal is input to the projection unit 24 and the marker recognition operation signal is input to the image input unit 10. , Supply each. These projection operation signal and marker recognition operation signal operate the projection unit 24 and the image input unit 10 in a time division manner regardless of the captured image and the related information to be projected. That is, each signal is permitted to operate with a negative logic, and is output complementarily so as to permit the operation of the projection unit 24 and the image input unit 10 in a time division manner.

  It should be noted that the recognition operation period of the marker 26 is controlled to be shorter than the projection period of the related information so that a person can visually recognize the information without any problem.

  Such control can be realized with a configuration with a small circuit scale, software amount, and the like.

  As described above, in the information presentation apparatus according to the second embodiment, the projection of the related information by the projector of the projection unit 24 is stopped when the marker detection unit 12 captures the camera of the image input unit 10 for detecting the marker 26. As a result, the shooting of the marker 26 and the projection of the related information are performed in a complementary manner. Therefore, even if the related information is projected on the marker 26, the marker detection period is the projection period of the related information. Marker detection is possible without interference.

  Further, by shortening the period (recognition operation period) during which the related information projection is stopped, the information can be projected without causing any visual trouble.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.

  Since the configuration of the information presentation apparatus according to the third embodiment is the same as the configuration of the information presentation apparatus according to the first embodiment, description thereof is omitted.

  FIG. 5 is a diagram illustrating an operation flowchart of the information presentation apparatus according to the third embodiment.

  That is, in the present embodiment, first, the interference prevention control unit 22 is set to an operation stop state (step S10).

  Thereafter, an image obtained by photographing the marker 26 with the camera of the image input unit 10 is input to the marker detection unit 12 (step S12). The marker detection unit 12 detects a predetermined shape of the marker 26 from the image, that is, a frame portion that is a square frame in this embodiment (the detection of the square frame is a known image processing technique, and thus the description thereof is omitted). To do). Here, if such a frame portion cannot be detected, that is, if the marker 26 as a feature point cannot be detected (step S14), the process returns to step S12.

  On the other hand, if the feature point can be detected, then the coordinates in the four corner images of the detected square frame are detected, the inside of the square frame is extracted, affine transformation is performed, and the affine transformation is performed. The marker 26 is identified by performing pattern matching between the image and the template image of the marker 26 registered in advance in the marker information storage unit 16 (step S16). If the image does not match the template image of the marker 26, that is, if it is not a feature point of the object 30 (step S18), the process returns to step S12 without detecting the marker 26.

  When the marker 26 is identified in this way, spatial localization information of the corresponding marker (area information to which the marker 26 belongs, position / orientation information where the marker 26 is arranged) can be obtained from the marker information storage unit 16. 14, the position and orientation of the camera of the image input unit 10 can be detected from the positions of the four corners of the marker 26 in the image (step S20).

  The related information generation unit 18 extracts information set in advance from the related information storage unit 20 according to the position and orientation of the camera of the image input unit 10 detected by the position and orientation detection unit 14, and generates related information. Supply to the projection unit 24 (step S22). Thereby, projection from the projector of the projection unit 24 is started.

  Thereafter, the operation of the interference prevention control unit 22 is started for the first time (step S24), and the interference prevention control unit 22 is operating while the marker 26 is being recognized.

  As described above, in the information presentation apparatus according to the third embodiment, the interference prevention control unit 22 is operated after the marker detection so that the detection of the marker 26 is not hindered. Therefore, it is possible to reduce current consumption without performing unnecessary control.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

(Appendix)
The invention having the following configuration can be extracted from the specific embodiment.

(1) Imaging means for imaging a feature point having a predetermined imaging range and having an optical feature fixed to an object in the imaging range;
An information projection unit that is arranged in a predetermined position and orientation with respect to the photographing unit and projects information related to the feature point;
Recognizing means for recognizing the feature point and the position and orientation of the photographing means;
An interference prevention control means for preventing interference between feature points in the image photographed by the photographing means and information projected by the information projecting means;
An information presentation apparatus comprising:

(Corresponding embodiment)
The first to third embodiments correspond to the embodiment related to the information presentation apparatus described in (1).

(Function and effect)
In the information presenting apparatus described in (1), the interference prevention control means prevents interference between the feature points in the image photographed by the photographing means and the information projected by the information projecting means.

  Therefore, according to the information presenting apparatus described in (1), it is possible to prevent the feature point in the photographed image from being interfered with the projected information. And related information based thereon can be presented so that many users such as many workers can confirm simultaneously and satisfactorily.

  (2) The information presentation apparatus according to (1), wherein the interference prevention control unit controls the information to be projected so that the information projection unit projects the information while avoiding the feature point. .

(Corresponding embodiment)
The embodiment relating to the information presentation apparatus described in (2) corresponds to the first embodiment.

(Function and effect)
According to the information presenting device described in (2), information is projected while avoiding the feature points, so that the projected information does not interfere with the feature points.

  (3) The interference prevention control unit stops the information projection by the information projection unit when capturing the feature point for the imaging unit to recognize the feature point, thereby capturing the feature point for recognizing the feature point. The information presenting apparatus according to (1), wherein the photographing unit and the information projecting unit are controlled such that the information projection and the information projection are performed in a complementary manner.

(Corresponding embodiment)
The embodiment relating to the information presentation device described in (3) corresponds to the second embodiment.

(Function and effect)
According to the information presenting apparatus described in (3), even if related information is projected on the feature point, the projection of the related information is stopped during the period for recognizing the feature point. The feature points can be recognized without interfering with each other.

FIG. 1 is a diagram showing a configuration of an information presentation apparatus according to the first embodiment of the present invention. 2A is a diagram showing an object in real space, FIG. 2B is a diagram showing related information generated from a related information generation unit, and FIG. 2C is a diagram showing related information in real space. FIG. 2D is a diagram showing the projected state, FIG. 2D is a diagram showing related information obtained by cutting information data projected on the marker portion generated by the interference prevention control unit, and FIG. It is a figure which shows the state which projected the relevant information of 2 (D) on real space. FIG. 3 is a diagram showing a configuration of an information presentation apparatus according to the second embodiment of the present invention. FIG. 4 is a diagram illustrating a timing chart of the projection operation signal and the marker recognition operation signal generated by the interference prevention control unit. FIG. 5 is a diagram showing an operation flowchart of the information presentation apparatus according to the third embodiment of the present invention.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 10 ... Image input part, 12 ... Marker detection part, 14 ... Position and orientation detection part, 16 ... Marker information storage part, 18 ... Related information generation part, 20 ... Related information storage part, 22 ... Interference prevention control part, 24 ... Projection 26: Marker, 28: Image of related information, 28A: Three-dimensional model, 28B: Annotation information, 30: Object.

Claims (3)

An imaging means for imaging a feature point having a predetermined imaging range and having an optical feature fixed to an object within the imaging range;
An information projection unit that is arranged in a predetermined position and orientation with respect to the photographing unit and projects information related to the feature point;
Recognizing means for recognizing the feature point and the position and orientation of the photographing means;
An interference prevention control means for preventing interference between feature points in the image photographed by the photographing means and information projected by the information projecting means;
An information presentation apparatus comprising:   2. The information presenting apparatus according to claim 1, wherein the interference prevention control unit controls information to be projected so that the information projection unit projects information while avoiding the feature points.   The interference prevention control means stops the information projection by the information projection means at the time of feature point photographing for the photographing means to recognize the feature points, and thereby performs feature point photographing and information projection for recognizing the feature points. The information presenting apparatus according to claim 1, wherein the photographing unit and the information projecting unit are controlled so as to be executed in a complementary manner.

Images