JP2006252036A - Projection image creation apparatus, program, projection image creation method, and image projection system - Google Patents

Abstract

When a worker moves an object in a space on an image projection side by a projection apparatus using a general apparatus, the displayed image is adjusted according to the movement of the object.
A projection image creating apparatus according to one aspect of the present invention stores an object-related data associated with an object in a space photographed by a photographing apparatus and display position data indicating the display position. The storage means 12, the hand area detection means 17 for creating the hand area data 25 that detects the area of the hand 61 of the worker from the photographed image data 10 photographed by the photographing device 5, the hand area and the display position data of the hand area data 25. 6 when the object 2 is held by the hand 61 and the hand region of the hand region data 25 is moved. , Position management means 22a for creating post-movement display position data 7 indicating the display position of the object-related data after movement along with the movement of the hand region, and a projection image for displaying the object-related data 4 at the position of the display position data 7 after movement Comprising the image data generating unit 22b for creating data 9.
[Selection] Figure 1

Description

  The present invention relates to a projection image creation device, a program, a projection image creation method, and an image projection system for creating image data projected by a projection device.

The projector projects an image based on the image data on the display surface.
Patent Document 1 (Japanese Patent Laid-Open No. 2003-140830) wirelessly transmits a mouse pointer operation and a click operation to a projector system, recognizes the position and projection state of a pointer image projected on a screen, and follows a mouse control signal. An invention for processing is disclosed.

  Non-Patent Document 1 ("Construction of a 3D remote instruction interface using a projector" (Tojo, Hiura, Iguchi; Osaka University), Transactions of the Virtual Reality Society of Japan, Vol.7, No.2, pp.169-176 , Jun. 2002) describes a technique for constructing a three-dimensional remote instruction interface using an apparatus for measuring an object such as a laser range finder and a projector.

JP 2003-140830 A

"Construction of a 3D remote instruction interface using a projector" (Tojo, Hiura, Iguchi; Osaka University), Transactions of the Virtual Reality Society of Japan, Vol.7, No.2, pp.169-176, Jun.2002

In Patent Document 1, the projection state by the projector system is changed by the operation of the mouse pointer and the click operation.
However, for example, even if some change or event occurs on the space where the image is projected, the projected image does not change at all.

In the said nonpatent literature 1, the special apparatus which measures objects, such as a laser range finder, is required.
The present invention has been made in view of the above circumstances, and a case where an operator moves an object in a space on the projection side of an image by a projection device using a general device such as a projection device and a photographing device. It is another object of the present invention to provide a projection image creation apparatus, a program, a projection image creation method, and an image projection system that easily adjust a displayed image according to the movement of an object.

  The projection image creating apparatus according to the aspect of the invention stores object-related data related to an object arranged in a space photographed by the photographing apparatus and object storage means for storing display position data indicating a display position of the object-related data. And hand region detecting means for creating hand region data for detecting the hand region of the worker based on the photographed image data photographed by the photographing device, and the hand region indicated by the hand region data and the display indicated by the display position data An object hold detecting means for detecting whether or not the object is held by the operator's hand based on the position, and the hand area indicated by the hand area data in a state where the object is held by the operator's hand The position management means for creating post-movement display position data indicating the display position of the object-related data after movement along with the movement of the hand area, and the display position data after movement Comprising an image forming means for forming a projection image data for displaying the object-related data to the location.

As a result, when an operator moves an object in a space on the projection side of an image by the projection apparatus using a general apparatus such as a projection apparatus and an imaging apparatus, object-related data is located at a position corresponding to the movement of the object. Can be displayed quickly, efficiently and easily.
In the projection image creating apparatus, the position management means may update the display position data stored in the object storage means with the display position data after movement.

  The projection image creating apparatus further includes a center of gravity calculating means for creating center of gravity data indicating the position of the center of gravity of the hand area based on the hand area data, and the object hold detecting means is configured to detect the hand area indicated by the center of gravity data. Even if it is determined that the object is held by the operator's hand when the distance between the position of the center of gravity and the display position of the object-related data indicated by the display position data is equal to or less than a predetermined holding detection threshold Good.

  In the projection image creating apparatus, the object hold detection means has a negative value for the movement acceleration of the hand area indicated by the hand area data, and the movement speed is equal to or less than a predetermined threshold value for detecting the end of movement. In such a case, it may be determined that the state in which the object is held by the operator's hand has ended.

Further, in the projection image creation device, the position management means obtains a vector from the position of the hand area at the start of movement to the display position of the object-related data, and displays after movement based on the position and vector of the hand area after movement. Position data may be created.
In addition, said aspect is expressed as an apparatus. However, the present invention is not limited to this, and the above aspect may be expressed by a method, a program, a computer-readable storage medium, and the like, and is expressed by a system including a projection image creation device, a photographing device, and a projection device. It is good.

  According to the present invention, when an operator moves an object in a space on the projection side of an image by the projection apparatus using a general apparatus such as a projection apparatus and an imaging apparatus, an image displayed in the space is displayed on the object. It can be easily adjusted according to movement.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected about the same part in each following figure, and description is abbreviate | omitted.
(First embodiment)
FIG. 1 is a block diagram showing an example of the configuration of the image projection system according to the present embodiment. In the present embodiment, description will be made assuming that the projection apparatus is a projector and the photographing apparatus is a camera. Further, in the present embodiment, the object-related data includes, for example, data of various contents related to the object, such as the handling method of the object, the alerting about the object, the position movement direction of the object, various marks, and the attribute data of the object. Used.

  In the image projection system 1 according to the present embodiment, first, the content 4 a of the object-related data 4 is displayed in a state associated with the object 2 by the projector 3 in the real space in which the object 2 is arranged. Next, the image projection system 1 captures this real space with the camera 5 and changes / updates the display position of the object-related data 4 according to the position of the hand 61 of the operator. Then, the image projection system 1 displays the content 4a of the object related data 4 at a position associated with the moved object 2 by the projector 3 with respect to the real space.

  More specifically, the image projection system 1 according to the present embodiment associates the object-related data 4 of the object 2 in the real space with the display position data (coordinate value) 6 indicating the display position, and this display position. The object-related data 4 is presented on the real space by the projector 3 according to the data 6, the operator's hand 61 is recognized, the post-movement display position data 7 is created based on the movement amount of the hand 61, and the movement in the space By displaying the object related data 4 at the position indicated by the post display position data 7, the object related data 4 corresponding to the moving object 2 is displayed.

The image projection system 1 includes a projector 3, a camera 5, and a projection image creation device 8.
The projector 3 receives the projection image data 9 created by the projection image creation device 8 and projects an image onto the real space based on the projection image data 9.
The camera 5 shoots the real space state such as the arrangement state of the object 2 and the state of the image projected by the projector 3 in the real space, and uses the captured image data 10 indicating the real space state as the projection image creation device 8. To pass.

The projection image creation device 8 includes an image recognition unit 11, an object database 12, an object hold detection unit 13, and an image creation unit 14.
The image recognition unit 11 further includes a captured image acquisition unit 15, a captured image data storage unit 16, a hand region detection unit 17, a hand region data storage unit 18, a centroid calculation unit 19, and a centroid data storage unit 20. Among these, the hand region detection unit 17, the hand region data storage unit 18, the centroid calculation unit 19, and the centroid data storage unit 20 constitute a hand position detection unit 21.

The image creation unit 14 includes a position changing unit 22, a projection image data storage unit 23, and a projection image providing unit 24.
The captured image acquisition unit 15 acquires the captured image data 10 of the space from the camera 5 and stores the captured image data 10 in the captured image data storage unit 16.
The hand region detection unit 17 reads the photographed image data 10 from the photographed image data storage unit 16 and detects a skin color region indicating the worker's hand 61 as a hand region by image analysis on the photographed image data 10. Then, the hand region detection unit 17 creates hand region data 25 indicating the hand region detection result and stores it in the hand region data storage unit 18.

Even when the object 2 is not moved by the hand 61, it is only necessary to detect an object that contacts the object 2 when the object 2 is moved. When an operator wears gloves, this glove is detected.
The center-of-gravity calculation unit 19 reads the hand region data 25 from the hand region data storage unit 18, calculates the position (coordinate value) of the center of gravity of the hand region in the hand region data 25, and creates the center-of-gravity data 26 indicating the position of the center of gravity. To do. The centroid calculating unit 19 stores the centroid data 26 in the centroid data storage unit 20.

The object database 12 stores the object-related data 4 and display position data 6 indicating the display position (coordinate values) of the object-related data 4 in association with each other.
FIG. 2 is a diagram illustrating an example of the contents stored in the object database 12.
In FIG. 2, object-related data 4 and display position data 6 are managed in a table format for each of the plurality of objects 2.

As the contents of the object related data 4, an object ID, an object name, a mark shape to be displayed in association with the object, and attribute data (for example, arrangement time in space, cautions on the object, features, operation method, etc.) are managed. ing.
As the display position data 6, an X coordinate value and a Y coordinate value indicating a position where the object related data 4 is displayed are managed.

The object hold detection unit 13 reads the centroid data 26 from the centroid data storage unit 20 and reads the display position data 6 of the object 2 from the object database 12.
The object hold detection unit 13 makes contact between the object 2 and the operator's hand 61 based on the position of the center of gravity of the hand region indicated by the center of gravity data 26 and the display position of the object related data 4 indicated by the object related data. It is determined whether the object 2 is in a state (whether the object is held by the operator's hand), that is, whether the object 2 is moved by the operator's hand 61 and notifies the position change unit 22 of the determination result.

  For example, when the distance between the coordinate value indicated by the gravity center data 26 and the coordinate value indicated by the display position data 6 is equal to or less than a predetermined threshold value for holding detection, the object hold detection unit 13 This is detected as a state of movement by the person's hand 61. On the other hand, the object hold detection unit 13 has a negative value for the movement acceleration of the center of gravity of the hand 61 represented by the coordinate value indicated by the center of gravity data 26, and the movement speed is equal to or less than a predetermined movement end detection threshold. In this case, the end of the state in which the object 2 is moving by the operator's hand 61 is detected.

The position changing unit 22 includes a position management function 22a and an image data creation function 22b.
When the movement of the object 2 is detected by the object hold detection unit 13, the position management function 22 a of the position changing unit 22 creates post-movement display position data 7 according to the moving direction and distance of the centroid data 26. The display position data 6 in the object database is updated with the display position data 7 after movement.

  The image data creation function 22b of the position changing unit 22 displays the content of the object-related data 4 in the object database 12 at the display position indicated by the display position data 6 (the display position data 7 after movement in the case of movement). Data 9 is created and stored in the projection image data storage unit 23.

The projection image providing unit 24 reads the projection image data 9 stored in the projection image data storage unit 24 and provides it to the projector 3.
By using the image projection system 1 as described above, when the operator moves the object 2 arranged in the space with the hand 61 using the general camera 5 and the projector 3, the movement of the hand 61 is performed. Accordingly, the display position of the object-related data 4 with respect to the object 2 can also be moved.

  FIG. 3 is a conceptual diagram showing an example of the relationship between the coordinate value of the center of gravity of the hand 61 and the coordinate value of the display position of the object-related data 4. In FIG. 3, only the camera 5, projector 3, image recognition unit 11, and image creation unit 14 are shown in the image projection system 1 for the sake of simplicity, and the other parts are omitted.

The display position of the content 4a of the object-related data 4 indicated by the display position data 6 is represented by O, and the coordinate value of the display position O of the content 4a is represented by O (X0, Y0).
The center of gravity of the hand 61 is represented as H, and the coordinate value of the center of gravity H of the hand is represented as H (Xh, Yh).
The image projection system 1 obtains the distance between the display position O of the object-related data 4 and the center of gravity H of the hand 61 based on the coordinate value O (X0, Y0) and the coordinate value H (Xh, Yh).

  In the image projection system 1, when the distance between the display position O of the content 4a and the center of gravity H of the hand 61 is equal to or less than a predetermined holding detection threshold, the relative position between the position O and the center of gravity H is constant. As described above, the display position of the object-related data 4 after the movement is obtained, and the coordinate value O (X0, Y0) of the display position O is updated with the display position after the movement.

The image projection system 1 displays the display position O of the object related data 4 when the value of the movement acceleration of the center of gravity H of the hand is a negative value and the movement speed is equal to or less than a predetermined movement end detection threshold. The update of the coordinate value O (X0, Y0) of is completed.
When the distance between the display position O of the object-related data 4 and the center of gravity H of the hand 61 exceeds the holding detection threshold, the image projecting system 1 returns to the position O again when the distance falls below the holding detection threshold. The coordinate value O (X0, Y0) of the display position O of the object related data 4 is updated so that the relative position between the center of gravity H and the center of gravity H is constant.

FIG. 4 is a flowchart illustrating an example of processing of the object hold detection unit 13.
In step S <b> 1, the object hold detection unit 13 sets a holding detection threshold value Dg used for detecting the state in which the object 2 is gripped by the hand 61, and whether or not the object 2 is in the state gripped by the hand 61. In the object hold flag Fg indicating the above, False representing a state in which the object 2 is not grasped by the hand 61 is set.

In step S <b> 2, the object hold detection unit 13 sets a movement end detection threshold value Df used for detecting a state in which the hand 61 releases the object 2.
In step S <b> 3, the object hold detection unit 13 sets a captured image acquisition period Tcap.
Note that the processing order of the steps S1 to S3 can be arbitrarily changed, and may be executed in parallel.

In step S <b> 4, the object hold detection unit 13 acquires the coordinate value H (Xh, Yh) of the current center of gravity H of the hand 61 calculated by the center of gravity calculation unit 19.
In step S5, the object hold detection unit 13 acquires the coordinate value O (X0, Y0) of the display position O indicated by the display position data 6 from the object database 12.

Note that steps S4 and S5 may be executed in reverse order or in parallel.
In step S6, the object hold detection unit 13 determines whether the distance between the center of gravity H of the hand 61 and the display position O indicated by the display position data 6 is equal to or less than the hold detection threshold value Dg.

If the distance between the center of gravity H and the position O is not less than or equal to the threshold value Dg, in step S7, the object hold detection unit 13 waits until the captured image acquisition period Tcap elapses, and then the process returns to step S4.
When the distance between the center of gravity H and the position O is equal to or less than the threshold value Dg, the object hold detection unit 13 indicates that the object 2 is held by the hand 61 in the object hold flag Fg in step S8. And the object hold flag Fg is provided to the position changing unit 22.

In step S9, the object hold detection unit 13 holds the coordinate value H (Xh, Yh) of the center of gravity H of the hand 61 at the coordinate stored value Htemp (Xh, Yh).
In step S10, the object hold detection unit 13 maintains the coordinate storage value variable Htemp (Xh, Yh) until the captured image acquisition cycle Tcap has elapsed.

In step S <b> 11, the object hold detection unit 13 acquires the coordinate value H (Xh, Yh) of the current center of gravity H of the hand 61 calculated by the center of gravity calculation unit 19.
In step S12, the object hold detection unit 13 has a negative value of the moving acceleration of the center of gravity H of the hand 61, and the distance between the coordinate value H (Xh, Yh) and the coordinate stored value Htemp (Xh, Yh). Is less than or equal to the movement end detection threshold value Df.

The movement acceleration value of the center of gravity H of the hand 61 is a negative value, and the distance between the coordinate value H (Xh, Yh) of the center of gravity H of the hand 61 and the coordinate stored value Htemp (Xh, Yh) is a threshold. If the condition of value Df or less is not satisfied, the process returns to step S9.
The movement acceleration value of the center of gravity H of the hand 61 is a negative value, and the distance between the coordinate value H (Xh, Yh) of the center of gravity H of the hand 61 and the coordinate stored value Htemp (Xh, Yh) is a threshold. If the value is less than or equal to the value Df, in step S13, the object hold detection unit 13 sets False to the object hold flag Fg, and provides the object hold flag Fg to the position change unit 22.

In step S14, the object hold detection unit 13 waits until the captured image acquisition cycle Tcap elapses, and then the process returns to step S4.
In the processing of the object hold detection unit 13 shown in FIG. 4, when there are a plurality of objects 2 and the respective object related data 4 is displayed for the plurality of objects 2, the processing after step S <b> 4 is performed for each of the plurality of objects 2. Just do it. Moreover, when displaying each object related data 4 about the some object 2, you may set threshold value Dg and Df for every some object 2. FIG.

  Further, in the present embodiment, the hand 61 and the object using the distance between the coordinate value H (Xh, Yh) and the coordinate value O (X0, Y0), the threshold values Dg, Df, and the movement acceleration of the center of gravity H of the hand 61. 2 is detected (a state of being gripped). However, the shape and size (the radius when the object is a circle) of the object 2 are registered in the object database 12 together with the coordinate value O (X0, Y0) of the display position for the object 2, and the coordinate value O (X0, Y0). It is also possible to determine whether or not the region of the object 2 and the hand 61 overlap from the shape and size of the object 2 and detect the contact state between the hand 61 and the object 2.

FIG. 5 is a flowchart illustrating an example of processing of the position changing unit 22.
In step T1, the position management function 22a of the position changing unit 22 sets a position update period Tprj.
In step T2, the position changing unit 22 acquires the coordinate value O (X0, Y0) of the display position O indicated by the display position data 6 from the object database 12, and also acquires the object related data 4.

  In step T3, the image data creation function 22b of the position changing unit 22 creates projection image data 9 for displaying the object related data 4 at the display position of the coordinate value O (X0, Y0) indicated by the display position data 6, The data is stored in the projection image data storage unit 23. Thereby, an image based on the projection image data 9 is displayed by the projector 3.

In step T4, the position management function 22a of the position changing unit 22 determines whether the object hold flag Fg received from the object hold detection unit 13 is False.
If the object hold flag Fg is False, this step T4 is repeated.
When the object hold flag Fg is not False, the position management function 22a of the position changing unit 22 acquires the coordinate value H (Xh, Yh) of the current center of gravity H of the hand 61 calculated by the center of gravity calculating unit 19 in Step T5. To do.

In step T <b> 6, the position management function 22 a of the position changing unit 22 calculates a vector HO starting from the center of gravity H of the hand 61 and having the display position O indicated by the display position data 6 as the end point.
In step T7, the position management function 22a of the position changing unit 22 waits until the position update cycle Tprj elapses.

In step T8, the position management function 22a of the position changing unit 22 acquires the coordinate value H (Xh, Yh) of the center of gravity H of the hand 61 after movement calculated by the center of gravity calculating unit 19.
In step T9, the position management function 22a of the position changing unit 22 uses the coordinate value of the end point of the vector HO starting from the coordinate value H (Xh, Yh) of the center of gravity H of the hand 61 after movement as the display position data after movement. 7 is obtained as a coordinate value O (X0, Y0) of the display position O shown in FIG. That is, the coordinate management function calculates the coordinate value H (Xh, Yh) + vector HO of the center of gravity H of the hand after movement, and obtains the coordinate value O (X0, Y0) of the display position O after movement.

  In step T10, the image data creation function 22b of the position changing unit 22 moves the projected image data after displaying the object related data 4 at the display position of the coordinate value O (X0, Y0) indicated by the moved display position data 7. 9 is created and stored in the projection image data storage unit 23. Thereby, an image based on the projection image data 9 after the movement of the object 2 is displayed by the projector 3.

In step T11, the image data creation function 22b of the position change unit 22 determines whether the object hold flag Fg received from the object hold detection unit 13 is False.
If the object hold flag Fg is not False, the process returns to Step T7.
When the object hold flag Fg is False, in step T12, the position management function 22a of the position changing unit 22 uses the coordinate value O (X0, Y0) of the display position O indicated by the post-movement display position data 7 obtained in step T9. Is registered in the object database 12. Thereafter, the process returns to step T4.

An example of a usage mode of the image projection system 1 according to the present embodiment will be described.
For example, if there is equipment that can be moved and should be handled with care at a work site such as a plant, the image projection system 1 can be used to display a handling method or an operation manual of the equipment. Display contents such as handling methods and operation manuals can be moved along with the movement.

Further, the object related data 4 can be created at the support center, and the object related data 4 created at the support center can be quickly presented to the work site. As a result, cooperative work between the site and the support center can be realized.
In the present embodiment described above, the object related data 4 related to the object 2 is displayed on the object 2 arranged in the space using the general camera 5 and the projector 3 which are not special devices. Can be made.

Even when the object 2 is moved by the operator, the object related data 4 can be displayed at the position of the object 2 after the movement.
In the present embodiment, the individual object 2 itself is not recognized from the captured image data 10, but the operator's hand 61 is recognized, so that the calibration can be easily performed and the movement of various objects 2 can be supported. The recognition accuracy can be improved by specifying the recognition target as the hand 2, and the object related data 4 and the display position data 6 are registered in the object database 12 even when a new object 2 is arranged in the space. This can be easily handled.

  In this embodiment, each component may be rearranged as long as the same operation can be realized, each component may be freely combined, and each component may be freely divided. Alternatively, some components may be deleted. That is, the present embodiment is not limited to the above-described configuration as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.

  For example, in the present embodiment, the object hold detection unit 13 determines whether or not the operator's hand 61 is grasping the object 2 based on the distance between the center of gravity H of the hand 61 and the display position O indicated by the display position data 6. Deciding. However, instead of this, for example, the object hold detection unit 13 recognizes the shape of the hand 61, and the shape of the hand 61 corresponds to a predetermined object holding shape pattern or corresponds to an object release shape pattern. It may be determined whether or not the operator's hand 61 is holding the object 2 using another method.

Further, for example, the storage units 16, 18, 20, 23 and the object database 12 can be freely combined. The data stored in the storage units 16, 18, 20, 23 and the object database 12 can be freely combined.
FIG. 7 is a block diagram showing an example of a configuration for realizing the projection image creation apparatus 8 of the image projection system 1 according to the present embodiment using a computer and software.

The computer 27 includes a processor 28, a storage device 29 such as an internal memory, and an internal bus 30. The processor 28 and the storage device 29 are connected via an internal bus 30.
A program 32 stored in the storage medium 31 is read into the computer 27 and executed by the processor 28. As a result, the computer 27 realizes the function as each component included in the projection image creating apparatus 8. The storage device 29 functions as each storage unit 16, 18, 20, 23 and the object database 12.

  In the present embodiment, the program 32 is written in a storage medium 31 such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, etc., and operates as the projection image creation device 8. The present invention can be applied to the computer 27. The program 32 can also be transmitted to a computer 27 and applied to the computer 27. The computer 27 reads the program 32 and controls the operation of the program 32 to realize the function as the projection image creation device 8. The program 32 may be distributed and arranged in a plurality of computers, and the processing may be executed while cooperating with each other between the plurality of computers.

(Second Embodiment)
In the present embodiment, a modification of the first embodiment will be described. In the example of the first embodiment, the display position data 6 in the object database 12 is updated after the moving state of the object 1 is completed.
However, the update timing of the object database 12 is not limited to this. For example, every time the post-movement display position data 7 is created, the display position data 6 of the object database 12 is updated by the post-movement display position data 7. Also good.

FIG. 7 is a flowchart showing an example of processing of the image projection system 1 according to the present embodiment.
In step U <b> 1, the captured image acquisition unit 15 acquires captured image data 10 captured by the camera 5.
In step U2, the hand region detection unit 17 detects the region of the hand 61 in the captured image data 10 by a method such as skin color detection.

In step U3, the center-of-gravity calculation unit 19 calculates the coordinate value H (Xh, Yh) of the center of gravity H of the region of the operator's hand 61.
In step U4, the object hold detector 13 determines the coordinate value O (X0, Y0) of the display position O indicated by the display position data 6 in the object database 12 and the coordinate value H (Xh, Yh) of the center of gravity H of the hand region. Is less than or equal to the threshold value for holding detection. If the distance is not equal to or smaller than the holding detection threshold, the process proceeds to step U9.

When the distance is equal to or smaller than the threshold value for holding detection, in step U5, the position management function 22a of the position changing unit 22 displays the post-movement display position according to the change in the coordinate value H (Xh, Yh) of the center of gravity H of the hand area. Data 7 is created.
In step U6, the object hold detection unit 13 determines whether the value of the movement acceleration of the center of gravity H of the hand region is a negative value and the movement speed is equal to or less than the movement end detection threshold value.

  When the value of the movement acceleration of the center of gravity H of the hand region is a negative value and the condition that the movement speed is equal to or less than the movement end detection threshold value is not satisfied, in step U7, the position management function 22a of the position changing unit 22 The display position data 6 in the object database 12 is updated with the post-movement display position data 7, and the process thereafter proceeds to step U9.

If the value of the movement acceleration of the center of gravity H of the hand region is a negative value and the movement speed is less than or equal to the movement end detection threshold, the position management function 22a of the position changing unit 22 displays the display position data 6 in step U8. Is finished, and then the process proceeds to step U9.
In step U <b> 9, the screen data creation function 22 b of the position changing unit 22 creates the projection image data 9 for displaying the object related data 4 in the object database 12 at the display position indicated by the display position data 6.

In step U10, the projection image providing unit 24 provides the projection image data 9 to the projector 3 and displays it on the space.
In step U11, the image projection system 1 repeats the processing from step U1 onward until the operation ends.
In the present embodiment, the content stored in the object database 12 can be updated even while the object 2 is moving, and the object related data 4 can be displayed for the moving object 2.

  The present invention is effective in the field of presenting information using a projector.

1 is a block diagram showing an example of the configuration of an image projection system according to a first embodiment of the present invention. The figure which shows the example of the memory content of an object database. The conceptual diagram which shows the example of the relationship between the coordinate value of the gravity center of a hand, and the coordinate value of the display position of object related data. The flowchart which shows the example of a process of the object hold | maintenance detection part which concerns on the embodiment. The flowchart which shows the example of a process of the position change part which concerns on the same embodiment. The block diagram which shows the example of a structure for implement | achieving the projection image production apparatus of the image projection system which concerns on the embodiment using a computer and software. The flowchart which shows the example of a process of the image projection system which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image projection system, 2 ... Object, 3 ... Projector, 4 ... Object related data, 4a ... Contents, 5 ... Camera, 6 ... Display position data, 61 ... Hand of operator, 7 ... Display position data after movement, 8 DESCRIPTION OF SYMBOLS ... Projection image production apparatus, 9 ... Projection image data, 10 ... Shooting image data, 11 ... Image recognition part, 12 ... Object database, 13 ... Object hold detection part, 14 ... Image creation part, 15 ... Shooting image acquisition part, 16 ... Photographic image data storage unit, 17... Hand region detection unit, 18... Hand region data storage unit, 19... Centroid calculation unit, 20 ... centroid data storage unit, 21 ... hand position detection unit, 22. Position management function, 22b ... image data creation function, 23 ... projection image data storage unit 23 ..., 24 ... projection image providing unit, 25 ... hand area data, 26 ... centroid data, 27 ... computer, 28 ... processor, 9 ... storage device, 30 ... internal bus 31 ... storage medium, 32 ... Program

Claims (8)

Object storage means for storing object-related data related to an object arranged in a space imaged by the imaging device and display position data indicating a display position of the object-related data;
Hand region detection means for creating hand region data that detects the region of the hand of the operator based on the captured image data captured by the imaging device;
Object hold detection means for detecting whether or not the object is held by the operator's hand based on the hand area indicated by the hand area data and the display position indicated by the display position data;
When the object is gripped by the operator's hand and the hand area indicated by the hand area data is moved, the display position of the object-related data after the movement is indicated along with the movement of the hand area. Position management means for creating display position data after movement;
An image data creating means for creating projection image data for displaying the object related data at a position indicated by the display position data after movement. The projection image creation apparatus according to claim 1,
The position management unit updates the display position data stored in the object storage unit with the post-movement display position data. In the projection image creation device according to claim 1 or 2,
Based on the hand area data, further comprising a center of gravity calculation means for creating center of gravity data indicating the position of the center of gravity of the hand area,
When the distance between the position of the center of gravity of the hand region indicated by the center of gravity data and the display position of the object-related data indicated by the display position data is equal to or less than a predetermined threshold for holding detection In addition, it is determined that the object is held by the hand of the operator. In the projection image creation device according to any one of claims 1 to 3,
The object hold detection means is configured to detect the operator when the movement acceleration of the hand area indicated by the hand area data is a negative value and the movement speed is equal to or lower than a predetermined movement end detection threshold. It is determined that the state in which the object is gripped by the hand is finished. In the projection image creation device according to any one of claims 1 to 4,
The position management means obtains a vector from the position of the hand area at the start of movement to the display position of the object-related data, and displays the post-movement display position data based on the position of the hand area after movement and the vector. Projection image creation device characterized by creating Computer
Object storage means for storing object-related data related to an object arranged in a space imaged by the imaging device and display position data indicating a display position of the object-related data;
Hand area detection means for creating hand area data for detecting the area of the hand of the operator based on the photographed image data photographed by the photographing device, and storing the hand area data in the hand area data storage means;
Based on the hand area indicated by the hand area data of the hand area data storage means and the display position indicated by the display position data of the object storage means, it is detected whether or not the object is held by the operator's hand. Object hold detection means for
When the object is gripped by the operator's hand and the hand area indicated by the hand area data is moved, the display position of the object-related data after the movement is indicated along with the movement of the hand area. Position management means for creating display position data after movement;
A program for creating projection image data for displaying object-related data in the object storage means at a position indicated by the post-movement display position data, and causing the projection image data storage means to function as image data creation means. In a projection image creation method implemented by a computer,
Storing object-related data related to an object arranged in a space photographed by the imaging device and display position data indicating a display position of the object-related data in the object storage means;
Based on the photographed image data photographed by the photographing device, create hand region data for detecting the hand region of the operator, and store it in the hand region data storage means,
Based on the hand area indicated by the hand area data of the hand area data storage means and the display position indicated by the display position data of the object storage means, it is detected whether or not the object is held by the operator's hand. And
When the object is gripped by the operator's hand and the hand area indicated by the hand area data is moved, the display position of the object-related data after the movement is indicated along with the movement of the hand area. Create display position data after moving,
Projection image data for displaying object-related data in the object storage unit at a position indicated by the post-movement display position data is generated and stored in the projection image data storage unit. A photographing device for photographing an object arranged in space;
Object storage means for storing object related data related to the object and display position data indicating a display position of the object related data;
Hand region detection means for creating hand region data that detects the region of the hand of the operator based on the captured image data captured by the imaging device;
Object hold detection means for detecting whether or not the object is held by the operator's hand based on the hand area indicated by the hand area data and the display position indicated by the display position data;
When the object is gripped by the operator's hand and the hand area indicated by the hand area data is moved, the display position of the object-related data after the movement is indicated along with the movement of the hand area. Position management means for creating display position data after movement;
Image data creation means for creating projection image data for displaying the object-related data at a position indicated by the display position data after movement;
An image projection system comprising: a projection device that projects an image onto the space based on the projection image data.

Images