JP2014178790A - Projection system, projection device, projection program, and projection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a reduced image from hindering a viewer when projecting a projection image in which the reduced image is arranged.SOLUTION: A portion detection part detects a predetermined portion of a user on the basis of a picked-up image obtained by picking up a first image projected on a projection object. A first position determination part determines a position for arranging a reduced image of the first image on the basis of detection by the portion detection part. A projection part projects a second image in which the reduced image is arranged at the position determined by the first position determination part. An instruction operation detection part detects an instruction operation of the user with respect to the second image projected by the projection part on the basis of the picked-up image. A second position determination part determines a position for arranging an instruction image on the basis of detection by the instruction operation detection part. A projection control part controls a projection of an image by the projection part on the basis of a comparison of the position determined by the second position determination part with the position of the reduced image in the second image.

Description

  The present invention relates to a projection system, a projection apparatus, a projection program, and a projection method.

  Conventionally, in a meeting or a store, a display device that displays an image may be used. In a meeting, a display device is used for displaying documents and giving presentations. In stores, display devices are used for displaying various contents to users. In addition, the display device can have a large screen, and the large screen makes it easier for the user to see.

  In addition, a technique is known in which a user operation is performed on an image displayed on a display device so that various instructions such as switching of images can be performed. That is, the user can give various instructions by performing user operations while being in the vicinity of the display device. Thereby, a presentation can be performed efficiently in a meeting, and each user can browse information useful for himself / herself quickly also in a store.

  However, when the display device has a large screen, it is difficult to perform a user operation at a position that cannot be reached by the user. Therefore, recently, there is a technique in which a reduced image obtained by reducing the original image is synthesized and displayed on the original image to be displayed, a user operation on the reduced image is detected, and various instructions are reflected on the original image. Thereby, even when using a large-screen display device, various instructions by user operations are realized.

  However, it is difficult to apply the above-described conventional technique to a technique that reflects a user operation on a projection image projected on a projection plane. In the conventional technique, the coordinates on the reduced image displayed on the display device are converted into the coordinates on the original image displayed on the display device based on the user operation. There is a match. On the other hand, in a technique for reflecting a user operation on a projection image projected on a projection surface, the operation location is a screen, a wall, or the like, and is not image data itself to be operated. As a result, it is difficult to apply the above-described conventional technique to a technique that reflects a user operation on a projection image projected on a projection plane.

  The present invention has been made in view of the above, and a projection system and a projection apparatus capable of preventing a reduced image from obstructing a viewer when projecting a projection image on which a reduced image is arranged. An object is to provide a projection program and a projection method.

  In order to solve the above-described problems and achieve the object, a projection system according to the present invention is based on an imaging unit that captures a first image projected on a projection target, and a captured image captured by the imaging unit. A part detecting unit for detecting a predetermined part of the user, a first position determining unit for determining a position at which a reduced image of the first image is arranged based on detection by the part detecting unit, and the first position A projection unit that projects the second image in which the reduced image is arranged at the position determined by the determination unit, and the second image that is projected by the projection unit based on the captured image captured by the imaging unit. An instruction operation detection unit that detects an instruction operation of a user with respect to an image, a second position determination unit that determines a position where an instruction image is to be arranged based on detection by the instruction operation detection unit, and a second position determination unit. It has a determined location, based on a comparison between the position of the reduced image in the second image, and a projection control unit that controls the projection of images by the projecting unit.

  According to one aspect of the present invention, it is possible to prevent the reduced image from obstructing the viewer when projecting a projection image in which the reduced image is arranged.

FIG. 1 is a functional block diagram illustrating a configuration example of the projection apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of information stored in the coordinate storage unit according to the first embodiment. FIG. 3 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the first embodiment do not overlap. FIG. 4 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the first embodiment. FIG. 5 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the first embodiment overlap. FIG. 6 is a flowchart illustrating an example of the flow of image generation processing according to the first embodiment. FIG. 7 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the modification of the first embodiment. FIG. 8 is a diagram illustrating an example of a projected image in a case where the arrangement area and the pointer according to the modification of the first embodiment overlap. FIG. 9 is a flowchart illustrating an example of the flow of image generation processing according to a modification of the first embodiment. FIG. 10 is a functional block diagram illustrating a configuration example of the projection apparatus according to the second embodiment. FIG. 11A is a diagram illustrating an example in which the pointer according to Embodiment 2 is changed to a ring shape. FIG. 11B is a diagram illustrating an example in which the pointer according to Embodiment 2 is changed to an arrow shape. FIG. 12 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the second embodiment. FIG. 13 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the second embodiment overlap. FIG. 14 is a flowchart illustrating an example of the flow of image generation processing according to the second embodiment. FIG. 15 is a functional block diagram illustrating a configuration example of the projection apparatus according to the third embodiment. FIG. 16 is a diagram illustrating an example of information stored in the coordinate storage unit according to the third embodiment. FIG. 17 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the third embodiment. FIG. 18 is a flowchart illustrating an example of the flow of image generation processing according to the third embodiment. FIG. 19 is a functional block diagram illustrating an apparatus configuration example of the projection system. FIG. 20 is a block diagram illustrating a hardware configuration example of the projection apparatus 100.

  Embodiments of a projection system, a projection apparatus, a projection program, and a projection method according to the present invention will be described below with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment. Moreover, each embodiment can be combined suitably as long as the content is not contradicted.

(Embodiment 1)
[Apparatus Configuration According to Embodiment 1]
The configuration of the projection apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a functional block diagram illustrating a configuration example of the projection apparatus according to the first embodiment.

  As illustrated in FIG. 1, the projection apparatus 100 includes a storage unit 110 and a control unit 120. Among these, the memory | storage part 110 memorize | stores the data utilized by the various processes by the control part 120, and the various process results by the control part 120, and the projection image memory | storage part 111, the captured image memory | storage part 112, and a coordinate memory | storage part 113. The storage unit 110 is, for example, a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), and a flash memory, or a storage device such as a hard disk or an optical disk.

  The projection image storage unit 111 stores a projection image projected on a projection target such as a projection surface. Specifically, the projection image storage unit 111 may store the projection image itself projected onto the projection plane, or may store configuration data for generating a projection image projected onto the projection plane. . For example, the configuration data of a projection image is data for configuring a projection image such as various objects arranged in the projection image.

  The captured image storage unit 112 stores a captured image obtained by capturing an imaged area representing an imageable area. Such a captured image is an image captured and generated by an imaging device such as a camera. The imaging device captures an imaging region determined according to the distance between the projection plane and the imaging device, and generates a captured image. Then, the imaging device transmits the generated captured image to the projection device 100. As a result, the captured image transmitted by the imaging apparatus is stored in the captured image storage unit 112. In the present embodiment, the imaging area is assumed to be the same size as the projection image projected on the projection plane.

  The coordinate storage unit 113 stores information related to the position and size of the arrangement area and information related to the position in the projection image designated by the user. Such an arrangement area is an area included in the projection image, and is an area where a reduced image obtained by reducing the projection image is arranged. The determination of the position of the arrangement area will be described later. Specifically, the coordinate storage unit 113 stores the coordinates of the arrangement area with respect to the projection image, and the horizontal width (X direction) and height (Y direction) of the arrangement area. In addition, the coordinate storage unit 113 stores user-designated coordinates for the projection image. In the following, the coordinates designated by the user may be referred to as “designated coordinates”.

  FIG. 2 is a diagram illustrating an example of information stored in the coordinate storage unit 113 according to the first embodiment. As shown in FIG. 2, the coordinate storage unit 113 stores the X coordinate and Y coordinate of the arrangement area with respect to the projection image, and the horizontal width and height of the arrangement area. For example, the coordinate storage unit 113 stores an X coordinate “580”, a Y coordinate “400”, a horizontal width “60”, and a height “40” for the arrangement region. In addition, the coordinate storage unit 113 stores an X coordinate and a Y coordinate of a user instruction for the projection image. For example, the coordinate storage unit 113 stores an X coordinate “300” and a Y coordinate “300” for the designated coordinate. The coordinate storage unit 113 does not hold the above information in a default state such as when the system is activated.

  The control unit 120 has an internal memory for storing a control program, a program defining various processing procedures, and required data. The control unit 120 includes a part detection unit 121, a first position determination unit 122, a projection image generation unit 123, a reduced image generation unit 124, an instruction image generation unit 125, a projection unit 126, and an operation recognition unit 127. And a second position determination unit 128 and a determination unit 129.

  The control unit 120 is, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), or an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). . In addition, about each said part, all or these may be software (program) and the above hardware circuits.

  The part detection unit 121 detects a predetermined part of the user using the captured image. More specifically, the part detection unit 121 acquires a captured image from the captured image storage unit 112 and detects that a predetermined part of the user is reflected in the acquired captured image. The predetermined part is, for example, a user's hand. Then, the part detection unit 121 outputs to the first position determination unit 122 the fact that the user's hand has been detected and the captured image when the user's hand has been detected. In addition, the detection process of the user's hand by the part detection unit 121 is sequentially performed, and the part detection unit 121 is in a state where it cannot be detected when the user's hand is detected and becomes undetectable. To the projected image generation unit 123.

  The first position determination unit 122 determines the position of the arrangement area according to the position where the predetermined part is detected. More specifically, the first position determination unit 122 acquires the projection image being projected from the projection image storage unit 111, and the acquired projection image and the imaging used for detection of the user's hand by the part detection unit 121. From the image, the coordinates of the arrangement area with respect to the projection image are calculated. Then, the first position determination unit 122 stores the calculated coordinates in the coordinate storage unit 113 together with the horizontal width and height of the arrangement area. The horizontal width and height of the arrangement area may use predetermined values or may be determined according to the size of the projection image. Thereafter, the first position determination unit 122 requests the projection image generation unit 123 to perform a projection image generation process.

  The coordinates of the arrangement area may be any coordinates in the arrangement area. For example, any one of the vertex coordinates of the arrangement area may be used, or a coordinate serving as the center of gravity of the arrangement area may be used. That is, when the coordinates calculated by the first position determining unit 122 are any of the vertex coordinates of the placement area, the position of the placement area with respect to the projection image is determined from one vertex coordinate and the horizontal width and height of the placement area. it can. In addition, when the coordinates calculated by the first position determination unit 122 are the coordinates that are the center of gravity of the arrangement area, the area having the horizontal width and the height of the arrangement area, and the area having the calculated coordinates as the center of gravity If comprised, the position of the arrangement | positioning area | region with respect to a projection image can be determined. In addition to these, in the case where the coordinates calculated by the first position determination unit 122 are arbitrary coordinates in the arrangement area, the area includes the horizontal width and height of the arrangement area, and includes an area including the arbitrary coordinates. Then, the position of the arrangement area with respect to the projection image can be determined.

  The projection image generation unit 123 that has received the request for the projection image generation process requests the reduced image generation unit 124 for the reduced image generation process and also requests the instruction image generation unit 125 for the instruction image generation process.

  The reduced image generation unit 124 generates a reduced image obtained by reducing the projection image. More specifically, the reduced image generation unit 124 acquires the projection image being projected from the projection image storage unit 111 and the horizontal width and height of the arrangement area from the coordinate storage unit 113. Then, the reduced image generation unit 124 generates a reduced image obtained by reducing the acquired projection image in accordance with the horizontal width and height of the arrangement area. Thereafter, the reduced image generation unit 124 outputs the generated reduced image to the projection image generation unit 123. The instruction image generation unit 125 generates a pointer to be arranged in the projection image, and outputs the generated pointer to the projection image generation unit 123. Such a pointer is an example of an instruction image. The size of the pointer may be determined in advance or may be determined according to the size of the projection image. If the size of the pointer is determined according to the size of the projection image, it is possible to suppress the placement of an unnatural pointer.

  The projection image generation unit 123 generates a projection image in which the reduced image is arranged in the arrangement area. More specifically, the projection image generation unit 123 acquires the projection image being projected from the projection image storage unit 111, and the coordinates of the arrangement region determined by the first position determination unit 122, the width of the arrangement region, The height is acquired from the coordinate storage unit 113. Then, the projection image generation unit 123 arranges the reduced image generated by the reduced image generation unit 124 in the arrangement area based on the coordinates, the horizontal width, and the height of the arrangement area, and is generated by the instruction image generation unit 125. A projection image in which the pointer is placed at an arbitrary position is generated. Here, since the position of the pointer is not operated by the user at this time, it may be arranged at an arbitrary position as described above. Thereafter, the projection image generation unit 123 outputs the generated projection image to the projection unit 126. Note that the projection image on which the reduced image is arranged is an example of the second image.

  The projection unit 126 projects a projection image on the projection surface. More specifically, the projection unit 126 projects the projection image generated by the projection image generation unit 123 onto a projection surface such as a screen. Note that the projection image projected on the projection plane is an example of the first image.

  The operation recognition unit 127 recognizes a user operation. More specifically, the operation recognizing unit 127 acquires a captured image from the captured image storage unit 112, detects the movement of the user's hand in the reduced image based on a plurality of consecutive captured images, and performs projection. A user operation that is an operation on the device 100 is recognized. Such user operations are sometimes referred to as gestures or gesture operations. Here, it is assumed that a user operation for moving the pointer is recognized. The operation recognition unit 127 is an example of an instruction operation detection unit.

  The user moves his / her hand to move the pointer in the reduced image included in the projection image projected onto the projection plane. That is, the operation recognition unit 127 recognizes a user operation for moving the pointer from the locus of the user's hand in the reduced image using the captured image. Further, the operation recognition unit 127 calculates the coordinates of the position of the user's hand with respect to the projection image using the captured image in which the user operation for moving the pointer can be recognized, and stores the calculated coordinates in the coordinate storage unit 113. The coordinates stored in the coordinate storage unit 113 by the operation recognition unit 127 may be referred to as designated coordinates. Then, the operation recognition unit 127 outputs to the second position determination unit 128 that the user operation for moving the pointer has been recognized. As a result, the coordinate storage unit 113 is in a state in which the position of the arrangement area determined by the first position determination unit 122 and the indicated coordinates calculated by the operation recognition unit 127 are stored (see FIG. 2). Note that the designated coordinates stored in the coordinate storage unit 113 are updated (overwritten) by the operation recognition unit 127 each time.

  The second position determination unit 128 determines the position of the instruction image to be arranged in the projection image in response to a user operation. More specifically, the second position determining unit 128 acquires the designated coordinates from the coordinate storage unit 113 when receiving an indication that the operation recognizing unit 127 has recognized the user operation to move the pointer. Then, the second position determination unit 128 determines the position of the pointer to be arranged in the projection image by converting the acquired designated coordinates from the coordinate system of the reduced image to the coordinate system of the projection image. The coordinate system may be converted based on the size (horizontal width or height) of the projected image and the reduced image. For the size of the projection image, the configuration data of the projection image stored in the projection image storage unit 111 may be referred to. Thereafter, the second position determination unit 128 outputs the position (coordinates) of the pointer to be arranged in the projection image to the determination unit 129.

  The determination unit 129 determines whether or not the arrangement area and the instruction image overlap. More specifically, when the determination unit 129 receives the position of the pointer arranged in the projection image from the second position determination unit 128, the determination unit 129 acquires the coordinates, the horizontal width, and the height of the arrangement area from the coordinate storage unit 113, It is determined whether or not the placement area and the pointer overlap in the projection image. In such determination, whether or not the placement area and the pointer overlap each other may be determined based on whether or not the coordinates of the pointer are included in the placement area. Then, the determination unit 129 outputs, to the projection image generation unit 123, a determination result as to whether or not the arrangement area and the pointer overlap with the position of the pointer arranged in the projection image.

  The projection image generation unit 123 generates a projection image in which the display method in the arrangement area is changed according to the determination result of whether or not the arrangement area received from the determination unit 129 overlaps the pointer. More specifically, the projection image generation unit 123 generates a projection image in which the pointer is arranged at a position corresponding to the user operation when the determination unit 129 determines that the arrangement area and the pointer do not overlap. Then, the projection image generation unit 123 outputs the generated projection image to the projection unit 126. Thereby, the projection unit 126 projects a projection image in which the pointer is arranged at a position corresponding to the user operation onto the projection plane. Note that the determination unit 129 and the projection image generation unit 123 are examples of a projection control unit.

  FIG. 3 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the first embodiment do not overlap. The upper part of FIG. 3 is a projection image projected in an initial state such as when the system is started, and the lower part of FIG. 3 is a projection image on which pointer display is performed according to a user operation. As shown in the lower part of FIG. 3, when the user puts his hand in the lower left place on the projection plane, the reduced image is displayed at the position where the hand is put. As shown in the lower part of FIG. 3, when a user operation for moving the pointer by the user is performed in the reduced image, a pointer (black circle) is displayed at a position corresponding to the user operation. In this embodiment, the case where the shape of the pointer is circular is taken as an example, but the shape of the pointer is not limited to a circle.

  On the other hand, when the determination unit 129 determines that the arrangement area and the pointer overlap, the projection image generation unit 123 generates a projection image in which the pointer is arranged and the reduced image that has been transmitted is arranged. Then, the projection image generation unit 123 outputs the generated projection image to the projection unit 126. Thereby, the projection unit 126 projects a projection image on which a pointer is arranged in an arrangement region that is a position corresponding to a user operation and a reduced image that has been transmitted is arranged on a projection plane.

  FIG. 4 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the first embodiment. In FIG. 4, a case where the arrangement area and the pointer overlap is taken as an example. As shown in FIG. 4, the arrangement area display method is “transparent”, the X coordinate is “580”, the Y coordinate is “400”, the horizontal width is “60”, and the height is “40”. Is. The X coordinate of the pointer is “600”, the Y coordinate is “420”, the horizontal width is “30”, and the height is “30”. Note that the horizontal width and height of the pointer represent the size of a rectangle circumscribing a circle as described above. The projection image generation unit 123 generates a projection image in which a reduced image is arranged at the position of the arrangement area and a pointer is arranged based on the display contents of various objects shown in FIG.

  FIG. 5 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the first embodiment overlap. In FIG. 5, the reduced image is represented by a broken line, but is not actually represented by a broken line. That is, the reduced image shown in FIG. 5 is an image of the transmitted reduced image. As shown in FIG. 5, when a user operation for moving the pointer by the user is performed in the reduced image and the destination of the pointer at this time overlaps with the arrangement area, the pointer is arranged in the transparent reduced image. It becomes a projected image of the state.

  Further, when the projection image generation unit 123 receives from the part detection unit 121 that the user's hand has not been detected, the projection image generation unit 123 generates a projection image in which the reduced image is not arranged. That is, the fact that the user's hand is not included in the captured image means that the user has stopped the user operation and the reduced image does not have to be displayed. A projection image that does not place the is generated. At this time, the information stored in the coordinate storage unit 113 is deleted.

[Image Generation Processing Flow According to Embodiment 1]
Next, the flow of image generation processing according to Embodiment 1 will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of the flow of image generation processing according to the first embodiment.

  As shown in FIG. 6, the part detection unit 121 acquires a captured image from the captured image storage unit 112 (step S101). Here, the part detection unit 121 attempts to detect the user's hand using the captured image (step S102). At this time, when the user's hand is detected by the part detection unit 121 (step S102: Yes), the first position determination unit 122 acquires the projection image being projected from the projection image storage unit 111, and acquires the obtained projection. The position of the arrangement area with respect to the projection image is determined from the image and the captured image used for detecting the user's hand by the part detection unit 121 (step S103). The determined position of the arrangement area is stored in the coordinate storage unit 113. On the other hand, if the user's hand is not detected by the part detection unit 121 (step S102: No), the process in step S101 is executed.

  The reduced image generation unit 124 acquires the projected image being projected from the projection image storage unit 111, acquires the horizontal width and height of the arrangement area from the coordinate storage unit 113, and matches the horizontal width and height of the arrangement area, A reduced image obtained by reducing the acquired projection image is generated (step S104). The instruction image generation unit 125 generates a pointer placed in the projection image (step S105). The projection image generation unit 123 acquires the projection image being projected from the projection image storage unit 111 and also acquires the position (coordinates), horizontal width, and height of the arrangement area from the coordinate storage unit 113, and the reduced image generation unit 124 A projection image in which the generated reduced image is arranged in the arrangement area is generated (step S106). The projection image generation unit 123 also arranges the pointer generated by the instruction image generation unit 125 in the projection image. The projection unit 126 projects the projection image generated by the projection image generation unit 123 (step S107).

  After projecting the projection image on which the reduced image is arranged, the part detection unit 121 acquires the captured image from the captured image storage unit 112 (step S108). Here, the part detection unit 121 attempts to detect the user's hand (step S109). When the user's hand is detected by the part detection unit 121 (step S109: Yes), the operation recognition unit 127 acquires a captured image from the captured image storage unit 112, and uses the captured image as a pointer. Attempts to recognize a user operation to move (step S110). When the operation recognizing unit 127 recognizes a user operation to move the pointer, the operation recognizing unit 127 calculates the coordinates (instructed coordinates) of the position of the user's hand with respect to the projection image, and stores the calculated coordinates in the coordinate storage unit 113.

  When the user operation for moving the pointer is recognized by the operation recognition unit 127 (step S110: Yes), the second position determination unit 128 acquires the indicated coordinates from the coordinate storage unit 113, and reduces the acquired indicated coordinates. By converting the coordinate system of the image to the coordinate system of the projected image, the position of the pointer placed in the projected image is determined (step S111). The determination unit 129 determines whether the placement area and the pointer overlap from the position of the pointer determined by the operation recognition unit 127 and the coordinates, width, and height of the placement area acquired from the coordinate storage unit 113 ( Step S112).

  At this time, when the determination unit 129 determines that the arrangement area and the pointer overlap (step S112: Yes), the projection image generation unit 123 arranges the pointer at the position determined by the second position determination unit 128. At the same time, a projection image in which the reduced image that has been transmitted is arranged is generated (step S113). Thereafter, the projection unit 126 projects the projection image on which the reduced reduced image generated by the projection image generation unit 123 is arranged on the projection surface (step S114).

  On the other hand, when the determination unit 129 determines that the arrangement area and the pointer do not overlap (step S112: No), the projection image generation unit 123 arranges the pointer at the position determined by the second position determination unit 128. A projection image is generated (step S115). After that, the projection unit 126 projects the projection image generated by the projection image generation unit 123 with the pointer placed at the position corresponding to the user operation (Step S114).

  Further, when the user's hand is not detected by the part detection unit 121 (step S109: No), the projection image generation unit 123 generates a projection image in which the reduced image is not arranged (step S116). Thereafter, the projection unit 126 projects a projection image on which the reduced image generated by the projection image generation unit 123 is not arranged on the projection plane (step S114).

[Effects of Embodiment 1]
The projection apparatus 100 arranges a reduced image obtained by reducing the projection image in an arrangement area that is a position of the projection image where the user puts a hand, and moves the pointer in the projection image based on a user operation of moving the pointer in the arrangement area. When the position is determined and the placement area and the pointer overlap, a projection image in which the reduced reduced image is placed is generated. As a result, the projection apparatus 100 can prevent the reduced image from obstructing the viewer.

  Further, the projection device 100 generates a projection image in which the reduced image is not arranged when the user's hand is not detected from the state where the user's hand is detected. As a result, the projection device 100 can improve the operability of the user operation because the position of the arrangement area differs depending on the position of the projected image with the hand for each user who performs the user operation.

(Modification of Embodiment 1)
In the first embodiment, the case where the projection image in which the reduced image that has been transmitted is arranged is generated when the arrangement area and the pointer overlap each other has been described. In the modification of the first embodiment, a case will be described in which a projection image in which a frame image representing an outer frame of a placement area is placed is generated when the placement area and the pointer overlap. The modification of the first embodiment will be described as appropriate using the configuration example of the projection apparatus 100 shown in FIG. Specifically, as described below, the processing by the projection image generation unit 123 is different from that of the first embodiment.

  The projection image generation unit 123 generates a projection image in which the display method in the arrangement area is changed according to the determination result of whether or not the arrangement area received from the determination unit 129 overlaps the pointer. More specifically, the projection image generation unit 123 arranges the pointer when the determination unit 129 determines that the arrangement area and the pointer overlap, and arranges the arrangement area without arranging the reduced image in the arrangement area. A projection image in which a frame image representing the outer frame is arranged is generated. Then, the projection image generation unit 123 outputs the generated projection image to the projection unit 126. Thereby, the projection unit 126 projects a projection image on which the pointer is arranged in the arrangement area that is a position corresponding to the user operation and the frame image of the arrangement area is arranged on the projection plane. That is, in the present embodiment, the user performs a user operation to move the pointer within the frame image of the arrangement area in the projection image in which the frame image of the arrangement area is arranged.

  FIG. 7 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the modification of the first embodiment. FIG. 7 shows an example where the arrangement area and the pointer overlap. As shown in FIG. 7, the arrangement area display method is “outer frame”, the X coordinate is “580”, the Y coordinate is “400”, the horizontal width is “60”, and the height is “ 40 ". The X coordinate of the pointer is “600”, the Y coordinate is “420”, the horizontal width is “30”, and the height is “30”. Based on the display contents of the various objects shown in FIG. 7, the projection image generation unit 123 arranges the frame image of the arrangement area without arranging the reduced image at the position of the arrangement area, and sets the pointer in the arrangement area. The arranged projection image is generated.

  FIG. 8 is a diagram illustrating an example of a projected image in a case where the arrangement area and the pointer according to the modification of the first embodiment overlap. As shown in FIG. 8, when a user operation for moving the pointer is performed by the user and the movement destination of the pointer at this time overlaps with the arrangement area, the reduced image is not arranged and the pointer is placed in the frame image of the arrangement area. Is a projected image in a state where is arranged.

[Image Generation Processing Flow According to Modification of First Embodiment]
Next, the flow of image generation processing according to a modification of the first embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating an example of the flow of image generation processing according to a modification of the first embodiment. Note that detailed description of the same processing as the flow of image generation processing according to Embodiment 1 is omitted. Specifically, steps S201 to S212 are the same as the processes in steps S101 to S112, and steps S214 to S216 are the same as the processes in steps S114 to S116.

  As illustrated in FIG. 9, when the determination unit 129 determines that the placement area and the pointer overlap (step S212: Yes), the projection image generation unit 123 is positioned at the position determined by the second position determination unit 128. A projection image in which a pointer is placed and a frame image in the placement area is placed is generated (step S213). At this time, the reduced image is not arranged in the projection image. After that, the projection unit 126 projects the projection image generated by the projection image generation unit 123 on which the frame image of the arrangement area is arranged on the projection plane (step S214).

[Effects of Modification of First Embodiment]
The projection apparatus 100 arranges a reduced image obtained by reducing the projection image in an arrangement area that is a position of the projection image where the user puts a hand, and moves the pointer in the projection image based on a user operation of moving the pointer in the arrangement area. When the position is determined and the arrangement area and the pointer overlap, a projection image in which the frame image of the arrangement area is arranged is generated without arranging the reduced image. As a result, the projection apparatus 100 can prevent the reduced image from obstructing the viewer.

(Embodiment 2)
In the first embodiment and the modification, when the arrangement area and the pointer overlap, a projection image in which the reduced image that has been transmitted is arranged is generated, or a projection image in which the frame image in the arrangement area is arranged is generated. Explained the case. In the second embodiment, a case where the pointer display method is changed will be described.

[Apparatus configuration according to Embodiment 2]
The configuration of the projection apparatus according to the second embodiment will be described with reference to FIG. FIG. 10 is a functional block diagram illustrating a configuration example of the projection apparatus according to the second embodiment. In the second embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description of the same components may be omitted. Specifically, functions, configurations, and processes other than the projection image generation unit 223 and the instruction image generation unit 225 described below are the same as those in the first embodiment.

  As illustrated in FIG. 10, the projection apparatus 200 includes a storage unit 110 and a control unit 220. Among these, the storage unit 110 includes a projection image storage unit 111, a captured image storage unit 112, and a coordinate storage unit 113. The control unit 220 includes a part detection unit 121, a first position determination unit 122, a projection image generation unit 223, a reduced image generation unit 124, an instruction image generation unit 225, a projection unit 126, and an operation recognition unit 127. The second position determination unit 128 and the determination unit 129 are included.

  The projection image generation unit 223 generates a projection image in which the instruction image display method is changed according to the determination result of whether or not the arrangement area received from the determination unit 129 overlaps the pointer. More specifically, when the determination unit 129 determines that the placement area and the pointer overlap, the projection image generation unit 223 instructs to generate a pointer having a different shape from that of the placement area outside the placement area. Is output to the instruction image generation unit 225.

  The instruction image generation unit 225 that has received the pointer generation instruction generates a pointer having a shape different from that when the pointer is generated outside the arrangement area, and outputs the generated pointer to the projection image generation unit 223. Accordingly, the projection image generation unit 223 generates a projection image in which the pointer generated by the instruction image generation unit 225 is arranged and the reduced image that has been transmitted is arranged. Note that, as in the modified example of the first embodiment, a projection image in which a frame image in an arrangement area is arranged may be generated without arranging a reduced image. Thereby, the projection unit 126 projects a projection image on which a pointer whose shape has been changed is arranged in an arrangement region that is a position according to a user operation and a reduced image that has been transmitted is arranged on a projection surface.

  Here, a pointer whose shape has been changed will be described with reference to FIGS. 11A and 11B. FIG. 11A is a diagram illustrating an example in which the pointer according to Embodiment 2 is changed to a ring shape. FIG. 11B is a diagram illustrating an example in which the pointer according to Embodiment 2 is changed to an arrow shape.

  When the pointer having the ring shape shown in FIG. 11A is generated, as one aspect, a circle inscribed in the square when the lateral width and height of the square circumscribing the original pointer are doubled is used as a ring. And remove the original pointer part to make a ring shape. The above calculation may be executed by the instruction image generation unit 225 or may be executed by the projection image generation unit 223. When executed by the projection image generation unit 223, the ring size information is output as a parameter to the instruction image generation unit 225.

When the pointer having the arrow shape shown in FIG. 11B is generated, the maximum arrow included in the horizontal width and the height is used, and the angle of the arrow is obtained as follows. The following calculation may be executed by the instruction image generation unit 225 or may be executed by the projection image generation unit 223. When executed by the projection image generation unit 223, the angle of the arrow is output as a parameter to the instruction image generation unit 225.
“X coordinate of center of gravity of placement area = X coordinate of placement area + width of placement area ÷ 2”
“Y coordinate of the center of gravity of the placement area = Y coordinate of the placement area + height of the placement area ÷ 2”
"Arrow angle (rad)
= Tan ⁻¹ [(Y coordinate of indicated coordinate−Y coordinate of centroid of arrangement area) ÷ (X coordinate of indicated coordinate−X coordinate of centroid of arrangement area)]

  FIG. 12 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the second embodiment. In FIG. 12, the arrangement area and the pointer overlap each other, and the arrangement area display method is a frame image arrangement, and the pointer display method is a ring shape.

  As shown in FIG. 12, the arrangement area display method is “outer frame”, the X coordinate is “580”, the Y coordinate is “400”, the horizontal width is “60”, and the height is “ 40 ". The pointer display method is “ring”, the X coordinate is “600”, the Y coordinate is “420”, the horizontal width is “60”, and the height is “60”. The instruction image generation unit 225 generates a pointer based on the display content of the pointer shown in FIG. Further, the projection image generation unit 223 generates a projection image in which a frame image of the arrangement area is arranged and a pointer whose shape is changed is arranged based on the display contents of various objects shown in FIG.

  FIG. 13 is a diagram illustrating an example of a projected image when the arrangement area and the pointer according to the second embodiment overlap. As shown in FIG. 13, when a user operation for moving the pointer by the user is performed in the reduced image and the destination of the pointer at this time overlaps with the arrangement area, the pointer whose shape has been changed to an arrow or the like is arranged. At the same time, the projected image is in a state where the frame image of the arrangement area is arranged.

[Image Generation Processing Flow According to Embodiment 2]
Next, the flow of image generation processing according to the second embodiment will be described with reference to FIG. FIG. 14 is a flowchart illustrating an example of the flow of image generation processing according to the second embodiment. Note that detailed description of the same processing as the flow of image generation processing according to the modification of the first embodiment is omitted. Specifically, steps S301 to S312 are the same as the processes in steps S201 to S212, and steps S314 to S317 are the same as the processes in steps S213 to S216. FIG. 14 illustrates an example in which a projection image in which frame images are arranged is generated.

  As illustrated in FIG. 14, when the determination unit 129 determines that the placement area and the pointer overlap (step S312: Yes), the instruction image generation unit 225 has a shape different from that when the placement image is placed outside the placement area. A pointer with a different display method is generated (step S313). Thereby, the projection image generation unit 223 generates the projection image in which the pointer generated by the instruction image generation unit 225 is arranged and the frame image of the arrangement area is arranged (step S314). At this time, the reduced image is not arranged in the projection image. Thereafter, the projection unit 126 projects the projection image generated by the projection image generation unit 223 on which the pointer with the changed shape is arranged and the frame image of the arrangement area is arranged on the projection surface (step S315). ).

[Effects of Embodiment 2]
When the arrangement area and the pointer overlap, the projection device 200 generates a projection image in which the pointer whose shape has been changed is arranged. As a result, the projection apparatus 200 can make the viewer easily recognize where the pointer is pointing.

(Embodiment 3)
In the first embodiment, the case where the image generation process is executed in response to a user operation by a single user has been described. In the third embodiment, a case where image generation processing is executed in response to user operations by a plurality of users will be described.

[Apparatus configuration according to Embodiment 3]
The configuration of the projection apparatus according to Embodiment 3 will be described with reference to FIG. FIG. 15 is a functional block diagram illustrating a configuration example of the projection apparatus according to the third embodiment. In the third embodiment, detailed description of the same configuration as in the first embodiment may be omitted. Specifically, the functions, configurations, and processes of the projection image storage unit 111, the captured image storage unit 112, the reduced image generation unit 124, and the projection unit 126 described below are the same as those in the first embodiment.

  As illustrated in FIG. 15, the projection apparatus 300 includes a storage unit 310 and a control unit 320. Among these, the storage unit 310 includes a projection image storage unit 111, a captured image storage unit 112, and a coordinate storage unit 313.

  The coordinate storage unit 313 stores, for each of a plurality of users, information related to the position and size of the arrangement area and information related to the position in the projection image instructed by each user. FIG. 16 is a diagram illustrating an example of information stored in the coordinate storage unit 313 according to the third embodiment. A plurality of users are referred to as “user A” and “user B”.

  For example, the coordinate storage unit 313 stores an X coordinate “0”, a Y coordinate “200”, a horizontal width “60”, and a height “40” for “arrangement area A” corresponding to “user A”. . In addition, the coordinate storage unit 313 stores an X coordinate “20” and a Y coordinate “220” for “instructed coordinates A” corresponding to “user A”. Similarly, the coordinate storage unit 313 stores the X coordinate “580”, the Y coordinate “400”, the horizontal width “60”, and the height “40” for the “arrangement region B” corresponding to “user B”. In addition, the coordinate storage unit 313 stores an X coordinate “300” and a Y coordinate “300” for the “instructed coordinate B” corresponding to “user B”.

  The control unit 320 includes a part detection unit 321, a first position determination unit 322, a projection image generation unit 323, a reduced image generation unit 124, an instruction image generation unit 325, a projection unit 126, and an operation recognition unit 327. , A second position determination unit 328 and a determination unit 329.

  The part detection unit 321 detects a predetermined part of a plurality of users using the captured image. More specifically, the part detection unit 321 acquires a captured image from the captured image storage unit 112, and detects that a predetermined part of the user is reflected in the acquired captured image. The part detection processing by the part detection unit 321 is appropriately executed, and the part detection unit 321 detects that a predetermined part is reflected for a plurality of users by the part detection process executed as appropriate. Then, each time the user's hand is detected, the part detection unit 321 outputs to the first position determination unit 322 that the user's hand has been detected and a captured image when the user's hand has been detected.

  The first position determination unit 322 determines the positions of the plurality of arrangement regions in accordance with the positions where the predetermined sites of the plurality of users are detected. More specifically, the first position determination unit 322 acquires the projection image being projected from the projection image storage unit 111, and the acquired projection image and the imaging used for detecting the user's hand by the part detection unit 321. A process of calculating the coordinates of the arrangement area with respect to the projection image from the image is executed every time the user's hand is detected. Then, the first position determination unit 322 stores the calculated coordinates in the coordinate storage unit 313 together with the horizontal width and height of the arrangement area. Thereafter, the first position determination unit 322 requests the projection image generation unit 323 to perform a projection image generation process.

  The recognition of multiple users may be recognized from the size of the user's hand included in the captured image, the state of the eyelids, etc., or the hand image is held in advance and held with the hand included in the captured image. It may be recognized by matching with a hand image. In addition, existing techniques may be used as the user recognition method.

  The projection image generation unit 323 that has received the request for the projection image generation process requests the reduced image generation unit 124 for the reduced image generation process and also requests the instruction image generation unit 325 for the instruction image generation process.

  The processing by the reduced image generation unit 124 is the same as in the first embodiment. That is, the reduced image generation unit 124 generates a reduced image obtained by reducing the projection image being projected in accordance with the size of the arrangement area, and outputs the generated reduced image to the projection image generation unit 323. Even if there are a plurality of users, if the size of the arrangement area is the same, the same reduced image can be used. That is, if the size of the arrangement area is different for each user, a plurality of reduced images are generated according to the size of each arrangement area.

  The instruction image generation unit 325 generates the same number of pointers as the number of users, and outputs the generated pointers to the projection image generation unit 323. The instruction image generation unit 325 may generate pointers having different shapes and colors for each user. As a result, each user can easily recognize which pointer the user operates.

  The projection image generation unit 323 generates a projection image in which reduced images are arranged in a plurality of arrangement areas. More specifically, the projection image generation unit 323 acquires the projection image being projected from the projection image storage unit 111, coordinates of the plurality of arrangement regions determined by the first position determination unit 322, and a plurality of arrangements. The width and height of the area are acquired from the coordinate storage unit 313. Then, the projection image generation unit 323 arranges the reduced images generated by the reduced image generation unit 124 in the plurality of arrangement regions based on the coordinates, the horizontal width, and the height of the plurality of arrangement regions, and the instruction image generation unit 325. A projection image is generated in which a plurality of pointers generated by the above are arranged at arbitrary positions. Thereafter, the projection image generation unit 323 outputs the generated projection image to the projection unit 126. Thereby, the projection unit 126 projects the projection image generated by the projection image generation unit 323 onto the projection plane.

  The operation recognition unit 327 recognizes a plurality of user operations. More specifically, the operation recognizing unit 327 acquires the captured image from the captured image storage unit 112, detects the movement of the user's hand in the reduced image based on a plurality of consecutive captured images, and Recognize operations. Such user operation is recognized for each user if there are a plurality of users. Here, it is assumed that a user operation for moving the pointer is recognized. In addition, the operation recognition unit 327 calculates the coordinates of the position of the user's hand with respect to the projection image using the captured image in which the user operation for moving the pointer can be recognized, and stores the calculated coordinates in the coordinate storage unit 313 for each user. Store. Then, the operation recognition unit 327 outputs to the second position determination unit 328 that the user operation for moving the pointer has been recognized. As a result, the coordinate storage unit 313 stores the position of the arrangement area for each user determined by the first position determination unit 322 and the instruction coordinates for each user calculated by the operation recognition unit 327. (See FIG. 16).

  The second position determination unit 328 determines the positions of a plurality of instruction images to be arranged on the projection image in response to a plurality of user operations. More specifically, when the second position determination unit 328 accepts that the user operation of moving the pointer is recognized by the operation recognition unit 327, the second position determination unit 328 obtains the instruction coordinates corresponding to the user who moved the pointer from the coordinate storage unit 313. get. Then, the second position determination unit 328 determines the position of the pointer to be arranged in the projection image by converting the acquired designated coordinates from the coordinate system of the reduced image to the coordinate system of the projection image. Thereafter, the second position determination unit 328 outputs the position (coordinates) of the pointer to be arranged in the projection image to the determination unit 329.

  The determination unit 329 determines whether or not the plurality of arrangement areas and the plurality of instruction images overlap. More specifically, when the determination unit 329 receives the position of the pointer to be arranged in the projection image from the second position determination unit 328, the determination unit 329 acquires the coordinates, the width, and the height of the plurality of arrangement areas from the coordinate storage unit 313. Then, it is determined whether or not the plurality of arrangement areas and the pointer overlap in the projection image. The determination method is the same as in the first embodiment. Then, the determination unit 329 outputs, to the projection image generation unit 323, a determination result as to whether or not the plurality of arrangement areas and the plurality of pointers overlap with the position of the pointer arranged in the projection image.

  The projection image generation unit 323 generates a projection image in which the display method in the arrangement area is changed according to the determination result of whether or not the plurality of arrangement areas received from the determination unit 329 overlap with the plurality of pointers. More specifically, when the determination unit 329 determines that the plurality of placement areas and the plurality of pointers overlap, the projection image generation unit 323 places the pointer and transmits the pointer to the placement area overlapping the pointer. A projection image in which the reduced images are arranged is generated. Then, the projection image generation unit 323 outputs the generated projection image to the projection unit 126. As a result, the projection unit 126 projects a projection image on which a pointer is arranged in each arrangement area, which is a position corresponding to a user operation by each user, and a reduced image that has been transmitted is arranged on a projection surface. As for the arrangement of the reduced image, a projection image may be generated in which the reduced image that has been transmitted is arranged in an arrangement area in which the outer frame has a different color for each user. Further, instead of arranging the reduced image that has been transmitted, a frame image of the arrangement area may be arranged.

  FIG. 17 is a diagram illustrating an example of display contents of various objects arranged in the projection image according to the third embodiment. In FIG. 17, a case where a user operation is performed by a plurality of users (user A, user B) is taken as an example. As shown in FIG. 17, the display method of “arrangement area A” corresponding to “user A” is “normal”, the X coordinate is “0”, the Y coordinate is “200”, and the horizontal width is “ 60 ”and the height is“ 40 ”. The display method “normal” indicates that the arrangement area display method is not particularly changed. The display method of “pointer A” corresponding to “user A” is “normal”, the X coordinate is “20”, and the Y coordinate is “220”. Similarly, the display method “normal” indicates that the display method of the pointer is not particularly changed.

  The display method of “arrangement area B” corresponding to “user B” is “outer frame”, the X coordinate is “580”, the Y coordinate is “400”, and the horizontal width is “60”. The height is “40”. The display method of “Pointer B” corresponding to “User B” is “Ring”, the X coordinate is “600”, the Y coordinate is “420”, the horizontal width is “60”, and the high The size is “60”. The instruction image generation unit 325 generates a pointer based on the display contents of each pointer shown in FIG. The projection image generation unit 323 generates a projection image in which arrangement areas and pointers having different display methods are arranged based on the display contents of various objects shown in FIG.

[Image Generation Processing Flow According to Embodiment 3]
Next, the flow of image generation processing according to the third embodiment will be described with reference to FIG. FIG. 18 is a flowchart illustrating an example of the flow of image generation processing according to the third embodiment. Note that FIG. 18 illustrates image generation processing in a state where a projection image on which a reduced image is arranged is projected on a projection plane.

  As illustrated in FIG. 18, after projecting a projection image in which a plurality of reduced images are arranged, the part detection unit 321 acquires a captured image from the captured image storage unit 112 (step S <b> 401). Here, the part detection unit 321 tries to detect the hands of all users (step S402). When all the user's hands are detected by the part detection unit 321 (step S402: No), the operation recognition unit 327 acquires a captured image from the captured image storage unit 112, and uses the captured image. Attempts to recognize a user operation for moving the pointer (step S403). When the operation recognizing unit 327 recognizes a user operation for moving the pointer, the operation recognizing unit 327 calculates the coordinates (instructed coordinates) of the position of the user's hand with respect to the projection image, and associates the calculated coordinates with the corresponding user information to the coordinate storage unit 313. To store.

  When a user operation for moving the pointer is recognized by the operation recognition unit 327 (step S403: Yes), the second position determination unit 328 acquires the designated coordinates from the coordinate storage unit 313, and reduces the obtained designated coordinates. By converting from the coordinate system of the image to the coordinate system of the projected image, the positions of a plurality of pointers arranged in the projected image are determined (step S404). Based on the positions of the plurality of pointers determined by the operation recognition unit 327 and the coordinates, width, and height of the plurality of arrangement areas acquired from the coordinate storage unit 313, the determination unit 329 determines whether the plurality of arrangement regions and the plurality of pointers are obtained. It is determined whether or not they overlap (step S405).

  At this time, when the determination unit 329 determines that the plurality of arrangement areas and the plurality of pointers overlap (step S405: Yes), the projection image generation unit 323 is positioned at the position determined by the second position determination unit 328. A projection image is generated in which a pointer is placed and a frame image of a placement area overlapping the pointer is placed (step S406). Thereafter, the projection unit 126 projects the projection image on which the frame image of the arrangement area overlapping the pointer generated by the projection image generation unit 323 is arranged on the projection plane (step S407).

  On the other hand, when the determination unit 329 determines that the plurality of placement areas and the plurality of pointers do not overlap (step S405: No), the projection image generation unit 323 is positioned at the position determined by the second position determination unit 328. A projection image on which the pointer is arranged is generated (step S408). Thereafter, the projection unit 126 projects the projection image generated by the projection image generation unit 323 and having the pointers arranged at positions corresponding to user operations by a plurality of users on the projection plane (step S407).

  In addition, when any part of the user's hand is not detected by the part detection unit 321 (step S402: Yes), the projection image generation unit 323 displays a projection image in which no reduced image is arranged in the corresponding arrangement region. Generate (step S409). At this time, the corresponding information stored in the coordinate storage unit 313 is deleted. Thereafter, the projection unit 126 projects onto the projection surface a projection image in which there is an arrangement area where the reduced image generated by the projection image generation unit 323 is not arranged (step S407).

[Effects of Embodiment 3]
The projection apparatus 300 arranges a reduced image obtained by reducing the projection image in each arrangement area, which is the position of the projection image obtained by a plurality of users, and projects the projection image based on a user operation of moving the pointer in each arrangement area. The positions of the plurality of pointers in the image are determined, and when the plurality of arrangement areas and the plurality of pointers overlap, a projection image in which the display method in the arrangement area is changed is generated. As a result, the projection apparatus 300 can prevent the reduced image from obstructing the viewer even when a plurality of users perform user operations.

(Embodiment 4)
Although the embodiments of the projection apparatus according to the present invention have been described so far, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, different embodiments of (1) pointer display method, (2) configuration, and (3) program will be described.

(1) Pointer Display Method In the second embodiment described above, a case has been described in which a pointer having a shape different from that when the placement area overlaps with the pointer is generated. About the display method of a pointer, you may make it change the effect in the case of a color and a display. Thereby, the operativity of user operation can be improved.

  The pointer display method can be changed by a user operation. For example, when the user performs a user operation for changing to a display that emphasizes the pointer, a parameter for emphasizing is added to the coordinate management unit 113, and the instruction image generation unit 125 performs the highlight display accordingly. Generated pointers are generated. This is useful when the user wants to pay attention to the position of the pointer or when the user loses sight of the pointer.

  The pointer display method can be changed according to the configuration of the projection image. For example, an edged pointer may be generated when the color of the projected image and the color of the pointer are similar. In other words, when the pointer is placed on the projected image, if it is detected that the color of the projected image and the pointer are the same even at one pixel at the position where the pointer is placed, the edged pointer may be placed. . Thereby, it is possible to provide a projection image that is easy for the user or viewer to see.

(2) Configuration In addition, information including processing procedures, control procedures, specific names, various data, parameters, and the like shown in the documents and drawings can be arbitrarily changed unless otherwise specified. Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution or integration of each device is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed in arbitrary units according to various burdens or usage conditions. Or they can be integrated.

  In the above embodiment, a captured image captured and generated by an imaging device such as a camera is held, a user operation is recognized using the captured image, a projection image is generated according to the recognized user operation, and the generated projection The projection apparatus 100 that projects an image on a projection plane has been described. The projection apparatus 100 can be distributed or integrated functionally or physically in an arbitrary unit.

  FIG. 19 is a functional block diagram illustrating an apparatus configuration example of the projection system. As illustrated in FIG. 19, the projection system includes an imaging device 10, a projection device 20, and an information processing device 30. The imaging device 10 is a camera or the like having an imaging unit 11. The imaging unit 11 generates a captured image and transmits it to the information processing apparatus 30. The projection device 20 is a projector or the like having a projection unit 21. The projection unit 21 projects the projection image generated by the information processing device 30 onto the projection plane.

  The information processing apparatus 30 includes a storage unit 31 and a control unit 32. Among these, the storage unit 31 includes a projected image storage unit 31a, a captured image storage unit 31b, and a coordinate storage unit 31c. Further, the control unit 32 includes a part detection unit 32a, a first position determination unit 32b, a projection image generation unit 32c, a reduced image generation unit 32d, an instruction image generation unit 32e, an operation recognition unit 32f, and a second A position determination unit 32g and a determination unit 32h are included. For each functional unit described above, a functional unit having the same name as that of the projection apparatus 100 performs the same process, and thus detailed description thereof is omitted. In addition to the configuration shown in FIG. 19, the imaging device 10, the projection device 20, and the information processing device 30 may be realized as an integrated device, and the combination thereof can be changed.

  In the above embodiment, the case where the imaging region has the same size as the projection image has been described. Depending on the distance between the projection plane and the imaging device 10, there is a possibility that the imaging area becomes a part of the projection image. In such a case, an area that is frequently operated by the user may be picked up so that only the area within the reach of the user's hand below the projection plane (for example, the lower left) can be an imaging area. As a result, the above-described image generation processing can be realized even when the distance between the projection surface and the imaging device 10 is short and the focal point is short.

  FIG. 20 is a block diagram illustrating a hardware configuration example of the projection apparatus 100. As illustrated in FIG. 20, the projection apparatus 100 includes a CPU 1010, a memory controller 1020, a main memory 1030, and a host-PCI bridge 1040. The memory controller 1020 is connected to the CPU 1010, the main memory 1030, and the host-PCI bridge 1040 via the host bus 1110.

  A CPU 1010 performs overall control of the projection apparatus 100. The memory controller 1020 controls reading / writing and the like with respect to the main memory 1030. The main memory 1030 is a system memory used as a memory for storing programs and data, a memory for developing programs and data, a memory for drawing, and the like.

  The host-PCI bridge 1040 is a bridge for connecting peripheral devices and PCI (Peripheral Component Interconnect) devices 1050. The host-PCI bridge 1040 is connected to the memory card 1060 via the HDD I / F 1120. The host-PCI bridge 1040 is connected to the PCI device 1050 via the PCI bus 1130. The host-PCI bridge 1040 is connected to the communication card 1070, the wireless communication card 1080, the video card 1090, and the like via the PCI bus 1130 and the PCI slot 1140.

  The memory card 1060 is used as an OS boot device. The communication card 1070 and the wireless communication card 1080 are used for connection to a network such as a LAN or a communication line. The video card 1090 is used for image projection and outputs a video signal to a display output. The program executed by the projection apparatus 100 is provided by being incorporated in advance in a storage memory of the main memory 1030.

(3) Program The projection program executed by the projection apparatus 100 is, as one form, a file in an installable format or an executable format as a CD-ROM, flexible disk (FD), CD-R, DVD ( And recorded on a computer-readable recording medium such as a digital versatile disk. The projection program executed by the projection apparatus 100 may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the projection program executed by the projection apparatus 100 may be provided or distributed via a network such as the Internet. Further, the projection program may be provided by being incorporated in advance in a ROM or the like.

  The projection program executed by the projection apparatus 100 includes the above-described units (the part detection unit 121, the first position determination unit 122, the projection image generation unit 123, the reduced image generation unit 124, the instruction image generation unit 125, the projection unit 126, and the operation. The module includes a recognition unit 127, a second position determination unit 128, and a determination unit 129). As actual hardware, a CPU (processor) reads out and executes a projection program from a storage medium, whereby each of the above-described units. Are loaded on the main storage device, and the part detection unit 121, the first position determination unit 122, the projection image generation unit 123, the reduced image generation unit 124, the instruction image generation unit 125, the projection unit 126, the operation recognition unit 127, the second A position determination unit 128 and a determination unit 129 are generated on the main storage device.

DESCRIPTION OF SYMBOLS 100 Projection apparatus 110 Memory | storage part 111 Projection image memory | storage part 112 Captured image memory | storage part 113 Coordinate memory | storage part 120 Control part 121 Site | part detection part 122 1st position determination part 123 Projection image generation part 124 Reduced image generation part 125 Instruction image generation part 126 Projection Unit 127 operation recognition unit 128 second position determination unit 129 determination unit

JP 2009-282737 A

Claims (12)

An imaging unit that captures the first image projected on the projection object;
A part detection unit that detects a predetermined part of the user based on a captured image captured by the imaging unit;
A first position determination unit that determines a position at which a reduced image of the first image is arranged based on detection by the part detection unit;
A projection unit for projecting a second image in which the reduced image is arranged at a position determined by the first position determination unit;
An instruction operation detection unit that detects an instruction operation of the user with respect to the second image projected by the projection unit based on a captured image captured by the imaging unit;
A second position determination unit that determines a position where the instruction image is to be arranged based on detection by the instruction operation detection unit;
A projection control unit that controls projection of an image by the projection unit based on a comparison between the position determined by the second position determination unit and the position of the reduced image in the second image. Projection system.   The second position determination unit determines the position of the instruction image converted from the coordinate system of the reduced image to the coordinate system of the first image when the instruction operation is detected. Item 4. The projection system according to Item 1.   The projection system according to claim 1, wherein the projection control unit controls projection of an image in which the reduced image that has been transmitted is arranged when the instruction image and the reduced image overlap each other.   When the instruction image and the reduced image overlap, the projection control unit projects an image in which a frame image representing an outer frame of an arrangement region in which the reduced image is arranged is arranged without arranging the reduced image. The projection system according to claim 1, wherein the projection system is controlled.   5. The projection system according to claim 3, wherein the projection control unit controls projection of an image obtained by changing a display method of the instruction image when the instruction image and the reduced image overlap each other. The part detection unit outputs that the predetermined part is not detected from the state where the predetermined part is detected,
The projection system according to claim 1, wherein the projection control unit controls projection of an image in which the reduced image is not arranged when the predetermined part is not detected. The part detection unit detects the predetermined part of a plurality of users,
The first position determination unit determines a position where a plurality of reduced images are arranged according to each position where the predetermined site of a plurality of users is detected,
The instruction operation detection unit detects the instruction operation of a plurality of users,
The second position determination unit determines a position where a plurality of the instruction images are arranged,
The projection control unit controls projection of an image by the projection unit based on a comparison between a plurality of positions determined by the second position determination unit and positions of the plurality of reduced images in the second image. The projection system according to claim 1, wherein:   The projection system according to claim 7, wherein the projection control unit controls projection of an image in which the reduced image having a different outer frame color is arranged for each user.   The projection system according to claim 7, wherein the projection control unit controls projection of an image in which the instruction image arranged in a display method different for each user is arranged. A part detection unit that detects a predetermined part of the user based on a captured image obtained by capturing the first image projected on the projection target;
A first position determination unit that determines a position at which a reduced image of the first image is arranged based on detection by the part detection unit;
A projection unit for projecting a second image in which the reduced image is arranged at a position determined by the first position determination unit;
An instruction operation detector that detects an instruction operation of a user with respect to the second image projected by the projector based on the captured image;
A second position determination unit that determines a position where the instruction image is to be arranged based on detection by the instruction operation detection unit;
A projection control unit that controls projection of an image by the projection unit based on a comparison between the position determined by the second position determination unit and the position of the reduced image in the second image. Projection device. A part detecting step for detecting a predetermined part of the user based on a captured image obtained by capturing the first image projected on the projection target;
A first position determining step for determining a position where a reduced image of the first image is arranged based on detection by the part detecting step;
A projecting step of projecting a second image in which the reduced image is arranged at the position determined by the first position determining step;
An instruction operation detecting step of detecting an instruction operation of a user with respect to the second image projected by the projecting step based on the captured image;
A second position determining step for determining a position where the instruction image is to be arranged based on the detection by the instruction operation detecting step;
Causing the computer to execute a projection control step of controlling projection of the image by the projection step based on a comparison between the position determined by the second position determination step and the position of the reduced image in the second image. Projection program for. A part detecting step for detecting a predetermined part of the user based on a captured image obtained by capturing the first image projected on the projection target;
A first position determining step for determining a position where a reduced image of the first image is arranged based on detection by the part detecting step;
A projecting step of projecting a second image in which the reduced image is arranged at the position determined by the first position determining step;
An instruction operation detecting step of detecting an instruction operation of a user with respect to the second image projected by the projecting step based on the captured image;
A second position determining step for determining a position where the instruction image is to be arranged based on the detection by the instruction operation detecting step;
A projection control step for controlling projection of the image by the projection step based on a comparison between the position determined by the second position determination step and the position of the reduced image in the second image. A characteristic projection method.

Images