JP2010151997A - Presentation system and program for the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a presentation system that projects, on a presenter's eye using a head-mounted display, an auxiliary image according to the description of a presenter among auxiliary images corresponding to a presentation image displayed by a display. <P>SOLUTION: The presentation system includes: a storage means 16 storing presentation data including a first display information for displaying the presentation image and a second display information for displaying an associated auxiliary image in a region within the presentation image; and the display 15 for displaying the presentation image based on the first display information. Based on the coordinates of a region designated by a user in a region on the displayed presentation image, an auxiliary image associated with the designated region is displayed on the head-mounted display 13 by reading the second display information from the storage means 16. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technology of a presentation system and its program. Specifically, a presentation system using a see-through head mounted display that projects image light according to display information onto a user's eyes while allowing external light to pass through and allows the user to visually recognize an image according to the image light. And the technology of the program.

  In recent years, in various conferences and product exhibitions, a computer and a display device (for example, a projector or a large display) are connected, and an image corresponding to an image signal output from the computer is displayed on the display device. More and more presentations are being made to a large audience.

  In this type of presentation, the presenter looks at the script and related materials (hereinafter referred to as supplementary materials) at hand, while pointing to the relevant part of the presentation image displayed on a display device such as a projector with a laser pointer or the like. Often the presentation progresses.

  However, in presentations while viewing supporting materials at hand, the presenter often has to stay in a certain place, and the movement of the presenter is limited. There is also the problem of having to keep an eye on the audience when it is necessary to check supporting materials.

Therefore, a head-mounted display is proposed that can project an auxiliary material image corresponding to the presentation image displayed on the display device to the presenter's eye so that the auxiliary material can be visually recognized according to the progress of the presentation. (See Patent Document 1).
JP 2002-165156 A

  However, in the said patent document 1, an auxiliary material is displayed for every presentation image displayed with the display apparatus. Therefore, it takes time and effort to search from the auxiliary material displaying the location corresponding to the presentation image currently being explained.

  For example, when a presentation image including a large number of charts is projected on a screen, auxiliary materials corresponding to the charts are displayed at a time on the head mounted display. Therefore, the presenter must find out the explanation part of the chart to be explained from many auxiliary materials, and there is a possibility that the presentation cannot be performed smoothly.

  Therefore, the present invention provides a head-mounted display to the presenter's eyes as an auxiliary image for presenting the presenter with necessary explanatory material among the explanatory images of the presentation image displayed on the display device. An object is to provide a presentation system capable of projecting.

  In order to solve the above-described problem, in the invention described in claim 1, image light corresponding to display information is projected on the user's eyes while transmitting external light, and an image corresponding to the image light is displayed to the user. In a presentation system having a see-through type head-mounted display for visual recognition, first display information for displaying a presentation image, and second display information for displaying an auxiliary image associated with an area in the presentation image Storage means for storing presentation data including: a display device for displaying a presentation image based on the first display information; and a region designated by a user among regions on the presentation image displayed on the display device Detection means for detecting the coordinates of the A selection to select second display information for displaying the auxiliary image associated with the designated area when it is determined that the area associated with the auxiliary image is designated by the user, And a display control unit that reads the second display information selected by the selection unit from the storage unit and displays the second display information on the head mounted display.

  According to a second aspect of the present invention, in the presentation system according to the first aspect, the detection means uses the coordinates of the area in which the designated time is equal to or longer than a predetermined time as coordinates of the designated area. It was decided to detect as.

  According to a third aspect of the present invention, in the presentation system according to the first or second aspect, the detection means uses the indication of the coordinates of an area where the designated number of times is a predetermined number or more per unit time. It was decided to detect it as the coordinates of the specified area.

  According to a fourth aspect of the present invention, in the presentation system according to any one of the first to third aspects, the display device is a projector that projects a presentation image based on the first display information. It was.

  According to a fifth aspect of the present invention, in the presentation system according to the fourth aspect of the present invention, the presentation system includes an imaging unit that is fixed to the head mounted display and images the front of the user, and the detection unit outputs from the imaging unit. Based on the captured image data, the coordinates of the designated area are detected.

  In the invention described in claim 6, the computer is a program that functions as each unit of the presentation system described in any one of claims 1 to 5.

  According to the first and sixth aspects of the invention, the detecting unit detects the coordinates of the region designated by the user among the regions on the presentation image, and the selecting unit instructs the region associated with the auxiliary image by the user. If it is determined that the auxiliary image is determined, the second display information for displaying the auxiliary image associated with the designated area is selected. Accordingly, among the explanatory materials of the presentation image displayed on the display device, the explanatory material required by the presenter can be projected on the user's eyes as an auxiliary image for presenting the presenter to the presenter. A presentation system can be provided.

  According to the second aspect of the present invention, since the coordinates of the area where the time designated by the user is equal to or longer than the predetermined time are detected, the undesired area is instantaneously pointed by the user's hand shake or the like. Even in such a case, it is possible to prevent an auxiliary image in an area not desired by the user from being unexpectedly displayed on the head mounted display.

  According to the third aspect of the present invention, in the presentation system according to the first or second aspect, the coordinates of the area where the number of times designated by the user is equal to or greater than the predetermined number per unit time are detected.

  This effect will be described with an example. During the presentation, there are cases where the indicated location is moved back and forth depending on the convenience of the explanation given by the user. In such a case, generally, a desired region, that is, a region serving as the center of the description is mainly designated, and other regions are appropriately designated. Then, the number of times a desired area is designated is greater than the number of times other areas are designated. Therefore, with the configuration described in claim 3 described above, even when the user's desired area and the outside of the same area are alternately indicated multiple times, the auxiliary image associated with the user's desired area is displayed. It can be displayed on a head mounted display.

  According to the fourth aspect of the invention, since the display device is a projector, it is possible to give a presentation to many viewers.

  According to the fifth aspect of the present invention, the coordinates of the area designated by the user are detected based on the captured image data output from the imaging unit fixed to the head mounted display. When the user is looking at the projected image, the auxiliary image can be displayed on the head mounted display.

  The present invention is a see-through type in which image light corresponding to display information is projected on the eyes of a presenter (hereinafter also referred to as a user) while allowing the user to visually recognize an image corresponding to the image light while transmitting external light. The present invention relates to a presentation system provided with a head mounted display (hereinafter also referred to as HMD).

  In particular, the presentation system according to the present embodiment stores an auxiliary image for presenting the explanatory material of the presentation image to the user, and the user of the presentation system draws the attention of the audience on the presentation image. When the set predetermined area is designated, only the auxiliary image associated with the part is displayed on the HMD worn by the user.

  That is, when there are multiple charts on the presentation image, each chart is divided into areas, and areas are set. By associating auxiliary images according to each chart for each area, the user can select a chart. Is displayed, only the auxiliary image associated with the instructed chart is displayed on the HMD.

  Therefore, the user can recognize the auxiliary image corresponding to the chart displayed in the area described by the user without being confused with the auxiliary image related to the chart in the other area.

  Hereinafter, the presentation system according to the present embodiment will be described with reference to the drawings.

[Outline of the Invention]
First, the outline of the presentation system A according to the present embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an audience field of view and a user field of view in a state where the presentation system A is actually used. In the present embodiment, as a presentation method, a description is given by taking as an example a style in which the user P explains the presentation image 11 projected on the large screen 12 by the projector while pointing with the laser pointer 10.

  As shown in FIG. 1A, the user P is presenting to the audience while holding the laser pointer 10 as an indicator and pointing to the presentation image 11 on the screen 12.

  At this time, the user P wears an HMD 13 to be described in detail later, and is in a state in which an auxiliary image that only the user P can visually recognize can be viewed.

  That is, when the user P directs the point light 27 to the area C1 on the presentation image 11 by using the laser pointer 10, the user P visually recognizes the auxiliary image H1 with the HMD 13, as shown in FIG. It will be.

  Here, the auxiliary image H1 is an auxiliary image associated with the area C1 designated by the laser pointer 10, and a description of a bar graph in the area C1 is displayed in the auxiliary image H1. Note that the auxiliary image in the present embodiment is a memorandum previously written by the user P before giving a presentation.

  Next, as shown in FIG. 1C, when the user P designates the area C2 with the laser pointer 10, the auxiliary image H2 is displayed by the HMD 13 as shown in FIG. The auxiliary image of the inner bar graph, that is, the memorandum will be visually recognized.

  At this time, the auxiliary image H2 is displayed on the left side in FIG. 1D and is displayed so as not to overlap the area C2 in the field of view of the user P.

  Next, as shown in FIG. 1 (e), the user P changes its position on the left side of the screen 12 when viewed from the audience's field of view, and designates the region C 3 by the laser pointer 10.

  Then, in the field of view of the user P, the auxiliary image H3 describing the explanation of the pie chart in the region C3 is displayed at a position that does not overlap with the region C3.

  As described above, according to the presentation system A according to the present embodiment, an auxiliary image corresponding to the description of the user P can be projected onto the eyes of the user P by the HMD 13 and displayed in the area described by the user. It is possible to recognize an auxiliary image corresponding to the chart being displayed.

  In addition, since the auxiliary image is displayed in an area other than the chart instructed by the laser pointer 10, it is possible to prevent the instructed chart and the auxiliary image from overlapping and making these images difficult to view.

[Configuration of the presentation system]
Next, a specific configuration of the presentation system A according to the present embodiment will be described.

  As shown in FIG. 2, the presentation system A according to the present embodiment is abbreviated to a personal computer (hereinafter referred to as a personal computer) 14, a screen 12, a projector 15, an HMD 13, and a controller 20 (described later) of the HMD 13. The display control unit 58 is connected so as to be integrated.

  The personal computer 14 is installed with a presentation program for displaying and creating the presentation image 11, and a hard disk drive (HDD) 16 built in the personal computer 14 has a first display for displaying the presentation image 11. Presentation data including information and second display information for displaying an auxiliary image associated with each region in the presentation image is stored. That is, the HDD 16 of the personal computer 14 functions as storage means for storing presentation data.

  In the present embodiment, the first display information is a presentation data file executed by the presentation program, and the second display information is data for displaying an auxiliary image in the HMD 13, such as a text file, JPEG, or the like. It can be interpreted as an image file.

  The projector 15 functions as a display device for the presentation image 11 based on the first display information.

  The projector 15 is electrically connected to the personal computer 14 via the projector connection cable 18. The projector 15 projects an image based on the image signal output from the personal computer 14 onto the screen 12, thereby giving a presentation image 11 to the audience. Can be displayed.

  Specifically, an image signal is generated by an image output circuit (not shown) built in the personal computer 14 based on the presentation data file described above, and this image signal is sent from the projector 15 via the projector connection cable 18. The presentation image 11 is displayed by receiving it.

  The HMD 13 is for allowing the user P to visually recognize the auxiliary image based on the second display information, and is electrically connected to the personal computer 14 via the display control unit 58 by the HMD connection cable 19.

  In the present embodiment, the HMD 13 includes a controller 20 that forms light whose intensity is modulated for each pixel of the auxiliary image (hereinafter referred to as auxiliary image forming light) based on the second display information transmitted from the personal computer 14; A transmission cable unit 22 having an optical fiber cable for transmitting auxiliary image forming light emitted from the controller 20, and a head-mounted unit for scanning the transmitted auxiliary image forming light and causing the user P to recognize the auxiliary image And 23. The transmission cable unit 22 also has an image signal transmission cable for transmitting an image signal (hereinafter referred to as a captured image signal) from the CCD camera 21 described later to the display control unit 58.

  As shown in FIG. 3, a display control unit 58 is connected to the controller 20. The display control unit 58 is provided with an external input / output terminal 31. By connecting the above-described HMD connection cable 19, a second display can be performed between the display control unit 58 or the controller 20 and the personal computer 14. Information and various control signals can be transmitted and received.

  The head mounting unit 23 includes an image forming unit 25 that scans the auxiliary image forming light transmitted by the support member 24 having a substantially glasses shape and the transmission cable unit 22 so that the user P can recognize the auxiliary image as an auxiliary image. And.

  The image forming unit 25 is provided with a half mirror 26 in front of the user P's eyes. The external light La is transmitted through the half mirror 26 to enter the eyes Y of the user P, and auxiliary image formation according to the second display information is performed. The light Lb is reflected by the half mirror 26 and is incident on the eyes of the user P. At this time, since the half mirror also transmits external light, the user P can also see the external world while viewing the image light. The image forming unit 25 enters light (hereinafter referred to as image light) obtained by scanning the auxiliary image forming light whose intensity is modulated for each color (R, G, B) in the two-dimensional direction into the eyes of the user P, and This is a retinal scanning display that allows the user P to visually recognize an auxiliary image corresponding to the second display information by scanning auxiliary image forming light in a two-dimensional direction on the retina of the eye of P. The image forming unit 25 is not a retinal scanning display, but a display that projects image light generated by transmission or reflection through a liquid crystal display (LCD) to the eyes of the user P and allows the user P to visually recognize the content. May be. At this time, the image forming unit 25 incorporates an LCD, a light source, etc., and outputs an image signal from the controller 20 to display an image corresponding to the image signal on the LCD. The light from the light source is incident on the LCD and transmitted. Alternatively, image light is generated by reflection.

  As described above, the HMD 13 is a see-through type HMD that projects the auxiliary image forming light onto the eyes of the user P while transmitting external light.

  Further, the head mounting unit 23 is provided with a CCD camera 21 for imaging on the support member 24 so that when the user P wears the head mounting unit 23, the front of the user P can be imaged. Yes. That is, the CCD camera 21 is configured to sample at least a part of the image of the field of view of the user P, and functions as an imaging unit that images the front of the user P.

  The captured image signal generated by the CCD camera 21 is transmitted to the controller 20 via the transmission cable unit 22.

[Electrical configuration and optical configuration of HMD 13 and display control unit 58]
Next, the electrical configuration and optical configuration of the HMD 13 and the display control unit 58 worn by the user P will be described with reference to FIG.

  As described above, the controller 20 of the HMD 13 is provided with the display control unit 58 for generating an image signal based on the second display information supplied from the personal computer 14 and emitting auxiliary image forming light. .

  The display control unit 58 is provided with a control unit 30, and executes a predetermined process described later based on a program stored therein, thereby detecting, selecting means, display control means, etc. described later. Function. The configuration of the control unit 30 and the operation as each means will be described in detail later. The program may be recorded on a recording medium such as a CD-R or DVD-R and then read into the control unit 30 via a recording medium drive (not shown).

  The controller 20 is provided with an image signal supply circuit 34. The image signal supply circuit 34 generates each signal as an element for synthesizing the auxiliary image based on the auxiliary image forming signal S generated by the control unit 30 of the display control unit 58. That is, the image signal supply circuit 34 outputs an R (red) image signal 35r, a G (green) image signal 35g, a B (blue) image signal 35b, a horizontal synchronization signal 36, and a vertical synchronization signal 37.

  Further, the controller 20 emits laser light whose intensity is modulated based on the image signals of the R image signal 35r, the G image signal 35g, and the B image signal 35b, respectively. An R laser driver 41, a G laser driver 42, and a B laser driver 43 for driving the B laser 40 are provided.

  Furthermore, it comprises a collimating optical system 44 provided so as to collimate the laser light emitted from each laser into parallel light, a dichroic mirror 45 for combining the collimated laser light, and the combined laser light. A coupling optical system 46 that guides image light, that is, auxiliary image forming light to the transmission cable unit 22 is provided.

  As the R laser 38, G laser 39, and B laser 40, a semiconductor laser such as a laser diode or a solid laser may be used.

  The HMD 13 scans the collimated auxiliary image forming light in the horizontal direction using the optical scanning element 49 and the collimating optical system 48 for guiding the auxiliary image forming light propagated from the controller 20 to the horizontal scanning system 47. A horizontal scanning system 47, a first relay optical system 51 for guiding auxiliary image forming light scanned by the horizontal scanning system 47 to the vertical scanning system 50, and a scanning by the horizontal scanning system 47, via the first relay optical system 51. A vertical scanning system 50 that scans the incident auxiliary image forming light in the vertical direction using the galvano mirror 52 and the auxiliary image forming light scanned by the vertical scanning system 50 to the pupil 53 of the eye Y of the user P A second relay optical system 54 (projection optical system) and a half mirror 26 are provided so as to be incident.

  The auxiliary image forming light that is two-dimensionally scanned, reflected by the half mirror 26 and incident on the pupil 53 of the user P is projected onto the retina 55, whereby the user P recognizes the auxiliary image forming light as an auxiliary image. Can do.

  The horizontal scanning system 47 includes an optical scanning element 49 that scans the auxiliary image forming light in the horizontal direction, and a horizontal scanning control circuit 56 that controls driving of the optical scanning element 49, and each scanning line of an image to be displayed. And an optical system for horizontally scanning the auxiliary image forming light in the horizontal direction.

  On the other hand, the vertical scanning system 50 includes a galvano mirror 52 that performs vertical scanning, and a vertical scanning control circuit 57 that performs drive control of the galvano mirror 52, and auxiliary image forming light for each frame of an image to be displayed. Is an optical system that performs vertical scanning vertically from the first scanning line toward the last scanning line.

  The horizontal scanning system 47 is designed to scan the auxiliary image forming light at a higher speed than the vertical scanning system 50, that is, at a high frequency. Further, as shown in FIG. 4, the horizontal scanning system 47 and the vertical scanning system 50 are respectively connected to the image signal supply circuit 34, and are respectively supplied to the horizontal synchronization signal 36 and the vertical synchronization signal 37 output from the image signal supply circuit 34. The auxiliary image forming light is scanned synchronously.

(Configuration of control unit 30)
Next, the configuration of the control unit 30 provided in the display control unit 58 will be described with reference to FIG.

  The control unit 30 includes a CPU 60, a ROM 61, a RAM 63, a CCD camera interface 64, a CCD camera VRAM 65, an image signal supply circuit interface 66, an image signal supply circuit VRAM 67, a personal computer connection interface 68, A peripheral device interface 69 is provided and is connected to each other via a system bus 70.

  The ROM 61 stores a program for realizing processing according to a flowchart described later by being executed by the CPU 60.

  The RAM 63 functions as a temporary storage area such as an auxiliary image association table, various flags, and image data that are referred to by the CPU 60.

  Here, the auxiliary image association table will be described with reference to FIG.

  The auxiliary image association table stored in the RAM 63 is generated when the user P creates the presentation data by the presentation program on the personal computer 14. When the display control unit 58 and the HMD 13 are activated, the instruction of the CPU 60 is generated. Thus, it is read into the RAM 63 from the personal computer 14.

  As shown in FIG. 6, this auxiliary image association table associates each presentation image over a plurality of pages with the set number of auxiliary images, the file name of the auxiliary image, and the coordinate area of the auxiliary image.

  That is, with this auxiliary image association table, each chart (each region) on the presentation image 11 is associated with the auxiliary image.

  For example, the auxiliary image associated with the coordinate (X1, Y1)-(X2, Y2) area of the third presentation image is the second display information with the file name “A3.jpg” I understand.

  The CPU 60 can select the second display information associated with the area designated by the user P by referring to the auxiliary image association table.

  Returning to the description of FIG. 5 again, the CCD camera interface 64 is responsible for connection with the CCD camera 21 and converts the captured image signal for each frame transmitted from the CCD camera 21 into captured image data. It plays a role of writing to the VRAM 65 for the CCD camera.

  Further, for example, as shown in FIG. 1B, the CCD camera interface 64 converts the captured image data of the screen 12 captured by the CCD camera 21 from a slightly oblique direction to the captured image data of the screen 12 in the front view. It has the function to convert to.

  An already known method can be applied to the conversion to the captured image data of the front view.

  For example, captured image data obtained by capturing the shape of the screen 12 in the front view is stored in advance in the ROM 61 or the like, and the shape of the screen of the captured image data captured during the presentation is based on the captured image data in the front view. The degree of deformation may be calculated by comparison, and the captured image data captured during the presentation may be converted to a front view by correcting according to the calculated degree of deformation.

  The CCD camera VRAM 65 has a function of storing the image converted into the front view by the CCD camera interface 64 for each frame. As will be described in detail later, the CPU 60 compares the captured image data converted into the front view stored in the CCD camera VRAM 65 with the image data of the presentation image obtained from the personal computer 14, and displays the image on the presentation image 11. The coordinates of the point light 27 are determined.

  The image signal supply circuit interface 66 is responsible for connection with the image signal supply circuit 34, refers to the image signal supply circuit VRAM 67, generates an auxiliary image formation signal S, and supplies it to the image signal supply circuit 34. .

  The image signal supply circuit VRAM 67 stores auxiliary image formation data for forming an auxiliary image visually recognized by the user P in the head-mounted unit 23. As described above, the image signal supply circuit interface 66 However, the auxiliary image forming signal S is supplied to the image signal supply circuit 34 with reference to the image signal supply circuit VRAM 67.

  The personal computer connection interface 68 is responsible for connection with the personal computer 14, and performs transmission / reception of data required by the CPU 60, writing of received data to the image signal supply circuit VRAM 67, and the like. When the page switching process of the presentation program executed on the personal computer 14 is performed, the page switching signal transmitted from the personal computer 14 is received, and the page switching process is performed at a predetermined address of the RAM 63. Set a flag to indicate that. This flag is returned to an initial value indicating that the page switching process is not performed after the CPU 60 refers to it.

  The peripheral device interface 69 is responsible for operation control and signal transmission / reception of peripheral devices connected to the control unit 30. The peripheral device interface 69 is used to activate the display control unit 58 and the HMD 13. A power lamp 33 is connected to display that the power switch 32, the display control unit 58, and the HMD 13 are in an activated state.

  For example, when the power switch 32 is pressed in a state where the display control unit 58 or the HMD 13 is not turned on, the peripheral device interface 69 transmits a signal indicating that the power is turned on to the CPU 60. Receiving the signal, the CPU 60 turns the HMD 13 on. At this time, the peripheral device interface 69 turns on the power lamp 33.

[Processing flow of control unit]
Next, processing in the control unit 30 in the presentation system A will be described with reference to FIGS. FIG. 7 is a flowchart showing main processing of the presentation system A according to the present embodiment, and FIG. 8 is a flowchart showing auxiliary image related processing executed in the main processing.

  First, the main flow will be described with reference to FIG. The CPU 60 of the control unit 30 first executes an initial setting process such as permitting access to the RAM 63 and initializing the work area (step S10). At this time, an auxiliary image related process execution flag is set in the RAM 63, and the value of the auxiliary image related process execution flag is set to “0” which means “auxiliary image related process execution unnecessary”.

  Further, the CPU 60 requests the auxiliary image association table from the personal computer 14 through the personal computer connection interface 68 and stores the obtained auxiliary image association table in the RAM 63.

  Next, the CPU 60 refers to a predetermined address in the RAM 63 and determines whether or not a page switching signal has been received from the personal computer 14 (step S11).

  If the CPU 60 determines that a page switching signal has been received (step S11: Yes), the process proceeds to step S12.

  In step S12, the CPU 60 deletes the data stored in the image signal supply circuit VRAM 67, initializes the image signal supply circuit VRAM 67, and moves the process to step S14.

  On the other hand, if the CPU 60 determines in step S11 that the page switching signal has not been received (step S11: No), the process proceeds to step S13.

  In step S13, the CPU 60 refers to a predetermined address in the RAM 63 and determines whether or not the value of the auxiliary image related process execution flag is “1” meaning “execution of auxiliary image related process”.

  If the CPU 60 determines that the value of the auxiliary image related process execution flag is “1” (step S13: Yes), the process proceeds to step S14.

  In step S14, auxiliary image related processing is executed. In this auxiliary image-related processing, the CPU 60 performs a process for making the user P visually recognize the auxiliary image by making a determination based on a plurality of conditions such as whether or not a predetermined area on the presentation image 11 is instructed by the user P. Do. The auxiliary image related processing will be described in detail later with reference to FIG.

  On the other hand, when the CPU 60 determines that the value of the auxiliary image related process execution flag is not “1” (step S13: No), the process proceeds to step S15.

  In step S15, the CPU 60 determines whether or not there is an input from the power switch 32.

  If the CPU 60 determines that there is an input from the power switch 32 (step S15: Yes), the process proceeds to step S16.

  In step S16, the CCD camera VRAM 65, the image signal supply circuit VRAM 67, and the RAM 63 are initialized, the power lamp 33 is turned off, and the process ends. Note that the CPU 60 monitors the input from the power switch 32 even after this process is finished, and when the power switch 32 is pressed again, the process starts again from step S10 of this flow.

  On the other hand, when the CPU 60 determines that there is no input from the power switch 32 (step S15: No), the process is moved again to step S11.

  Next, the auxiliary image related process executed in step S14 will be described with reference to FIG.

  As shown in FIG. 8, first, the CPU 60 refers to the auxiliary image association table stored in the RAM 63 and confirms the number of auxiliary images set in the presentation image 11 currently displayed by the projector 15. (Step S20).

  Next, the CPU 60 determines whether or not an auxiliary image is set for the currently displayed presentation image 11 (step S21). The CPU 60 determines that the auxiliary image is not set when the number of auxiliary images set in the presentation image 11 is “0”, and the auxiliary image is determined when the number of auxiliary images is other than “0”. Judge that it is set.

  If the CPU 60 determines that an auxiliary image is not set (step S21: No), the process proceeds to step S22.

  In step S22, the CPU 60 sets the value of the auxiliary image related process execution flag stored in the RAM 63 to “0”, and moves the process to step S15.

  On the other hand, if the CPU 60 determines in step S21 that an auxiliary image is set (step S21: Yes), the process proceeds to step S23.

  In step S23, the CPU 60 sets the value of the auxiliary image related process execution flag stored in the RAM 63 to “1”, and moves the process to step S24.

  In step S24, the CPU 60 requests the personal computer connection interface 68 to obtain the image data of the presentation image 11 currently displayed on the screen 12 from the personal computer 14, and writes the obtained image data to the RAM 63. Then, the process proceeds to step S25.

  In step S25, the CPU 60 refers to a predetermined address of the CCD camera VRAM 65, and captures the captured image data for one frame, which is captured by the CCD camera 21 and converted into a front view image by the CCD camera interface 64, into the predetermined address of the RAM 63. make a copy.

  Next, the CPU 60 refers to the captured image data stored on the RAM 63 and determines whether or not the entire presentation image 11 has been captured (step S26).

  If the CPU 60 determines that the entire presentation image 11 has not been picked up (step S26: No), it moves the process to step S27, initializes the image signal supply circuit VRAM 67, and moves to step S15.

  On the other hand, if the CPU 60 determines that the entire presentation image 11 has been captured (step S26: Yes), the process proceeds to step S28.

  In step S28, the CPU 60 compares the image data of the presentation image 11 stored on the RAM 63 with the captured image data, and determines the coordinates of the point light 27 on the presentation image 11 displayed on the screen 12. decide.

  Then, the CPU 60 refers to the auxiliary image association table and determines whether or not the coordinates of the point light 27 determined in step S28 are within the range of the coordinate area set in the auxiliary image association table ( Step S29). As described above, the CPU 60 detects the coordinates of the area designated by the laser pointer 10 by the user P among the areas on the presentation image 11 displayed on the screen 12 by executing the processes of steps S28 and S29. It will function as a detecting means.

  If the CPU 60 determines that the coordinates of the point light 27 are not within the range of the coordinate area (step S29: No), the process proceeds to step S27, where the image signal supply circuit VRAM 67 is initialized and processed. To step S15.

  On the other hand, when the CPU 60 determines that the coordinates of the point light 27 are within the range of the coordinate area (step S29: Yes), the process proceeds to step S30.

  In step S30, the CPU 60 determines whether or not the point light 27 has elapsed for a predetermined time within the same region.

  If the CPU 60 determines that the predetermined time has elapsed (step S30: Yes), the process proceeds to step S32.

  On the other hand, if the CPU 60 determines that the predetermined time has not elapsed (step S30: No), the process proceeds to step S31.

  In step S31, the CPU 60 determines whether or not an area on the presentation image 11 in which the same auxiliary image is set has been instructed more than a predetermined number of times by the point light 27 emitted from the laser pointer 10 by the user P. Make a decision.

  If the CPU 60 determines that a predetermined number of times has been instructed (step S31: Yes), the process proceeds to step S32.

  On the other hand, if the CPU 60 determines that it has not been instructed more than a predetermined number of times (step S31: No), the process proceeds to step S27, the image signal supply circuit VRAM 67 is initialized, and the process proceeds to step S15.

  In step S32, the CPU 60 refers to the auxiliary image association table, determines the file name of the auxiliary image associated with the coordinate area designated by the user, and selects the second display information. That is, when the CPU 60 determines that the area associated with the auxiliary image among the areas of the presentation image 11 is instructed by the user P by executing the processes of steps S30 to S32, the CPU 60 associates the area with the instructed area. It functions as a selection means for selecting the second display information for displaying the auxiliary image.

  Next, the CPU 60 instructs the personal computer connection interface 68 to receive the second display information corresponding to the determined file name from the personal computer 14 (step S33).

  Then, the CPU 60 writes data on the image signal supply circuit VRAM 67 based on the file obtained by the personal computer connection interface 68 (step S34). At this time, the CPU 60 refers to the captured image data, and forms an auxiliary image in the image signal supply circuit VRAM 67 so that the auxiliary image is displayed in a region that does not overlap with the area designated by the user P in the field of view of the user P. Write data. As described above, the CPU 60 functions as a display control unit that reads the second display information selected by the selection unit from the storage unit and displays the second display information on the HMD 13 by executing steps S33 and S34. When this step ends, the CPU 60 moves the process to step S15.

  As described above, the presentation system A according to the present embodiment uses the HMD to display the auxiliary image corresponding to the presenter's explanation among the auxiliary images corresponding to the presentation image displayed on the display device. I am going to project.

[Other Embodiments]
Next, a presentation system B according to another embodiment will be described.

  FIG. 9 is an explanatory diagram showing an overall image of a presentation system B according to another embodiment. In the following description, the same components as those of the above-described presentation system A are denoted by the same reference numerals and description thereof is omitted.

  The presentation system B has substantially the same configuration as the presentation system A described above, but the head mounted unit 23 is not provided with the CCD camera 21, the personal computer 14 is provided with the mouse 80, and the user P Is different in that the region is indicated by operating the mouse 80 and moving the mouse pointer 82 on the presentation image 11 displayed on the display 81. In this presentation system B, the viewer can understand which area the user P is pointing to by viewing the mouse pointer 82 displayed on the screen 12.

  Also with this presentation system B, the user P projects the auxiliary image corresponding to the explanation of the user P among the auxiliary images corresponding to the presentation image 11 displayed on the screen 12 onto the eyes of the user P by the HMD 13. Can do.

  In particular, this presentation system B differs from the presentation system A in the auxiliary image-related processing executed by the control unit 30 of the controller 20.

  Therefore, here, the auxiliary image related processing executed in step S14 of the main flow described above with reference to FIG. 7 in the present presentation system B will be described. The processes in steps S40 to S43, S47, S51, and S52 are the same as the processes in steps S20 to S23, S27, S33, and S34 shown in FIG.

  In step S44, the CPU 60 requests the personal computer connection interface 68 to obtain the coordinates of the mouse pointer 82 on the presentation image 11 displayed on the display 81 from the personal computer 14, and the obtained coordinates of the mouse pointer 82 are obtained. Is written to the RAM 63, and the process proceeds to step S45.

  Next, the CPU 60 refers to the coordinates of the mouse pointer 82 and the coordinate area set in the presentation image 11 currently displayed in the auxiliary image association table (step S45).

  Then, the CPU 60 determines whether the coordinates of the mouse pointer 82 are within the auxiliary coordinate area (step S46). As described above, the CPU 60 detects the coordinates of the area designated by the mouse pointer 82 by the user P among the areas on the presentation image 11 displayed on the screen 12 by executing the processes of steps S45 and S46. It will function as a detecting means.

  If the CPU 60 determines that the coordinates of the mouse pointer 82 are not within the coordinate area (step S46: No), the process proceeds to step S47, where the coordinates of the mouse pointer 82 are within the coordinate area. If it is determined that there is any (step S46: Yes), the process proceeds to step S48.

  In step S48, the CPU 60 determines whether the mouse pointer 82 has passed a predetermined time within the same area.

  When the CPU 60 determines that the predetermined time has elapsed (step S48: Yes), the process proceeds to step S50, and when it is determined that the predetermined time has not elapsed (step S48: No). Moves the process to step S49.

  In step S49, the CPU 60 determines whether or not the user P has instructed by the mouse pointer 82 a region on the presentation image 11 where the same auxiliary image is set more than a predetermined number of times.

  If the CPU 60 determines that a predetermined number of times has been instructed (step S49: Yes), the process proceeds to step S50. If the CPU 60 determines that the predetermined number of times has not been instructed (step S49: No), The process moves to step S47.

  In step S50, the CPU 60 refers to the auxiliary image association table, determines the auxiliary image file associated with the coordinate area designated by the user P with the mouse pointer 82, and selects the second display information. As described above, when the CPU 60 determines that the area associated with the auxiliary image among the areas of the presentation image 11 is instructed by the user P by executing the processes of steps S48 to S50, the CPU 60 associates the area with the instructed area. It functions as a selection means for selecting the second display information for displaying the received auxiliary image. Thereafter, the CPU 60 executes the processes of steps S51 and S52, and returns the process to step S15.

  Thus, the presentation system B according to another embodiment operates according to the flow described above.

  As described above, according to the presentation system A and the presentation system B according to the present embodiment, image light corresponding to display information is projected on the user's eyes while transmitting external light, and the image light is transmitted to the user. In a presentation system having a see-through type head mounted display for visually recognizing an image according to the first display information, first display information for displaying a presentation image and a second image for displaying an auxiliary image associated with an area in the presentation image Storage means for storing presentation data including two display information, a display device for displaying a presentation image based on the first display information, and a user among regions on the presentation image displayed on the display device Detect the coordinates of the area indicated by When it is determined based on the detection means and the detection result of the detection means that the area associated with the auxiliary image is designated by the user, the auxiliary image associated with the designated area is selected. Selecting means for selecting second display information for display; and display control means for reading the second display information selected by the selecting means from the storage means and displaying the second display information on the head mounted display. Therefore, a presentation system capable of projecting an auxiliary image corresponding to the description of the user (presenter) among the auxiliary images corresponding to the presentation image displayed on the display device onto the user's eyes by a head mounted display (HMD). Can be provided.

  Finally, the description of each embodiment described above is an example of the present invention, and the present invention is not limited to the above-described embodiment. For this reason, it is a matter of course that various modifications can be made in accordance with the design and the like as long as they do not depart from the technical idea according to the present invention other than the embodiments described above.

  For example, in the presentation system B according to the other embodiment described above, the coordinates of the mouse pointer 82 are directly read from the personal computer 14, but the present invention is not limited to this. The unit 23 includes a CCD camera 21, and the coordinates of the mouse pointer 82 are calculated by capturing the presentation image 11 displayed on the screen 12 and the display 81 and recognizing the shape of the mouse pointer 82. May be.

  In the presentation system A and the presentation system B, the storage means for storing the presentation data uses the HDD 16 provided in the personal computer 14, but the installation location of the storage means is not limited to this. For example, the presentation data may be stored in an external connection type EEPROM such as a memory card, and this EEPROM may be inserted into a slot of the display control unit 58 to function as a storage unit.

It is explanatory drawing which showed the visual field of the audience in the state which actually uses the presentation system which concerns on this embodiment, and a user's visual field. It is explanatory drawing which showed the specific structure of the presentation system which concerns on this embodiment. It is explanatory drawing which showed the structure of the head mounted display. It is the block diagram which showed the electrical structure and optical structure of the head mounted display. It is the block diagram which showed the structure of the control part. It is explanatory drawing of an auxiliary image correlation table. It is a flowchart which shows the operation | movement of the process performed in the presentation system which concerns on this embodiment. It is a flowchart which shows the operation | movement of the process performed in the presentation system which concerns on this embodiment. It is explanatory drawing which showed the specific structure of the presentation system which concerns on other embodiment. It is a flowchart which shows the operation | movement of the process performed in the presentation system which concerns on other embodiment.

Explanation of symbols

10 Laser pointer
11 Presentation image
12 screen
13 Head mounted display (HMD)
14 PC
15 Projector
16 Hard disk drive (HDD)
18 Projector connection cable
19 HMD connection cable
20 Controller
21 CCD camera
22 Transmission cable
23 Head mounted unit
24 Support member
25 Image forming section
26 Half mirror
27 point light
30 Control unit
31 External input / output terminals
32 Power switch
33 Power indicator
34 Image signal supply circuit
35r R (red) image signal
35g G (green) image signal
35b B (blue) image signal
36 Horizontal sync signal
37 Vertical sync signal
38 R laser
39 G laser
40 B laser
41 R laser driver
42 G laser driver
43 B laser driver
44 Collimating optics
45 Dichroic mirror
46 Coupled optics
47 Horizontal scanning system
48 Collimating optics
49 Optical scanning element
50 Vertical scanning system
51 First relay optical system
52 Galvano mirror
53 pupil
54 Second relay optical system (projection optical system)
55 Retina
56 Horizontal scan control circuit
57 Vertical scan control circuit
58 Display control unit
58 Display control unit
60 CPU
61 ROM
63 RAM
64 CCD camera interface
65 VRAM for CCD camera
66 Interface for image signal supply circuit
67 VRAM for image signal supply circuit
68 Interface for PC connection
69 Interface for peripheral devices
70 System bus
80 mice
81 display
82 Mouse pointer A Presentation system B Presentation system C1 area C2 area C3 area H1 auxiliary image H2 auxiliary image H3 auxiliary image La outside light Lb auxiliary image forming light P user

Claims (6)

In a presentation system having a see-through type head-mounted display that projects external light and projects image light according to display information onto the user's eyes and allows the user to visually recognize an image according to the image light.
Storage means for storing presentation data including first display information for displaying a presentation image and second display information for displaying an auxiliary image associated with an area in the presentation image;
A display device for displaying a presentation image based on the first display information;
Detecting means for detecting coordinates of a region designated by a user among regions on the presentation image displayed on the display device;
Based on the detection result of the detection means, when it is determined that the region associated with the auxiliary image is designated by the user among the regions of the presentation image, the auxiliary image associated with the designated region is displayed. Selecting means for selecting second display information;
A display system comprising: display control means for reading the second display information selected by the selection means from the storage means and displaying the second display information on the head mounted display.   The presentation system according to claim 1, wherein the detection unit detects coordinates of an area where the instructed time is a predetermined time or more as coordinates of the instructed area.   3. The presentation system according to claim 1, wherein the detection unit detects coordinates of a region where the designated number of times is equal to or greater than a predetermined number of times per unit time as coordinates of the designated region.   The presentation system according to any one of claims 1 to 3, wherein the display device is a projector that projects a presentation image based on the first display information. An imaging unit fixed to the head mounted display and imaging the front of the user,
The presentation system according to claim 4, wherein the detection unit detects coordinates of the designated area based on captured image data output from the imaging unit.   A program for causing a computer to function as each unit of the presentation system according to any one of claims 1 to 5.