JP2010224207A - Visual stimulus applying device, and biological information measurement system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve a stimulus effect by adding dynamic properties to visual stimulus peculiar to a projection image, and to prevent the occurrence of failure that a projected image is distorted or the projection image is not reflected on a screen because it is hidden behind a subject when projecting an image from a projector even if the projector or the screen is moved. <P>SOLUTION: The visual stimulus applying device which applies the visual stimulus to the subject OB includes: a projection device 150 projecting the image that is the visual stimulus; devices to be projected 160, 161, 162 and 163 each having the screen to which the image is projected from the projection device; and moving mechanisms 151 and 165 moving at least either the projection device or the device to be projected so that two-dimensional positional relationship in a horizontal plane among the subject, the projection device and the screen may change. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a visual stimulus applying device, a biological information measuring system, and the like.

  Environmental stimuli such as lighting, sound, and image presentation, biological responses such as blood pressure, electrocardiogram, pulse wave, electroencephalogram, and cerebral blood flow of the subject to the environmental stimulus, and subjective reports on how the subject feels A system for controlling and measuring in an integrated manner is disclosed in Patent Document 1. The system includes a stimulus presentation unit that presents stimuli that generate the motivation of the subject's behavior, a physiological response data collection unit that collects physiological response data of the subject, and a subjective amount / collection of subject's subjective amount data and behavior amount data. An action amount data collection unit. Then, based on at least one of the physiological response data, the subjective amount data, the behavior amount data, and the stimulus, the psychosomatic state of the subject is evaluated by the control calculation unit.

JP 2007-167105 A

  In the above system, as a visual stimulus that is one of the environmental stimuli given to the subject, an image-specific visual stimulus can be given to the subject by projecting an image from the projector onto the screen. However, simply projecting an image on the screen has limitations as a visual stimulus and lacks dynamicity.

  According to some aspects of the present invention, it is possible to improve the stimulation effect by adding dynamics to the visual stimulus unique to the projection image. In addition, according to some aspects of the present invention, even if the projector or the screen is moved, the projected image is distorted or the projected image is behind the subject when the image is projected from the projector. It is possible to prevent problems that are not projected on the screen.

  One aspect of the present invention is a visual stimulus imparting device that imparts a visual stimulus to a subject, the projection device projecting an image serving as the visual stimulus, and a screen on which the image is projected from the projection device A moving mechanism that moves at least one of the projection device and the projection target device so that a two-dimensional positional relationship of the subject, the projection device, and the screen in a horizontal plane changes. It relates to the visual stimulus applying device.

  According to one aspect of the present invention, the two-dimensional positional relationship of the subject, the projection device, and the screen in the horizontal plane is changed by moving at least one of the projection device and the projection target device. Thereby, even if the same image is projected on the screen from the projection apparatus, the visual stimulation effect can be dynamically changed by changing the above-described two-dimensional position.

  At this time, in one aspect of the present invention, a seating portion on which the subject sits is provided, and the moving mechanism is arranged along the direct viewing direction when the subject seated on the seating portion faces the screen. A projection device moving mechanism for moving the projection device in a space above the seating portion may be included.

  In this way, the distance between the projection device and the screen changes, the size of the image projected on the screen changes, and the visual stimulus changes. As another significance of moving the projection device, the projection device moving mechanism can move the projection device so that the visual point corresponding to the eye position of the subject is outside the projection area of the projection device.

  Further, in one aspect of the present invention, the screen of the projection apparatus includes a wide-angle screen that partitions a surrounding space surrounding the subject at a wide angle of 180 ° or more around the subject seated on the seat portion, An omnidirectional screen that divides a surrounding space surrounding the subject at all angles of 360 ° centering on the subject seated on the seating portion may be provided. In this case, the projection device projects a wide-angle image or an omnidirectional image on the wide-angle screen or the omnidirectional screen of the projection target device.

  In this way, the image projected on the screen by the movable projection device becomes a wide-angle image or an omnidirectional image, and the presence of the change in the visual stimulus by the projection device is enhanced.

  Further, according to one aspect of the present invention, it may further include a seating portion moving mechanism that moves the seating portion along the direct viewing direction.

  In this way, the visual stimulus can be changed by changing the distance between the seating portion and the screen, and the relative positional relationship between the seating portion and the projection device is changed, corresponding to the eye position of the subject. Since the frequency with which the visual point is set in the projection area of the projection apparatus increases, the significance of moving the projection apparatus is also increased. The seating part can also have a reclining mechanism. By changing the reclining angle, it is possible to give a feeling of openness and pressure, and depending on the reclining angle, the frequency of the visual point corresponding to the eye position of the subject being set in the projection area of the projection device increases. In addition, the significance of moving the projection apparatus increases.

  Further, in one aspect of the present invention, a seating unit on which the subject sits is provided, and the moving mechanism is configured so that the subject in the direct view direction when the subject seated on the seating unit faces the screen and the subject A screen moving mechanism for moving the screen of the projection apparatus so as to change at least the distance to the screen can be included.

  In this way, the distance between the projection device and the screen changes, the size of the image projected on the screen changes, and the visual stimulus changes. Further, since the distance between the subject and the screen changes, the feeling of opening, pressing, and blocking the subject's front changes as compared with the case where only the projector is moved, and the visual stimulation effect is further enhanced.

  Further, in one aspect of the present invention, the movable screen is divided into a wide-angle screen that partitions a surrounding space surrounding the subject at a wide angle of 180 ° or more around the subject seated on the seat, and the seating portion The omnidirectional screen can define a surrounding space surrounding the subject at all angles of 360 [deg.] With the subject sitting on the center. In this case, the projection device projects a wide-angle image or an omnidirectional image on the wide-angle screen or the omnidirectional screen of the projection target device.

  In this way, the image projected on the moving screen whose distance from the subject is variable becomes a wide-angle image or an omnidirectional image, and the presence of changes in visual stimulation by the projection device is further enhanced.

  In one embodiment of the present invention, as an example of the movable omnidirectional screen, a first screen and a second screen movable in the direct viewing direction with respect to the first screen are included. be able to. In this way, the size of the surrounding space is varied by the wide-angle screen moving mechanism while maintaining the surrounding space surrounding the entire circumference of the subject by the first and second screens.

  In one aspect of the present invention, the wide-angle screen moving mechanism includes a take-up roll. As an example of the movable wide-angle screen, the wide-angle screen taken up by the take-up roll and pulled out from the take-up roll is used. Can be mentioned. A free end portion of the wide-angle screen is developed in a curved shape so as to define the surrounding space of the subject, and the size of the surrounding space is variable according to the amount of winding on the winding roll. .

  Moreover, in one aspect of the present invention, the moving mechanism may further include the screen moving mechanism, and the moving mechanism may further include the moving portion of the seating portion along a direct viewing direction when the subject sitting on the seating portion faces the screen. A projection device moving mechanism for moving the projection device in the upper space may be included.

  In this way, when either the projection device or the screen moves, the other device is also moved by the same distance in the same direction, so that the relative positional relationship between the projection device and the screen can be maintained. In this way, the same image can be projected on the screen in the same form as before the movement without necessarily performing image correction.

  In one aspect of the present invention, in addition to the projection device and the screen moving mechanism, a seating portion moving mechanism that moves the seating portion along the direct viewing direction can be further provided.

  In this way, the two-dimensional positional relationship in the horizontal plane of the subject, the projection device, and the screen can be arbitrarily changed, and the range of variations in visual stimulation can be further widened.

  In one aspect of the present invention, the image processing apparatus further includes an image correction device that corrects an image supplied to the projection device based on a movement parameter when the projection device or the projection target device is moved by the movement mechanism. it can.

  In this way, the image projected on the screen after movement can be made equivalent to that before the movement, and the projected image can be distorted, projected only on a part of the screen, or projected beyond the screen. Can be prevented.

  In one aspect of the present invention, the image projection device is supplied with an image captured by an external image capturing device, and the image correction device receives movement parameters of the image capturing device as well as the image from the image capturing device. The image can be corrected based on the movement parameter of the image capturing apparatus.

  In this way, it is possible to prevent the projected image from being distorted, projected onto only a part of the screen, or projected from the screen even if camera shake or other movement factors occur in the image capturing apparatus.

  In one aspect of the present invention, an omnidirectional image is projected on the screen, and the image correction device is configured to display the omnidirectional image corresponding to the lower end point and the upper end point of the screen when the radius of the screen is enlarged from d0 to d1. The radiuses r1 and r2 of the polar coordinates of the azimuth image are calculated, and black pixel values can be assigned to regions where the radius r is smaller than r1 and regions where the radius r is larger than r2.

  In this way, since the area projected from the screen is a black pixel value, unnecessary projection outside the screen can be prevented.

In one aspect of the present invention, an omnidirectional image is projected on the screen, and the image correction device sets the optical characteristic of the projection device to f (r) when the radius of the screen is changed from d0 to d1. It is given by the following equation so that the radius r in the polar coordinates (r, φ) of the conversion source of the omnidirectional image becomes the radius r ′ of the polar coordinates (r ′, φ ′) after the conversion. Image conversion can be performed.

  In this way, even if the distance from the projection device to the screen changes, the original image can be enlarged or reduced and projected without losing part of the image content.

In one aspect of the present invention, the projector further includes a vertical movement mechanism that moves the projection device in a vertical direction, and an omnidirectional image is projected onto the screen, and the projection device is perpendicular to the height tz ′ from the height tz. The image correction device of the projection device uses f (r) as a function representing optical characteristics, and the radius r in the polar coordinates (r, φ) of the conversion source of the omnidirectional image is Image conversion given by the following equation can be performed so that the radius r ′ of the polar coordinates (r ′, φ ′) is obtained.

  In this way, an erect image can be projected even when the projection apparatus is moved up and down.

In one aspect of the present invention, an omnidirectional image is projected on the screen, and the image correction device projects f (r) when the imaging position of the image capturing device has a change of Δz in the vertical direction. As a function representing the optical characteristics of the apparatus, the radius r of the original polar coordinates (r, φ) of the omnidirectional image becomes the radius r ′ of the polar coordinates (r ′, φ ′) after the conversion, Image conversion given by the following equation can be performed.

  In this way, it is possible to accurately correct the projected image in response to the vertical fluctuation or movement of the image capturing apparatus.

  According to another aspect of the present invention, there is provided a biological information measurement system for measuring biological information of a subject, the visual stimulus applying device according to any one of the above that applies a visual stimulus to the subject, and attached to the subject This relates to a biological information measurement system including a measurement device that measures the biological information of the subject and a recording device that records the biological information of the subject measured by the measurement device in response to the environmental stimulus. .

  If it does in this way, living body information can be measured, giving an effective visual stimulus to a subject as mentioned above.

1 is a functional block diagram of a biological information measurement system according to a first embodiment of the present invention. It is a schematic block diagram of the biological information measuring system of the embodiment. 3A and 3B are a schematic plan view and a schematic front view of an embodiment in which the projector can be moved with respect to the omnidirectional screen. 4A and 4B are a schematic plan view and a schematic front view of an embodiment in which both the omnidirectional screen and the projector are movable. FIGS. 5A to 5C are schematic plan views of an embodiment in which both the wide-angle screen and the projector are movable. It is a block diagram of a moving mechanism and its control system. 7A and 7B are schematic explanatory diagrams for explaining the necessity of moving the projector alone. 8A and 8B are schematic explanatory diagrams showing the necessity of moving the projector when the screen is moved. FIGS. 9A and 9B are schematic explanatory diagrams showing a state in which the problem is solved by moving the projector from the state of FIGS. 8A and 8B. FIGS. 10A to 10C are block diagrams showing the relationship between the omnidirectional image capturing (generating) apparatus and the image projecting apparatus. It is a top view of the omnidirectional image which an omnidirectional image photographing (generation) device generates. It shows a state of projection onto a cylindrical surface. FIGS. 13A and 13B are schematic explanatory views showing the necessity of image correction when the projector is moved. It is a block diagram of the image correction apparatus provided in the front | former stage of an image projector. It is a schematic explanatory drawing of the image correction operation | movement of the omnidirectional image by an image correction apparatus. It is a block diagram of the other embodiment of the image correction apparatus provided in the front | former stage of an image projector. It is a block diagram of the other embodiment of the image correction apparatus provided in the front | former stage of an image projector. It is a block diagram of the other embodiment of the image correction apparatus provided in the front | former stage of an image projector. It is a block diagram of the other embodiment of the image correction apparatus provided in the front | former stage of an image projector. It is sectional drawing of the omnidirectional optical system which shows the imaging | photography range of an omnidirectional camera, or the projection range of an omnidirectional projector. It is a schematic perspective view of the cylindrical imaging | photography object used as the imaging | photography object of an omnidirectional camera. It is a schematic plan view of the omnidirectional image image | photographed with the omnidirectional camera. It is a perspective view which shows schematic structure of an omnidirectional screen. It is explanatory drawing of the coordinate system which defines the omnidirectional image image | photographed with the omnidirectional camera. It is explanatory drawing of the relationship between the projection possible angle of an omnidirectional projection apparatus, and the projectable range in an omnidirectional image. It is explanatory drawing for defining the coordinate system of an omnidirectional screen. It is a specific operation explanatory diagram of image correction. It is a specific operation explanatory diagram of image correction. It is a schematic explanatory drawing which shows the necessity for carrying out image correction when the radius of a cylindrical screen is expanded. FIG. 10 is an operation explanatory diagram of image correction by the image correction method 1; FIG. 10 is an operation explanatory diagram of image correction by the image correction method 2; It is a schematic explanatory drawing which shows the necessity for carrying out image correction when moving an omnidirectional projection apparatus to a perpendicular direction. FIG. 10 is an operation explanatory diagram of image correction by an image correction method 3; FIG. 10 is an operation explanatory diagram of image correction by an image correction method 4; FIG. 10 is an operation explanatory diagram of image correction by an image correction method 5; 36A and 36B are schematic plan views of omnidirectional images before and after image correction by the image correction method 5. FIG. 10 is an operation explanatory diagram of image correction by an image correction method 7;

  Hereinafter, preferred embodiments of the present invention will be described in detail. The present embodiment described below does not unduly limit the contents of the present invention described in the claims, and all the configurations described in the present embodiment are essential as means for solving the present invention. Not necessarily.

  FIG. 1 is a functional block diagram of the biological information measuring system according to the first embodiment of the present invention, and FIG. 2 is a schematic configuration diagram of the biological information measuring system of the same embodiment.

1. Biological Information Measurement System The biological information measurement system 100 according to the present embodiment includes an environmental stimulus from outside the subject, a biological reaction of the subject to the environmental stimulus, and a subjective report on how the subject feels with respect to the environmental stimulus. The system controls and measures in an integrated manner, and includes an environmental stimulus applying device 102, a measuring device 104, a recording device 106, and a control device 112, as shown in FIG.

  The environmental stimulus imparting device 102 imparts environmental stimuli such as illumination, sound, room temperature, and image presentation to the subject OB to be measured. In the present embodiment, as shown in FIG. 2, the environmental stimulus applying device 102 is an air conditioner 140 that functions as a temperature stimulus applying device provided inside the biological information measurement chamber 101 or an image that functions as a visual stimulus applying device. A projection apparatus (also called a projection apparatus, which is a projector, for example) 150, a screen 160 as a projection apparatus, and the like are included. As will be described later, the screen 160 may be a dome-shaped spherical screen in addition to a wide-angle screen having a projection surface of 180 ° or more, an omnidirectional screen having a 360 ° omnidirectional projection surface. The image supplied to the image projection apparatus 150 includes both a case where the image is formed artificially by a computer graphic or the like and a case where the image is taken by a camera such as an omnidirectional camera. The subject OB is seated on a seat (sitting unit) 103 installed at a position where the projection image of the image projection device 150 is viewed, or moved to a position where the subject can be arbitrarily touched by the operation input unit 120. Answer the subjective report on the environmental stimulus.

  In addition, the environmental stimulus applied to the subject OB from the environmental stimulus applying device 102 is converted into an electrical signal or the like by the environmental stimulus information conversion unit 108, and the converted environmental stimulus information is recorded in the memory 110. Then, the environmental stimulus information converted into an electrical signal or the like by the environmental stimulus information conversion unit 108 is supplied to the control device 112. Details of the projector 150 and the screen 160 serving as the visual stimulus applying device will be described later.

  As shown in FIG. 2, the measuring device 104 is attached to a subject OB in the living body information measurement chamber 101, and the subject OB with respect to the environmental stimulus applied from the environmental stimulus applying device 102 (140, 150, 160). Biometric information obtained from biological reactions such as blood pressure, electrocardiogram, pulse wave, electroencephalogram, and cerebral blood flow is measured. In the present embodiment, the measurement device 104 includes a measurement probe 114 that is directly attached to the subject OB via a conductive paste (not shown) or the like, and a biological amplifier 116 that amplifies an electrical signal from the measurement probe 114. Including. Then, the electrical signal of biological information amplified by the biological amplifier 116 is transmitted to the recording PC 130 including the recording device 106 installed in the experimenter console.

  The recording PC 130 includes the recording device 106, the control device 112, and the like, and the biological information included in the electrical signal transmitted to the recording PC 130 is recorded on the recording medium 110. At this time, the operation information is output from the control device 112 to the recording device 106 and recorded. In this case, it is preferable to record the operation information in synchronization with the biological information. The operation information and biometric information recorded in synchronism are recorded in a recording medium (memory) 110 such as an HDD (Hard Disk Drive) or a semiconductor memory. Although FIG. 2 shows electrocardiogram measurement using induction with an arm, connection methods by various induction methods may be substituted, and not only electrocardiography but also skin electrical resistance, electroencephalogram, electromyogram. Alternatively, electrical biological information such as electrooculogram may be measured.

  As shown in FIG. 2, the recording device 106 is provided in a recording PC 130 installed in an experimenter console outside the biological information measurement chamber 101. Then, the biological information of the subject OB measured by the measurement probe 114 and output from the biological amplifier 116 is recorded according to the environmental stimulus applied from the environmental stimulus applying device 102. Further, the recording device 106 records the reply information input by the subject OB via the operation input unit 120 in response to the environmental stimulus.

  The operation input unit 120 is a pressure-sensitive touch panel whose operation display screen is covered with an insulator 118, and inputs information by pressing an operation display screen on which various questions and control buttons are displayed. Information input by the operation input unit 120 is converted into an electrical signal or the like by the operation input conversion unit 122 and transmitted to the control device 112 via the network. Then, the display control unit 124 outputs the presentation information to be displayed on the display unit 126 based on the operation information input from the operation input unit 120 to the control device 112 via the operation input conversion unit 122. Specific examples of networks connected to transmit operation information input from the operation input unit 120 include various types such as Ethernet (registered trademark), wireless LAN, Zigbee, Bluetooth, USB, IEEE 1394, RS232C, and GPIB. Examples include networks such as communication standards.

  The example of the operation input unit 120 is not limited to a pressure-sensitive touch panel in which the surface of the operation display screen is covered with an insulator 118. In order to prevent a short circuit between the subject OB and the operation input unit 120, the operation input unit 120 is insulated. If it is a type touch panel, an optical switch type touch panel may be used. In addition to the touch panel, the operation input unit 120 can be replaced with various mechanical switches, sliders, knobs, etc., or an electromagnetic induction stylus pen made of an insulator. May be.

2. Next, the configuration of the visual stimulus applying device included in the environmental stimulus applying device 102 of the biological information measuring system of this embodiment will be described with reference to the drawings. FIG. 3A is a plan view seen from above the visual stimulus applying device of the biological information measuring system of the present embodiment, and FIG. 3B is the visual stimulus applying device of the biological information measuring system of the present embodiment. It is the front view seen from the side.

  The visual stimulus applying device 141 is an image projecting device that is provided inside the biological information measuring room 101 (see FIG. 2) and projects image data captured (generated) by an omnidirectional camera (image generating device). A projector, for example, an omnidirectional projector 150 and a screen, for example, a cylindrical screen 161, to which image data is projected from the projector 150 are included. In addition, the cylindrical screen 161 is good also as a screen not only the side part of the biometric information measurement chamber 101 but the ceiling and floor surface of the measurement chamber 101. FIG. The cylindrical screen 161 is an example of a wide-angle screen that partitions a surrounding space surrounding the subject OB at a wide angle of 180 ° or more around the subject OB seated on the seat 103. The cylindrical screen 161 is also an example of an omnidirectional screen that defines a surrounding space surrounding the subject OB at all angles of 360 ° with the subject OB seated on the seat 103 as the center.

  Further, the projector 150 can be moved in the horizontal direction over the head of the subject OB by a projector suspension 152 that is held movably with respect to the cylindrical screen 161. That is, as shown in FIG. 3A, the projector 150 moves from the circular center O of the cylindrical screen 161 at least in the radial direction. This moving direction can be made to coincide with the direct viewing direction A (see FIG. 2) when the subject OB seated on the seat 103 faces the cylindrical screen 161.

  In the example of FIG. 3A, the cylindrical screen 161 is an embodiment that does not move. In FIG. 3A, the seat 103 which is a seating portion may be moved along the direct viewing direction A in FIG. Further, the seat 103 may have a reclining mechanism at the backrest portion. Thus, the eyes (visual points) of the subject OB can be moved.

  FIG. 4A is a plan view of a modification of the visual stimulus imparting device of the biological information measurement system of the present embodiment as viewed from above, and FIG. 4B is the vision of the biological information measurement system of the present embodiment. It is the front view which looked at the modification of the stimulus imparting apparatus from the side.

  In the present embodiment, the omnidirectional screen 162 is the same wide-angle screen as in FIGS. 3A and 3B, and the first screen 162a and the first screen 162a are in the direct viewing direction A (FIG. 2). And a movable second screen 162b. The first and second screens 162a and 162b make the surrounding space variable while maintaining the surrounding space surrounding the entire circumference of the subject OB. Note that both the first and second screens 162a and 162b may be movable.

  Each of the first and second screens 162a, 162b includes a semicircular portion 162a1 (162b1) and two straight portions 162a2, 162a3 (162b2, 162b3) in cross section, and the first and second screens 162a, 162b3 The size of the surrounding space surrounding the entire circumference of the subject OB is varied by varying the amount of overlap between the two linear portions 162b (162a2 and 162b2, 162a3 and 162b3).

  FIG. 5A to FIG. 5C show still another visual stimulus applying device. As shown in FIGS. 5A to 5C, a wide-angle screen 163 made of a flexible material including a take-up roll 164 can be taken up by the take-up roll 164. The free end portion side of the wide-angle screen 163 drawn out from the take-up roll 164 is developed in a curved shape so as to partition the surrounding space of the subject OB, and the surrounding space is in accordance with the winding amount to the take-up roll 164. Variable size. The size of the surrounding space is “small” in FIG. 5A, “medium” in FIG. 5B, and “large” in FIG. 5C, and the size of the surrounding space is stepless or stepped. Variable.

  Thus, by seamlessly partitioning the size of the space using the roll-type screen 163, there is no seam, and the appearance upon entry into the room can be prevented from changing. By using the screen 163 made of an elastic body or a single cardboard-shaped resin panel, it can be configured to have a cylindrical surface shape while ensuring the rigidity in the vertical direction. Vertical fixing is performed between the free end of the screen 163 and the portion of the roll 164, and anchors can be fixed to the ceiling and floor as necessary. When the anchor portion is formed, it may be a cart that travels on the floor surface or the ceiling surface, or may be supported by an arm using a pantograph structure or the like on the ceiling surface or the floor surface.

  In the embodiment shown in FIGS. 4A and 4B or FIG. 5A to FIG. 5C, the projector 150 is in the direct viewing direction A (see FIG. 3A) as in the embodiment shown in FIGS. 2), the seat 103, which is a seating portion, may be moved along the direct viewing direction A in FIG. Further, the seat 103 may have a reclining mechanism at the backrest portion. Thus, the eyes (visual points) of the subject OB can be moved.

  As shown above, in the form of FIGS. 4A and 4B and FIGS. 5A to 5C, in addition to the function of the visual stimulus applying device shown in FIGS. 3A and 3B, It becomes possible to control an environmental factor such as the size of the surrounding space of the subject OB, and to realize video presentation corresponding to the environmental factor.

3. FIG. 6 shows a movement mechanism of the environmental stimulus applying apparatus and a control system for controlling movement of the movement mechanism. The control device 112 (see also FIG. 2) that controls the biological information measurement system receives movement information of the projector 150 and screens 162 and 163 from an operation unit such as an operator keyboard 131 connected to the recording PC 130 of FIG. It is possible. The subject OB seated on the seat 103 in FIG. 2 can input the amount of movement of the seat 103 in the direction of arrow A in FIG. 2 and the reclining amount of the backrest portion as necessary from the subject operation unit 105 provided on the seat 103. It is configured. The movement amount and reclining amount of the seat 103 can also be input from the operator keyboard 131. Further, as will be described later, a movement detection device for detecting various movement amounts may be provided and provided as a movement parameter to an image correction device to be described later. Moreover, the test subject operation part 105 can apply the operation input part 120. FIG.

  The control device 112 to which these pieces of movement information are input includes a sheet movement mechanism 103A, a sheet reclining mechanism 103B, a projector vertical movement mechanism 151, a projector horizontal movement mechanism (projector suspension device) 152, a screen movement via a movement control unit 170. A mechanism 165 is connected as necessary. The subject operation unit 105, and the seat movement mechanism 103A and the seat reclining mechanism 103B that are controlled by the input thereof are the above-described FIGS. 3A, 3B, 4A, 4B, and 5A to 5A. It can arrange | position as needed in each embodiment of FIG.5 (C). The projector horizontal movement mechanism 152 is provided in each of the embodiments shown in FIGS. 3A, 3B, 4A, 4B, or 5A to 5C. The screen moving mechanism 165 is provided in each of the embodiments shown in FIGS. 4A and 4B and FIGS. 5A to 5C.

  The screen moving mechanism 165 can also be referred to as a wide-angle screen moving mechanism because the screens 162 and 163 in FIGS. 4A and 4B and FIGS. 5A to 5C are wide-angle screens. . The screen moving mechanism 165 can also be called an omnidirectional screen moving mechanism because the screen 162 in FIGS. 4A and 4B is an omnidirectional screen. Although the operation mode of the projector vertical movement mechanism 151 is not particularly shown, an example in which the projector 150 is vertically moved will be described as an example of an image correction method described later.

  The reason for providing the sheet moving mechanism 103A, the seat reclining mechanism 103B, the projector horizontal moving mechanism 152, and the screen moving mechanism 165 is to change the visual stimulus to the subject OB. That is, if any one of these is moved, the two-dimensional positional relationship of the subject OB, the projector 150, and the screens 161 to 163 in the horizontal plane changes, and the visual stimulus reflected in the eyes of the subject OB changes. is there. Below, the movement mode by each moving mechanism 103A, 103B, 152, 165 is demonstrated.

3.1 Single Movement of Projector In each of the embodiments shown in FIGS. 3A, 3B, 4A, and 5B and FIGS. 5A to 5C, the projector 150 is connected to the screens 161 to 163. On the other hand, if the projector 150 is moved forward / backward along the arrow A direction in FIG. 2, the distance between the projector 150 and the screens 161 to 163 changes. Therefore, even if the projector 150 projects based on the same projection image data, the size of the image projected on the screens 161 to 163 changes, and the visual stimulus for the subject OB can be changed. This is because the two-dimensional positional relationship within the horizontal plane of the subject OB, the projector 150, and the screens 161 to 163 changes.

  Other reasons for moving the projector 150 are necessary to prevent the projection light from the projector 150 from entering the eyes of the subject OB and to prevent the projection image from being blocked by the head of the subject OB. Become. For example, when moving from the sitting position shown in FIG. 3B to the sitting position shown in FIG. 7A, a visual point corresponding to the eyes of the subject OB enters the projection area A2 from the non-projection area A1. Alternatively, as shown in FIG. 7B, when tilting from a state where the backrest is raised, similarly, a visual point corresponding to the eyes of the subject OB enters the projection area A2 from the non-projection area A1. In addition, even if the seat 103 does not necessarily move or recline, the same situation occurs depending on the sitting height of the subject OB. As described above, when the visual point of the subject OB enters the projection area A2, the projector 150 is moved so that the visual point of the subject OB enters the non-projection area A1. Thereby, the projector 150 can be set outside the visual field of the subject OB.

3.2 Single Movement of Screen In each of the embodiments of FIGS. 4A and 4B and FIGS. 5A to 5C, the screens 162 and 163 are moved from the projector 150 to the arrow A in FIG. If the projector is moved forward / backward along the direction, the distance between the projector 150 and the screens 162, 163 changes. Therefore, even if the projector 150 projects based on the same projection image data, the size of the image projected on the screens 162 and 163 changes, and the visual stimulus for the subject OB can be changed. At the same time, since the distance between the screens 162 and 163 and the subject OB changes, the sense of presence due to image projection also changes. This is because the two-dimensional positional relationship of the subject OB, the projector 150, and the screens 161 to 163 in the horizontal plane changes.

  As other reasons for moving the screens 162 and 163, especially in the case of the omnidirectional screen 162 of FIGS. 4A and 4B and the wide-angle screen 163 of FIGS. Since the size of the surrounding space changes, it is possible to give a feeling of opening, feeling of pressure, or feeling of obstruction, thereby changing the visual stimulus.

3.3 Both Projector and Screen Movement FIGS. 8A and 8B show a state where the second screen 162b is moved closer to the subject OB side from the state of FIGS. 4A and 4B. As shown in FIG. 8 (B), the lower portion B of the second screen 162b becomes the non-projection area A1, so that the projection surface on the second screen 162b is reduced and the sense of reality is lost.

  Therefore, as shown in FIGS. 9A and 9B, the projector 150 can be moved backward to, for example, the overhead position of the subject OB. In this way, the positional relationship between the projector 150 and the second screen 162b becomes the same as that in FIGS. 4A and 4B, and an image can be displayed on the entire surface of the second screen 162b without necessarily performing image correction to be described later. Can be projected. Further, as pointed out in FIG. 7A, in FIG. 8B, a visual point corresponding to the eye of the subject OB enters the projection area A2, and the projector 150 enters the field of view of the subject OB. After the movement of the projector 150, as shown in FIG. 9B, the visual point of the subject OB can be set to enter the non-projection area A1.

  That is, when one of the projector 150 and the screens 162 and 163 is moved, if the other is also moved in the same direction by the same amount, the positional relationship between the projector 150 and the screens 162 and 163 is the same. This is the same as before 163 movement. In this way, the same image as before the movement can be projected on the screen after the movement without necessarily performing the image correction described later.

  The projector 150 can also be provided with a shift mechanism that shifts the projection lens in the same direction as the movement direction of the projector 150. In this case, a part of the movement amount of the projector 150 may be supplemented by the shift amount of the projection lens.

4). As described above, the image projected onto the screen includes both a case where the image is formed artificially by a computer graphic or the like and a case where the image is photographed by an image photographing device such as a camera such as an omnidirectional camera. 10A to 10C, these are collectively referred to as an image capturing (generating) apparatus 200. As shown in FIG. 10A, an omnidirectional image captured (generated) by an image capturing (generating) apparatus is projected on an omnidirectional screen or a spherical screen by an image projecting apparatus 150 such as an omnidirectional projector. At this time, the captured image may be directly projected, but the omnidirectional image is once stored in the image storage device 210 and then projected by the image projection device 150 as shown in FIG. Also good. Alternatively, when the user is in a remote location, the image projection device 150 may project the image via the image communication device 220 as shown in FIG.

  An image generated by the omnidirectional image capturing (generating) apparatus 200 is, for example, a concentric image 230 shown in FIG. In this case, the omnidirectional image may be an omnidirectional image generated by pasting partial images, or may be a partial image of the omnidirectional (for example, a half-angle or 270 degree wide-angle image). .

  FIG. 12 shows a state of projection onto a cylindrical surface. FIG. 12 shows a cylindrical surface having a height H (m) and a radius R (m) with the Z axis of three orthogonal axes X, Y, and Z as the vertical axis, and the projector 150 is positioned on the Z axis.

5). Problems caused by changes in the position and orientation of the image projection apparatus and image correction As described above, when the position of the image projection apparatus (projector) 150 changes, an image that does not stand upright is projected on the projection plane (for example, a cylindrical surface) depending on the position. It will be. When an appropriate image is projected when the image projection device 150 is positioned at the center of the cylindrical screen 160 as shown in FIG. 13A, the image projection device 150 is displaced from the center as shown in FIG. The proper image is not projected.

  For this reason, as shown in FIG. 14, an image correction device 250 can be disposed in front of the image projection device 150. When the image correction apparatus 250 corrects the omnidirectional image, even in the case of FIG. 13B, an image projected on a projection surface such as a cylindrical surface as shown in FIG. ). At this time, image correction is performed by the image correction apparatus 250 by using parameters (position and orientation parameters of the image projection apparatus 150) describing the movement amount and movement direction of the omnidirectional projector (image projection apparatus) 150. .

  Such image correction is not limited to the case where the image projection apparatus 150 moves. As shown in FIG. 16, in the case of using a real image photographed by an omnidirectional image photographing device (for example, omnidirectional camera) 200A, a slight positional deviation or posture deviation may occur due to photographing camera shake or the like. is there. Therefore, a movement measuring device 260 that measures the movement amount of the omnidirectional image capturing device 200A is provided, and the image correction device 250 is input with the parameter of the position deviation or the posture deviation (image photographing device movement parameter). Then, the image correction device 250 in front of the image projection device 150 corrects the positional deviation or the posture deviation so as to adapt to the shape of the projected screen. Alternatively, as shown in FIG. 17, the image storage device 210 is arranged in the front stage of the image correction device 250, and the image from the image capturing device 200 </ b> A and the image capturing device movement parameter from the movement detection device 260 are stored in the image storage device 210. You can also keep it.

  FIG. 18 shows a modified example of FIG. 14, in which the omnidirectional image capturing (generating) apparatus 200 and the image correcting apparatus 250 are connected by the image communication apparatus 220, and such an image can be transferred from a remote place. It is said. FIG. 19 shows a modification of FIG. 17, in which a movement detection device 270 that detects the movement of the image projection device 150, that is, the movement amount and the movement parameter value, for example, is provided. Yes.

  According to the embodiment shown in FIGS. 14 to 19, the following two problems can be solved. One is to correct the omnidirectional image in order to project an image equivalent to that when the omnidirectional image projector 150 is moved, even when the omnidirectional image projector 150 is not moved. The other is to correct the omnidirectional image to be projected in order to project an image equivalent to when the omnidirectional image capturing device 200A does not move even if the omnidirectional image capturing device 200A moves, It is possible to display with the omnidirectional image projector 150 (can be used for camera shake correction in the omnidirectional image photographing device 200A).

Here, the position of the projector is inherently fixed with respect to the Z axis, and the center axis of the cylindrical screen and the center axis of the projector coincide with the X / Y axis. Conditions are desirable, but in the following cases, such conditions may not be satisfied.
1) The case where the central axis of the projector cannot coincide with the central axis of the omnidirectional screen (cylindrical screen) due to mechanical constraints or the position of the projector hinders viewing by the viewer.
2) This is a case where the projector is a movable type and it is desired to project video content having the same effect as when the movable projector is not movable.
3) This is a case where calibration between the central axis of the projector and the central axis of the omnidirectional screen (cylindrical screen) is desired to produce a precise projection effect.
4) Since the photographing environment of the omnidirectional camera and the omnidirectional projector is different, the central axis of the projector and the central axis of the omnidirectional screen (cylindrical screen) must be changed. For example, even if the omnidirectional camera causes a camera shake at the time of shooting, when the absolute movement component of the camera shake is known, the camera shake component is to be corrected by the image correction.

  Therefore, in the present embodiment, a cylindrical screen is assumed as the omnidirectional screen, and an omnidirectional projector capable of projecting a 360-degree image is installed therein, and video content stored in advance in the projector is projected. By doing so, image correction is performed on the omnidirectional screen in order to generate an erect image equivalent to that at the time of content shooting or content creation.

6). Next, the principle of cylindrical projection by the omnidirectional optical system used in the visual stimulus applying device included in the biological information measurement system of the present embodiment will be described. It is assumed that an omnidirectional camera or omnidirectional projector that is an omnidirectional optical system is configured by an axis target optical system or the like.

  In general, an omnidirectional projector is designed using an optical system such as a mirror or a prism, and an omnidirectional camera and an omnidirectional projector are combined so that an image captured by the omnidirectional camera can be directly used as an omnidirectional screen. By projecting, it becomes possible to appreciate images from all directions.

  Specifically, taking the optical system as an example, an omnidirectional camera or omnidirectional projector is used, and in order to show the shooting range or projection range, the cross section shown in FIG. be able to. At this time, if the projection or shooting possible angle θa above the reference horizontal position and the shooting or projection possible angle θb below the reference horizontal position are defined as shown in FIG. The possible region can be determined as an angle θ in FIG.

  In recent years, with the development of computer graphics, it is possible to produce still images and moving images that are the same as those shot with an omnidirectional camera, without being photographed or captured with an omnidirectional camera. Alternatively, a video content can be created by cylindrically converting a rectangular video shot with a normal digital camera or the like.

  In general, when an omnidirectional camera is constructed using a rectangular imaging device such as a CCD or CMOS, and an environment having a cylindrical lattice pattern CA1 as shown in FIG. In the photographed image, as shown in FIG. 22, a radial pattern CB1 is arranged from the center of the image in an ambient environment of 360 degrees. The image shown in FIG. 22 is called an omnidirectional image. When the omnidirectional image (FIG. 22) photographed as described above is photographed on the plain cylindrical omnidirectional screen 160 as shown in FIG. 23 from the projector 150, the omnidirectional screen 160 is similar to FIG. A grid pattern CA1 can be projected.

  If a computer graphics technique such as the omnidirectional image shown in FIG. 22 is created, a desired upright image can be projected onto the cylindrical screen as shown in FIG. it can. Conversely, a group of images photographed by an imaging device such as a digital camera are polar-coordinate-transformed and combined to synthesize a circular image as shown in FIG. 22 without using an omnidirectional camera. It is also possible to produce projection images such as

  In the following description, the description will be made on the assumption that there is video content shot by a shooting unit capable of shooting the 360-degree surrounding environment such as an omnidirectional camera. At that time, the omnidirectional image as shown in FIG. 22 can define the coordinate system as an image, and the omnidirectional screen (cylindrical screen) 160 having the configuration shown in FIG. 23 is used as the projection target device. Explain the mathematical relationship.

First, for the omnidirectional image shown in FIG. 24 taken by an omnidirectional camera, the coordinate system in the image is represented by P (u, v). Here, u represents the horizontal coordinate axis, v represents the vertical coordinate axis, and the upper left corner of the image is the origin (0, 0). Also, the image center when the omnidirectional image is generated is expressed by Oi (u0, v0). At this time, an arbitrary point P (u, v) in the omnidirectional image can be expressed by the following equation 1 using the radius r of the polar coordinates (r, φ) and the in-image angle φ.

Range of radius r is, r _a ≦ r ≦ _{r b} becomes a -π ≦ φ ≦ 0. FIG. 24 shows this state. Here, the relationship between FIG. 20 and FIG. 24 will be described. In Figure 20, imaging or projectable angle, since it is defined by θa and .theta.b, in FIG. 24, the radius vector r corresponding to θa corresponds to r _a, radius r is r _b corresponding to .theta.b Corresponding to This is shown in FIG. In Figure 25, as an example of the omnidirectional projector, a projection range corresponding to the minimum value r _a and the maximum value r _b that can be taken of the radius vector r, and the radius r0 corresponding to the horizontal projection position is shown .

  At this time, a relational expression of what point on the cylindrical screen 160 the point P (u, v) is projected to when projected onto the cylindrical screen 160 using the omnidirectional projector 150 will be described. To do.

As shown in FIG. 26, a coordinate system (x, y, z) of a cylindrical screen 160 (height h0, radius d0) is defined, and an omnidirectional projector is arranged at a position of M (0, 0, tz). . At this time, when the point P (u, v) in the omnidirectional image shown in FIG. 24 is represented by (r, φ) in the polar coordinate system, the ray in the projection direction defined by (u, v) is the parameter t. Can be expressed by the following formula 2.

Here, f (r) is a function representing the optical characteristics of the projector. For example, in the case of conformal mapping as shown in FIGS.

Since the radius of the cylindrical screen is d0, and the height difference between the maximum moving radius rb and the horizontal moving radius r0 on the cylindrical screen is tz, the following equation 4 can be calculated.

Expression 2 can be expressed by Expression 5 below using Expression 3.

Therefore, the relationship between the point P (u, v) = P (r, φ) in the omnidirectional image and the image point Q formed on the cylindrical screen 160 is expressed by Equation 5 and x _w ² + y _w ² = d ₀ ^2. Can be expressed by the following formula 6.

  Therefore, the method for calculating the relationship between the omnidirectional image and the cylindrical screen from the above results is performed by either the following method A or method B depending on which side is used.

Method A) When a point P (u, v) in an omnidirectional image is given, a method for calculating a point (xw, yw, zw) at which the point is projected onto a cylindrical screen is as follows. ) And A-2).
A-1) (r, φ) corresponding to (u, v) is calculated. That is, as a function h,
A-2) (xw, yw, zw) corresponding to (r, φ) is calculated by Equation 6.

Method B) Given a point (xw, yw, zw) on a cylindrical screen, to calculate the corresponding omnidirectional image point P (u, v), the following B-1), B- It is calculated by the method 2).
B-1) (r, φ) corresponding to (xw, yw, zw) is obtained from the inverse transformation of Equation 6.
B-2) Using Equation 1, (u, v) is calculated from (r, φ).

7). Image correction Up to the above description, the relationship between an omnidirectional image and a cylindrical screen has been described. However, image correction accompanying the movement of an image projection device (omnidirectional projector) or an omnidirectional image photographing device (omnidirectional camera) This is performed by the image correction device 250 described above. Both the input image and the output image to the image correction apparatus 250 are based on a rectangular image (or a square shape equivalent to that) as shown in FIG. That is, the image correction apparatus 250 performs correction between an image in the coordinate system defined by the (u, v) coordinate system and an image defined by the (u ′, v ′) coordinate system.

  Hereinafter, a general technique for such image correction will be described. Note that the image correction method described here is one method, and when the original image is defined in another coordinate system such as a polar coordinate system display, the same effect can be obtained by this method. It goes without saying that it is possible.

  In the image correction apparatus included in the visual stimulus applying apparatus of the present embodiment, when an image correction conversion formula is given, appropriate image conversion can be performed based on the conversion. For example, when a conversion from an orthogonal coordinate system to an orthogonal coordinate system of an image g: (u, v) → (u ′, v ′) is given, the image correction is “an image before correction that is an original image”. Thus, a “corrected image as a result image” is generated.

Therefore, in actual work, each coordinate value (u ′, v ′) (integer value) of the result image is subjected to g ⁻¹ which is the inverse transformation of the above-described transformation g, and (u ′, v ′). (U, v) corresponding to is first calculated, and appropriate interpolation processing is performed on the corresponding (u, v) (which is not necessarily an integer value corresponding to a real number coordinate) Thus, the image correction can be calculated with respect to the result image without any gap. That is, as shown in FIG. 27, while scanning each coordinate value (u ′, v ′) (integer value) of the result image, inversely corresponding (u, v) is calculated, and the (u, v) Interpolate with the values of surrounding pixels.

Further, in many cases, it is more convenient to express an image using a polar coordinate system as a medium. In this case, as shown in FIG. 28, g ⁻¹ is expanded in three functional forms represented by Equation 9. Accordingly, an image correction method using polar coordinate display as a medium is also possible.

  In the following description, the image correction method using the polar coordinate display described above will be mainly described. That is, a method for correcting an image by combining a parametric display parameter and the above methods A and B will be described with examples.

(Image correction method 1)
The image correction method 1 is an image correction method when the radius of the cylindrical screen is enlarged from d0 to d1 when an omnidirectional camera and an omnidirectional projector having the same optical system are used. When the radius of the cylindrical screen is enlarged from d0 to d1, the image is projected so as to protrude above and below the cylindrical screen 160 as shown in FIG. Such a problem is required due to a difference in magnification between optical systems of the omnidirectional camera and the omnidirectional projector.

  Therefore, it is possible to erect the projected image on the cylindrical screen 160 by deforming the original omnidirectional image by a method as shown in FIG. That is, only the image of the part projected on the cylindrical screen 160 is cut out from the omnidirectional image, and the area of the other part is filled with a color such as black, whereby the video content other than the cylindrical screen 160 is displayed. Eliminate the reflections. In this case, the radial radius of each polar coordinate of the omnidirectional image corresponding to the lower end point Q1 (d1, 0, 0) and the upper end point Q2 (d1, 0, h0) of the cylindrical screen with the radius d1 by the above method A. Calculate r1 and r2. Then, only the portion of the radius r including r1 and r2 is cut out, and the other regions, that is, the radius radius r shown in FIG. 30 is smaller than the radius r1 and the radius radius r is larger than the radius r2 are black. Embedding with pixel values. As can be seen from the example of the case where the radius of the cylindrical screen 160 is enlarged, even if the radius of the cylindrical screen 160 is changed, image correction that matches the size of the screen 160 is performed. Is possible. For this reason, an image can be effectively projected on the screen 160, and a realistic image can be provided to the viewer.

(Image correction method 2)
Image correction method 2 converts an omnidirectional image so that an area of (ra, rb), which is an effective range of omnidirectional image content, can be projected onto a cylindrical screen having a radius d1 of (r1, r2). It is characterized by that. That is, as shown in FIG. 31, coordinate conversion of an omnidirectional image that reduces the video content portion may be performed.

Specifically, when the integer coordinates of the omnidirectional image of the conversion destination are (u ′, v ′), the real number coordinates (u, v ′) of the omnidirectional image of the conversion source corresponding to the integer coordinates (u ′, v ′) v). Then, pixel values that should be filled in (u ′, v ′) are calculated by performing appropriate image interpolation on the pixel values around (u, v). For this purpose, a relational expression between real number coordinates (u, v) and integer coordinates (u ′, v ′) is obtained. If the polar coordinate representation for the real number coordinates (u, v) is (r, φ) and the polar coordinate representation for the integer coordinates (u ′, v ′) is (r ′, φ ′), the transformation source coordinate system (r , Φ) is a conversion such that r in the coordinate system (r ′, φ ′) after conversion is r ′. Therefore, the image conversion given by Equation 10 below may be performed. By converting the image in this way, it is possible to project onto the cylindrical screen without losing a part of the video content.

(Image correction method 3)
The image correction method 3 is a method for correcting an image when the omnidirectional projector moves in the z-axis direction. This image correction method is characterized by providing a means for projecting a video that fits and stands upright on a cylindrical screen by providing a means for correcting an image of an omnidirectional image taken in advance. In other words, when the position of the omnidirectional projector is changed from tz to tz ′, by performing image correction conversion, a projection image similar to the case where the omnidirectional projector is arranged at the position of tz can be obtained. .

  Specifically, as shown in FIG. 32, the omnidirectional projector 150 is positioned at the position of S ′ (0, 0, tz ′) in the coordinate system (xw, yw, zw) defined by the cylindrical screen 160. Assume that they are in place. At this time, by converting the image correction, it is a problem to create a projection image as if the omnidirectional projector is arranged at the position S (0, 0, tz) before the movement. Become. That is, even if the position of the omnidirectional projector 150 in the z direction is changed, it is necessary to prevent the height of the image projected on the cylindrical screen 160 from changing.

If the coordinate value of the erect image that should be before moving is (xw, yw, zw) and the coordinate value on the screen of the corresponding point that should be corrected after moving is (xw ', yw', zw '), In Expressions 11 and 12, (xw, yw, zw) = (xw ′, yw ′, zw ′) must be established.

From the relational expression described above, the following expression 13 is obtained.

  The relationship of the above equation 13 will be described with reference to FIGS. 33 (A) to 33 (C). In the present embodiment, the omnidirectional projector 150 is moved in the z-axis direction from the upright position of the image before the omnidirectional projector 150 is moved as shown in FIG. 33A, as shown in FIG. Image correction. When the omnidirectional projector 150 is moved in the z-axis direction, as shown in FIG. 33C, as a correction of the omnidirectional image after the movement of the omnidirectional projector 150, the projection position on the omnidirectional screen 160 is corrected after the image correction. The omnidirectional image is converted so as to be equivalent to the projection position on the omnidirectional screen 160 before correction.

(Image correction method 4)
The image correction method 4 is a method for correcting movement in the z-axis direction during shooting with an omnidirectional camera. In other words, when the omnidirectional camera moves and changes occur in the image during omnidirectional camera photography, the change is corrected by the image. More specifically, when an image is picked up by an omnidirectional camera and the image pickup position changes with Δz in the z direction, that is, when a change in the z-axis direction occurs by the magnitude of Δz, the change is the amount. The image is corrected so that it can be observed so that Δz is not a projection image of a cylindrical screen with a radius d0, that is, Δz becomes zero. Image conversion in this case, in the above formula 13, obtained from the following formula 14 was replaced with Δz = t _'z -t _z.

  More specifically, as shown in FIGS. 34 (A) to 34 (C), a virtual cylindrical screen surface 160a is installed so as to have a radius d0, and can be processed by the above equation (14). In addition, image correction and interpolation processing are performed by inverse transformation of r → r ′. That is, the correction of the omnidirectional image after the omnidirectional camera is moved is that the projection position on the screen surface 160a virtually provided after the image correction shown in FIG. 34C is the same as that before the correction shown in FIG. The omnidirectional image of FIG. 34B is converted so as to be equivalent to the projection position on the virtually provided screen surface 160a.

(Image correction method 5)
The image correction method 5 is a method for correcting an omnidirectional image when the omnidirectional projector moves in the x-axis direction. That is, when the position of the omnidirectional projector is moved in the x-axis direction, position correction is performed on the cylindrical screen on which the image is projected as if there was no position movement.

If the movement in the x direction is Δx, xw = between the coordinate system (xw, yw, zw) and (xw ′, yw ′, zw ′) on the cylindrical screen before and after the correction. Since xw ′, yw = yw ′ and zw = zw ′ must be established, (φ, r ′) with respect to (φ ′, r ′) is satisfied so that the relationship between the following equations 15 and 16 is satisfied with the parameter as e. The image is corrected by obtaining r).

Specifically, the following expression 17 is obtained from the above-described expressions 15 and 16.

  More specifically, as shown in FIG. 35B, the projector 150 is moved in the x-axis direction by a distance Δx from the state in which the projected image before the movement of the omnidirectional projector 150 shown in FIG. When moved, f (r) is changed to f (r ′) so that the upright image of the projection image after correction becomes the same height as before movement, as indicated by symbol E in FIG. Convert. At this time, the omnidirectional image is corrected from the state shown in FIG. 36A to the state shown in FIG.

(Image correction method 6)
The image correction method 6 is a method for correcting an omnidirectional image when the omnidirectional camera is rotated by a minute angle ΔR or moved by a minute distance ΔT. In the present embodiment, the omnidirectional camera is equipped with a sensor capable of measuring changes in the position and orientation of the omnidirectional camera, such as a gyro sensor capable of measuring a rotational change and an acceleration sensor capable of measuring a positional change. . Therefore, when video content is shot with an omnidirectional camera, if the omnidirectional camera records the minute rotation and minute position movement parameters of the omnidirectional camera with respect to the shooting time for each shooting frame, the video content is cylindrical. The camera shake correction function of an omnidirectional camera can be realized when projecting onto a screen or the like.

For example, when an omnidirectional image at a certain time is described with a minute rotation matrix described by ΔR and a minute position movement parameter represented by ΔT with respect to the previous time, and before moving on the cylindrical screen, Since the projected position coordinates after the movement need to coincide with each other, the following Expression 18 must be satisfied.

Here, ΔR is composed of (Ψx, Ψy, Ψz) (multiplication of Ψx, Ψy, Ψz which are Yaw, Pitch, and Roll angles), which can be defined by the following Expression 19.

ΔT can be defined by the following equation 20.

Therefore, in Equation 18, the unknown e may be eliminated and (r, φ) may be expressed as (r ′, φ ′). That is, a function system represented by the following expression 21 may be obtained.

  In this way, even when the position and orientation of the omnidirectional camera (omnidirectional image capturing device) changes minutely, the change in position and orientation is canceled out (removed) by measuring the change in position and orientation. Image correction that can be performed can be performed. In other words, in the image correction method 6, even when there is a minute change in the position or orientation of the omnidirectional camera, the image projected on the projection surface on the cylindrical screen stands upright with the change in the position or orientation. Correct the image as follows. By doing so, camera shake correction or movement correction can be realized as if there were no minute changes in the position or orientation of the omnidirectional camera. For this reason, it is possible to reduce the occurrence of symptoms such as motion sickness in the user who is viewing the projection image from the omnidirectional projector.

(Image correction method 7)
The image correction method 7 is a method for correcting an omnidirectional image when the screen surface is the spherical surface shown in FIG. 37 or a part of the spherical surface. Each of the image correction methods 1 to 6 described above has been described for the cylindrical screen. However, the same effect can be obtained by using a spherical or hemispherical dome-shaped screen instead of the cylindrical screen. In that case, Expression 22 may be used instead of Expression 2.

  As described above, an omnidirectional screen can be defined as a projection if the geometric characteristics of the screen can be described by parameters, whether it is a cylindrical screen or a screen composed of a part of a spherical surface. Even if the image projection apparatus (omnidirectional projector) or the image photographing apparatus (omnidirectional camera) moves, the movement itself can be removed on the screen surface by calculating or observing parameters related to the movement. become.

  Although the present embodiment has been described in detail as described above, those skilled in the art can easily understand that many modifications can be made without departing from the novel matters and effects of the present invention. Let's go. Therefore, all such modifications are included in the scope of the present invention.

  For example, a term described with a different term having a broader meaning or the same meaning at least once in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configurations and operations of the visual stimulus applying device and the biological information measuring system are not limited to those described in this embodiment, and various modifications can be made.

DESCRIPTION OF SYMBOLS 100 Biological information measurement system, 101 Biological information measurement room, 102 Environmental stimulus provision apparatus,
103 Seating part (seat), 103A, 103B Seating part moving mechanism, 104 Measuring device,
106 recording device, 110 memory, 112 control device, 114 measuring probe,
116 biological amplifier, 118 insulator, 120 operation input unit, 122 operation input conversion unit,
124 display control unit, 126 display unit, 130 recording PC, 140 air conditioner,
150 image projector, 151, 152 projector moving mechanism,
160, 161, 162, 163 Projection device (screen),
164 winding screen, 165 screen moving mechanism, 170 movement control unit,
200 omnidirectional image capturing (generating) device, 200A omnidirectional image capturing device,
210 image storage device, 220 image communication device, 250 image correction device,
260, 270 Movement detector, OB Subject

Claims (21)

A visual stimulus applying device for applying a visual stimulus to a subject,
A projection device for projecting an image serving as the visual stimulus;
A projection apparatus having a screen on which the image is projected from the projection apparatus;
A moving mechanism for moving at least one of the projection device and the projection target device so that a two-dimensional positional relationship in the horizontal plane of the subject, the projection device, and the screen changes;
A visual stimulus imparting device comprising: In claim 1,
A seating section on which the subject sits;
The moving mechanism includes a projection device moving mechanism that moves the projection device in a space above the seating portion along a direct viewing direction when the subject seated on the seating portion faces the screen. A visual stimulus imparting device. In claim 2,
The projection device moving mechanism moves the projection device so that the visual point corresponding to the eye position of the subject is outside the projection area of the projection device. In claim 3,
The screen of the projection apparatus is a wide-angle screen that partitions a surrounding space surrounding the subject at a wide angle of 180 ° or more around the subject seated on the seating portion,
The visual stimulation imparting device, wherein the projection device projects a wide-angle image onto the wide-angle screen of the projection target device. In claim 4,
The wide-angle screen of the projection apparatus is an omnidirectional screen that partitions a surrounding space surrounding the subject at all angles of 360 ° with the subject seated on the seating portion as a center,
The projection device is characterized by projecting an omnidirectional image onto the omnidirectional screen of the projection target device. In any of claims 2 to 5,
A visual stimulus imparting device, further comprising a seating portion moving mechanism for moving the seating portion along the direct viewing direction. In any one of Claims 2 thru | or 6.
The seating unit further includes a reclining mechanism, wherein the visual stimulus imparting device is provided. In claim 1,
A seating section on which the subject sits;
The moving mechanism moves the screen of the projection apparatus so as to change at least a distance between the subject and the screen in a direct viewing direction when the subject sitting on the seating unit faces the screen. A visual stimulus imparting device comprising a screen moving mechanism. In claim 1,
A seating section on which the subject sits;
The screen of the projection apparatus is a wide-angle screen that partitions a surrounding space surrounding the subject at a wide angle of 180 ° or more around the subject seated on the seating portion,
The projection device projects a wide-angle image on the wide-angle screen of the projection target device,
The visual stimulus imparting device according to claim 1, wherein the moving mechanism includes a wide-angle screen moving mechanism that moves the wide-angle screen of the projection target device so as to vary the size of the surrounding space. In claim 9,
The wide-angle screen of the projection apparatus is an omnidirectional screen that partitions a surrounding space surrounding the subject at all angles of 360 ° centered on the subject seated on the seating portion,
The projection device is characterized by projecting an omnidirectional image onto the omnidirectional screen of the projection target device. In claim 9,
The omnidirectional screen includes a first screen and a second screen that is movable in the direct viewing direction with respect to the first screen, and the first and second screens make the entire circumference of the subject. A visual stimulus imparting device, wherein the surrounding space surrounding the frame is maintained and the size of the surrounding space is varied by the wide-angle screen moving mechanism. In claim 9,
The wide-angle screen moving mechanism includes a take-up roll, the wide-angle screen is taken up by the take-up roll, and a free end side of the wide-angle screen drawn out from the take-up roll defines the surrounding space of the subject. The visual stimulus imparting device, which is developed in a curved shape as described above, and changes the size of the surrounding space in accordance with the amount of winding on the winding roll. In any one of Claims 8 thru | or 12.
The moving mechanism includes a projection device moving mechanism that moves the projection device in a space above the seating portion along a direct viewing direction when the subject seated on the seating portion faces the screen. A visual stimulus imparting device. In any one of Claims 8 thru | or 13.
A visual stimulus imparting device, further comprising a seating portion moving mechanism for moving the seating portion along the direct viewing direction. In any one of Claims 1 thru | or 14.
A visual stimulus imparting device, further comprising an image correction device that corrects an image supplied to the projection device based on a movement parameter when the projection device or the projection target device is moved by the movement mechanism. . In claim 15,
The image projection device is supplied with an image captured by an external image capturing device,
The visual stimulus imparting device, wherein the image correction device receives a movement parameter of the image photographing device together with an image from the image photographing device, and corrects the image based on the movement parameter of the image photographing device. In claim 15,
An omnidirectional image is projected on the screen,
When the radius of the screen is enlarged from d0 to d1, the image correction device calculates the radius r1, r2 of each polar coordinate of the omnidirectional image corresponding to the lower end point and the upper end point of the screen, and the radius r A visual stimulus applying device, wherein black pixel values are assigned to a region where is smaller than r1 and a region where the radius r is larger than r2. In claim 15,
An omnidirectional image is projected on the screen,
When the radius of the screen is changed from d0 to d1, the image correction device uses f (r) as a function representing the optical characteristics of the projection device, and polar coordinates (r, φ) from which the omnidirectional image is converted A visual stimulus imparting device that performs image conversion given by the following expression so that the radial radius r in the graph becomes the radial radius r ′ of the converted polar coordinates (r ′, φ ′).
In any one of Claims 1 thru | or 15,
A vertical movement mechanism for moving the projection device in a vertical direction;
An omnidirectional image is projected on the screen,
When the projection device moves in the vertical direction from height tz to height tz ′, f (r) is the light of the projection device. Visual transformation characterized by performing image conversion given by the following equation so that the radial radius r in the polar coordinates (r, φ) of the same becomes the radial radius r ′ of the converted polar coordinates (r ′, φ ′) Stimulation device.
In claim 16,
An omnidirectional image is projected on the screen,
When the imaging position of the image capturing device has a change of Δz in the vertical direction, the image correction device uses f (r) as a function representing the optical characteristics of the projection device, and polar coordinates of the conversion source of the omnidirectional image Visual stimulus application characterized by performing image conversion given by the following equation so that the radius r at (r, φ) becomes the radius r ′ of the converted polar coordinates (r ′, φ ′) apparatus.
A biological information measurement system for measuring biological information of a subject,
The visual stimulus imparting device according to any one of claims 1 to 20, which imparts a visual stimulus to the subject;
A measuring device for measuring the biological information of the subject by being attached to the subject;
A recording device for recording the biological information of the subject measured by the measuring device in response to the environmental stimulus;
A biological information measuring system comprising: