WO2018101229A1 - Display system, video communication system, and method of detecting position coordinate - Google Patents

Abstract

Provided are a lightweight display system, a video communication system, and a method of detecting position coordinates, with which it is possible for a user to artificially experience bodily sensations of body contact with a familiar person and freely switch between a use state and a non-use state in a simple manner. This display system 1 is provided with: a projector 10; a transmission screen 40 arranged in front of the eyes of a user U; a first reflecting mirror 20 and a second reflecting mirror 30 for reflecting a main image outputted from the projector and projecting the reflected image onto the transmission screen; a cushion 50 arranged in front of the user's eyes and having a transparent part in at least a portion thereof; and a support mechanism 60 for supporting the projector, the transmission screen, the first reflecting mirror, the second reflecting mirror, and the cushion.

Description

Display system, video communication system, and position coordinate detection method

The present invention relates to a lightweight display system and video communication system that can simulate body contact with a person close to a user and can switch between a use state and a non-use state freely and simply.

Increasingly, it is difficult to meet frequently because close family members, friends, lovers, etc. live in remote areas or are busy or hospitalized.
In this case, in general, a person often carries a picture of a close person or communicates via a telephone or video call, but there is a problem of lack of realism.
Therefore, for example, if a head mounted display with a display mounted on the user's head is used, it is possible to view a realistic image as if a close person actually exists in front of the eyes.
Or you may utilize the bed with a projector screen (patent document 1) and the pillow (patent document 2) which this inventor developed.
A bed with a projector screen allows the user to lie on the bed and view the projector image on a large screen. The body pillow allows the user to view images in various postures such as lying down, lying down, or lying sideways. Therefore, the user can have a realistic experience that is close to a close person.

Japanese Patent No. 5991606 Japanese Patent No. 6019463

However, the above prior art has the following problems.
In other words, in any of the bed with a projector screen, the pillow, and the head mounted display, the user can only perform non-contact communication via an image. In other words, there is a problem that the user cannot experience bodily contact such as shaking hands, embrace, and sexual intercourse with a close person, and it is difficult to obtain sufficient satisfaction.
In addition, in the case of a bed with a projector screen, there is a problem that the user can view images only in a lying state.
In the case of a body pillow, it is configured to attach a video playback device such as a liquid crystal monitor to the body of the body pillow, and it is assumed that multiple video playback devices are attached as necessary, so the weight is heavy and difficult to handle There is.

In the case of a head-mounted display, the user can view the image in a desired posture, but when the user wears glasses or reading glasses, the display and glasses are in contact with each other and it is difficult to use, the problem of being prone to video sickness, In the case of elderly people, there is a problem that it is difficult to support the display with the strength of the neck.
In the case of a head mounted display, due to its nature, an image is always displayed in front of the eyes even when the user turns his / her neck or twists his / her body to change the direction of the face. Therefore, for example, when it is desired to view an image between meals, writings, etc., once the display is removed from the head, there is a problem that it is troublesome to attach the display to the head again when viewing again. .

In view of the above problems, the present invention is a lightweight display system, video communication system, and position coordinate detection that enables a user to experience bodily contact with a close person and can switch between a use state and a non-use state freely and easily. It is an object to provide a method.

The display system of the present invention includes a projector, a transmissive screen disposed in front of the user's eyes, and a first reflecting mirror and a second reflecting mirror that reflect a main image output from the projector and project the reflected image onto the transmissive screen. A cushion having at least part of a transparent portion and disposed in front of the user's eyes, and a support mechanism for supporting the projector, the transmission screen, the first reflecting mirror, the second reflecting mirror, and the cushion The transmissive screen is disposed between the second reflecting mirror and the cushion, the support mechanism is mounted on a user's trunk, and the user projects onto the transmissive screen. The main image can be seen through the transparent part and the surroundings of the subject can be seen by changing the direction of the face. Characterized in that the kill.
Further, the support mechanism includes an optical path adjusting mechanism for changing an optical path from the projector to the transmissive screen.

The image processing apparatus includes a first camera that captures an image around the user, and a processing unit that combines the image captured by the first camera and the main image and outputs the combined image from the projector.
Further, the first reflecting mirror and the second reflecting mirror reflect only a part of the main image and project it on the transmission type screen, and the first reflecting mirror and the second reflecting mirror are not reflected by the first reflecting mirror and the second reflecting mirror. The remaining image of the main image is projected onto a reflective screen arranged in front of the user.
Further, the support mechanism supports the projector, the transmission screen, the first reflecting mirror, the second reflecting mirror, and the cushion so that they can be retracted from the front of the user without changing the relative positions of the projector, the transmission screen, the first reflecting mirror, the second reflecting mirror, and the cushion. It is characterized by doing.
The image processing device further includes an image changing unit that detects at least one of a facial expression of the user, a position where the user touches the cushion, and a deformation state of the cushion when the user touches, and changes the main image based on a detection result. It is characterized by.

Moreover, the recessed part for storing a sperm extractor or a small massager is provided in a part of said cushion, It is characterized by the above-mentioned.
In addition, a storage member having a recess for storing a sperm collector or a small massage device is provided separately from the cushion, and the storage member is supported by the support mechanism. .
Further, a sound collecting device is provided in the recess.
The video communication system of the present invention includes the display system at the first point and the second point, and each display system includes a self-portrait camera that shoots each user, and the self-portrait camera captures the image. It is characterized in that an image of a user at one point is transmitted via the Internet line and projected onto the transmissive screen via the projector used by the user at the other point.
The position coordinate detection method of the present invention is a position coordinate detection method for detecting a position where a user touches the cushion in the display system and a deformation state of the cushion when touched by the image changing means. Silhouette image data obtained by projecting the cushion covered with the colored bag onto two coordinate planes among the three coordinate planes defined by the three coordinate axes of the three-dimensional coordinate system. Projecting the image into the image changing means, removing the colored bag from the cushion, and projecting a state in which a user who has colored fingers or palm is touching the cushion onto the two coordinate planes. Capturing the obtained moving image data into the image changing means; and the image changing means includes the silhouette image And a step of detecting in real time the position of the cushion touched by the user and the deformation of the cushion when touched by analyzing the color data by superimposing the data and the moving image data. And
Further, the cushion is divided into a plurality of regions, each region is covered with a bag of a different color, the silhouette image data of each region is obtained and taken into the image changing unit, and the image changing unit is Detecting in real time which area the user is touching.

In the present invention, a main image from a projector is projected onto a transmission screen arranged in front of the user's eyes. The user can simulate the physical contact with a close person by touching the cushion while viewing the main image through the cushion.
In particular, the display system does not change the relative distance between the projector and the transmissive screen even if the user moves in any direction, back and forth, left and right, sits on a chair, assumes a supine posture, or moves. Therefore, there is no need to correct the focus of the lens of the projector, and the user can always see the main image of a close person or the like in a clear state in front of his eyes regardless of the posture / state.
In general, since a main image is projected onto a transmissive screen without using a liquid crystal monitor or the like that is a heavy object, the weight of the display system can be reduced.

In addition, by using two reflectors, the first reflector and the second reflector, the optical path from the projector to the transmissive screen can be lengthened, and the main image can be enlarged to a large size to make a transmissive screen. Can project.
Also, by using a transmissive screen instead of a reflective screen, the first and second reflective mirrors can be placed on the back side (the side farther from the user) than the transmissive screen, so it must be placed in front of the user's eyes. Space for certain cushions and transmission screens can be reserved.
Moreover, the size of the main image can be freely changed from the minimum to the maximum by using the projector. Furthermore, when used in a dark environment where the illuminance is low in the room, it is possible to enjoy a fantastic sensation of touching the main image in the dark.

By wearing this display system other than the head, the user can see the main image when the face is turned forward, and can see the surroundings of the user when the face is turned sideways. In this way, the use state and the non-use state can be switched freely and simply.
Further, when the user turns sideways with the body, the display, the transmissive screen, and the like also turn together, so that the interval at which the main image moves with the user can be tasted.
If this display system is mounted on the back of a chair, the weight applied to the user can be distributed, and the burden on the user can be reduced. Further, even when the chair is rotated or reclined, the relative position between the user and the transmission screen does not change, so that the main image can always be seen in front of the eyes.
Further, by using the optical path adjusting mechanism, it is possible to adjust the size of the main image projected onto the transmission screen in accordance with the content of the main image, the size and shape of the cushion, and the like.

Also, an AR (Augmented Reality) system can be constructed by taking an image around the user with the first camera and combining the taken image with the main image by the processing means. In particular, by mounting the display system other than the head, it is possible to construct an AR system that synchronizes only with the direction of the torso without synchronizing with the turning direction of the face (neck). By projecting the image of the user's surroundings (especially in front) on the transmissive screen, safety when walking with the display system attached can be improved. Note that the image captured by the first camera is not limited to the front of the user, but may be an image of a foot or sky.
Further, if the transmission type screen is disposed inside or on the surface of the cushion, a mechanism for supporting the transmission type screen becomes unnecessary, and thus the weight can be reduced. Moreover, since the transmissive screen comes into contact with the cushion, the reality can be improved.
In addition, if a part of the main image is projected onto a transmissive screen and the remaining image is projected onto a reflective screen placed in front of the user, the user can project a distant landscape on the reflective screen, for example. Can enjoy images with perspective.

Further, if the support mechanism supports the projector and the transmissive screen and the like so as to be retractable from the user's eyes without changing the relative positions of the projector and the transmissive screen, the support mechanism can be easily retracted during work. Further, if the transmissive screen is moved to the side of the user, it is possible to obtain a sense of realism that a close person exists beside the user.
Further, if the image changing means is provided, it is possible to obtain a display system having an interactive property in which an image reacts to a user's action.

Moreover, if a recess for storing a sperm collector or small massager is provided in a part of the cushion or separately from the cushion, sexual body contact with a close person in various postures is simulated. You can experience it.
In addition, if the sound collecting device is provided in the recess, it is possible to analyze the frictional sound generated when the user moves the male or female genital in the recess, based on the volume, sound pressure, waveform, etc. The main image projected from the projector can be changed. In addition, when the user is a male, there is an advantage that insertion of male genitals can be detected regardless of the size of male genitals. In addition, since the sound collecting device is close to the source of frictional sound, there is an advantage that it is difficult to pick up other sound noise. Note that a simple analysis process may be used in which the power is turned on when a frictional sound of a certain level or more is detected, and the others are turned off.
In addition, if a display system equipped with a camera for self-shooting is arranged at the first point and the second point and these are connected by an Internet line, it is possible to experience bodily contact while observing each other's moving images.
In addition, according to the position coordinate detection method of the present invention, the position where the user touches the cushion and the degree of deformation when the cushion is touched can be detected in real time simply by covering the cushion which is entirely transparent or partially transparent. be able to.
Further, if the cushion is divided into a plurality of regions and each region is covered with a bag of a different color, the position touched by the user can be detected for each region.

Overall side view (a), enlarged view (b) showing the user wearing the display system, and figure (c) showing the user touching the cushion Diagram showing cushion variations Overall side view (a) showing the user wearing the display system and (b) showing the state where the transmissive screen is provided inside the cushion, etc. Enlarged view (a) showing the user wearing the display system and enlarged view 状態 (b) showing the state where the position of the transmissive screen and cushion is switched The figure which shows the variation of the state where the user looked through the transparent part of the cushion which the main picture which is shown in the transmission type screen Figure (a) showing the user standing upright and leaning forward (a), figure showing the bed lying on his back (b), figure showing the reclining while sitting on a chair (c) A figure showing a state in which the main image is projected on the ceiling surface of the room lying on the bed (a) and a figure showing a state in which the main image is projected on the wall surface of the room while reclining while sitting on a chair (b) The figure which shows the state which mounted | wore the chair with the support mechanism using the fixing tool An overall side view (a) showing a user wearing a display system, etc., a plan view (b) showing a state where only the neck is turned, and a plan view (c) showing a state where the body is turned. Overall side view (a) showing the user wearing the display system, etc. and (b) showing the enlarged first and second reflecting mirrors Diagram for explaining composite image Diagram for explaining composite image A diagram showing the structure of the arm rotation support mechanism (a) and a diagram for explaining the operation (b) The figure which shows the state which moved the transmission type screen etc. to the user's left side (a) and the top view which shows the state which moved the transmission type screen etc. right and left (b) Diagram showing the structure of the connecting rod Side view showing a display system with an XZ coordinate detection camera and a YZ coordinate detection camera The figure which shows the state which covered the colored bag on the cushion Figures (a) to (d) showing how to calculate the XYZ coordinates and displacement in the XYZ directions at the position where the user touched the cushion A diagram showing the case of using one camera (a), a diagram (b) showing a state before changing the main image, and a diagram (c) showing a state after changing Flow chart showing an example of a program for detecting displacement in the XYZ coordinates and XYZ directions The figure which shows the state which projected the live image of the close person who is in the remote place on the transmission type screen via the Internet line Diagram for explaining the recess Diagram showing how to attach a sperm collector or small massager to a recess using a spacer Figure (a) showing the display system being used by men (a), Figure (b) showing the state being used by women Figure (a) showing the user standing upright and leaning forward (b), figure showing the user lying on his back (b), and sitting on a chair (c) The figure which shows the state which formed the recessed part in the storage member The figure which shows the state which attached the sound collector in the recessed part Figures (a) and (b) showing a configuration in which a transmission screen is sandwiched between front and rear cushions. Diagram showing video communication system

[First embodiment]
Hereinafter, a first embodiment of a display system of the present invention will be described with reference to the drawings.
As shown in FIG. 1, the display system 1 is roughly composed of a projector 10, a first reflecting mirror 20, a second reflecting mirror 30, a transmissive screen 40, a cushion 50, and a support mechanism 60.
As the projector 10, a commercially available projector such as a liquid crystal system, a DLP (Digital Light Processing) system, or an LCOS (Liquid Crystal On Silicon) system may be used, and it is particularly preferable to use a small and light projector.
In the present embodiment, the projector 10 is disposed above the head of the user U. However, the position of the projector 10 is not particularly limited, and for example, the projector 10 may be disposed on either the left or right side of the user U's head or torso. Good.
The projector 10 projects light (main image M) forward in the horizontal direction.
It is assumed that the main image M data is supplied to the projector 10 by known means such as a computer 72, a smartphone, a USB memory, or the like.

The first reflecting mirror 20 is disposed in front of the projector 10 with its reflecting surface inclined at 45 degrees with respect to the horizontal direction. The light from the projector 10 is reflected by the first reflecting mirror 20, so that the path is changed downward in the vertical direction.
The second reflecting mirror 30 is disposed below the first reflecting mirror 20 and its reflecting surface is inclined 45 degrees with respect to the horizontal direction. The light reflected by the first reflecting mirror 20 is reflected by the second reflecting mirror 30, so that the path is changed backward in the horizontal direction.
Although normal mirrors may be used as the first reflecting mirror 20 and the second reflecting mirror 30, for example, a thin film mirror backed by a thin plastic resin plate such as hard urethane or soft vinyl chloride is used. It is preferable because the weight can be reduced.

The transmissive screen 40 is disposed along the vertical direction in front of the user U's eyes. The light reflected by the second reflecting mirror 30 is projected onto the back surface of the transmissive screen 40 (the surface opposite to the surface facing the user U). The material and properties of the transmissive screen 40 are well known and will not be described.
When projecting light with the optical axis of the projector 10 parallel to the horizontal plane, the inclination angle of the first reflecting mirror 20 and the second reflecting mirror 30 with respect to the horizontal plane is set to 45 degrees as in the present embodiment. Is preferred. By setting the inclination angle to 45 degrees, the light reflected by the second reflecting mirror 30 can be made to enter the transmissive screen 40 arranged along the vertical direction vertically, and the distortion of the projected image is suppressed as much as possible. be able to. In other words, when the transmissive screen 40 is disposed so as to be inclined with respect to the vertical direction, the projector 10, the first reflecting mirror 20, and the second reflecting mirror are arranged so that light enters the transmissive screen 40 vertically. Adjust the mounting angle of 30.

The cushion 50 is a member having a transparent part at least in part, and is disposed in front of the user U's eyes. In the present embodiment, the entire cushion 50 is formed in a hollow shape by a material having transparency and flexibility, such as soft vinyl chloride. That is, in the present embodiment, the entire cushion 50 constitutes a transparent portion. When a part of the cushion 50 is a transparent part, the part other than the transparent part may be manufactured with colored soft vinyl chloride or rubber, or the part may be covered with a cover such as cloth.
The cushion 50 may be filled with a transparent liquid such as water or a substance such as a gel having transparency, but is preferably filled with air in order to reduce the weight.
As shown in FIG. 2, the shape of the cushion 50 is not particularly limited, and can be changed according to the main image M to be projected on the transmission screen 40. Further, as shown in FIG. 3, the transmission screen 40 may be provided inside the cushion 50, or the transmission screen 40 may be attached to the surface of the cushion 50. Furthermore, as shown in FIG. 4 (b), the transmission screen 40 may be disposed closer to the user U than the cushion 50.

The support mechanism 60 is provided to support the projector 10, the transmissive screen 40, the first reflecting mirror 20, the second reflecting mirror 30, and the cushion 50. Specifically, the support mechanism 60 includes a column 61, a projector arm 62, a first reflector arm 63, a second reflector arm 64, a cushion arm 65, a shoulder belt 66, and a waist belt 67.
The column 61 is a rod-like or plate-like member that extends in the vertical direction at the back of the user U. The column 61 is provided with a stretchable portion 68 at a part thereof.
The projector arm 62 is supported on the upper portion of the support 61 so as to be rotatable in a vertical plane. The projector 10 is fixed to the upper part of the projector arm 62 by known fixing means such as bolts and nuts.

The first reflector arm 63 includes a first member 63a that is rotatably supported in the vertical plane by the projector arm 62, a second member 63b that is rotatably supported in the vertical plane by the first member 63a, The third member 63c is rotatably supported by the two members 63b in a vertical plane.
The second member 63b and the third member 63c are each provided with a stretchable portion 68 so that the length can be adjusted. The first reflecting mirror 20 is supported by the third member 63c so as to be rotatable in the vertical plane.
The second reflector arm 64 includes a first member 64a that is rotatably supported by the support 61 within a vertical plane, a second member 64b that is rotatably supported by the first member 64a within a vertical plane, and a second member. The third member 64c is rotatably supported on the vertical plane by 64b, and the fourth member 64d is rotatably supported by the third member 64c on the vertical plane.
The second member 64b, the third member 64c, and the fourth member 64d are each provided with a stretchable portion 68 so that the length can be adjusted. The transmissive screen 40 is supported by the third member 64c so as to be rotatable in the vertical plane. The second reflecting mirror 30 is supported by the fourth member 64d so as to be rotatable in the vertical plane.

The cushion arm 65 includes an upper holding member 65a that holds the upper portion of the cushion 50 and a lower holding member 65b that holds the lower portion. One end of each of the upper holding member 65a and the lower holding member 65b is supported by the column 61 so as to be rotatable in the vertical plane, and the cushion 50 is held at the other end. The upper holding member 65a and the lower holding member 65b are provided with a stretchable part 68 at a part thereof.
The shoulder belt 66 has an arm hole 66a for passing the user U's arm in the center thereof, and is fixed to the support 61.
The waist belt 67 has an annular shape by fixing both ends to the support 61, and holds the waist of the user U.
In addition, each extension / contraction part 68 with which the support | pillar 61, the 1st reflector arm 63, and the 2nd reflector arm 64 are equipped functions as a part of optical path adjustment mechanism for changing the optical path from the projector 10 to the transmissive screen 40. .

According to the display system 1 of the present embodiment, as shown in FIG. 5, the user U can view the main image M reflected on the transmissive screen 40 through the transparent portion of the cushion 50. If the main image M is a moving image of a close person, the user U can feel the physical contact with the close person by touching the cushion 50 in a pseudo manner.
Also, by adjusting the length of each telescopic part 68 as the optical path adjustment mechanism and adjusting the mounting angle of each arm 62, 63, 64, 65, the size of the user U, the size of the cushion 50, the main image M Various adjustments according to dimensions and the like are possible. The user U may select the size and shape of the cushion 50 and the size of the main image M according to his / her preference.

FIG. 6 shows variations of postures that can be taken by the user U with the display system 1 attached. Fig. 6 (a) shows the user U standing upright and leaning forward, Fig. 6 (b) is lying on the bed, Fig. 6 (c) is sitting on the chair and reclining. In either case, the distance from the eyes of the user U to the transmissive screen 40 does not change.
As shown in FIG. 7 (a), the first reflector 20 and, if necessary, the second reflector 30 in a state of lying on the bed or sitting in a chair and reclining as shown in FIG.7 (b). The main image M may be projected onto the indoor ceiling or wall surface by rotating.
Further, as shown in FIG. 8, a support mechanism 60 is attached to a U-shaped fixture 69 having an opening 69a below, and the backrest of the chair is inserted into the opening 69a so that the support mechanism 60 is attached to the chair. You may wear it.

Further, as shown in FIG. 9 (b), the user U can remove the line of sight from the transmission screen 40 by turning only the neck to the left and right, and the torso turns to the left and right as shown in FIG. 9 (c). By doing so, it is possible to continue viewing the main image M without removing the line of sight from the transmissive screen 40.
FIG. 10 shows a modification of the display system 1 of the present embodiment. As shown in FIG. 10 (b), it is possible to view the large main image M by enlarging the first reflecting mirror 20 and the second reflecting mirror 30, and extending each telescopic portion 68 to a position far from the user U. it can. In this case, the second reflecting mirror 30 may be supported using a caster 70.

Further, as in the modified example of FIG. 11, the main image is obtained by the first reflecting mirror 20 and the second reflecting mirror 30 by adjusting the inclination angle of the first reflecting mirror 20 with respect to the horizontal plane or the distance to the second reflecting mirror 30. Only a part of M (the heart image, FIG. 11 (a)) is reflected and projected onto the transmissive screen 40, and the remaining image of the main image M that is not reflected by the first reflecting mirror 20 and the second reflecting mirror 30. (Child image FIG. 11B) may be projected onto the reflective screen 41 arranged in front of the user U. In this case, the main image M is projected from the projector 10 with a heart image on the upper side and a child image on the lower side. User U can see the composite image shown in FIG.
The main image M is an image projected directly without a mirror, while the heart image is a projected image that passes through the first reflecting mirror and the second reflecting mirror. This can be corrected on the output device side such as a computer so that the projector 10 projects a heart image that is inverted upside down in advance.
In addition, as in the modification of FIG. 12, the first camera 71 that captures an image around the user U is attached via the arm mechanism attached to the support 61, and the image (tree image diagram) taken by the first camera. The composite image (FIG. 12 (c)) obtained by combining the image 12 (a) and the main image M (the child image FIG. 12 (b)) by the computer 72 (processing means) may be output from the projector 10. .

[Second Embodiment]
A second embodiment of the present invention will be described, but the same reference numerals are given to portions having the same configuration as the first embodiment, and the description thereof is omitted.
In the display system 2 according to the present embodiment, the support mechanism 60 integrates the projector 10, the transmission screen 40, the first reflecting mirror 20, the second reflecting mirror 30, and the cushion 50 without changing the relative positions thereof. It is characterized in that it is supported so as to be retractable from the front of the user U.
Specifically, as shown in FIG. 13, the projector arm 62, the first reflector arm 63, the second reflector arm 64, and the cushion arm 65 that constitute the support mechanism 60 are supported by the arm rotation support mechanism 73. 61 is rotatably supported in a horizontal plane. As the arm rotation support mechanism 73, it is preferable to use a mechanism such as a torque hinge that can maintain the position after rotation.
As shown in FIG. 13 (b) and FIG. 14 (b), the user U can change the position of the transmissive screen 40 and the like in the horizontal plane, so that it is transparent while sitting on a chair and working on a personal computer or the like. If the main image M is continuously projected with the mold screen 40 rotated to the left side of the mold screen 40, the user can work while feeling the presence of a close person on the left side.
As shown in FIG. 15, the projector arm 62 may be attached to the cushion arm 65 and the arm rotation support mechanisms 73 may be connected to each other by a connecting rod 74. In this case, the rigidity of the support mechanism 60 can be increased and fine vibration can be suppressed.

[Third embodiment]
A third embodiment of the present invention will be described, but the same reference numerals are given to portions having the same configuration as the above-described embodiments, and the description thereof is omitted.
The display system 3 according to the present embodiment detects at least one of the facial expression of the user U, the position where the user U touches the cushion 50, and the deformation state of the cushion 50 when touched. It is characterized in that it includes image changing means (computer 72) for changing the image M.
Specifically, as shown in FIG. 16, an XZ coordinate detection camera 75 for photographing a horizontal plane (XZ plane) and a YZ coordinate detection camera 76 for photographing a vertical plane (YZ plane) are provided. . Further, as shown in FIG. 17, the cushion 50 is covered with a colored bag 78 (hereinafter also referred to as “colored bag 78”). The shape and dimensions of the cushion 50 may be selected as appropriate according to the contents of the main image M, and different colors of bags 78 may be put on the cushion 50 according to the part whose coordinates are to be detected.

Next, an example of the position coordinate detection method by the image changing means will be described.
The method includes a step of covering a cushion with a colored bag, and projecting the cushion covered with the colored bag onto two coordinate planes among three coordinate planes defined by three coordinate axes of a three-dimensional coordinate system. A step of taking the obtained silhouette image data into the image changing means, a step of removing the colored bag from the cushion, and a state where a user in a state of coloring a finger or palm is touching the cushion, the two coordinates The step of capturing the moving image data obtained by projecting on a plane into the image changing unit, and the image changing unit superimposing the silhouette image data and the moving image data to analyze the color data, thereby allowing the user to And a step of detecting in real time the position where the cushion is touched and the degree of deformation of the cushion when touched. With a butterfly.

Specifically, first, as shown in the upper diagram of FIG. 18 (a), a cushion 50 simulating the upper body of a person is used, and a bag 78 of a different color is put on the head and torso. As shown in the lower diagram of FIG. 18 (a), only the head of the cushion 50 may be covered with the coloring bag 78.
Then, as shown in FIG. 18 (b), the cushion 50 before being touched by the user U is photographed by the XZ coordinate detection camera 75 and the YZ coordinate detection camera 76, and taken into the computer 72 as silhouette image data.
Since the colored bag 78 is unnecessary after the silhouette image is once taken into the computer 72, it is detached from the cushion 50 and returned to the original cushion 50 as shown in the upper part of FIG. 18 (c). Furthermore, as shown in the upper and lower views of FIG. 18 (c), when the user U touches the cushion 50 in that state with finger bags 79 of different colors attached to the left and right fingertips (or the entire palm), The moving image in such a state is taken by the XZ coordinate detecting camera 75 and the YZ coordinate detecting camera 76, and each moving image data is transmitted to the computer 72.
As shown in FIG. 18 (d), the computer 72 superimposes the silhouette image data and the moving image data previously captured and analyzes the color data to analyze the XYZ coordinates of the position where the user U is touching the cushion 50 and the cushion 50. The degree of deformation (displacement in the XYZ directions) is detected in real time.

In other words, from the XZ coordinate detection camera 75 and the YZ coordinate detection camera 76 once covered with the colored bag 78 in order to detect the position of touching the cushion 50 that is entirely transparent or partially includes a transparent portion. The position and shape of the cushion 50 are determined through the silhouette photographed image and stored in advance. Then, when the user U is touching the transparent cushion 50 in normal use with the colored bag 78 removed, the position and shape of the cushion 50 stored in advance are superimposed on the computer 72 to touch it. The XYZ coordinates and the degree of deformation of the cushion 50, that is, how much the fingertip is buried in the cushion 50 by touching is detected from the XYZ coordinates of the same fingertip. Is simply a device for facilitating extraction of color data by the computer 72. With this device, it is possible to identify and acquire the silhouette of the transparent cushion 50, which is difficult to identify on the computer by color data analysis. Is possible. In addition, this coloring mechanism can be colored only by attaching and detaching the bag, and a mechanism that can easily color the cushion 50 as described above is desirable when it is assumed that the cushion 50 having various shapes is switched as in this display system. If necessary, it may be expanded to obtain a silhouette by dividing it into finer parts and covering with different colored bags 78.
The computer 72 changes the main image M based on the detection result (XYZ coordinates and displacement in the XYZ directions). For example, when the main image M is an image of a child, if the user U detects that the child's head is stroked, the smiling child from the state of FIG. 19 (b) to the state of FIG. 19 (c) The image is changed and projected from the projector 10.
In detecting the displacement in the XYZ coordinates and the XYZ directions, the cushion 50 and the user U's as shown in FIG. 19 (a), not the two cameras of the XZ coordinate detection camera 75 and the YZ coordinate detection camera 76. A single camera 77 that can simultaneously photograph facial expressions may be used and detected by a well-known motion capture technique.

FIG. 20 shows an example of a program for detecting displacement in the XYZ coordinates and XYZ directions by the computer 72, but the program is not necessarily limited to this program example.
In addition, as shown in FIG. 21, a live image of a close person at a remote location is projected onto the transmissive screen 40 via the Internet line, and the position where the user U touches the cushion 50 is the remote location. It may be displayed on the display D.

The configuration shown in the present embodiment has the following advantages.
-Even when the cushion 50 is changed according to the dimensions of the main image M, it is possible to cope with it only by re-taking the silhouette image of the cushion 50.
Even when the distance from the XZ coordinate detection camera 75 and the YZ coordinate detection camera 76 to the cushion 50 is changed, it can be dealt with by simply taking a silhouette image of the cushion 50 again.
-Since the cushion 50 is transparent, if the cross section along the XZ plane or the YZ plane does not change, such as the shape of the cushion 50 is cylindrical, rectangular parallelepiped, cross-section kamaboko, etc., only silhouette image data from the above two cameras The displacement in the XYZ coordinates and the XYZ directions at the position touched by the user U can be calculated.
-By giving the coloring bag 78 flexibility, silhouette image data can be created simply by analyzing the color data of each pixel of the image, so that silhouette image data of the cushion 50 of various shapes can be created easily.
-Since it is only necessary to determine whether the color of the fingertips overlaps only the pixels of the coordinates on the silhouette that have been read in advance, because the colors of the left and right fingertips are different, the discrimination of the right and left fingertips is also Since only numerical data comparison is necessary, the speed can be increased.
-Compared to shape comparison and pattern recognition, it is faster because it only compares and determines the numerical value of the color data of each pixel.
・ If you use colored bags 78 of different colors for each part of the cushion 50 and create silhouette image data for each part in advance, for example, if you have silhouette image data for the head and torso, only the head When it is desired to know whether or not it is touching, it is only necessary to analyze whether or not only the pixel portion on the silhouette of the head is the color of the fingertip.
-The cushion 50 after the coloring bag 78 is removed can be used as it is.
Even if the cross section along the ZX plane does not change in the Y direction, for example, like a cylindrical cushion 50, the head and the body are forcibly covered with different colored bags 78 when creating a silhouette image. It can be handled as if it were a cushion 50 having a head and a torso.

[Fourth embodiment]
The fourth embodiment of the present invention will be described, but the same reference numerals are given to portions having the same configuration as the above-described embodiments, and the description thereof is omitted.
The display system 4 of the present embodiment is characterized in that a recess 90 for storing the sperm collector 91 or the small massager 92 is provided in a part of the cushion 50.
Specifically, as shown in FIG. 22, a recess 90 extending in the front-rear direction is formed in a portion where the cushion 50 is extended downward. As shown in FIG. 23, a sperm collector 91 or a small massager 92 is inserted into the recess 90. Several types of spacers 93 which are cylindrical members made of a low resilience material such as urethane and whose thickness is changed may be prepared. The spacer 93 may be appropriately replaced so as to match the size of the recess 90 with the size of the sperm collector 91 or the small massager 92. Since the spacer 93 fitted in the recess 90 is fixed by the pressure P from the surroundings, it is difficult to be displaced during use. The spacer 93 may be completely fixed to the recess 90 with a hook-and-loop fastener or a hook.

FIG. 24 (a) shows a state where the male is using the display system 4, and FIG. 24 (b) shows a state where the female is using it. As shown in FIG. 25, the user can give stimulation to his / her body part using the sperm collector 91 and the small massager 92 in various postures while viewing the main image M.
As shown in FIG. 26, a storage member 94 in which a recess 90 is formed may be provided separately from the cushion 50. The storage member 94 is supported by the support mechanism 60.

Alternatively, a sound collecting device 95 may be provided in the recess 90 as shown in FIG. The sound collecting device 95 picks up the friction sound generated when the user U moves the male genital in the concave portion 90 or presses the female genital against the small massage device 92 attached to the concave portion 90. Send to (computer 72). The computer 72 analyzes the audio data, and changes the main image M projected by the projector 10 based on the volume, sound pressure, waveform, and the like. As shown in FIG. 28, the transmission screen 40 may be sandwiched between the front and rear cushions 50. In this case, the end surfaces of the front and rear cushions 50 may be connected by known connecting means such as a hook-and-loop fastener, a button hook, and a chuck. Further, in order to prevent a situation in which wrinkles occur on the transmissive screen 40, it is preferable to dispose a transparent plate 96 made of acrylic or glass in front of or behind the transmissive screen 40.

Next, an embodiment of the video communication system of the present invention will be described.
As shown in FIG. 29, the video communication system 100 includes the display system 1 at the first point P1 and the second point P2. Each display system 1 includes a self-portrait camera 97 that captures each user U1, U2, and transmits an image of the user U1 at one point captured by the self-portrait camera 97 via the Internet line. The image is projected onto the transmissive screen 40 via the projector 10 used by the user U2 at the other point.

Specifically, moving image data captured by the self-portrait camera 97 of one user U1 at the first point P1 is taken into the computer 72. Then, after performing known processing such as compression and encryption for security, it is converted into packet data and sent to the computer 72 of the other user U2 at the second point P2 via the Internet.
The computer 72 of the other user U2 performs well-known processing such as decompression and decoding on the received moving image data, and projects it onto the transmissive screen 40 via the projector 10. The moving image data may include audio data. Further, when the sound collecting device 95 is disposed in the recess 90, the frictional sound may be transmitted as it is to the other display system 1.
Similarly, on the other user U2 side, the moving image data of the other user U2 is sent to one user U1 side and projected onto the transmissive screen 40 via the projector 10 of one user U1.
According to this video communication system 100, for example, if a sperm collector 91 is attached to the concave portion 90 of one user U1 and a small massage device 92 is attached to the concave portion 90 of the other user U, pseudo-action between remote locations is possible. become.

The present invention relates to a lightweight display system, a video communication system, and a position coordinate detection method capable of simulating bodily contact with a person close to a user and switching between a use state and a non-use state freely and easily. Yes, it has industrial applicability.

D display
M Main image
P pressure
P1 1st point
P2 Point 2
U user
1 Display system
2 Display system
3 Display system
4 Display system
10 Projector
20 First reflector
30 Second reflector
40 transmission screen
41 reflective screen
50 cushion
60 Support mechanism
61 prop
62 Projector arm
63 Arm for first mirror
63a 1st member
63b Second member
63c 3rd member
64 Second reflector arm
64a 1st part
64b Second part
64c 3rd part
64d 4th member
65 Cushion arm
65a Upper holding member
65b Lower holding member
66 Shoulder Belt
66a armhole
67 waist belt
68 Telescopic part
69 Fixing
69a opening
70 casters
71 1st camera
72 Image change means
73 Arm rotation support mechanism
74 Connecting rod
75 XZ coordinate detection camera
76 YZ coordinate detection camera
77 cameras
78 Colored bags
79 Finger Bag
90 recess
91 Sperm collector
92 Small massage device
93 Spacer
94 Containment member
95 Sound collector
96 plates
97 Selfie camera
100 video communication system

Claims (12)

1. A projector,
   A transmissive screen placed in front of the user's eyes;
   A first reflecting mirror and a second reflecting mirror that reflect the main image output from the projector and project it on the transmissive screen;
   A cushion having a transparent part at least in part and arranged in front of the user's eyes;
   A support mechanism for supporting the projector, the transmission screen, the first reflecting mirror, the second reflecting mirror, and the cushion;
   The transmission screen is disposed between the second reflecting mirror and the cushion;
   The support mechanism is attached to the user's torso,
   A display system, wherein a user can see the main image projected on the transmissive screen through the transparent portion and can see the surroundings by changing the direction of the face.
2. 2. The display system according to claim 1, wherein the support mechanism includes an optical path adjusting mechanism for changing an optical path from the projector to the transmissive screen.
3. 2. A first camera that captures an image around the user, and a processing unit that combines the image captured by the first camera and the main image and outputs the combined image from the projector. Or the display system of 2.
4. The main image in which the first reflecting mirror and the second reflecting mirror reflect only a part of the main image and project on the transmission screen and are not reflected by the first reflecting mirror and the second reflecting mirror. 4. The display system according to claim 1, wherein the remaining image is projected onto a reflective screen disposed in front of the user.
5. The supporting mechanism supports the projector, the transmissive screen, the first reflecting mirror, the second reflecting mirror, and the cushion so that they can be retracted from the front of the user without changing the relative positions of the projector, the transmissive screen, the first reflecting mirror, the second reflecting mirror, and the cushion. The display system according to any one of claims 1 to 4, wherein:
6. Image change means for detecting at least one of a facial expression of the user, a position where the user touches the cushion, and a deformation state of the cushion when touched, and changing the main image based on a detection result is provided. The display system according to any one of claims 1 to 5.
7. The display system according to any one of claims 1 to 6, wherein a part of the cushion is provided with a recess for storing a sperm collector or a small massager.
8. The storage member in which a concave part for storing a sperm collector or a small massage device is formed separately from the cushion, and the storage member is supported by the support mechanism. The display system according to any one of 1 to 6.
9. 9. A display system according to claim 7, wherein a sound collecting device is provided in the recess.
10. 10. The display system according to any one of claims 1 to 9 is provided at a first point and a second point, and each display system is provided with a self-shooting camera for photographing each user. A video communication system, wherein an image of a user at one point photographed by a camera is transmitted via an Internet line and projected onto the transmissive screen via the projector used by a user at the other point.
11. The position coordinate detection method for detecting the position where the user touched the cushion and the degree of deformation of the cushion when touched by the image changing means in the display system according to claim 6,
    Covering the cushion with a colored bag;
    Capturing the silhouette image data obtained by projecting the cushion covered with the colored bag onto two coordinate planes defined by the three coordinate axes of the three-dimensional coordinate system into the image changing means; ,
    Removing the colored bag from the cushion;
    Capturing the moving image data obtained by projecting on the two coordinate planes a state in which the user in a state of coloring a finger or palm is touching the cushion;
    The image changing means superimposes the silhouette image data and the moving image data to analyze color data, thereby detecting in real time the position where the user touched the cushion and the deformation of the cushion when touched. A position coordinate detection method comprising the steps of:
12. Dividing the cushion into a plurality of regions and covering each region with a different color bag; and
    Obtaining the silhouette image data of each region and taking it into the image changing means;
    12. The position coordinate detection method according to claim 11, wherein the image changing means includes a step of detecting in real time which area the user is touching.

Images