JP7410523B1 - display device - Google Patents

Abstract

The present invention provides a display device capable of expressing a pseudo three-dimensional space in a soundproof space. [Solution] A display device that displays an image in a preset display range, and includes a control unit and an image forming unit that outputs a concatenated image generated from a plurality of captured images in a visible format. , the display range is provided in a soundproof room equipped with a housing 32 that forms a cubic or rectangular parallelepiped soundproof space into which the user can enter, and the control unit is configured to cover three consecutive inner walls of the housing surrounding the user. With the three side wall surfaces (projection surfaces SCL, SCF, and SCR) as a display range, a cutout image corresponding to the display range is generated from the connected image, and an output command for the cutout image is given to the image forming section. [Selection diagram] Figure 9

Description

Article 30, Paragraph 2 of the Patent Act applies Website publication date: April 18, 2020 Website address: https://jp. vcube. com/news/release/20220418-1530. html https://www. youtube. com/watch? v=ItFn3Dnbdpw

The present invention relates to a display device, and more particularly to a display device that displays images in all directions in space.

A spherical camera has been developed that can capture 360 degrees of the top, bottom, left, and right as a single spatial image. For example, if a part of a spherical image is displayed on a head-mounted display (also referred to as HMD) and the range of the display screen changes according to the movement of the user's head, the user can feel as if he or she is inside the space of the displayed image. You can feel as if you have entered the world (for example, see Patent Document 1).

Additionally, in recent years, as companies have become increasingly interested in telework, soundproof rooms have been developed for teleworkers. These soundproof rooms are equipped with IoT (connection of furniture and networks), and users who use the soundproof rooms are users who use external devices with video conference functions using video conference terminals installed in the room. (For example, see Patent Document 2.)

JP 2017-55173 Publication JP2019-28995A

When viewing spherical images on an HMD, a highly immersive viewing experience can be obtained, but the display is limited depending on the effective viewing angle for both eyes (approximately 120 degrees), and as shown in Figure 24, in reality The area displayed (HMD display range) is not wide. Therefore, it would be convenient if a naturally visible image could be reproduced in a soundproof space, and in this case, the hassle of preparing and wearing an HMD would be eliminated. However, in the field of soundproof rooms, there has been no idea of creating a three-dimensional spatial image inside the room, and although there has been an idea of emphasizing a closed-in feeling suitable for work or study, on the contrary, there has been no idea of creating a sense of spatial unity with the outside. No consideration had been given to fostering this.

Therefore, an object of the present invention is to provide a display device capable of expressing a pseudo three-dimensional space in a soundproof space.

In order to achieve the above object, the display device of the present invention is a display device that displays an image in a preset display range, and includes a control unit and a format that allows visualizing a concatenated image generated from a plurality of captured images. the display range is provided in a portable soundproof room equipped with a casing forming a rectangular parallelepiped soundproof space into which a user can enter, and the soundproof space is The width direction is larger than the depth direction from a door provided on one side of the body, and the height direction is larger than the depth direction, and the control unit controls the user. Among the inner wall surfaces of the surrounding housing, excluding one side having the door, three consecutive side wall surfaces are set as a display range, and a cutout image corresponding to the display range is generated from the connected image, and the image forming unit an output command for the cut out video, and the display range is arranged such that the top end thereof is aligned with the height of the ceiling surface of the casing, and the display range is provided to cover the entire width direction of each of the three consecutive side wall surfaces. It is characterized by being

Further, the controller includes a controller capable of changing a display viewpoint in response to the user's operation, and a detection unit that detects the position of the user's head, and the control unit, based on the detection result of the detection unit , The present invention is characterized in that the display viewpoint is corrected so that the height of the display viewpoint becomes the user's line of sight .

According to the display device of the present invention, three consecutive side wall surfaces among the wall surfaces within the housing forming a cubic or rectangular soundproof space are set as the display range, and a cutout video corresponding to the display range is generated from the connected video. Therefore, it is possible to make the display area to be simultaneously displayed larger than the viewing angle of the user, and it is possible to create a sense of spatial unity with the outside as a pseudo three-dimensional space expressed in a soundproof space.

FIG. 1 is a configuration diagram of a display device showing a first embodiment of the present invention. It is a schematic diagram showing a connected image as well. It is an explanatory view showing an example of the case where a connected video is similarly converted into a cube map. It is an explanatory view showing a display object in a cube similarly. FIG. 6 is an explanatory diagram showing an example of a case where a cutout video is similarly projected onto a projection surface. It is a perspective view showing the appearance of a soundproof room similarly. It is also a front view. Similarly, it is a plan sectional view. Similarly, it is a front sectional view. FIG. 1 is a diagram for explaining an overview of a display system. It is a figure which shows an example of the image (scenery of a tourist spot) similarly displayed in a soundproof room. It is a figure showing an example of an image (model room) similarly displayed in a soundproof room. FIG. 3 is a diagram showing an example of an image (meeting room) similarly displayed in a soundproof room. FIG. 7 is a plan sectional view of a soundproof room in a second embodiment of the present invention. It is also a front cross section. FIG. 7 is a plan sectional view of a soundproof room in a third embodiment of the present invention. It is also a front cross section. FIG. 7 is a plan sectional view of a soundproof room in a fourth embodiment of the present invention. It is also a front cross section. It is a figure for explaining the outline of the display system in the 5th example of the present invention. FIG. 7 is an image diagram showing an example of a case where a space inside a soundproof room is shared between users who are also located in remote locations. It is a figure which shows an example of the image (meeting room of virtual space) displayed in the soundproof room in the 6th embodiment of this invention. FIG. 6 is a diagram showing an example of a video (entrance of a virtual space office) similarly displayed in a soundproof room. FIG. 2 is a diagram for explaining the characteristics of an HMD as a conventional technology.

1 to 13 show a first embodiment of a display device of the present invention. First, the basic configuration and operating principle of the display device shown in this embodiment will be explained. "Video" in this specification means an image or a moving image. As shown in FIG. 1, the display device 11 mainly includes three projectors 12, 13, and 14 that project images onto three walls defining a soundproof space as projection surfaces SC, and an information processing device 15. , a hand-held controller (remote control) 16 for operating these, and a head detection camera 17 for detecting the position of the head of the user who is the user of the display device 11.

The projectors 12, 13, and 14 are short-focus projectors that are installed directly below or above the projection surface SC and project images onto a projection surface at a short distance. , a video input section 20, a Pr video processing section 21, a video forming section 22, and the like. Although not shown, the projectors 12, 13, and 14 include, for example, a power supply circuit that obtains commercial power and supplies it to each part, an audio processing unit that processes audio information, and a speaker that outputs audio based on the audio information. It is equipped with an audio output section such as.

The Pr control unit 18 is configured to include one or more processors, and centrally controls the operations of the projectors 12, 13, and 14 by operating according to a control program stored in the Pr storage unit 19.

The Pr storage unit 19 includes a RAM (Random Access Memory) that is a volatile memory and a ROM (Read Only Memory) that is a nonvolatile memory. The RAM is used to temporarily store various data, and the ROM stores control programs and data for controlling the operations of the projectors 12, 13, and 14. The control program also includes a program for determining priorities and performing processing according to the contents of the control signal transferred from the information processing device 15, and accompanying data.

The video input section 20 includes a plurality of connection terminals, an interface circuit, and the like. The plurality of connection terminals are HDMI (registered trademark, High-Definition Multimedia Interface) terminals and USB (Universal Serial Bus) terminals. An information processing device 15 is connected to the cable connected to the HDMI terminal. The projectors 12, 13, and 14 and the information processing device 15 are connected to each other so that they can communicate mutual control signals in both directions using a link function provided by HDMI. In other words, both can be controlled by one controller 16 through HDMI connection.

The Pr video processing unit 21 is a control block that is substantially the same as the Pr control unit 18, and performs various processing on the video information input from the video input unit 20, and converts the processed video information into a video image. It is output to the forming section 22. For example, the Pr video processing unit 21 executes processing for adjusting image quality such as brightness and contrast, processing for superimposing images such as menu images and various messages on part or all of the supplied image, and the like, as necessary.

The image forming section 22 includes a light source, a liquid crystal light valve as a light modulating section, a projection optical system, a light valve driving section, and the like. For example, the image forming section 22 receives an output command from the Pr control section 18 and modulates the light emitted from the light source with a liquid crystal light valve to form image light. The formed image lights of each color are combined pixel by pixel by a color synthesis optical system to become image light LT representing a color image, and the image light LT is enlarged and projected from a projection optical system constituted by a projection lens, and then projected onto a projection surface. (Three walls) Images are displayed in the display range provided in the SC. The display range will be described in detail later, but is set in advance based on the shape and size of the soundproof space.

The information processing device 15 is, for example, a personal computer (hereinafter referred to as PC), and includes a control section 23, a storage section 24, a content playback section 25, a video processing section 26, a video output section 27, a user interface section 28, etc. It consists of The information processing device 15 also includes, for example, a power supply circuit that obtains commercial power and supplies it to each unit, and input units such as a mouse and a keyboard that accept user operations. In this embodiment, a stationary PC is exemplified, but the present invention is not limited to this. Devices equipped with a control unit and capable of outputting video signals, such as notebook PCs, tablet PCs, and smartphones, are terminals owned by users. It may be a device.

The control unit 23 is configured with one or more processors, and operates according to a control program stored in the storage unit 24, thereby controlling the operation of the projectors 12, 13, and 14 in cooperation with the Pr control unit 18. General control.

The storage unit 24 includes a RAM and a ROM. The RAM is used to temporarily store various data, and the ROM stores control programs, data, etc. for controlling the operation of the information processing device 15. The control program also includes a program that transmits an operation signal from the controller 16 to the projectors 12, 13, and 14 according to the content of the operation signal, and includes a process that follows instructions from the projector. The control program also includes a program for processing video and audio information of various contents acquired by the content reproduction section 25.

The content reproduction unit 25 has a function of, for example, acquiring various contents distributed from a server of a content distribution service via a communication interface and reproducing them as display contents. Content includes product information, real estate property information, tourism information, event information, news, recorded and live videos of presentations in education, welfare, medical care, and business settings, CAD (Computer Aided Design) data, or BIM (Building Information Modeling). In addition to CG (Computer Graphics) created from data, scents are added to the content by using means (for example, air cannons) to convey information (fragrance information) about the scent of the site (for example, a travel destination) or product. Various types of content can be mentioned, such as video content with odor information. The content playback unit 25 includes, for example, a DVD (Digital Versatile Disc) playback device, a BD (Blu-ray (registered trademark) Disc) playback device, a card reader device that reads data from a memory card, and a small device also called a streaming media player. This includes STB (Set Top Box).

The video processing unit 26 is a control block that is substantially the same as the control unit 23, and performs various processing on information of various contents reproduced by the content reproduction unit 25, and outputs the processed content information to the projector. The video information is converted into a format that can be used in 12, 13, and 14. The content information includes a so-called spherical video (video having a field of view in all directions) as a concatenated video generated from a plurality of captured videos. The spherical image will be described in detail later, but is used as an original when generating a desired cutout image.

The video output section 27 outputs the video information generated by the video processing section 26 to the video input section 20 of the projectors 12 , 13 , and 14 . The video output unit 27 also relays input and output of control signals using the HDMI link function. Thereby, the control section 23 can transmit and receive control signals to and from the Pr control sections 18 of the projectors 12 , 13 , and 14 via the video output section 27 . Here, in FIG. 1, for convenience of drawing, arrow lines are drawn between the control unit 23 and the Pr control units 18 of the projectors 12, 13, and 14 to indicate the exchange of data between the two, but in reality, the cables Control signals including operation signals are transmitted and received via the connected hub.

The user interface unit 28 is a communication interface between the information processing device 15 and the controller 16, and uses a short-range wireless communication function such as an industry standard infrared communication interface or Bluetooth (registered trademark). For example, the user interface section 28 receives a Bluetooth operation signal transmitted from the controller 16, decodes it, and transmits it to the control section 23.

The controller 16 encodes an operation signal corresponding to an input operation on the operation button under the control of a built-in control section, and transmits the encoded operation signal to the information processing device 15 as a predetermined operation signal. Operation buttons include directional buttons such as cross buttons or stick buttons, and are assigned to, for example, power, menu, content selection, home, return, decision, mute, volume up, volume down, play, and fast. Settings include rewind, fast forward, distortion correction, and viewpoint change.

The menu button is an operation key that displays a menu screen for selecting the projector to be operated and its setting items when setting the projectors 12, 13, and 14. Media such as a content recording medium is assigned to the content selection button. The media also includes, for example, identification information such as a specific predetermined video distribution service and a base (meeting room number) specified as a communication destination. The distortion correction button is an operation key used when performing trapezoidal distortion correction for adjusting the shape of images projected by the projectors 12, 13, and 14.

The viewpoint change button is an operation key for keeping the virtual space of the content within the user's visual field. When the selection of the viewpoint change button is accepted, the video processing unit 26 generates video information with the coordinates of the viewpoint changed according to the accepted change details for various content information based on the control of the control unit 23. do. In this way, the user can view the object from various viewpoints and angles.

The head detection camera 17 is a detection unit that detects the spatial position of the user's head, and is installed in the front or side of the user in a soundproof space. For example, in the case of a sensing method such as an RGB camera or an infrared camera, the location of the face, the outline of the face, facial parts, etc. are detected from the input image, and the image of the user's head is captured as information. It is output to the control unit 23 of the processing device 15. On the other hand, the control unit 23 specifies the eye position and line-of-sight direction within a predefined three-dimensional space, that is, within a soundproof space, based on the output. This series of controls is executed using known image analysis and face recognition techniques. Then, the video processing unit 26 generates video information in which the display viewpoint, for example, the viewpoint height, is corrected to a different height for each person with respect to information on various contents.

Next, please refer to FIGS. 2 to 5 for an overview of the process of generating a desired cut-out video from the connected video and displaying the video in the display range provided on the projection surface SC in the display device 11 of this embodiment. I will explain while doing so.

For example, a spherical camera is used as a photographing means to generate the connected video. Although not shown in the drawings, the omnidirectional camera includes a plurality of lens optical systems and a plurality of image sensors provided in correspondence with the camera body. The image sensor converts the received light distribution into an image signal, and sequentially outputs the image to the image processing block of the control section. The images captured by the image sensors are combined using a known calculation method, thereby generating a spherical image as a connected image. A spherical image is one shot in all directions that can be seen from the shooting point. In this embodiment, the explanation will be given assuming that a 360-degree spherical image is generated, but it may also be a 360-degree panoramic image (panoramic format) that captures only a 360-degree horizontal plane, or a 360-degree panoramic view of a 360-degree horizontal plane or a 360-degree horizontal plane. It may be a video (dome format) of a part of the house. Further, the spherical image can be acquired as a still image or as a moving image. The acquired spherical images can be archived and used for secondary purposes if necessary.

FIG. 2 shows an example of the format of the spherical image, which is an equirectangular image in which the spherical image is projected onto a plane using the equirectangular projection method. For convenience, the image is a spherical image with a range of 180 degrees (vertical angle φ = -90 degrees to +90 degrees) in 90 degree increments in the latitude direction (vertical direction in Figure 2) centered on the equator, The image is shown as an image extending over a range of 360 degrees (horizontal angle θ=-180 degrees to +180 degrees) in the longitude direction (horizontal direction in FIG. 2). Here, since the spherical image is an image pasted to cover a spherical surface, it is processed into an image that can be viewed without causing any discomfort on display. That is, the image is corrected as a flat image with less curvature and then cropped, that is, cut out. This cutout video corresponds to the display range provided on the projection surface SC, and for example, the masked area (shaded area) in FIG. 2 is extracted. Such a rectangular extraction region is used to extract a region near the equator and use it as a target region, thereby eliminating the influence of image distortion in high latitude regions. The extraction area includes an area (attention area) that attracts a high degree of user attention for each piece of content information used.

Here, the display range of the projectors 12, 13, and 14 is set within a cubic soundproof room or a similar rectangular parallelepiped soundproof room. Therefore, as pre-processing for displaying the video in the display range, a cube map, which is an environment map assuming the user's viewpoint, is created. Cubemap is a method that assumes that a spherical camera is housed in a cube (box), and it is a method that assumes that a spherical camera is housed in a cube (box). It is expressed as a developed view of a cube so that the orientation, that is, the front surface (FRONT), rear surface (BACK), left surface (LEFT), right surface (RIGHT), top surface (TOP), and bottom surface (BOTTOM) are arranged. In this case, when acquiring a cut-out video, the upper and lower surfaces that are inappropriate for three-dimensional reconstruction have already been removed, and the masked area (shaded area) becomes the mapping target. Mapping is then performed on the four sides of the cube using a 90 degree mapping angle. In this example, the rear surface (BACK), which is assumed to be the entrance and exit of the soundproof space, is removed, and as shown in FIG. mapping is done. Note that mapping may be performed on the back surface (BACK) if necessary.

FIG. 5 schematically shows an example of a case where a cutout image is projected onto the projection surface SC. As shown in FIGS. 5(a) to (c), for convenience of explanation, the projection surface SC consisting of three wall surfaces is individually expressed as a left projection surface SCL, a front projection surface SCF, and a right projection surface SCR. However, it actually consists of a continuous wall surface that is U-shaped in plan view and is formed within the same soundproof space. The display range of the cutout video is set, for example, to the upper half of the entire projection surface SC, taking into account the height of the user's viewpoint and the like. When a user in a soundproof space looks with his/her back to the entrance/exit, the left projector 12 of the three projectors 12, 13, 14 is the one that projects the left side of the user's left hand side, as shown in FIG. 5(a). The front projector 13 is placed on the front projection surface SCF on the user's front side as shown in FIG. 5(b), and the right projector 14 is placed on the user's right side as shown in FIG. 5(c). The corresponding cutout images are projected onto the right projection surface SCR of the screen. As a result, an image of the environment surrounding the user (view of all surroundings) can be expressed as a natural image with a sense of distance.

Below, the soundproof room forming the projection surface SC of the display device 11 and the display mode on the projection surface SC will be described with reference to FIGS. 6 to 13.

As shown in FIGS. 6 and 7, the soundproof room 31 is formed by a casing 32 and a door 33, and has a space into which people can enter, such as a station premises, an airport, a commercial facility, an exhibition hall, etc. They are installed in places where various people pass, such as outdoor plazas, inside stores, hotels, apartment complexes, or offices, and even infrastructure facilities such as charging stations for electric vehicles (EVs). Further, the soundproof room 31 can transmit and receive data to and from external devices via a network such as the Internet. A user using the soundproof room 31 may use, for example, the video conference function of the information processing device 15 installed in the soundproof room 31 or a video conference terminal installed independently of the video conference function to connect an external device having a video conference function. You can hold video conferences with other users.

The housing 32 is made of a member such as metal, wood, or resin, and has a cube shape or a rectangular parallelepiped in appearance, and has a correspondingly large interior so that the user does not feel oppressed. A cubic or rectangular parallelepiped soundproof space (indoor space) is formed. The shape of the casing 32 will be described in detail later, but by changing at least one of the dimension in the depth direction from the entrance and the dimension in the width direction, the shape of the casing 32 can be adjusted according to the number of people accommodated in the soundproof space. It has variations in size.

The front side (FIG. 7), which is one side of the housing 32, is provided with an opening for the user to enter and exit. Furthermore, casters and adjusters are provided on the bottom surface of the housing 32, respectively. The installer of the soundproof room 31 uses casters to move the soundproof room 31 to the installation location, and after setting the door 33 in a direction that matches the installation location, adjusts the length of the adjuster to match the bottom surface of the soundproof room 31 and the floor surface. The soundproof room 31 can be fixed in a fixed position by adjusting the distance so that it is equal to the distance between Regarding the installation of the soundproof room 31, it is conceivable that the soundproof room 31 be installed in the form of parts that are transported to the installation site and assembled on the spot.

A soundproof panel (for example, sound-absorbing material) is provided inside the casing 32, so that the voices spoken by users inside the soundproof room 31 and the voices uttered by users using external equipment are transmitted to the outside of the soundproof room 31. Not easy to leak. Therefore, even if the soundproof room 31 is installed in a public place where an unspecified number of people pass, the user can talk about highly confidential content with peace of mind. In addition, the soundproof panels used as the projection surfaces SC (left projection surface SCL, front projection surface SCF, right projection surface SCR) are uniformly white in color to ensure good visibility of the projected image. During construction, the boundaries between panels are finished so that they are inconspicuous.

A power supply circuit for distributing power to electronic devices is provided inside the wall surface of the housing 32. For example, the power supply circuit electrically connects a power plug on the power transmission side and a connection port on the power reception side. This makes it possible to supply power from a power outlet that takes commercial power to the information processing device 15, projectors 12, 13, 14, computers brought in from outside, mobile information terminals, etc., through the connection port. Here, electronic devices include electronic locks and their peripheral devices (e.g., card readers for authentication, touch panel displays), video conferencing equipment, lighting (e.g., downlights), air conditioning (e.g., air conditioners, ventilation fans), sensors, etc. (a human sensor including a head detection camera 17).

The door 33 is coupled to the casing 32 so as to be able to close the opening of the casing 32, and is configured to be openable and closable around the joint to the casing 32 by operating a handle. . The door 33 is provided with a door glass, making it possible to visually check the inside of the soundproof room 31 from the outside. A lock (electronic lock) is provided between the casing 32 and the door 33, and the door 33 can be opened in an unlocked state and the door 33 can be opened under the control of a control unit built into the casing 32. Switch between the locked state and the locked state. The lock is controlled to unlock or lock the soundproof room 31 by switching between an unlocked state and a locked state by operating the lock from inside the casing 32 by the user, or by switching by remote control. Ru.

As shown in FIGS. 8 and 9, the projection surface SC is three continuous side wall surfaces of the inner wall surface of the casing 32 surrounding the user, excluding one side wall surface having the door 33. For example, the display range corresponding to the front projection surface SCF has a rectangular shape with a large width, as shown by the shaded area in FIG. The position of E2 is aligned with the height of the upper and lower bisectors of the side wall surface. On the other hand, the left end E3 and right end E4 of the display range are aligned with the corner portions CN1 and CN2 of the side wall surfaces. The left projection surface SCL and the right projection surface SCR are similarly formed into rectangular shapes with a large width, and the display range is provided throughout the width direction of the side wall surface. The information processing device 15 and each of the projectors 12, 13, and 14 are used while being placed on a pedestal (indicated by broken lines) provided near a wall.

When the image light LT is enlarged and projected from each of the projectors 12, 13, and 14 set near the center of the corresponding display range, images are displayed on three sides of the display range. At this time, keystone distortion correction is performed on the projected image, and distortion caused by the projector projecting from an oblique direction with respect to the display range is reduced. After the initial setting is performed in this manner, during operation, each of the projectors 12, 13, and 14 displays the video content desired by the user as a cutout video in the display range. Then, the image of the cube map is reflected in the corners CN1 and CN2 of the projection surface SC, that is, in the continuous portions of the left projection surface SCL, front projection surface SCF, and right projection surface SCR that are perpendicular to each other. The cut out images are smoothly joined together (see also FIGS. 11 to 13).

Here, as can be imagined from FIG. 8, the user in the soundproof space places himself in a position facing the front projection surface SCF with his back to the door 33. In other words, the user is naturally guided to a standing position near the door 33 while being surrounded by the displayed images on the three side wall surfaces (projection surfaces SCL, SCF, and SCR), and the user moves around the space without being particularly conscious of it. It will be occupied. As a result, various displayed video contents are displayed over a wide range beyond the effective viewing angle for both eyes (approximately 120 degrees), and images that can be viewed naturally are expressed in a soundproof space.

As shown in FIG. 10, the display device 11 configured in this manner is incorporated into a soundproof room 31 to form a single device, and a plurality of display devices 11 can be installed at various locations along the user's flow line in daily life or business situations. , a display system capable of communication connection with a server SV on a network NW and various photographing means including a spherical camera SI is constructed.

Here, the various videos acquired by the content playback unit 25 of the information processing device 15 are not limited to their originals; for example, the videos to be input include live videos distributed from a remote location, and recorded videos. There are two possible output methods: one is to perform video synthesis on the server SV, and the other is to perform it on the information processing device (local machine) 15. Therefore, four cases are possible in which two types of input and two types of output are combined.

The spherical camera SI is not only a simple type that can be held hand-held, but also a type that can be attached to the human body such as the head or torso, a vehicle, a drone (flying robot) that flies autonomously, etc. Small-sized devices that emphasize compactness are used, such as those that can be attached to moving objects. Drones can hover above a specific area, such as a tourist spot, and perform straight, circling, and rotational movements, making it easy to capture even images shot from above.

As an example of the display mode on the projection surface SC, FIG. 11 shows a state in which the scenery of a tourist spot is displayed in the display range. It is travel guide content that guides sightseeing trips, and is set in the order of recommended routes from the distribution destination as a specific travel plan. When the user performs a predetermined operation on the controller 16, the viewpoint in space is moved and related information is displayed. This allows the user to imagine what it would be like to travel before the trip, making it easier for the user to empathize with the user and attract interest in the trip. These tourist destinations include various facilities such as those in the amusement and leisure industry. In addition, in the case of performance events such as sporting events and concerts that are streamed live, participating users (audiences) can experience the atmosphere of the venue and the reactions of the audience in real time without having to go to the event venue. This will make the performance event more exciting. The person with whom the space is shared may be a living creature other than a human, or a character imitating a human.

FIG. 12 shows a model room displayed in the display range. The content of the real estate property information service is to support real estate transactions, and is used in housing exhibition halls, model room exhibition halls, meetings before construction work or renovation work, etc., when users (owners) are looking inside a building where construction is planned. It is set up so that you can experience the space in a walk-through format. For example, a drone is used for imaging by the omnidirectional camera SI. After determining the self-position and attitude of the drone as a reference, the drone is made to orbit while changing its height position while tracing the flow line set within the building based on the three-dimensional design information.

At the preview site, for example, the floor height of the kitchen matches the floor height of the actual space (soundproof space) where the user is standing, and by performing a predetermined operation on the controller 16, the construction plan can be adjusted. To realize a walk-through that allows users to experience life in a kitchen. In addition, the display video is subjected to viewpoint correction in advance through sensing by the head detection camera 17, and a cut-out video is displayed in which the viewpoint height is reflected to the user's eye level. This allows the user to experience and confirm the depth of the kitchen, the height of kitchen equipment, etc. from his or her own perspective. The same applies to spaces such as the entrance, living room, and bedroom.

FIG. 13 shows a state in which a meeting room is displayed in the display range. The information processing device 15 executes an application (video conference application) that supports video conferencing, and transmits and receives video data and audio data for calls, content data for sharing, etc. to and from the communication destination via the network NW. is set to be possible. The spherical camera SI is used to take pictures of participants during video conferences, but since it can take pictures of a wide range of people, it can be used not only to take pictures of participants but also people around them. good. Further, for photographing, a camera group including a plurality of cameras with narrow viewing angles may be used. Regarding camera designation, you can switch to one camera from two, main and sub, or three or more. This makes it possible to communicate with people in different spaces at once while sharing the same space in a natural manner, that is, in a manner in which the sense of distance, texture, responsiveness, etc. are close to those of the real thing.

As described above, according to the first configuration of the display device of the present invention, among the inner wall surfaces of the housing 32 forming a cubic or rectangular parallelepiped soundproof space, three continuous side wall surfaces (projection surfaces SCL, SCF, SCR) is set as the display range, and a cutout video corresponding to the display range is generated from the connected video, so it is possible to make the display area that is displayed simultaneously larger than the user's viewing angle, and it is possible to create a pseudo-image expressed in a soundproof space. As a three-dimensional space, it is possible to create a sense of spatial unity with the outside world.

14 and 15 show a second embodiment of the display device of the present invention. In addition, in the following description, the same reference numerals are given to the same components as those shown in the above embodiment, and detailed description thereof will be omitted.

The display device according to the second embodiment basically has the same configuration as the display device 11 according to the first embodiment, and has a seat 34 for a user to sit in a soundproof space. The soundproof space is a rectangular parallelepiped, and for example, the width direction dimension W is larger than the depth direction dimension D from the entrance/exit (door 33), and the height direction dimension H is larger than the depth direction dimension D. It is formed. The seat 34 is, for example, a rotary chair that can rotate about a rotation axis PV along the vertical direction, and is stably installed on the floor surface of the housing 32. The display range can be set as necessary by setting the height position of the upper end E1 to a lower position away from the ceiling surface, or setting the height position of the lower end E2 to a lower position accordingly. The height position is taken into consideration so that the display is easy for the user to see.

Since the arrangement of the seat 34 is an important factor to consider in determining the user's viewpoint, the seat 34 is arranged, for example, on a vertical plane (two vertical planes passing through the center C) including the diagonals of the rectangular parallelepiped forming the soundproof space. ) and one triangular prism-like space having an entrance and exit, and three triangular prism-like spaces corresponding to three consecutive side wall surfaces (projection surfaces SCL, SCF, SCR), making a total of four triangular prism-like spaces defined. It is installed in one triangular prism-shaped space with an entrance and exit. Particularly, in the case of a rotating chair, it is preferable that the rotation axis PV is installed at the widthwise center (W/2) of the soundproof space and a predetermined distance d1 away from the doorway in the depth direction. is more preferably a value obtained by dividing the dimension D in the depth direction by 4 (D/4).

As described above, according to the second configuration of the display device of the present invention, by sitting on the seat 34 and viewing the video content, the user can experience the pseudo three-dimensional space expressed in the soundproof space in a relaxed posture. be able to. Furthermore, if a rotating chair is used as the seat 34, the viewing angle of the seated user can be changed from the binocular viewing angle θ1 (approximately 120 degrees), which is the central visual field, to the left viewing angle θ2 (approximately 50 degrees) obtained by rotating the seat 34. It becomes possible to shift the peripheral visual field including the right viewing angle θ3 (about 50 degrees) and the right visual angle θ3 (approximately 50 degrees) by a simple rotational movement, and the central visual field of the seated user can be expanded in a rational and well-balanced manner.

16 and 17 show a third embodiment of the display device of the present invention. In the display device in the third embodiment, the projectors 12, 13, and 14 are installed on the ceiling of the housing 32, and a vacant space near the wall is used as a sofa and a table 35 and a seat 34 used as a pair. This is a configuration in which a The main bodies of the projectors 12, 13, and 14 are fixed to the outer space above the ceiling surface using mounting brackets (not shown), and are projected onto the side wall surfaces (projection surfaces SCL, SCF, and SCR) of the housing 32. The image light LT is set as close to the ceiling as possible. Note that the specifications of the projector are selected in consideration of workability for assembling it to the ceiling, etc., and the functions are the same as those of each of the above-mentioned embodiments.

The table 35 has a long table surface in the depth direction, and also serves as a pedestal for the information processing device 15. The seats 34 are arranged in pairs for two people, and are arranged to face each other in the width direction with a table 35 in between. Thereby, seats 34 for four people are secured at predetermined positions, and even if four users enter the soundproof space at the same time, they will not feel cramped. The soundproof space is defined into an upper space and a lower space by a horizontal plane that includes the table surface of the table 35, and when the seat 34 is accommodated in the lower space, the position of the lower end E2 of the display range is at the level of the table surface of the table 35. Aligned to height position. Thereby, the display range of the projectors 12, 13, and 14 is set in the upper space. When the image light LT is enlarged and projected from each of the projectors 12, 13, and 14 set directly above the center of the corresponding display range, images (cut out images) are displayed on three sides of the display range.

As described above, according to the third configuration of the display device of the present invention, the projectors 12, 13, and 14 are installed on the ceiling of the housing 32, and their main bodies are located in the space outside the ceiling, so that soundproofing is achieved. The space utilization efficiency of the space can be improved, and since the projectors 12, 13, and 14 themselves become inconspicuous, the aesthetic appearance of the real space inside the soundproof room 31 and the aesthetic appearance of the pseudo three-dimensional space expressed in the real space are improved. can also be improved. Furthermore, in addition to other users who are physically separated, users can also share content experiences in real time with other users who are in the same soundproof space while feeling the presence of the other user. In particular, users sitting facing each other can share information about the core parts of the image by focusing on the projected image with the central side wall surface (projection surface SCF) as the display range, making it highly practical. It is also excellent.

18 and 19 show a fourth embodiment of the display device of the present invention. In the display device in the fourth embodiment, a rectangular parallelepiped soundproof space is defined by three side wall surfaces (projection surfaces SCL, SCF, and SCR) of a common size. The soundproof room 31 with a soundproof space having a square planar shape has, for example, a characteristic appearance in which one side of the housing 32 is largely occupied by a door 33 (see also FIG. 20), and can be moved on casters. It has no directionality and can be handled maneuverably, and it has the flexibility of being able to rotate in the same place and only change the direction of the entrance and exit. Therefore, based on its technical features that facilitate easy installation, it has rapidly become popular in recent years as a soundproof room for teleworkers installed in public places. Note that the shape of the housing 32 can also be formed as a substantially rectangular parallelepiped with some rounded corners.

Although the seat 34 is illustrated as a simple stool, it may also include a rotating chair. The soundproof space is a relatively narrow space where the four corners (corners CN1 and CN2 of the projection surface SC on the front side) can be reached even when the user is seated. The display range is set to three sides of the upper space partitioned by the table surface, and when the image light LT is enlarged and projected from each of the projectors 12, 13, and 14 installed on the ceiling of the housing 32, three sides of the display range are set. The image (cut out image) is displayed on the screen.

As described above, according to the fourth configuration of the display device of the present invention, by being incorporated into the small soundproof room 31 for one person and making it into one device, it is possible to use the display device in normal use in the soundproof space for work, study, etc. A caged feeling suitable for this can be obtained. On the other hand, once the display device is functional, it becomes possible to display various video contents in the display range and express a pseudo three-dimensional space as shown in Figs. 11 to 13. However, here too, it is possible to create a sense of spatial unity with the outside.

20 and 21 show a fifth embodiment of the display device of the present invention. In the display device in the fifth embodiment, as shown in FIG. 20, a display system (video conference system) is constructed that enables mutual communication with users in a soundproof space at the communication destination by connecting a plurality of display devices to a network. There is. The various soundproof rooms 31 shown in the above embodiments are set as platforms for expressing a pseudo three-dimensional space. By photographing and transmitting the soundproof space in real time using the omnidirectional camera SI or a group of cameras, predetermined video processing is performed on the cut-out video at the server SV on the network NW. Then, the video and audio are live distributed to each soundproof room 31 through two-way communication. Meeting styles include, in addition to commonly held formal meetings, sudden emergency meetings, casual meetings to discuss matters, and business negotiations. Applications to the fields of education, welfare, and medical care are also possible.

Since each soundproof space is a cube or a rectangular parallelepiped that has a high affinity for interconnection, no matter where the soundproof room 31 is installed and in which direction, it will act as if it were a single cube, as shown in the image of FIG. They are combined to form a shape space, and the coordinates are transformed into a predetermined space coordinate system. In other words, peripheral elements are linked to a specific element by an axis (for example, you may imagine a Rubik's Cube (registered trademark), which is a three-dimensional puzzle whose orientation and arrangement are matched by a rotation operation). In the three soundproof spaces each equipped with a seat 34 and a table 35, display ranges are set on three sides of the upper space partitioned by the table surface, and images (cut out images) are displayed on three sides of each display range. As a result, one space can be viewed from the other two spaces through the display image displayed on one or more of the three side wall surfaces (projection surfaces SCL, SCF, SCR). This is a virtual reality (VR) space that is an extension of the previous space. As a result, a plurality of seated users can share the table 35 even though they are in different spaces, and can experience not only the presence of the user facing them, but also the presence of the user next to them (diagonally opposite). can.

As described above, according to the fifth configuration of the display device of the present invention, as a display system that can share content experiences in real time, it is constructed based on a plurality of soundproof rooms 31 connected to the network NW. It is now possible to recreate a person in a different space next to you, and yourself next to a person in a different space, including the entire space, without relying on the conventional method (HMD). It is possible to foster a sense of unity. This will contribute to providing new added value in remote communication.

22 and 23 show a sixth embodiment of the display device of the present invention. In the display device according to the sixth embodiment, a display system is constructed that enables mutual communication with users of the virtual space by connecting the various devices shown in the above embodiments to a metaverse (virtual space on the Internet). ing. For example, when a meeting room (conference room) is represented as a virtual space, as shown in FIG. It is possible to share the space with A1 to A5 and deepen communication by sharing objects such as the whiteboard WB created within the space.

The spaces in which activities take place in the Metaverse can be various spaces where social activities linked to the real world can take place.For example, in a business setting, as shown in Figure 23, spaces of the visited company, seminar/training halls, and exhibition spaces can be considered. It is expressed in imitation of a venue, etc. Also, although not shown in the drawings, if a group of cameras is built into each side wall surface (projection surface SCL, SCF, SCR) of the soundproof space, that is, multiple cameras arranged around the user can be used to take pictures. It will also be possible to create a humanoid avatar that looks just like you, making it possible to create a realistic virtual space via a soundproof space. In this way, expectations are high for the company to use the soundproof room 31, which forms the core of the display device, as a platform to enter the new digital world (metaverse).

Although the present invention has been explained based on the above-mentioned embodiments, the technical scope of the present invention is not limited to the range described in the above-mentioned embodiments, and various modifications and changes can be made within the scope of the gist. is possible. For example, specific embodiments for distributing and integrating devices are not limited to the above-mentioned embodiments, and all or part of the devices may be configured by functionally or physically distributing and integrating them in arbitrary units. can. Further, new embodiments resulting from arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiment resulting from the combination have the effects of the original embodiment.

DESCRIPTION OF SYMBOLS 11... Display device, 12, 13, 14... Projector, 15... Information processing device, 16... Controller, 17... Head detection camera, 18... Pr control unit, 19... Pr storage unit, 20... Video input unit, 21... Pr video processing unit, 22...Video forming unit, 23...Control unit, 24...Storage unit, 25...Content playback unit, 26...Video processing unit, 27...Video output unit, 28...User interface unit, 31...Soundproof room, 32...Housing, 33...Door, 34...Seat, 35...Table

Claims (2)

A display device that displays images in a preset display range,
It has a control unit and an image forming unit that outputs a concatenated image generated from a plurality of captured images in a visible format,
The display range is provided in a portable soundproof room equipped with a casing forming a rectangular parallelepiped soundproof space into which a user can enter,
The soundproof space is formed such that the dimension in the width direction is larger than the dimension in the depth direction from a door provided on one side of the casing, and the dimension in the height direction is larger than the dimension in the depth direction,
The control unit sets a display range of three consecutive side wall surfaces of the inner wall surfaces of the casing surrounding the user , excluding one side having the door , and extracts an image corresponding to the display range from the connected image. and giving an instruction to the image forming unit to output the cut out image ,
The display device is characterized in that the display range has an upper end aligned with the height of a ceiling surface of the housing, and is provided to cover the entire width of each of the three continuous side wall surfaces. A controller capable of changing a display viewpoint in response to an operation by the user, and a detection unit that detects the position of the user's head,
The display device according to claim 1, wherein the control unit corrects the display viewpoint based on the detection result of the detection unit so that the height of the display viewpoint becomes the eye level of the user. .

Images