JP7073116B2 - Office virtual reality system and office virtual reality program - Google Patents

Description

The present invention relates to virtual reality techniques, especially virtual reality systems for offices.

In recent years, in order to implement work style reforms, many companies are allowing employees to work from home without requiring them to go to the office.
On the other hand, when working from home, if confidential materials can be taken home, there is a risk that confidential information will be leaked from the personal computer at home. In addition, if you put a confidential file in a laptop computer and take it home, the risk may increase from the viewpoint of information security, such as the leakage of confidential information as a result of the laptop computer being lost or stolen. There is.
Therefore, in order to solve such a problem, a mechanism called a thin client has been conventionally proposed.

Japanese Unexamined Patent Publication No. 9-288645 Japanese Unexamined Patent Publication No. 10-254851 Japanese Unexamined Patent Publication No. 2001-265872 Japanese Unexamined Patent Publication No. 2007-228294

However, the mechanism of the thin client is generally to simply transfer the screen of the computer in the office to the computer at home or the laptop computer that you carry around, which is much different from the environment where you actually work at the office. It's different.
That is, for example, if you go to work in the office, a colleague is working next to you, and you can immediately have a conversation and exchange information or have a meeting. In addition, it is possible to easily gather in a conference room and have a meeting with more people.

Furthermore, in terms of attendance management, when going to work in the office, it is possible to clearly separate work time and private time by a mechanism such as a time card, but when working from home, work is done. There was a problem that the distinction between work and private time became ambiguous because no one could see him doing it.

Therefore, in order to eliminate the drawbacks of working from home using a conventional thin client, the present inventor can feel as if the employee is working from home while working at an actual office. We researched a mechanism that does not dilute communication or obscure attendance management.

Here, as a technique related to a virtual office system, the virtual office system described in Patent Document 1 can be mentioned. According to this technique, it is said that the video information and audio information of other users can be displayed three-dimensionally in the user's personal office. Further, in the virtual office system described in Patent Document 2, it is said that the concentration level and the sign of the user can be transmitted to other users. However, these did not provide the user with an environment in which they felt as if they were working in an actual office.

Further, the virtual office described in Patent Document 3 makes it possible to confirm who is working by displaying the situation in the office in a plane, and it is as if the employee is working from home. I felt like I was going to work in the actual office, and it was not possible to eliminate the weak communication and ambiguous attendance management.
Further, the virtual office system described in Patent Document 4 relates to a technique for establishing a VPN (virtual private network) between information processing devices via a network in a thin client type information processing system, and solves the above-mentioned problems. It wasn't possible.

Therefore, the inventor of the present invention can study diligently and feel that the employee is working at the actual office even though the employee is working from home, and communication is weakened or attendance management is performed. We have completed the present invention by developing a system for virtual offices that does not obscure.
Specifically, using VR (Virtual Reality) technology, a virtual office (hereinafter, may be referred to as VR office) is constructed by using an office computer and a computer of an employee who works from home. , Developed an office virtual reality (VR) system and an office virtual reality (VR) program that can provide employees with an environment as if they were actually working in the office. bottom.

According to the office VR system and the office VR program of the present invention, it is possible to go to the VR office from anywhere in the world wherever it is possible to connect to the Internet. That is, for example, it is currently impossible to go to the actual office every day from the other side of the earth, but it is possible if it is a VR office.
In addition, there are many countries and regions in the world where there are few companies and jobs due to various circumstances, and it is possible to provide an environment where people in those countries and regions can work in the office.

Furthermore, in the future, as a work style of people, not only working at a company, but also talented people temporarily get to know each other using a network etc. and start work, and when the work is completed, it will be disbanded. It is predicted to happen all over the world.
Until now, it was necessary to prepare an actual office as a place for that, but by using the VR office, you can easily secure a place to work, and when the work is completed, delete the VR office. Is also easy, which is beneficial for such work styles.

In the present invention, the above circumstances are taken into consideration, and even though the user is working from home, he / she can feel as if he / she is working in an actual office, and communication is weakened. The purpose is to provide an office VR system and an office VR program that do not obscure attendance management.

In order to achieve the above purpose, VR (Virtual Reality) image of the office is displayed on the user side information processing device, and VR for office to provide the user with a VR office that can perform business outside the office. The system includes an office VR device group having an office side storage device and a VR processing device, and a plurality of user side information processing devices connected to the office VR device group via a communication line. The office side storage device stores the user's authentication information, the user's position information in the VR office, and the information indicating that the user has accessed the VR office for each user's identification information, and the VR office. Office storage unit that stores the layout information of the three-dimensional image of one or more rooms in the VR office for each room identification information, and stores the layout information of the three-dimensional images of multiple equipment in the VR office for each equipment identification information. The VR processing device determines the success or failure of authentication based on whether or not the authentication information transmitted from the user-side information processing device matches the user's authentication information stored in the user storage unit. The authentication processing unit, the access recording unit that records information indicating that the user has accessed the VR office in the user storage unit when the authentication is successful, and the user in the VR office stored in the user storage unit. The VR information creation unit that inputs the position information and the arrangement information stored in the office storage unit, creates the VR information including the location information and the arrangement information, and transmits the VR information to the user-side information processing device. The user-side information processing device stores the user's three-dimensional image for each user's identification information, stores the three-dimensional image of one or more rooms in the VR office for each room identification information, and stores the user's three-dimensional image for each room identification information in the VR office. An image storage unit that stores three-dimensional images of a plurality of equipment for each identification information of the equipment, VR information transmitted from the VR information creation unit, and three dimensions of the user stored in the image storage unit. VR synthesis processing unit that synthesizes a VR image based on an image, a three-dimensional image of the room, and a three-dimensional image of the equipment and displays it on a display device connected to the information processing device on the user side, user authentication information The authentication information acquisition unit that inputs and sends to the VR processing device, and the position information in the VR office of the user who is using the user-side information processing device is acquired, and the position information of the user is processed in the VR. Outfit It is configured to include a user position information transmitting unit for transmitting to a device.

Further, the VR program for an office of the present invention includes a VR device group for an office having an office-side storage device and a VR (virtual reality) processing device, and a plurality of VR devices for the office connected via a communication line. An office VR program for displaying a VR image of an office on the user side information processing device using a user side information processing device and providing a VR office capable of performing business outside the office to the user. A user storage unit that stores the user's authentication information, the user's position information in the VR office, and the information indicating that the user has accessed the VR office for each user's identification information in the office-side storage device. In addition, the arrangement information of the three-dimensional images of one or more rooms in the VR office is stored for each room identification information, and the arrangement information of the three-dimensional images of the plurality of equipments in the VR office is stored for each identification information of the equipments. The VR processing device is made to function as an office storage unit, and the VR processing device is authenticated based on whether or not the authentication information transmitted from the user-side information processing device matches the user authentication information stored in the user storage unit. The authentication processing unit that determines the success or failure of the VR office, the access recording unit that records information indicating that the user has accessed the VR office in the user storage unit, and the user storage unit that stores the information indicating that the user has accessed the VR office when the authentication is successful. VR that inputs the position information of the user in the VR office and the arrangement information stored in the office storage unit, creates VR information including the position information and the arrangement information, and transmits the VR information to the user side information processing device. It functions as an information creation unit, and the user-side information processing device stores the user's three-dimensional image for each user's identification information, and stores the three-dimensional image of one or more rooms in the VR office for each room identification information. An image storage unit that stores three-dimensional images of a plurality of equipment in a VR office for each identification information of the equipment, VR information transmitted from the VR information creation unit, and storage in the image storage unit. A VR synthesis processing unit that synthesizes a VR image based on the user's three-dimensional image, the room's three-dimensional image, and the equipment's three-dimensional image and displays it on a display device connected to the user-side information processing device. , The authentication information acquisition unit that inputs the user's authentication information and sends it to the VR processing device, and the location information in the VR office of the user who is using the user's information processing device is acquired. It is configured to function as a user position information transmitting unit that transmits the user's position information to the VR processing device.

According to the present invention, the user can feel as if he / she is working at an actual office even though he / she is working from home, and communication is weakened or attendance management is ambiguous. It is possible to provide an office VR system and an office VR program that do not have to be done.

It is a block diagram which shows the structure of the VR system for office of 1st Embodiment of this invention. It is a block diagram which shows the structure of the office side storage apparatus and VR processing apparatus in the office VR system of 1st Embodiment of this invention. It is a block diagram which shows the structure of the user-side information processing apparatus in the office VR system of 1st Embodiment of this invention. It is a flowchart which shows the processing procedure of the VR processing apparatus and the user side information processing apparatus in the office VR system of 1st Embodiment of this invention. It is a block diagram which shows the structure of the office VR system of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the thin client processing apparatus in the office VR system of the 2nd Embodiment of this invention. It is a flowchart which shows the processing procedure of the thin client processing apparatus and the user side information processing apparatus in the office VR system of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the VR processing apparatus in the office VR system of the 3rd Embodiment of this invention. It is a flowchart which shows the processing procedure of the VR processing apparatus in the office VR system of the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the user-side information processing apparatus in the office VR system of 4th Embodiment of this invention. It is a flowchart which shows the processing procedure of the user-side information processing apparatus in the office VR system of 4th Embodiment of this invention. It is a block diagram which shows the structure of the VR processing apparatus in the office VR system of the 5th Embodiment of this invention. It is a flowchart which shows the processing procedure of the VR processing apparatus in the office VR system of the 5th Embodiment of this invention. It is a block diagram which shows the structure of the user-side information processing apparatus in the office VR system of the 6th Embodiment of this invention. It is a block diagram which shows the structure of the VR processing apparatus in the office VR system of the 6th Embodiment of this invention. It is a flowchart which shows the processing procedure of the VR processing apparatus and the user side information processing apparatus in the office VR system of the 6th Embodiment of this invention.

Hereinafter, preferred embodiments of the office VR system and the office VR program of the present invention will be described with reference to the drawings.

[First Embodiment]
First, the first embodiment of the present invention will be described with reference to FIGS. 1 to 4. FIG. 1 is a block diagram showing a configuration of an office VR system of the present embodiment, FIG. 2 is a block diagram showing a configuration of an office-side storage device and a VR processing device in the system, and FIG. 3 is a block diagram of the system. It is a block diagram which shows the structure of the information processing apparatus on the user side. FIG. 4 is a flowchart showing a processing procedure of the VR processing device and the user-side information processing device in the system.

The office VR system of the present embodiment is an office VR system for displaying an office VR image on a user-side information processing device and providing a VR office capable of performing business outside the office to the user. As shown in FIG. 1, the office VR device group 10 having the office side storage device 100 and the VR processing device 200 is connected to the office VR device group 10 via a network (communication line) such as the Internet. It is equipped with a plurality of user-side information processing devices 20 (20-1, 20-2). Although only two user-side information processing devices 20 are shown in FIG. 1, three or more user-side information processing devices 20 may be used.

Further, the office VR device group 10 may have a WEB processing unit 300 and an IP telephone device 400. Further, all or part of the functions of the office side storage device 100 may be provided in the VR processing device 200, and all or part of the functions of the WEB processing unit 300 and / or the IP telephone device 400 may be provided in the VR processing device 200. It may be configured to be provided collectively.

As shown in FIG. 2, the office side storage device 100 includes a user storage unit 101 and an office storage unit 102.
The user storage unit 101 stores user identification information, user biometric information, user location information in the VR office, and information indicating that the VR office has been accessed (user access time information, etc.). Further, it is also preferable to store the identification information of the user-side information processing device used by the user, the attendance status (whether or not he / she is at work, whether it is a VR office or an actual office), and the like in the user storage unit 101.

As the biometric information of the user, for example, the face image of the user can be used, and the face authentication can be performed based on the face image. It is also possible to use information such as fingerprints, vein information, and retina as the biometric information of the user, and perform authentication based on these. Further, the user's authentication information (password, etc.) can be used instead of the user's biometric information.

The office storage unit 102 stores the arrangement information of the three-dimensional images of one or more rooms in the VR office (position information in the three-dimensional space of the VR office) for each room identification information, and stores a plurality of equipment in the VR office. The arrangement information of the three-dimensional image is stored for each identification information of the equipment. In addition, this "room" is used generically for real estate in an office such as a corridor or an entrance. In addition, "equipment" is a general term for movables in the office.

These three-dimensional images can be created in advance using 3D CAD and stored in the image storage unit 21 of the user-side information processing apparatus 20 described later. Further, the 3D CAD may be operated by the WEB processing device 300 so that the user can edit the above three-dimensional image by using the user-side information processing device 20 using a browser. In the second embodiment described later, such a 3D CAD is operated by the thin client processing device 500 so that the user can edit the above three-dimensional image by using the user-side information processing device 20 using a browser. It can also be.

As shown in FIG. 2, the VR processing device 200 includes a biometric authentication unit 201, an access recording unit 202, and a VR information creation unit 203.
The biometric authentication unit 201 determines the success or failure of biometric authentication based on whether or not the biometric information transmitted from the user-side information processing device 20 matches the user's biometric information stored in the user storage unit 101. ..
Further, instead of the biometric authentication unit 201, an authentication processing unit that performs authentication based on the user's identification information (ID) and the user's authentication information (password) may be provided. In this case, the user ID and password are transmitted from the user-side information processing apparatus 20 instead of biometric information, and the authentication processing unit determines whether or not these match the user authentication information stored in the user storage unit 101. The success or failure of the authentication is judged based on the information processing.

The access recording unit 202 records information or the like indicating that the user has accessed the VR office in the user storage unit 101.
For example, the access recording unit 202 can record the latest location information of the user in the VR office, the access time information of the user to the VR office, and the like in the user storage unit 101.

The VR information creation unit 203 includes information on the user's position in the VR office stored in the user storage unit 101 and information on the arrangement of three-dimensional images of one or more rooms in the VR office stored in the office storage unit 102. And, the arrangement information of the three-dimensional images of a plurality of equipments in the VR office is input, the VR information including these position information and the arrangement information is created, and the user side information processing apparatus 20 via the WEB processing unit 300. Send to.

Here, the VR information created by the VR information creation unit 203 includes the position information of all the users existing in the VR office, and the VR information creation unit 203 creates the VR information and the VR information. The transmission of the VR information from the creation unit 203 to the user-side information processing device 20 is performed in real time.
Then, each user-side information processing device 20 sets the position of the user using the user-side information processing device 20 in the VR office as the current position of the first-person viewpoint, and sets the image of another user in the VR office as the VR synthesis processing unit. 22 is synthesized in the VR office. As a result, each user can visually recognize a user other than himself / herself in the VR office.

Further, in the VR office, the user's own image is not displayed on the display device 32 connected to the user-side information processing device 20 used by the user, and is not displayed on the display device 32 used by the other user. The information processing device 20 displays the information on the display device 32 connected to the other user-side information processing device 20.

As shown in FIG. 3, the user-side information processing apparatus 20 includes an image storage unit 21, a VR synthesis processing unit 22, a biological information acquisition unit 23, and a user position information transmission unit 24.
The image storage unit 21 stores the user's three-dimensional image for each user's identification information, and stores the three-dimensional image of one or more rooms in the VR office for each room identification information, and stores a plurality of facilities in the VR office. A three-dimensional image of the product is stored for each identification information of the equipment. The "three-dimensional image" referred to here also includes a two-dimensional image (future three-dimensional image) that is a source for creating a three-dimensional image by a three-dimensional conversion process.

The VR synthesis processing unit 22 contains VR information transmitted from the VR information creation unit 203 in the VR processing device 200 via the network and the data input / output unit 25, and a user's three-dimensional image stored in the image storage unit 21. , The VR image is synthesized based on the three-dimensional image of the room and the three-dimensional image of the equipment, and displayed by the display device 32 connected to the user-side information processing device 20.
Such VR image composition can be performed using, for example, Unity (Unity Technologies Japan GK) or the like.

Here, in the office VR system of the present embodiment, in order to allow the user to feel as if he / she is actually in the office, when the user moves from one place to another, he / she moves in the VR office. If you do not take the action of, you will not be able to move. In other words, it is basically not allowed to suddenly jump (warp) to another place.

The instruction of the movement action can be given by the input device 31 connected to the user-side information processing device 20. As the input device 31, a game controller can be used in addition to the keyboard and mouse, as will be described later. It is also possible to express movement by swinging it using a device equipped with an acceleration sensor or the like.

When the goggles are used as the display device 32 connected to the information processing device 20 on the user side and the user wears the goggles, if the goggles are equipped with a sensor for acquiring position information such as an acceleration sensor or GPS, It is possible to determine which direction the user is currently facing. Therefore, the VR synthesis processing unit 22 can acquire a three-dimensional image of the user, the room, and the equipment in that direction from the image storage unit 21 and display it on the display device 32.

In addition, the user-side information processing device 20 is provided with a movement direction detection unit (not shown), and an image input device 35 connected to the user-side information processing device 20 captures a user's moving image, and the eyeball in this moving image is taken. It is also possible to detect the movement direction of the user by the movement direction detection unit based on the movement of the head or the neck, and display the image in that direction on the display device 32 by the VR synthesis processing unit 22.

In this way, in the office VR system of the present embodiment, by allowing the user to move the user without moving it instantaneously, he / she can meet or greet other people in the VR office. It causes things to happen, which promotes a kind of communication.

In the office VR system of the present embodiment, it is technically possible to provide a button or the like for moving the user instantly, and the function is not excluded.
For example, in the information processing device 20 on the user side, a conference room instantaneous movement button display unit (not shown) is provided, the button is displayed by the display device 32, and when the button is pressed, the VR synthesis processing unit is used. 22 acquires an image of the conference room from the image storage unit 21 and displays it on the display device 32, sets the user's position information in the conference room, outputs the image to the user position information transmission unit 24, and outputs the user position information transmission unit 24. This allows the user's position information to be transmitted to the VR information creation unit 203. Then, the VR information creation unit 203 can record the user's position information in the user storage unit 101 and transmit the VR information including the user's position information to the user-side information processing apparatus 20.
This makes it possible for users to immediately gather in the conference room in the event of an emergency.

The VR synthesis processing unit 22 can store real-time position information in the VR office of the user of the user-side information processing apparatus 20 in a user position storage unit (not shown). Further, the VR composition processing unit 22 synthesizes an image of the VR office from the first-person viewpoint with the position of the user as the current position.
That is, when the user inputs a movement action such as looking back by the input device 31, the display device 32 displays an image of the landscape behind which is displayed when looking back.
At this time, when a 360-degree projection projector is used as the display device 32, it is not necessary to rotate the image of the VR office, but when a monitor or VR goggles are used as the display device 32, VR is used according to the amount of movement. Rotate the image of the office.
Even when a 360-degree projection projector is used as the display device 32, if an action is performed to move the field of view with a keyboard or the like, the image of the VR office is rotated and displayed according to the amount of movement. It is also possible to change the landscape.

Here, if a three-dimensional image in a VR office is created on the office VR device group 10 side and transferred to the network as image information, the amount of information in the network increases, so that the network load increases and the transfer speed is slow. In the case of, the display becomes slow and it becomes difficult to work in the VR office.
Therefore, in the office VR system of the present embodiment, a module (VR synthesis processing unit 22) for creating a final three-dimensional image is arranged on the user side information processing apparatus 20 side, and the user's network is connected to the user. By adopting a format that transfers numerical information such as location information and layout information of rooms and equipment, it is possible to avoid deterioration of performance.

The biometric information acquisition unit 23 acquires the user's biometric information by an image input device 35 (CDCD video camera, CMOS video camera, etc.) connected to the user side information processing device 20, and uses this biometric information as the data input / output unit 25. And transmission to the VR processing device 200 via the network.
Specifically, for example, a gate display unit (not shown) is provided in the user-side information processing device 20, and the gate display unit starts the office VR program in the user-side information processing device 20 and then goes to the VR office. The entrance of is displayed by the display device 32. Next, the biometric information acquisition unit 23 photographs the user's face, which is supposed to be virtually present in front of the entrance, by the image input device 35, and the photographed user's face image is captured by the VR processing device 200. Can be sent to.

Further, the authentication method is not limited to face authentication, and other biometric authentication can also be performed. For example, the image input device 35 is configured to be acquired by the biological information acquisition unit 23 using a device for inputting fingerprint and vein information, a device for inputting information such as the retina, and the like, and similarly transmitted to the VR processing device 200. be able to.

At this time, the display of the entrance to the VR office by the gate display unit can be performed by inputting a three-dimensional image of the entrance image stored in the image storage unit 21. The function of this gate display unit may be included in the VR synthesis processing unit 22 or the biometric information acquisition unit 23.
Then, the biometric authentication unit 201 in the VR processing device 200 performs biometric authentication as described above using the input user's face image or the like, and transmits the determination result to the user-side information processing device 20.

In this way, whether or not the user is the person himself / herself by using biometric authentication technology such as face authentication by, for example, sitting in front of the display device 32 connected to the information processing device 20 on the user side or wearing the display device 32. Can be determined.
Therefore, the person himself / herself can recognize that the user has come to the VR office and is working, and can record the state of coming to the office in the user storage unit 101.

On the contrary, when the user disappears from the front of the information processing apparatus 20 for a certain period of time or longer, it can be recorded as going out or leaving the office.
That is, the biometric information acquisition unit 23 inputs the user's face information or the like at any time by the image input device 35, transmits it to the biometric authentication unit 201 in the VR processing device 200, and records the user's attendance status by the access recording unit 202. This makes it possible to manage the attendance of the user.

Further, instead of the biometric information acquisition unit 23, an authentication information acquisition unit is provided, and a user ID and password are input by an input device 31 connected to the user side information processing device 20, and these authentication information is input / output. It may be configured to transmit to the VR processing device 200 via the unit 25 and the network.

The user position information transmission unit 24 acquires the position information in the VR office of the user using the user side information processing device 20 from the VR synthesis processing unit 22 at any time, and transmits the user position information to the VR processing device 200. .. This location information includes the user's three-dimensional location information in the VR office and the direction in which the user is facing.

The data input / output unit 25 inputs / outputs data to / from various devices (including those built-in and connected) connected to the user-side information processing device 20 and the network. For example, the information input from the input device 31 or the voice input device 34 can be output to the VR synthesis processing unit 22. Further, the information input from the image input device 35 can be output to the biological information acquisition unit 23. Further, the information input from the VR synthesis processing unit 22 can be output from the display device 32 and the voice output device 33, and the information input from the biometric information acquisition unit 23 and the user position information transmission unit 24 can be output from the VR processing unit via the network. It can be sent to 200.

The input device 31 is an input device such as a keyboard, a mouse, a game controller, and a device equipped with an acceleration sensor. That is, in the office VR system of the present embodiment, the keyboard, mouse, etc. required for operating the user-side information processing device 20 are not visualized in the VR office, but are actually placed in the user's hand. The operability is not lowered by operating the.

The display device 32 is a display device for allowing the user to visually recognize an image (video) of a VR office such as a monitor, a 360-degree projection type projector, and VR goggles.
When a monitor is used as the display device 32, for example, a total of three monitors can be arranged in front of the display device and on both sides thereof so that the user can sit in front of them and work. On these three monitors, not only the screen of a PC like a conventional thin client but also the image of the VR office is projected by the VR synthesis processing unit 22. When other users in the VR office are also using the office VR system, the user on the right side is displayed on the right side, and the user on the left side is displayed on the left side.

When a 360-degree projection projector is used as the display device 32, the image of the office can be displayed in any direction of 360 degrees, and it is possible to get closer to the feeling of being in a real office. be.
When VR goggles are used as the display device 32, it is not necessary to use a monitor, a projector, a screen, or the like, so that the VR goggles can be easily installed.
As described above, as the display device 32 of the office VR system of the present embodiment, various forms can be used and can be selected depending on the convenience of the installation location and the like.

The audio output device 33 is an audio output device such as a speaker or headphones, the audio input device 34 is an input device such as a microphone, and the image input device 35 is an input device such as a CCD video camera.

The WEB processing unit 300 is an information processing device such as a WEB server. The WEB processing unit 300 is a device for enabling communication with the user-side information processing device 20 via a network such as the Internet using an http protocol or the like. The http protocol is preferable from the viewpoint of versatility, but when speed and confidentiality are important, it is possible to use a device that communicates using another communication protocol instead of the http protocol.

The IP telephone device 400 is an information processing device such as a telephone protocol server, is connected to a public network, performs call control by the SIP protocol, and communicates actual voice by the VoIP protocol. In-house calls inside the VR office can also be made by the IP telephone device 400. Companies using VR offices can make calls without going through the public line.

Next, the processing procedure of the VR processing device 200 and the user-side information processing device 20 in the office VR system of the present embodiment will be described with reference to FIG.
First, after the office VR program is started by the user-side information processing device 20, the entrance to the VR office is displayed on the display device 32 (step 10).
Next, the biometric information acquisition unit 23 of the user-side information processing apparatus 20 acquires the user's biometric information by the image input device 35 or the like, and transmits the biometric information to the VR processing apparatus 200 (step 11).

Then, the biometric authentication unit 201 in the VR processing device 200 performs biometric authentication using the input biometric information of the user, and transmits the determination result to the user side information processing device 20 (step 12).

Further, the biometric authentication unit 201 in the VR processing device 200 provides information indicating that the user of the user-side information processing device 20 has accessed the VR office by the access recording unit 202 when the biometric authentication is successful. (Step 13).
That is, the access recording unit 202 can record the user's access time information to the VR office and the like in the user storage unit 101 for each user's identification information.

If the biometric is successful, the user will be able to pass through the entrance. At this time, the VR synthesis processing unit 22 in the user-side information processing device 20 inputs an image passing through the entrance from the image storage unit 21 and displays it on the display device 32 (step 14).
Further, the user position information transmission unit 24 in the user side information processing device 20 converts the position information into the VR synthesis processing unit 22 as the position of the user in the VR office using the user side information processing device changes. And transmit it to the VR processing apparatus 200 (step 15).
The VR information creation unit 203 in the VR processing device 200 causes the access recording unit 202 to record the input user's position information in the user storage unit 101 for each user's identification information (step 16).
Further, the VR information creation unit 203 acquires the position information of all the users existing in the VR office from the user storage unit 101, creates VR information including this, and converts the VR information into the information processing device on each user side. It is transmitted to 20 (step 17).
Then, the VR synthesis processing unit 22 in the user-side information processing apparatus 20 synthesizes the VR image using the VR information, the user's three-dimensional image stored in the image storage unit 21, the room three-dimensional image, and the like. It is displayed by the display device 32 (step 18).

As described above, according to the VR system for offices of the present embodiment, when the user works from home, by using the VR office, the user feels as if he / she is going to work in the actual office. be able to. Further, in the VR office, other users can also be displayed, and it is possible to prevent communication from being diluted. In addition, attendance management can be appropriately performed by inputting the biometric information of the user at any time.

[Second Embodiment]
Next, the second embodiment of the present invention will be described with reference to FIGS. 5 to 7. FIG. 5 is a block diagram showing the configuration of the office VR system of the present embodiment, and FIG. 6 is a block diagram showing the configuration of the thin client processing device in the system. FIG. 7 is a flowchart showing a processing procedure of the thin client processing device and the user-side information processing device in the system.

In addition to the configuration of the first embodiment, the office VR system of the present embodiment has a configuration in which a thin client computer screen is displayed on the computer screen displayed in the VR office. Other points can be the same as that of the first embodiment.
That is, as shown in FIG. 5, the office VR system of the present embodiment is a thin client processing device 500 for converting the office side storage device 100, the VR processing device 200, and the user side information processing device 20 into thin clients. It is provided with a VR device group 10 for offices, and a plurality of user-side information processing devices 20 (20-1, 20-2) connected to the VR device group 10 for offices via a network (communication line) such as the Internet.

As shown in FIG. 6, the thin client processing device 500 includes a user-side information processing device virtual machine (1) 501, a user-side information processing device virtual machine (2) 502, a screen adjustment processing unit (1) 503, and a screen adjustment processing unit (1) 503. The screen adjustment processing unit (2) 504 is provided. Although two user-side information processing devices are shown in FIG. 5, three or more user-side information processing devices 20 are provided, and in FIG. 6, each user-side information processing device virtual machine is provided. The screen adjustment processing unit may be provided in the thin client processing apparatus 500.

The user-side information processing device virtual machine (1) 501 and the user-side information processing device virtual machine (2) 502 have the corresponding functions of the user-side information processing device 20-1 and the user-side information processing device 20-2, respectively. Is a virtual computer that runs.

Here, in the office VR system, the user-side information processing device 20 stores the user's position information transmitted from the VR processing device 200, the arrangement information of the three-dimensional images of the room and the equipment, and the image storage unit 21. The thin client processing device 500 is provided with a function of synthesizing a VR image based on a stored three-dimensional image and a function of a computer used for business.

That is, the user-side information processing device virtual machine (1) 501 and the user-side information processing device virtual machine (2) 502 provide the functions of the computer used for business corresponding to each user-side information processing device 20. , It is configured to execute. In this way, since the functions of the computers used for business are operated as virtual machines in the thin client processing device 500, it is possible to simultaneously operate the functions of a plurality of computers in the thin client processing device 500.

The user-side information processing device virtual machine (1) 501 and the user-side information processing device virtual machine (2) 502 display their respective computer screens (desktop images used by the users of each user-side information processing device). create.
The screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 have the user's position information stored in the user storage unit 101 and the arrangement information such as equipment stored in the office storage unit 102. Based on this, the resolution of the computer screen created by the virtual machine for the user-side information processing device is determined for each user's identification information, and the computer screen processed to that resolution is the user-side information corresponding to each user's identification information. It is transmitted to the processing device 20.

That is, the computer screens created by the virtual machine (1) 501 for the user-side information processing device and the virtual machine (2) 502 for the user-side information processing device are the screen adjustment processing unit (1) 503 and the screen adjustment processing, respectively. After being processed by the unit (2) 504, it is transmitted to the user-side information processing apparatus 20 by the virtual machine image transmission module (not shown) in the WEB processing unit 300, and the virtual machine image reception in the user-side information processing apparatus 20. It is input to the VR synthesis processing unit 22 via a module (not shown). Then, the VR synthesis processing unit 22 synthesizes the processed computer screen with the computer screen in the VR office, and the display device 32 displays the computer screen in the VR office.
At this time, the composition of the computer screen in the VR office can be performed by using, for example, Unity (Unity Technologies Japan GK) or the like.

The above computer screen is created on the video memory of the thin client processing device 500, transferred to the main memory by the GPU, and further transmitted from the main storage to the user-side information processing device 20 via the WEB processing unit 300. Will be done. At this time, it is preferable to compress the image information in order to reduce the network communication load, and to encrypt and transmit the image information for security.

In this way, important information such as confidential files used in the user-side information processing device virtual machine in the thin client processing device 500 is not transferred to the user-side information processing device 20, and the user-side information processing device virtual machine. Since only the computer screen of the above is transmitted to the information processing apparatus 20 on the user side, it is possible to contribute to the prevention of information leakage.
That is, in the office VR system of the present embodiment, it is possible to make a thin client.

Further, more specifically, in the office VR system of the present embodiment, the VR office displayed on the user side information processing device 20 side is only the computer screen of the user who is using the user side information processing device 20. Instead, the computer screen of another person placed on a desk near the user may also be displayed. Therefore, in order to bring the display of the computer screen closer to the actual appearance, the resolution of the computer screen can be adjusted as described above.

Specifically, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 are a virtual machine for the user-side information processing device (1) 501 and a virtual machine for the user-side information processing device (2), respectively. ) From 502, the computer screen is input, the user's position information updated in real time is input from the user storage unit 101, and the arrangement information of the three-dimensional image of the computer is input from the office storage unit 102 (screen in FIG. 6). Adjustment processing unit (2) Only the input to 504 is shown), the resolution of the computer screen is determined for each user's identification information according to the distance from the computer of another user.
Then, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 transmit the computer screen processed to the determined resolution to the corresponding user-side information processing device 20 for each user's identification information.

For example, the resolution of the computer screen displayed in front of the user is increased, and the resolution of the computer screen at a remote location is decreased according to the distance.
In this way, in an actual office, the screen of a distant personal computer can be seen somewhat vaguely, and it is possible to express a state similar to the clear view of the screen of a personal computer at hand.
Further, by adopting such a display method, the amount of information on the screen of a personal computer placed far away can be reduced, and the amount of information on the computer screen via the network can be reduced.

Further, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 can not only adjust the resolution of the computer screen but also control whether or not another person can look into the computer screen. can.
In this case, the user storage unit 101 stores the user's authority information for each user's identification information.

Specifically, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 are authorized by the user from the user storage unit 101 based on the user's identification information corresponding to the user-side information processing device 20. Information is input (in FIG. 6, only input to the screen adjustment processing unit (2) 504 is shown), and it is determined for each user's identification information which user has the authority to display the computer screen. , Display only the computer image of the authorized user, and do not display the computer screen for the unauthorized user.

That is, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 process the computer screen based on the resolution only for the computer screen to be displayed, and the processed computer screen is processed for each user's identification information. Is transmitted to the user-side information processing apparatus 20 corresponding to the above. Further, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 correspond to the information indicating that the computer screen to be not displayed is not displayed for each user's identification information 20. Can be sent to.

As described above, according to the VR system for offices of the present embodiment, in the VR office, for each user-side information processing device 20, only the computer screen that may be looked into is displayed based on the user's authority information, and the reference authority is displayed. It is possible to hide the computer screen without information. It can be said that this is an advantage peculiar to the VR office that exceeds the security of the actual office.

The VR synthesis processing unit 22 of the user-side information processing device 20 normally synthesizes a VR image so as to display a computer screen corresponding to each user-side information processing device 20 in a part of the VR office, but inputs the VR image. When the full screen display instruction information is input from the device 31, the computer screen may be displayed on the entire screen of the display device 32. Further, when the input device 31 inputs the full-screen display cancellation instruction information, the display can be returned to the normal VR office display.
As a result, the amount of information that can be displayed on the screen displayed by the display device 32 can be temporarily increased.

Next, the processing procedure of the thin client processing device 500 and the user-side information processing device 20 in the office VR system of the present embodiment will be described with reference to FIG. 7.
First, the virtual machine (1) 501 for the user-side information processing device and the virtual machine (2) 502 for the user-side information processing device in the thin client processing device 500 work on behalf of the corresponding user-side information processing device 20. Perform the functions of the computer used for (step 20).
Further, the user-side information processing device virtual machine (1) 501 and the user-side information processing device virtual machine (2) 502 create their respective computer screens (step 21).

Next, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 in the thin client processing device 500 store the user's position information stored in the user storage unit 101 and the office storage unit 102. The resolution of the computer screen created by the virtual machine for the information processing device on the user side is determined for each identification information of the user based on the arrangement information of the equipment and the like (step 22).

Further, the screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 input the user's authority information from the user storage unit 101 based on the user's identification information corresponding to the user-side information processing device 20. Then, for each user's identification information, it is determined which user has the authority to display the computer screen (step 23). The order of step 22 and step 23 may be changed.

The screen adjustment processing unit (1) 503 and the screen adjustment processing unit (2) 504 process the computer screen based on the resolution only for the computer screen to be displayed, and the processed computer screen corresponds to each user's identification information. It is transmitted to the user-side information processing apparatus 20 (step 24).
Then, the VR synthesis processing unit 22 in the user-side information processing apparatus 20 synthesizes the processed computer screen with the computer screen in the VR office, and displays the computer screen in the VR office (step 25).

As described above, according to the VR system for offices of the present embodiment, it is possible to display a computer screen as a thin client in a part of the VR office.
In addition, the resolution of the computer screen can be adjusted according to the position from the user in the VR office.
Further, it is possible to display or hide the computer screen of another user according to the user's authority information.

[Third Embodiment]
Next, the third embodiment of the present invention will be described with reference to FIGS. 8 and 9. FIG. 8 is a block diagram showing a configuration of a VR processing device in the office VR system of the present embodiment. FIG. 9 is a flowchart showing a processing procedure of the VR processing device in the system.

In addition to the configuration of the first embodiment, the office VR system of the present embodiment has a configuration of adding a facial expression according to the user's emotion to the facial image in the user's three-dimensional image. Other points can be the same as that of the first embodiment.
That is, as shown in FIG. 8, the VR processing device 200a in the office VR system of the present embodiment includes a facial expression memory unit 204, a facial expression image creation unit 205, and an emphasized image addition unit 206.

Here, in general, people who work from home often do not bother to get dressed or make up, and do not want others to see the image of their face while working from home.
However, for example, in a normal video conference system, a person's face is shot as a moving image in real time and displayed to other employees. For this reason, when using the video conference system while working from home, it was necessary to dress and make up even at home.
In order to eliminate such a burden, the office VR system of the present embodiment introduces a mechanism for transmitting only the facial expression to other employees, instead of simply taking a picture of the face and displaying it in the VR office.

The facial emotion storage unit 204 stores facial images of various facial expressions in association with a numerical value indicating emotions and the degree of emotions (hereinafter, this may be referred to as a facial emotion value).
The facial emotion storage unit 204 is created by a so-called "supervised learning" method using a neural network.
Specifically, a neural network is trained by setting a large number of various facial images (laughing face, angry face, sad face, surprised face, etc.) and numerical values indicating the degree of emotion at that time. .. For example, the facial emotion value is a numerical value from 1 to 10, where 1 is the smallest emotion and 10 is the largest emotion.

In creating the facial emotion storage unit 204, the data inputters subjectively declare what kind of emotion a certain facial expression expresses and which facial emotion value corresponds to by a self-reporting method. Then, for example, emotions such as a laughing face 10, an angry face 5, and a sad face 3 and facial emotion values can be learned by a neural network together with a facial image, and this neural network can be obtained as the facial emotion storage unit 204. In this way, a large amount of various facial images and corresponding emotions and facial emotion values are accumulated in the facial emotion storage unit 204.
Learning by such a neural network can be performed using, for example, TENSORFLOW (registered trademark) manufactured by Google.

The facial expression image creating unit 205 inputs the user's identification information and the face image transmitted from the user-side information processing apparatus 20 via the WEB processing unit 300. Then, based on the input facial image, the image closest to the facial image and the emotions and facial emotion values associated with the image are acquired from the facial emotion storage unit 204.

Further, the facial expression image creating unit 205 acquires a facial image (expressionless) corresponding to the identification information of the user from the user storage unit 101, and deforms the facial image based on emotions and facial emotion values. By applying the processing, a facial expression image with a facial expression can be obtained.
In this case, the user storage unit 101 stores in advance a face image (expressionless) of the user's three-dimensional image for each user's identification information.

Specifically, the facial expression image creating unit 205 extracts each part (partial images such as eyes, nose, and eyebrows and their position information) in the expressionless face image. Further, the facial expression image creating unit 205 is pre-programmed to change the movement, resizing, color, etc. of each part according to emotions and facial emotion values based on the empirical rules of general facial expression changes. It is possible to add facial expressions by applying deformation processing to the expressionless face image. This makes it possible to express that each part of a person's face image moves depending on emotions, the size changes (eyes become larger, mouth opens, etc.), and the complexion changes. ..

The user storage unit 101 stores various facial expression images according to emotions and facial emotion values for each user identification information, and the facial expression image creation unit 205 stores the user based on the emotions and facial emotion values. It is also possible to acquire the corresponding emotional image from the unit 101.

Then, the facial expression image creating unit 205 outputs the obtained facial expression image and the user's identification information to the VR information creating unit 203, and the VR information creating unit 203 includes the facial expression image corresponding to the user's identification information in the VR information. Then, the information is transmitted to the information processing apparatus 20 on the user side.
The VR synthesis processing unit 22 in the user-side information processing apparatus 20 inputs VR information including a facial expression image, acquires a three-dimensional image of each user from the image storage unit 21, and obtains the corresponding facial expression image as a tertiary of each user. Create an image of the VR office by synthesizing it with the original image.
Specifically, this synthesis can be performed by, for example, Unity (Unity Technologies Japan GK).

Further, instead of including the facial expression image in the VR information, the emotion and the facial emotion value may be included and transmitted to the user-side information processing apparatus 20, and the facial expression image may be created by the VR synthesis processing unit 22. .. In this case, the expressionless face image of each user is stored in the image storage unit 21, and the expressionless face image is deformed by the same method as the above-mentioned facial expression image creation unit 205 to obtain a facial expression image. be able to. Then, the obtained facial expression image can be converted from a two-dimensional image to a three-dimensional image, combined with the three-dimensional image of each user, and displayed in the image of the VR office.

Since the image of the user himself / herself who uses the information processing apparatus 20 on the user side is not displayed in the image of the VR office, he / she cannot usually see his / her facial expression image. However, it is also preferable to make it possible to confirm that one's emotion and facial emotion value correspond appropriately to one's facial expression image, and if they are different, make it possible for the facial emotion storage unit 204 to learn. ..

For example, the VR processing device 200a causes the user-side information processing device 20 to include the user's facial expression image in the VR information and transmit the user's facial expression image, and the facial expression image confirmation unit (not shown) in the user-side information processing device 20 transmits the user's facial expression image. Can be displayed. Then, when it seems that the facial expression image does not properly display one's own emotions, the corrected values of the emotions and the facial emotion values can be input and transmitted to the VR processing device 200a together with the facial expression images.

Then, the facial emotion updating unit (not shown) in the VR processing device 200a can input the received emotion, the facial emotion value, and the facial expression image into the facial emotion storage unit 204 for learning.
This makes it possible to improve the ability of the facial emotion storage unit 204 and the facial expression image creation unit 205 to extract emotions from the facial image of the user or the like by learning.

Further, in addition to storing a face image composed of a user's face photograph in the user storage unit 101, it is also possible to store a face image composed of a picture such as a caricature for each user's identification information. Then, using a face image composed of such a picture, it is also possible to obtain a facial expression image by the facial expression image creating unit 205 by the same method as described above.

The highlighted image addition unit 206 inputs emotions and facial emotion values from the facial expression image creating unit 205, acquires the corresponding highlighted image, outputs it to the facial expression image creating unit 205, and highlights it by the facial expression image creating unit 205. It is possible to create a facial expression image with an image added.
For example, in the VR processing device 200a, the highlighted image storage unit (not shown) for storing the highlighted image corresponding to the emotion and the facial emotion value is provided, and the emphasized image addition unit 206 has input the emotion and the facial emotion value. Based on this, it is possible to acquire the corresponding highlighted image from the highlighted image storage unit and output it to the emotional image creating unit 205.
The facial expression image creating unit 205 can create a facial expression image to which a highlighted image is added by superimposing a highlighted image on the facial expression image.

For example, the highlighted image addition unit 206 may add a highlighted image that adds steam to the head in association with the angry face 10 (maximizing the angry emotion), or the sad face 10 (maximizing the sad emotion). It is possible to add a highlight image that sheds tears in association with (represented in). The highlighted image may be a picture or a photograph.

Next, the processing procedure of the VR processing device 200a in the office VR system of the present embodiment will be described with reference to FIG.
The facial expression image creating unit 205 in the VR processing device 200a inputs the user's face image from the user-side information processing device 20 together with the user's identification information (step 30). Then, based on the input facial image, the image closest to the facial image and the emotions and facial emotion values associated with the image are acquired from the facial emotion storage unit 204 (step 31).

Further, the facial expression image creating unit 205 acquires a face image corresponding to the identification information of the user from the user storage unit 101 (step 32).
Then, the facial expression image creating unit 205 processes the facial image based on the emotion and the facial emotion value to create a facial expression image to which the facial expression is added (step 33).

Further, the enhanced image addition unit 206 in the VR processing device 200a inputs emotions and facial emotion values from the facial expression image creating unit 205, acquires the corresponding highlighted image, and outputs it to the facial expression image creating unit 205 (step). 34).
The facial expression image creating unit 205 creates a facial expression image to which a highlighted image is added, outputs the obtained facial expression image and user identification information to the VR information creating unit 203, and the VR information creating unit 203 converts the obtained facial expression image into VR information. The facial expression image is included and transmitted to the user side information processing apparatus 20 side (step 35).

As described above, according to the VR system for office of the present embodiment, not only the facial expression can be added to the face image of the user, but also the highlight image can be added by quantifying the degree of emotion. It is possible to express various emotions more explicitly.
These effects cannot be obtained by the conventional method of quantifying the movement of facial feature points and changing the facial image based on the numerical values.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIGS. 10 and 11. FIG. 10 is a block diagram showing a configuration of a user-side information processing device in the office VR system of the present embodiment. FIG. 11 is a flowchart showing a processing procedure of the user-side information processing apparatus in the system.

The office VR system of the present embodiment has, in addition to the configuration of the first embodiment, a configuration of displaying an elevator prior to displaying the VR office. Other points can be the same as that of the first embodiment.
That is, as shown in FIG. 10, the user-side information processing device 20a in the office VR system of the present embodiment includes an elevator display unit 26.
Further, the image storage unit 21 in the user-side information processing apparatus 20a of the present embodiment stores a three-dimensional image of the elevator.

The elevator display unit 26 acquires a three-dimensional image of the elevator from the image storage unit 21 and displays an elevator (hereinafter, may be referred to as a VR elevator) for going to the floor where the VR office is located on the display device 32. Let me.
At this time, the elevator display unit 26 displays an image in which the elevator door changes from the closed state to the open state, and when the user moves into the elevator and turns to the door side, the elevator door opens. It is possible to display an image that shifts from the closed state to the closed state.
Similar to the VR synthesis processing unit 22, such an elevator display unit 26 displays using, for example, Unity (Unity Technologies Japan GK) based on the information input to the user-side information processing device 20a. Can be done.

Further, the elevator display unit 26 displays an open / close button and a floor number button in the elevator in the same manner as a general elevator, and is based on information indicating either open / close input to the information processing apparatus 20a on the user side. The opening and closing of the door can be displayed. Further, the elevator display unit 26 can also turn on and display the designated floor number button in the VR elevator based on the information indicating the floor number input to the user-side information processing apparatus 20a.

Then, the elevator display unit 26 displays an image in which the elevator door changes from the closed state to the open state for each designated number of floors after a certain period of time has elapsed from the display of closing the door.
Next, the elevator display unit 26 displays an image moving outward from the elevator, and the VR synthesis processing unit 22 displays the VR office.

Further, the elevator display unit 26 inputs the user's position information from the VR synthesis processing unit 22, and when another user virtually exists at the elevator position, the image storage unit 21 displays the user's three-dimensional image. Is acquired and the other user is displayed in the elevator.
The display of the elevator by the elevator display unit 26 can be performed after the biometric authentication of the user is successful and the image when passing through the entrance is displayed.

Further, the elevator display unit 26 displays the elevator, acquires the current time information, and displays the opening / closing of the door only when the current time is within the working hours, and when the current time is not within the working hours. Can also display to that effect on the display device 32 or make an announcement by the audio output device 33 to prevent the user from entering the elevator without displaying an image that opens the elevator door.

Next, the processing procedure of the user-side information processing apparatus 20a in the office VR system of the present embodiment will be described with reference to FIG.
First, the elevator display unit 26 in the user-side information processing device 20a acquires a three-dimensional image of the elevator from the image storage unit 21 and displays the elevator for going to the floor where the VR office is located on the display device 32 (step). 40).

Further, the elevator display unit 26 acquires the current time information and determines whether or not the current time is within the working hours (step 41).
Then, the elevator display unit 26 does not display an image for opening the elevator door when the current time information is not within working hours, and prevents the user from entering the VR elevator (step 42).

On the other hand, when the current time information is within working hours, the elevator display unit 26 displays the opening and closing of the elevator door and allows the user to enter the VR elevator (step 43).
Further, the elevator display unit 26 inputs the user's position information from the VR synthesis processing unit 22 (step 44), and when another user virtually exists at the elevator position, the user from the image storage unit 21. The three-dimensional image of the above is acquired, and the other user is displayed in the elevator (step 45).
Then, the elevator display unit 26 displays an image that arrives at the designated number of floors and opens the door, then displays an image that moves outward from the elevator, and the VR synthesis processing unit 22 displays the VR office. (Step 46).

As described above, according to the VR system for offices of the present embodiment, the user can go to work in the VR office as if he / she actually goes to work in the elevator, and switches his / her feelings to the work mode. be able to.
In addition, when the elevator is shared with other users, it is possible to communicate by giving a greeting.
Furthermore, it is possible to prevent overtime work by preventing the elevator from operating outside working hours.

[Fifth Embodiment]
Next, the fifth embodiment of the present invention will be described with reference to FIGS. 12 and 13. FIG. 12 is a block diagram showing a configuration of a VR processing device in the office VR system of the present embodiment. FIG. 13 is a flowchart showing a processing procedure of the VR processing apparatus in the system.

In the office VR system of the present embodiment, in addition to the configuration of the first embodiment, when a user can talk with another user in the VR office and the languages used by the users are different. It has a configuration for translating and outputting audio. Other points can be the same as that of the first embodiment.
That is, as shown in FIG. 12, the VR processing device 200b in the office VR system of the present embodiment includes a conversation transmission unit 207 and a translation creation unit 208.

The conversation transmission unit 207 inputs the voice information transmitted from the user-side information processing apparatus 20. Further, the conversation transmission unit 207 contains the position information of the user who uses the user-side information processing device 20 stored in the user storage unit 101 and the position information of another user who uses the other user-side information processing device 20. Is input from the VR information creation unit 203, the volume and output position of the user's voice information are determined for each user's identification information, and the information is transmitted to another user-side information processing device 20.

Specifically, the conversation transmission unit 207 sets the volume of voice information from a user who is far away to be low, and sets the volume of voice information from a user who is near to be high. Further, the voice information from the user on the right side is set to be output from the voice output device 33 arranged on the right side, and the voice information from the user on the left side is set to be output from the voice output device 33 arranged on the left side.

Further, it is also preferable to set the audio information from the user behind to be stereophonic, such as being output from the audio output device 33 arranged on the rear side. Stereophonic sound can be realized by using a method of placing speakers in the front and back, a stereophonic headphone that expresses the front and back sound in a pseudo manner, and the like.

Further, the conversation transmission unit 207 prevents the user-side information processing apparatus 20 from transmitting voice information from another user based on the input of the conversation transmission stop instruction information from the user-side information processing apparatus 20. You can also do it.
By doing so, when the users want to concentrate, it is possible to prevent the user-side information processing apparatus 20 from outputting the voice information from other users.

Further, from the viewpoint of security, it is possible to prevent the voice information from a specific user from being transmitted to the user-side information processing apparatus 20 of another user.
For example, the conversation transmission unit 207 can prevent the voice information corresponding to the identification information of the president or the director from being transmitted to the user-side information processing apparatus 20 used by the user of the identification information of a general employee.
Furthermore, the conversation transmission unit 207 is the user-side information used by other users with respect to the voice information transmitted from the user-side information processing apparatus 20 of the user when the user's position information exists in a specific place. It is possible to prevent transmission to the processing device 20. This makes it possible to prevent, for example, conversations in a conference room from leaking to the outside without providing soundproofing equipment or the like.

The translation creation unit 208 can input voice information from the conversation transmission unit 207, determine the type of language, and translate each user's identification information existing in the VR office into a specific language.
Specifically, the user storage unit 101 stores the type of language for each user identification information. The translation creation unit 208 acquires the identification information of the users existing in the VR office from the VR information creation unit 203, and acquires the language type corresponding to these users from the user storage unit 101. Then, when the type of the language determined above is different from the type of the acquired language, the translation creation unit 208 converts the voice information input from the conversation transmission unit 207 into text, and the language corresponding to each user's identification information. The translated text can be machine-translated into the above types, and the obtained translated text can be voice-converted and output to the conversation transmission unit 207.

At this time, the language type determination processing, the voice information text conversion processing, the machine translation processing, and the voice conversion of the translated text by the translation creation unit 208 are performed by machine translation using a neural network such as TENSORFLOW (registered trademark). It can be done by a system or the like.
As a result, even when the user uses various languages, the voice information to be transmitted to the user-side information processing apparatus 20 of another user can be translated into the language used by the user and then transmitted.

Next, the processing procedure of the VR processing device 200b in the office VR system of the present embodiment will be described with reference to FIG.
First, the conversation transmission unit 207 in the VR processing device 200b inputs the voice information transmitted from the user-side information processing device 20 (step 50).
The conversation transmission unit 207 determines whether or not the conversation transmission stop instruction information has been input from the user-side information processing device 20 (step 51), and if the stop instruction information has been input (Yes in step 51), the user side. The voice information from another user is not transmitted to the information processing apparatus 20 (step 52).

Further, the conversation transmission unit 207 determines whether or not the input voice information is from the user-side information processing device 20 used by a specific user based on the identification information of the corresponding user (step 53). If it is from a specific user (Yes in step 53), the voice information is not transmitted to the user-side information processing apparatus 20 used by another user (step 52).
Further, the conversation transmission unit 207 determines whether or not the user's position information exists in a specific place (step 54), and if it exists in a specific place (Yes in step 54), the voice information is transmitted. It is not transmitted to the user-side information processing apparatus 20 used by another user (step 52).

When none of the above applies (No in step 51, No in step 53, No in step 54), the translation creation unit 208 in the VR processing device 200b inputs voice information from the conversation transmission unit 207 to input the type of language. Is determined (step 55).
Further, the translation creation unit 208 acquires the identification information of the user existing in the VR office from the VR information creation unit 203, acquires the type of the language corresponding to these users from the user storage unit 101, and determines the language described above. When the type of is different from the type of the acquired language, the voice information input from the conversation transmission unit 207 is converted into text (step 56), and machine-translated into the corresponding language type for each user's identification information. The obtained translated text is voice-converted (step 57), and the obtained voice information is output to the conversation transmission unit 207.

Then, the conversation transmission unit 207 contains the position information of the user who uses the user-side information processing device 20 stored in the user storage unit 101 and the position information of another user who uses the other user-side information processing device 20. Is input from the VR information creation unit 203, and the volume and output position of the voice information of the user are determined for each identification information of the user (step 58).
Then, the conversation transmission unit 207 transmits voice information (voice information if translated), its volume and output position to another user-side information processing device 20 (step 59).

As described above, according to the VR system for offices of the present embodiment, it is possible to have a conversation with other users in the VR office as if they were actually in the office. In addition, since meetings can be held while looking at the other person's appearance and facial expression, it is possible to prevent feeling lack of communication with other users even at home.
In addition, it is possible to have a conversation with a user who is actually in a remote place such as overseas as if he / she is there.
Furthermore, even if there is a user who speaks a foreign language in the VR office, it is possible to make himself / herself speak in Japanese, for example.

[Sixth Embodiment]
Next, the sixth embodiment of the present invention will be described with reference to FIGS. 14 to 16. FIG. 14 is a block diagram showing the configuration of the user-side information processing device in the office VR system of the present embodiment, and FIG. 15 is a block diagram showing the configuration of the VR processing device in the system. FIG. 16 is a flowchart showing a processing procedure of the VR processing device and the user-side information processing device in the system.

In addition to the configuration of the first embodiment, the office VR system of the present embodiment makes it possible to display various rooms, determines whether or not each room can be used based on the user's authority information, and also determines the user's work status. Has a structure that makes it manageable. Other points can be the same as that of the first embodiment.
That is, the user-side information processing device 20b in the office VR system of the present embodiment includes a room display unit 27 as shown in FIG. Further, as shown in FIG. 15, the VR processing device 200c includes an authority determination and status recording unit 209, and the office side storage device 100a includes a user status storage unit 103. Further, the user storage unit 101 in the present embodiment stores the user's authority information for each user's identification information.

The room display unit 27 inputs the position information of the user using the information processing device 20b on the user side (hereinafter, may be referred to as the user position information) from the VR synthesis processing unit 22, and this position information. If there is a room arranged at a position corresponding to, a three-dimensional image of the room is acquired from the image storage unit 21 and displayed on the display device 32. At this time, the room display unit 27 can display an image in which the door of the room is closed.
Further, the user position information and the room identification information are transmitted from the room display unit 27 (or the VR synthesis processing unit 22) to the VR processing device 200c, input to the VR information creation unit 203, and the corresponding user identification information. At the same time, it is output to the authority determination and status recording unit 209.

When the authority determination and status recording unit 209 inputs the user identification information, the user location information, and the room identification information from the VR information creation unit 203, the user authority information is input from the user storage unit 101 based on the user identification information. It is acquired, and it is determined whether or not the user can enter the room based on the user's authority information and the room identification information. At this time, the VR processing device 200c is provided with an entry authority storage unit (not shown) that stores the authority information of the user who can enter the room for each room identification information, and is accessed by the authority determination and status recording unit 209 for determination. It may be configured to make it.

The authority determination and status recording unit 209 transmits the determination result of admission to the room to the user-side information processing apparatus 20b.
Then, the room display unit 27 of the user-side information processing device 20b can display the image that shifts from the closed state to the open state of the room door on the display device 32 when the determination result is possible to enter the room. This allows the user to move into the room. Then, the VR synthesis processing unit 22 (or the room display unit 27) acquires the three-dimensional image in the room from the image storage unit 21 and displays it on the display device 32.

Further, the VR synthesis processing unit 22 (or the room display unit 27) determines whether or not the position information of another user indicates that it exists in the room, and if it exists, the image. A three-dimensional image of the user is acquired from the storage unit 21, and the other user is displayed in the room.
On the other hand, when the determination result is that the room cannot be entered, the room display unit 27 displays the fact on the display device 32 or announces it by the voice output device 33.

Further, the authority determination and status recording unit 209 is made to transmit the instruction information for photographing and transmitting the user's face image to the user-side information processing device 20b, and the biometric information acquisition unit 23 is made to acquire the user's face image. It may be configured to transmit to the VR processing device 200c, perform face authentication of the user by the biometric authentication unit 201, and determine whether or not to enter the room only when the authentication is successful.

Further, the authority determination and status recording unit 209 can record the user's work status in the user status storage unit 103 based on the user position information and the room identification information.
Specifically, the user status storage unit 103 stores the type of the user's work status, location information (or room identification information), time information, and the like for each user identification information.
Examples of the type of work status of the user include the following.
(1) I am seated at my desk (2) I go to the meeting room and participate in the meeting (3) I sit down or talk in other offices (president's room, room of each department, etc.) (4) Walking in the aisle between the offices (5) Seated in the break room (6) Boarding the elevator (7) Authenticating at the entrance (8) Going out (9) Server maintenance in the server room (10) Customer support at the reception

In this way, the authority determination and status recording unit 209 records the user's work status in the user status storage unit 103, so that what kind of work each user is doing and how long it takes is automatically recorded. It is possible to record and manage it.

Further, the three-dimensional image of the room displayed by the room display unit 27 (or the VR synthesis processing unit 22) is not only an image of a room in an office that actually exists, but also, for example, a grassland, a forest, or a seaside space. In reality, it is difficult to realize, but it is possible to create an image of an environment that makes you feel at ease when you can work in such a space.
Further, when the VR synthesis processing unit 22 displays a three-dimensional image of a conference room, in the conference room, for each user, the computer screen used by each user is displayed in front of the user, or the screen on which the material is displayed is displayed in the air. It is also possible to float it on the screen and display it.
In these cases, the image storage unit 21 stores the three-dimensional image of the above environment for each room identification number, the screen or the like is stored for each equipment identification number, and the VR synthesis processing unit 22 stores these images. It can be acquired and displayed.

Next, the processing procedure of the VR processing device 200c and the user-side information processing device 20b in the office VR system of the present embodiment will be described with reference to FIG.
First, the room display unit 27 in the user-side information processing device 20b inputs the position information of the user who is using the user-side information processing device 20b from the VR synthesis processing unit 22, and at the position corresponding to this position information. When the arranged room exists, a three-dimensional image of the room is acquired from the image storage unit 21 and displayed on the display device 32 (step 60).
Further, the room display unit 27 transmits the user position information and the room identification information to the VR processing device 200c (step 61).

The authority determination and status recording unit 209 in the VR processing device 200c acquires the user's authority information of the user from the user storage unit 101 (step 62).
Then, the authority determination and status recording unit 209 determines whether or not the user can enter the room based on the user's authority information and the room identification information, and transmits the determination result to the user-side information processing apparatus 20b (step 63).
Further, the limit determination and status recording unit 209 records the user's work status in the user status storage unit 103 based on the user position information and the room identification information (step 64).

When the determination result is accessible, the room display unit 27 in the user-side information processing device 20b causes the display device 32 to display an image in which the door of the room changes from the closed state to the open state, whereby the user is displayed. Make it movable within the room (step 65).
Then, the room display unit 27 (or the VR synthesis processing unit 22) inputs the user's position information (step 66), and determines whether or not the position information of another user exists in the room. If it exists, a three-dimensional image of the user can be acquired from the image storage unit 21 and the other user can be displayed in the room (step 67).

As described above, according to the VR system for offices of the present embodiment, various rooms can be displayed in the VR office, and it is possible to determine whether or not to enter each room according to the user's authority. can. Therefore, it is possible to appropriately implement information leakage prevention and information control.
In recent years, improvement in productivity has been desired in work style reforms, and according to the VR system for offices of this embodiment, what kind of work and how long employees are doing is It can be recorded and managed automatically. For example, when an employee in charge of programming is attending a meeting, the user status storage unit 103 can store that he / she is not working on the program, and can stop recording the work time of the program. Then, when the meeting is over and the user returns to his / her desk again, the recording of the work time of the program can be resumed in the user status storage unit 103. This makes it possible to more appropriately evaluate the productivity of employees.

The office VR system of each of the above embodiments can be realized by using a computer controlled by the office VR program of the present invention. The CPU / GPU of the computer sends a command to each component of the computer based on the office VR program, and predetermined processing required for the operation of each processing device in the office VR system, for example, biometric authentication processing and access recording. Processing, VR information creation processing, VR synthesis processing, biological information acquisition processing, screen adjustment processing, facial expression image creation processing, and the like are performed. As described above, each process and operation in the office VR system of the present invention can be realized by concrete means in which the program and the computer cooperate.

The program is stored in a recording medium such as a ROM or RAM in advance, and the program is read and executed by the computer from the recording medium mounted on the computer. However, the program can also be read by the computer via a communication line, for example.
Further, the recording medium for storing the program can be configured by, for example, a semiconductor memory, a magnetic disk, an optical disk, or any other recording means that can be read by any computer.

As described above, according to the office VR system and the office VR program according to the embodiment of the present invention, it seems that the user actually goes to work by working from home using the VR office. It is possible to prevent communication with other employees from becoming weak.
In addition, a thin client computer screen can be displayed in a part of the VR office, and the computer screen of another user can be adjusted and displayed according to its position, or not displayed according to the user's authority information. be able to.
Further, in the VR office, instead of displaying the user's current face image, a face image having a facial expression according to the emotion at that time can be created and displayed, and the emotion is highlighted on the image. You can also do it.

In addition, you can display the elevator to go to the VR office so that you can virtually get on the elevator and go to the floor where your room is located, you can communicate with other users in the elevator, and you can work hours. By preventing the elevator from operating outside, it is possible to prevent overtime work.
Further, in the VR office, it is possible to have a conversation with another user as if he / she is actually in the office, and even if there is a user who speaks a foreign language, the voice can be output in his / her own language.
In addition, various rooms can be displayed in the VR office, and it is possible to judge whether or not to enter each room according to the user's authority, and what kind of work the employee is doing and how long it takes. It is also possible to record and manage automatically.

It goes without saying that the present invention is not limited to the above embodiments, and various modifications can be made within the scope of the present invention.
For example, it is possible to appropriately change the configuration such that each of the first to sixth embodiments is combined in various ways.

The present invention can be suitably used in a company or the like that works from home.

10 Office VR device group 100 Office side storage device 101 User storage unit 102 Office storage unit 200, 200a, 200b, 200c VR processing device 201 Biometric authentication unit (authentication processing unit)
202 Access recording unit 203 VR information creation unit 204 Face emotion memory unit 205 Facial expression image creation unit 206 Emphasis image addition unit 207 Conversation transmission unit 208 Translation creation unit 209 Entrance judgment unit 300 WEB processing device 400 IP telephone device 500 Thin client processing device 501 Virtual machine for user-side information processing device (1)
502 User-side information processing device virtual machine (2)
503 Screen adjustment processing unit (1)
504 Screen adjustment processing unit (2)
20, 20a, 20b (20-1, 20-2) User-side information processing device 21 Image storage unit 22 VR synthesis processing unit 23 Biometric information acquisition unit (authentication information acquisition unit)
24 User position information transmission unit 25 Data input / output unit 26 Elevator display unit 27 Room display unit 31 Input device 32 Display device 33 Voice output device 34 Voice input device 35 Image input device

Claims (11)

It is an office VR system for displaying a VR (Virtual Reality) image of an office on a user-side information processing device and providing a VR office capable of performing business outside the office to the user.
The office VR device group having the office side storage device and the VR processing device, and a plurality of the user side information processing devices connected to the office VR device group via a communication line are provided.
The office storage device
A user storage unit that stores user authentication information, user location information in the VR office, and information indicating that the user has accessed the VR office for each user identification information, and
An office that stores the layout information of three-dimensional images of one or more rooms in a VR office for each room identification information, and stores the layout information of three-dimensional images of multiple equipment in a VR office for each equipment identification information. Equipped with a storage unit
The VR processing device
An authentication processing unit that determines the success or failure of authentication based on whether or not the authentication information transmitted from the user-side information processing device matches the user's authentication information stored in the user storage unit.
If the authentication is successful, an access recording unit that records information indicating that the user has accessed the VR office in the user storage unit, and an access recording unit, and
The user's position information in the VR office stored in the user storage unit and the arrangement information stored in the office storage unit are input, and the VR information including the location information and the arrangement information is created and the user. It is equipped with a VR information creation unit that sends information to the information processing device on the side.
The user-side information processing device
The user's three-dimensional image is stored for each user's identification information, the three-dimensional image of one or more rooms in the VR office is stored for each room identification information, and the three-dimensional image of multiple equipment in the VR office is installed. Image storage unit that stores each item identification information,
A VR image based on the VR information transmitted from the VR information creation unit, the user's three-dimensional image stored in the image storage unit, the room three-dimensional image, and the equipment three-dimensional image. VR synthesis processing unit, which synthesizes and displays on the display device connected to the information processing device on the user side.
The authentication information acquisition unit that inputs the user's authentication information and sends it to the VR processing device, and
The office is provided with a user position information transmission unit that acquires the position information of the user using the user-side information processing device in the VR office and transmits the position information of the user to the VR processing device. VR system for. The office VR device group has a thin client processing device for making the user-side information processing device a thin client.
The thin client processing device
Each user-side information processing device is provided with a virtual machine for the user-side information processing device.
The resolution of the computer screen of the virtual machine for the information processing apparatus on the user side is identified by the user based on the user's position information stored in the user storage unit and the arrangement information stored in the office storage unit. Claim 1 is provided with a screen adjustment processing unit that determines each piece of information and processes the computer screen to a determined resolution to send the computer screen to the corresponding user-side information processing apparatus for each identification information of the user. The listed office VR system. The user storage unit stores a user's face image for each user's identification information.
The VR processing device
A facial emotion memory unit that stores facial images of various facial expressions in association with emotions and numerical values indicating the degree of emotions, and
Based on the user's identification information and the face image transmitted from the user-side information processing device, the image closest to the face image from the face emotion storage unit and the emotions and the magnitudes of the emotions associated with the image are determined. The indicated numerical value is acquired, and the facial image of the user stored in the user storage unit is acquired and processed based on the emotion and the numerical value indicating the magnitude of the emotion to create a facial expression image, which is used as the VR information. Equipped with an emotion image creation unit to include
The office use according to claim 1 or 2, wherein the VR synthesis processing unit processes a three-dimensional image of the user acquired from the image storage unit using the facial expression image included in the VR information. VR system. The VR processing device
Based on the emotion and the numerical value indicating the magnitude of the emotion, a highlighted image for emphasizing the three-dimensional face image of the user is acquired, and the facial expression image to which the highlighted image is added is obtained by the facial expression image creating unit. The VR system for an office according to claim 3, further comprising an enhanced image addition portion to be created by the above. The image storage unit stores a three-dimensional image of the elevator in the VR office.
The user-side information processing device
A three-dimensional image of the elevator is acquired from the image storage unit and displayed by a display device connected to the information processing device on the user side, and current time information is acquired so that the time information is within working hours. In any case, any of claims 1 to 4, further comprising an elevator display unit for displaying an image of opening the door of the three-dimensional image of the elevator by a display device connected to the user-side information processing device. VR system for office described in Crab. The VR processing device
The voice information transmitted from the user-side information processing device is input, and the position information of the user who uses the user-side information processing device stored in the user storage unit and the other user-side information processing device. A conversation transmission unit that determines the volume and output position of the voice information of the user for each user's identification information based on the position information of another user who uses the above, and transmits the information to the other user-side information processing device. The office VR system according to any one of claims 1 to 5, wherein the VR system is provided with. The user storage unit stores the type of language for each user identification information.
The VR processing device
Voice information is input from the conversation transmission unit to determine the language type of the voice, the voice information is text-converted, and machine-translated into the corresponding language type for each user's identification information, and the obtained translation is performed. The VR system for an office according to claim 6, further comprising a translation transmission unit that converts a sentence into voice and outputs the text to the conversation transmission unit. The user storage unit stores the user's authority information for each user's identification information.
The user-side information processing device
The position information of the user who is using the information processing device on the user side is input, the three-dimensional image and the identification information of the room arranged at the corresponding position are acquired from the image storage unit, and the information processing on the user side is performed. The display device connected to the device is provided with a room display unit for displaying a three-dimensional image of the room.
The VR processing device
When the user's position information and the room identification information are transmitted from the user-side information processing device, the user's authority information is acquired from the user storage unit based on the user's identification information, and the user's authority information is acquired. It is provided with an authority determination unit that determines whether or not to enter the room based on the authority information and the identification information of the room and transmits the determination result to the information processing device on the user side.
The room display unit
Any of claims 1 to 7, wherein when the determination result is acceptable, the image that opens the door of the three-dimensional image of the room is displayed by the display device connected to the information processing device on the user side. VR system for office described in the information processing. An office is used by using an office VR device group having an office side storage device and a VR (virtual reality) processing device, and a plurality of user side information processing devices connected to the office VR device group via a communication line. It is an office VR program for displaying a VR image of the above on the user side information processing device and providing a VR office capable of performing business outside the office to the user.
The office side storage device
A user storage unit that stores user authentication information, user location information in the VR office, and information indicating that the user has accessed the VR office for each user identification information, and
An office that stores the layout information of three-dimensional images of one or more rooms in a VR office for each room identification information, and stores the layout information of three-dimensional images of multiple equipment in a VR office for each equipment identification information. To function as a storage unit,
The VR processing device
An authentication processing unit that determines the success or failure of authentication based on whether or not the authentication information transmitted from the user-side information processing device matches the user's authentication information stored in the user storage unit.
If the authentication is successful, an access recording unit that records information indicating that the user has accessed the VR office in the user storage unit, and an access recording unit, and
The user's position information in the VR office stored in the user storage unit and the arrangement information stored in the office storage unit are input, and the VR information including the location information and the arrangement information is created and the user. It functions as a VR information creation unit to be transmitted to the side information processing device.
The user-side information processing device
The user's three-dimensional image is stored for each user's identification information, the three-dimensional image of one or more rooms in the VR office is stored for each room identification information, and the three-dimensional image of multiple equipment in the VR office is installed. Image storage unit that stores each item identification information,
A VR image based on the VR information transmitted from the VR information creation unit, the user's three-dimensional image stored in the image storage unit, the room three-dimensional image, and the equipment three-dimensional image. VR synthesis processing unit, which synthesizes and displays on the display device connected to the information processing device on the user side.
The authentication information acquisition unit that inputs the user's authentication information and sends it to the VR processing device, and
An office characterized by acquiring position information in a VR office of a user using the user-side information processing device and functioning as a user position information transmission unit that transmits the position information of the user to the VR processing device. VR program for. Using the office VR device group having a thin client processing device for converting the user-side information processing device into a thin client, a VR office capable of performing business outside the office is provided to the user. The office VR program according to claim 9 for the purpose.
The thin client processing device
Each user-side information processing device is made to function as a virtual machine for the user-side information processing device.
The resolution of the computer screen of the virtual machine for the information processing apparatus on the user side is identified by the user based on the user's position information stored in the user storage unit and the arrangement information stored in the office storage unit. An office VR characterized by functioning as a screen adjustment processing unit that determines each piece of information and processes the computer screen to a determined resolution and transmits the computer screen to the corresponding user-side information processing device for each identification information of the user. program. The user storage unit stores a user's face image for each user's identification information.
The VR processing device
A facial emotion memory unit that stores facial images of various facial expressions in association with emotions and numerical values indicating the degree of emotions, and
Based on the user's identification information and the face image transmitted from the user-side information processing device, the image closest to the face image from the face emotion storage unit and the emotions and the magnitudes of the emotions associated with the image are determined. The indicated numerical value is acquired, and the facial image of the user stored in the user storage unit is acquired and processed based on the emotion and the numerical value indicating the magnitude of the emotion to create a facial expression image, which is used as the VR information. Functions as an emotion image creation unit to include
The office according to claim 9 or 10, wherein the VR synthesis processing unit processes the three-dimensional image of the user acquired from the image storage unit by using the facial expression image included in the VR information. VR program for.

Images