JP6888854B1 - Remote work support system and remote work support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a remote work support system and a method which can intuitively recognize the state of each worker with a sense of realism as if he / she is working in the company even during remote work. A remote work support system 5 includes a server 200, a worker terminal 300, and a detection device 100 for detecting the state of a worker including a remote worker. The server 200 acquires the detection information detected by the detection device 100, generates each virtual worker reflecting the movement of each worker, and transmits the information of each virtual worker to each worker terminal 300. Each worker terminal 300 acquires the information of each virtual worker from the server 200, and acquires the information of the virtual room corresponding to the room in the company where each worker works and the information of the virtual object arranged in the virtual room. When the operation unit of each worker terminal 300 detects an operation on the displayed virtual worker or virtual object displayed in the first display area of the display unit included in each worker terminal 300, the operation corresponds to the operation. The information is displayed in the second display area of the display unit. [Selection diagram] Fig. 1

Description

The present disclosure relates to a remote work support system and a remote work support method for supporting remote work.

Conventionally, a method for holding a conference in a virtual environment has been known. For example, a method of holding a conference among multiple people, a virtual conference room for each of multiple people, including an avatar representing each person and a virtual display that displays a three-dimensional model (3D model) of the manufacturing equipment. A step to output on the display, a step to receive the selection of the 3D model object, a step to display information about the selected 3D model object on the virtual display, and a step for each person of multiple people. A method of holding a conference including a step of outputting a virtual conference room and a virtual display showing information about a selected three-dimensional model object, and a method of holding a conference is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2011-591438

Patent Document 1 describes that a three-dimensional model object can be selected in a virtual environment, but does not consider how each person's movement is specifically reflected in the avatar. Recently, due to the spread of infectious diseases, remote work (remote work) has been recommended.

For example, when the technology of Patent Document 1 is applied to the reproduction of remote work (for example, desk work, remote meeting) in a virtual environment, the movement of the worker is not reflected, so that the work is performed in the company where the worker works. There is a possibility that the sense of presence is insufficient. In addition, due to insufficient cooperation between the workers in the real space and the avatars in the virtual space corresponding to the workers, it may be difficult to understand the work status of each worker during remote work, for example. Further, it is preferable that the worker can obtain the necessary information in conjunction with the operation of the virtual environment because it is easy to understand intuitively and the work efficiency can be expected to be improved.

This disclosure was made in view of the above circumstances, and a remote work support system and remote work that can intuitively recognize the status of each worker, giving the feeling of being working in-house even during remote work. Provide support methods.

One aspect of the present disclosure is a remote work support system that supports remote work, and detects the state of a server, one or more worker terminals, and one or more workers including a remote worker. The server includes the above detection device, and the server acquires the detection information detected by the detection device, and based on the detection information, corresponds to each worker and reflects the movement of each worker. A virtual worker is generated, the information of each virtual worker is transmitted to each worker terminal, each worker terminal acquires the information of each virtual worker from the server, and each worker obtains the information of each virtual worker. Acquire the information of the virtual room corresponding to the room in the company where the employee works, acquire the information of the virtual object arranged in the virtual room, and at least one of the virtual room, each virtual worker, and the virtual object. Is displayed in the first display area of the display unit included in each worker terminal, and each worker terminal includes an operation on the virtual worker or the virtual object displayed in the first display area. When the operation unit detects it, the information corresponding to the operation is displayed in the second display area of the display unit , and each worker terminal acquires the information of the hierarchical organization to which each worker belongs. The information of the organization is displayed in the first display area, an arbitrary organization among the hierarchical organizations is designated via the operation unit, and the designated organization is arranged in the virtual room. It is a remote work support system that displays each virtual worker corresponding to each worker belonging to the organization based on the detection information of each worker belonging to the organization.
One aspect of the present disclosure is a remote work support system that supports remote work, and detects the state of a server, one or more worker terminals, and one or more workers including remote workers. The server includes the above-mentioned detection device, and the server acquires the detection information detected by the detection device, and based on the detection information, corresponds to each worker and reflects the movement of each worker. A virtual worker is generated, the information of each virtual worker is transmitted to each worker terminal, each worker terminal acquires the information of each virtual worker from the server, and each worker obtains the information of each virtual worker. Acquire the information of the virtual room corresponding to the room in the company where the employee works, acquire the information of the virtual object arranged in the virtual room, and at least one of the virtual room, each virtual worker, and the virtual object. Is displayed in the first display area of the display unit included in each worker terminal, and each worker terminal includes an operation on the virtual worker or the virtual object displayed in the first display area. When the operation unit detects it, the information corresponding to the operation is displayed in the second display area of the display unit, and each worker terminal detects the start of a remote dialogue in which a plurality of the workers attend. Based on the detection of the start of the remote dialogue, the virtual dialogue room for performing the remote dialogue as the virtual room and the virtual workers corresponding to each attendee attending the remote dialogue are designated as the first. It is a remote work support system that displays in the display area of 1.

One aspect of the present disclosure is a remote work support method for supporting remote work, in which the state of one or more workers including a remote worker is detected to generate detection information, and based on the detection information, the detection information is generated. Generate each virtual worker corresponding to each worker and reflect the movement of each worker, acquire the information of the virtual room corresponding to the room in the company where each worker works, and place it in the virtual room. The information of the virtual object to be generated is acquired, and at least one of the virtual room, the virtual worker, and the virtual object is displayed in the first display area of the display unit included in one or more worker terminals. When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. The information of the hierarchical organization to which each worker belongs is acquired, the information of the organization is displayed in the first display area, and the information of the hierarchical organization is displayed via the operation unit. In the virtual room where any of the organizations is designated and the designated organization is arranged, each worker belonging to the organization corresponds to each worker based on the detection information of each worker belonging to the organization. This is a remote work support method that displays each virtual worker.
One aspect of the present disclosure is a remote work support method for supporting remote work, in which the state of one or more workers including a remote worker is detected to generate detection information, and based on the detection information, the detection information is generated. Generate each virtual worker corresponding to each worker and reflect the movement of each worker, acquire the information of the virtual room corresponding to the room in the company where each worker works, and place it in the virtual room. The information of the virtual object to be generated is acquired, and at least one of the virtual room, the virtual worker, and the virtual object is displayed in the first display area of the display unit included in one or more worker terminals. When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. With a virtual dialogue room for displaying in the display area of, detecting the start of a remote dialogue attended by a plurality of the workers, and performing the remote dialogue as the virtual room based on the detection of the start of the remote dialogue. , The remote work support method for displaying each virtual worker corresponding to each attendee attending the remote dialogue in the first display area.

According to this disclosure, even during remote work, there is a sense of realism as if working in the company, and the state of each worker can be intuitively recognized.

The figure which shows the configuration example of the remote work support system in 1st Embodiment Block diagram showing a configuration example of the detection device Block diagram showing a server configuration example Block diagram showing a configuration example of a worker terminal The figure which shows the display example of the virtual space area when a worker is seated The figure which shows the display example of the virtual space area when a worker is away from the seat Sequence diagram showing an operation example related to the generation of worker movement information by the remote work support system Sequence diagram showing an operation example related to the display of the virtual office corresponding to the organization by the remote work support system Sequence diagram showing an operation example related to the display of the virtual office corresponding to the organization by the remote work support system (continued from FIG. 7A) Diagram showing a display example including a virtual office corresponding to an organization Sequence diagram showing an operation example when starting a remote dialogue by the remote work support system Sequence diagram showing an operation example when starting a remote dialogue by the remote work support system (continued from FIG. 9A) Sequence diagram showing an operation example at the end of remote dialogue by the remote work support system Diagram showing a display example showing the state of a remote conference Diagram showing an example of virtual material displayed in a specific area Diagram showing a display example showing the state of a remote interview The figure which shows the display example which showed the virtual worker corresponding to a remote worker and the virtual worker corresponding to an in-house worker in different display modes.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations for substantially the same configuration may be omitted. This is to avoid unnecessary redundancy of the following description and to facilitate the understanding of those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

For example, the "part" or "device" in the embodiment is not limited to a physical configuration realized by hardware, but also includes a function realized by software such as a program. Further, the functions of one configuration may be realized by two or more physical configurations, or the functions of two or more configurations may be realized by, for example, one physical configuration.

(First Embodiment)
FIG. 1 is a diagram showing a configuration example of the remote work support system 5. The remote work support system 5 includes a detection device 100, a server 200, and a worker terminal 300. The detection device 100, the server 200, and the worker terminal 300 may be communicably connected to each other via a network, or may be directly communicably connected to each other without going through a network.

The remote work support system 5 is a system that supports the work of worker HM including at least remote worker HM1. The worker HM may include an in-house worker HM2 who works in the company to which the remote worker HM1 belongs. The remote worker HM1 works remotely (business) using, for example, the worker terminal 300 used by the remote worker HM1. The in-house worker HM2 works (business) using, for example, the worker terminal 300 used by the in-house worker HM2.

The detection device 100 detects data for detecting the state of the worker HM. The detection device 100 includes at least one sensor. The sensor may include, for example, a camera, a microphone, an infrared sensor, a laser scanner, and the like. The detection device 100 is a room RS in the company (for example, an office room, a dialogue room (for example, a conference room, an interview room, an interview room, a counseling room)), a remote room RR (for example, a home where a remote worker HM1 performs desk work). Room), etc. The detection device 100 may be installed on the wall, ceiling, or other position of each room RS, RR. The worker HM is located in the detection range by the detection device 100. The detection device 100 transmits the detection information detected by various sensors to the server 200 or the worker terminal 300. In the present embodiment, the detection device 100 includes a camera, but other detection devices 100 as described above can also be adopted.

The server 200 performs a process for supporting the work by the worker HM. Work by worker HM includes in-house work and remote work (for example, work from home). In-house work may include in-house desk work work by in-house worker HM2, various dialogue work (for example, meetings, interviews, interviews, consultations), and the like. Remote work includes remote desk work at a remote location (for example, the home of remote worker HM1) by remote worker HM1 via a communication line, and various remote dialogue (for example, remote meetings, remote interviews, remote interviews). Etc. may be included. The remote dialogue is, for example, to perform various dialogues via a communication line in a state where each worker HM is located in a different place.

The server 200 supports the display of the virtual room RV arranged in the virtual space and the virtual worker HV located in the virtual room RV in order to express the working situation by the worker HM on the virtual space. The virtual room RV is a virtual room corresponding to each room RS in the company. The virtual room RV may include a virtual office room RV1, a virtual dialogue room RV2, and the like. The virtual worker HV is a virtual worker corresponding to the worker HM and corresponds to an avatar. In this way, the server 200 manages the information in the virtual space.

The worker terminal 300 is a terminal used by the worker HM during work, supports the work by the worker, and performs processing related to the work. The worker terminal 300 may include, for example, a desktop PC (Personal Computer), a notebook PC, a tablet, a smartphone, and other information processing terminals. The worker terminal 300 is installed in a room RS used for work in the company, a room RR used for remote work in a remote place, and the like. The worker terminal 300 cooperates with the server 200 to display the virtual room RV and the virtual worker HV in the virtual space.

FIG. 2 is a block diagram showing a configuration example of the detection device 100. The detection device 100 includes a control unit 110, a communication unit 120, a storage unit 130, a camera 140, a position acquisition unit 160, and a direction acquisition unit 170. The detection device 100 does not have to include the position acquisition unit 160 and the direction acquisition unit 170. Further, the detection device 100 may include a sensor (for example, a laser scanner) other than the camera 140.

The control unit 110 realizes various functions by executing, for example, a program stored in the storage unit 130, including a processor. The processor may include an MPU (Micro processing Unit), a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and the like. The control unit 110 comprehensively controls each unit of the detection device 100 and performs various processes.

The communication unit 120 communicates various data and information. The communication method by the communication unit 120 may be a wired communication or a wireless communication method, for example, communication such as WAN (Wide Area Network), LAN (Local Area Network), power line communication, infrared communication, and cellular communication for mobile phones. The method may be included. The communication unit 120 transmits, for example, detection-related data including detection data (for example, an image) detected by the detection device 100 and additional data thereof to the server 200.

The storage unit 130 includes a primary storage device (for example, a RAM (Random Access Memory) or a ROM (Read Only Memory)). The storage unit 130 may include a secondary storage device (for example, HDD (Hard Disk Drive) or SSD (Solid State Drive)) or a tertiary storage device (for example, an optical disk or SD card). The storage unit 130 may include other storage devices. The storage unit 130 stores various data, information, programs, and the like. The storage unit 130 may hold the identification information (detection device ID) of the detection device 100. The identification information of the detection device 100 may include the identification information (camera ID) of the camera 140, and the like.

The camera 140 captures the imaging range of the camera 140 and obtains an captured image. The imaging range of the camera 140 is based on the angle of view and the imaging direction of the camera 140. The captured image may be a moving image or a still image. The camera 140 may be constantly imaged, periodically imaged, or may be imaged at a predetermined timing. The captured image is an example of the detection data detected by the detection device 100. The imaging range includes rooms RS and RR in which the camera 140 is installed. A worker HM may be located in the rooms RS and RR. The angle of view of the camera 140 may be a wide angle of view such as 180 ° or omnidirectional (360 °), or may be another angle of view. The angle of view and the imaging direction of the camera 140 may be changeable, that is, the imaging range of the camera 140 may be changeable.

The position acquisition unit 160 includes, for example, a GPS (Global Positioning System) receiver, and acquires information on the position (arrangement position) of the detection device 100. The position acquisition unit 160 may acquire a two-dimensional position (for example, latitude, longitude) or a three-dimensional position (for example, latitude, longitude, altitude).

The direction acquisition unit 170 may include a magnetic sensor, an IMU (Inertial Measurement Unit), a gyro sensor, an acceleration sensor, and the like. The direction acquisition unit 170 acquires information on the detection direction by the detection device 100. The detection direction may include the north, south, east, and west directions (directions), and the vertical direction with reference to the vertical upward direction and the vertical downward direction (gravity direction). The detection direction may be the imaging direction by the camera 140. The imaging direction of the camera 140 may coincide with, for example, the direction in which the lens of the camera 140 is facing.

The control unit 110 has a clock function and acquires time data. For example, the control unit 110 may acquire data at the detection time when the detection data is detected. Further, the control unit 110 may acquire the detection device ID and the camera ID from the storage unit 130.

The control unit 110 may generate detection-related data including the detection data and its additional data. The detection data includes a captured image. The additional data may include at least one of data such as the arrangement position of the detection device 100, the detection direction, the detection time, the detection device ID, the camera ID, and the like. Therefore, the additional data may include data such as an imaging position by the camera 140 as the arrangement position, an imaging direction by the camera 140 as the detection direction, and a detection time by the camera 140 as the detection time.

The detection device 100 (for example, a camera) may obtain detection data from various positions in the rooms RS and RR in various directions, that is, while changing the detection position and the detection direction. This detection data may be the original data for generating 3D (three-dimensional) point cloud data of the rooms RS and RR in which the detection device 100 is installed.

The detection device 100 may simply detect the detection data (for example, an image captured by the camera 140) and provide the detection data to the server 200. In this case, the additional data is not generated and the detection-related data may not include the additional data.

FIG. 3 is a block diagram showing a configuration example of the server 200. The server 200 includes a control unit 210, a communication unit 220, and a storage unit 230.

The control unit 210 realizes various functions by executing a program stored in the storage unit 230, including a processor, for example. The processor may include an MPU, a CPU, a DSP, a GPU (Graphical Processing Unit), and the like. The control unit 210 comprehensively controls each unit of the server 200 and performs various processes.

The communication unit 220 communicates various data and information. The communication method by the communication unit 220 may be a communication method of wired communication or wireless communication, and may include communication methods such as WAN, LAN, power line communication, infrared communication, and cellular communication for mobile phones.

For example, the communication unit 220 receives detection-related data from the detection device 100. The detection-related data includes a captured image captured by the camera 140. The communication unit 220 receives detection-related data from all or a part of the plurality of detection devices 100 included in the remote work support system 5. The communication unit 220 may acquire a plurality of images captured at the same or different imaging times by each camera 140 of each detection device 100.

For example, the communication unit 220 communicates various data and information with each worker terminal 300. For example, data and information related to the display of information in the virtual space may be communicated, and information such as a virtual room RV, a virtual worker HV, and a virtual object OV which is a virtual object may be communicated. The communication unit 220 may receive an image captured by the camera 380 of each worker terminal 300 and related data thereof (for example, data indicating an imaging position, an imaging time, and an imaging direction).

For example, the communication unit 220 receives the voice information and the image obtained by each worker terminal 300 from each worker terminal 300. Then, the communication unit 220 transmits the received voice information and image to each worker terminal 300 other than the worker terminal 300 that has transmitted the voice information and image. Thereby, each worker terminal 300 can share the voice information and the image obtained by each worker terminal 300, and for example, each worker HM participating in the remote dialogue can share the voice and the image.

The data received by the communication unit 220 may be stored in the storage unit 230, read from the storage unit 230 in a timely manner, and used for various processes. Further, these data may be acquired via a storage medium or the like without going through the communication unit 220, stored in the storage unit 230, read from the storage unit 230 in a timely manner, and used for various processes.

The storage unit 230 includes a primary storage device (for example, RAM or ROM). The storage unit 230 may include a secondary storage device (for example, HDD or SSD) or a tertiary storage device (for example, an optical disk or SD card). The storage unit 230 may include other storage devices.

The storage unit 230 stores various data, information, programs, and the like. The storage unit 230 may hold and store data acquired from the detection device 100 or each worker terminal 300 by the communication unit 220, for example. The storage unit 230 may hold data generated by the control unit 210 (for example, 3D point cloud data and a 3D model described later).

The storage unit 230 may hold information constituting the company (for example, company organization information described later, worker information regarding the worker HM, virtual room RV information, virtual worker HV information). The organization information may be hierarchical organization information (for example, information including departments and sections), and may be expressed in the form of an organization chart. The worker information may include information such as the name, department name, job title, etc. of the worker HM. The virtual room RV information may include virtual room RVs of various layouts. The layout of the virtual room RV may be prepared for each type of remote dialogue (for example, remote conference, remote interview). As information on the virtual worker HV, virtual worker HVs having various appearances (for example, various body shapes and various clothes) may be prepared. The appearance of the virtual worker HV may be prepared for each worker information of the corresponding worker HM.

The control unit 210 includes a motion processing unit 212, an organization display processing unit 214, a remote dialogue processing unit 215, a work information management unit 216, and a 3D model processing unit 217 as constituent units that realize various functions. The control unit 210 does not have to include the 3D model processing unit 217.

The motion processing unit 212 sequentially acquires captured images from the communication unit 220 or the storage unit 230. The motion processing unit 212 obtains information about the object O reflected in the captured image and the object O by image recognition for each captured image. The object O reflected in the captured image includes the object O existing in the rooms RS and RR in which the captured image is captured. For example, the worker HM, the desk DK, the chair C, and the desk for the worker HM to work. The worker terminal 300 placed in the DK, materials, stationery, etc. may be included. Information about the object includes, for example, the position where the object O exists in the room where the captured image is captured (three-dimensional position in the three-dimensional space or the two-dimensional position on the image plane), the name of the object O, the color of the object O, and the like. It may contain information.

The motion processing unit 212 determines the motion of the object O in the rooms RS and RR based on at least one of the position of the object O and the shape of the object O in each captured image, and generates motion information related to this motion. The motion information may be information indicating how the object O has moved based on the type of the object O (for example, the worker HM), the position of the object O, the shape of the object O, and the like. Further, the motion processing unit 212 determines whether each worker HM is seated or absent from the chair C for each worker HM based on the detection data and the detection-related data, and determines the determination result. It may be included in the movement information. Therefore, the movement information may include information on whether the worker HM is seated or absent from the chair C for the worker HM. In addition, the movement information may include information indicating the movement of at least a part of the body (for example, face, mouth, hands, feet) of the worker HM.

The motion processing unit 212 generates a virtual object OV that reflects the motion of the object O based on the motion information of the object O. For example, the movement processing unit 212 may generate each virtual worker HV that reflects the movement of each worker HM based on the movement information of each worker HM. In this case, when the predetermined worker HM is away from the seat, the movement processing unit 212 corresponds to the worker HM and is away from the virtual chair CV for this virtual worker HV in the virtual room RV. A virtual worker HV in the state may be generated. Further, the motion processing unit 212 corresponds to this worker when a predetermined worker is seated, and the virtual worker in the state of being seated in the virtual chair CV for this virtual worker HV in the virtual room RV. HV may be generated.

The motion processing unit 212 transmits information on the virtual object OV reflecting the motion of the object O to the worker terminal 300 via the communication unit 220. For example, the motion processing unit 212 may transmit the information of each virtual worker HV reflecting the movement of each worker HM to the worker terminal 300 via the communication unit 220.

The organization display processing unit 214 supports the display of information about the organization in the company where each worker HM works. For example, it supports the display of the virtual room RV corresponding to the room RS in the company where an arbitrary organization is arranged. For example, the organization display processing unit 214 provides information on the organization of the company (for example, organization chart SZ) and information on the virtual office RV1 to each worker terminal 300 via the communication unit 220. A template of the virtual office RV1 may be prepared as the information of the virtual office RV1.

The organization display processing unit 214 details the layout and size of the virtual office RV1 (for example, the number and arrangement of virtual chairs) in which the virtual worker HV corresponding to the worker HM belonging to the organization is arranged based on the number of people in the organization. May be decided. The number of organizations may be included in the organization information. The organization display processing unit 214 associates the organization display processing unit 214 with each of the plurality of virtual chair CVs and each of the plurality of virtual worker HVs. This association information may be stored in the storage unit 230. Further, each of the plurality of virtual worker HVs and each of the plurality of worker HMs in the real space are associated with each other. This association information may be stored in the storage unit 230. Therefore, the virtual chair CV in which the virtual worker HV corresponding to the worker HM can be seated is determined.

The organization display processing unit 214 transmits the determined virtual office RV1 information, virtual worker HV information, association information, and the like to each worker terminal 300 via the communication unit 220. The layout and size of the virtual office RV1 can be changed by the operation unit 340 of each worker terminal 300. The virtual office RV1 may be generated as a 2D model or a 3D model based on the captured image. The virtual office RV1 may be generated by CAD (Computer-Aided Design).

The remote dialogue processing unit 215 performs processing related to the remote dialogue attended by at least two workers HM. Remote dialogue includes remote meetings, remote interviews, remote interviews, remote consultations, and the like.

The remote dialogue processing unit 215 detects the start of the remote dialogue. For example, the remote dialogue processing unit 215 stores the storage unit 230 with the start time of the remote dialogue as schedule information, measures the current time, and starts the remote dialogue when the current time becomes the start time of the remote dialogue. May be detected. Further, the remote dialogue processing unit 215 may detect the start of the remote dialogue when, for example, the start information of the remote dialogue is acquired from at least one worker terminal 300. The schedule information of the remote dialogue may include identification information of the attendees of the remote dialogue (for example, the name of the attendee, an e-mail address), the attendees of the remote dialogue, the start time and the end time of the remote dialogue, and the like.

The remote dialogue processing unit 215 detects the end of the remote dialogue. For example, the remote dialogue processing unit 215 stores the storage unit 230 with the end time of the remote dialogue as schedule information, clocks the current time, and terminates the remote dialogue when the current time becomes the end time of the remote dialogue. To detect. Further, the remote dialogue processing unit 215 may detect the end of the remote dialogue when, for example, the end information of the remote dialogue is acquired from at least one worker terminal 300.

The remote dialogue processing unit 215 determines the virtual dialogue room RV2 as the virtual room RV in which the remote dialogue is performed. The remote dialogue processing unit 215 may determine the virtual dialogue room RV2 (for example, virtual conference room RV21, virtual interview room RV22) according to the type of remote dialogue (for example, remote conference, remote interview). The layout of the virtual dialogue room RV2 may be different depending on the type of remote dialogue. As the virtual dialogue room RV2, a template of the virtual dialogue room RV2 may be prepared.

For example, when the remote dialogue is a remote conference, the virtual dialogue room RV2 may have a layout imitating a conference room (for example, a layout in which a virtual desk DKV or a virtual whiteboard is arranged). When the remote dialogue is a remote interview, the virtual dialogue room RV2 may have a layout that imitates the interview room (for example, a layout that facilitates discussions around the virtual desk DKV by the remote interview participants). When the remote dialogue is a remote interview, the virtual dialogue room RV2 may have a layout that imitates the interview room (for example, a layout that does not have a virtual desk DKV and makes it easy to check the whole body of the person to be interviewed). When the remote dialogue is a remote consultation, the virtual dialogue room RV2 may have a layout imitating the consultation room. When the remote dialogue is a remote consultation, a special virtual dialogue room RV2 is not prepared, the room is not changed from the virtual room RV located before the remote consultation, and the remote consultation participant does not move the virtual room RV. May be good.

The remote dialogue processing unit 215 determines the layout and size of the virtual dialogue room RV2 in which the virtual remote dialogue attendee HV3 corresponding to the remote dialogue attendee HM3 is arranged based on the number of remote dialogue attendees HM3 participating in the remote dialogue. Details (eg, the number and placement of virtual chairs) may be determined. The number of remote dialogue attendees HM3 may be included in the schedule information of the remote dialogue, the start information of the remote dialogue, and the like. The organization display processing unit 214 associates each of the plurality of virtual chair CVs with each of the plurality of virtual remote dialogue attendees HV3. This association information may be stored in the storage unit 230. Further, each of the plurality of virtual remote dialogue attendees HV3 and each of the plurality of remote dialogue attendees HM3 in the real space are associated with each other. This association information may be stored in the storage unit 230. Therefore, the virtual chair CV in which the virtual remote dialogue attendee HV3 corresponding to the remote dialogue attendee HM3 can be seated is determined. The layout and size of the virtual dialogue room RV2 can be changed by the operation unit 340 of each worker terminal 300.

The remote dialogue processing unit 215 transmits the determined information of the virtual dialogue room RV2, the information of the virtual remote dialogue attendee HV3, the association information, and the like to each worker terminal 300 via the communication unit 220. The layout and size of the virtual dialogue room RV2 can be changed by the operation unit 340 of each worker terminal 300. The virtual dialogue room RV2 may be generated as a 2D model or a 3D model based on the captured image. The virtual dialogue room RV2 may be generated by CAD.

When the remote dialogue processing unit 215 detects the start of the remote dialogue, the remote dialogue processing unit 215 starts measuring the remote dialogue time. When the remote dialogue processing unit 215 detects the end of the remote dialogue, the remote dialogue processing unit 215 ends the measurement of the remote dialogue time. As a result, the remote dialogue processing unit 215 calculates the remote dialogue time.

The remote dialogue processing unit 215 acquires each voice information picked up by the microphone 360 of each worker terminal 300 during the remote dialogue from the communication unit 220 or the storage unit 230. This voice information may include utterances by the remote dialogue attendee HM3, who is the worker HM attending the remote dialogue. The remote dialogue processing unit 215 generates minutes data summarizing the contents of the remote dialogue based on each acquired voice information. Minutes data may include information such as the title of the remote dialogue, participant information (eg, participant name), implementation date, start time, end time, and the like. The participant name may include the name of the company to which the participant belongs. When generating the minutes data, the remote dialogue processing unit 215 may integrate the acquired voice information to generate the integrated voice information. Then, the remote dialogue processing unit 215 may perform voice recognition of the integrated voice information and convert it into character information (sentence conversion). Then, the remote dialogue processing unit 215 may analyze the character information, summarize the character information, and generate the minutes data. The remote dialogue processing unit 215 transmits the minutes data to each worker terminal 300 corresponding to each remote dialogue attendee HM3 via the communication unit 220. The remote dialogue attendee HM3 may be a remote worker HM1 or an in-house worker HM2.

The remote dialogue processing unit 215 may estimate the emotions of each remote dialogue attendee HM3 during the remote dialogue. The presumed emotions may include any emotions of emotions, or other emotions. The remote dialogue processing unit 215 may analyze each acquired voice information or integrated voice information, and estimate the emotion of each remote dialogue attendee HM3 as the analysis result. Analysis based on voice information may include, for example, voice color analysis. Further, the remote dialogue processing unit 215 may analyze each acquired image and estimate the emotion of each remote dialogue attendee HM3 as the analysis result.

The remote dialogue processing unit 215 may generate minutes data based on the emotions of each remote dialogue attendee HM3. In this case, the remote dialogue processing unit 215 may add, for example, supplementary information indicating the emotions of the attendees (for example, the worker A seemed happy at the time of the remark A) to the minutes data. As a result, the server 200 can improve the effectiveness and value of the minutes data. The supplementary information may include voice information.

Here, the following can be considered as specific examples relating to the generation of minutes data.

The remote dialogue processing unit 215 converts the voice information representing the entire remote dialogue into a summary sentence by natural language processing, and stores it in the storage unit 230 as the minutes data of the remote dialogue. Speech recognition may be performed according to the following procedure. First, the remote dialogue processing unit 215 performs acoustic analysis and creates feature data such as sound intensity, sound frequency decomposition, sound interval, and sound time series based on voice information. The remote dialogue processing unit 215 uses, for example, a DNN-HMM (Deep Neural Network-Hidden Markov Model) as an acoustic model to determine phonemes (minimum units of speech) from feature data. The remote dialogue processing unit 215 adds a pattern matching model and converts the sequence of phonemes into words by pattern matching using an utterance dictionary. The remote dialogue processing unit 215 uses, for example, an RNN (Recurrent Neural Network) as a language model to create a sentence from a sequence of words.

Further, the remote dialogue processing unit 215 may also perform emotion recognition by voice recognition using, for example, an RNN. In this case, the remote dialogue processing unit 215 uses voice information including an utterance in one phrase as input data, and machine-learns the emotions and emotions corresponding to this utterance as the correct answer of the teacher data, thereby performing voice recognition. Emotion recognition may be performed.

Further, the remote dialogue processing unit 215 may use a CNN (Convolutional Neural Network) for natural language processing for creating a summary sentence. For example, the remote dialogue processing unit 215 converts the sentence indicated by the voice information into a One-Hot expression vector in which the presence or absence of the target word is represented by 0 or 1, and uses this One-Hot expression vector as input data of the CNN. , The summary sentence may be created by machine learning using the summary sentence prepared corresponding to the One-Hot expression vector as the teacher data.

In this way, the remote dialogue processing unit 215 may generate the minutes data by performing voice recognition, document conversion, and document summarization by AI.

The remote dialogue processing unit 215 extracts the information of self-introduction by each worker HM based on each acquired voice information or integrated voice information, and the worker information which is information about each worker HM included in the self-introduction. May be generated. For example, the remote dialogue processing unit 215 extracts the self-introduction part (section) included in the voice information by the creation AI (Artificial Intelligence) of the summary sentence, and analyzes the name of each remote dialogue attendee HM3. , This analysis result may be used to detect self-introduction. The information of the preparation AI of the summary sentence may be stored in the storage unit 230. By generating worker information based on self-introduction, the server 200 can obtain information on each remote dialogue attendee HM3 without actually face-to-face business card exchange. , Remote dialogue can proceed smoothly. The remote dialogue processing unit 215 may transmit the worker information to the server 200 via the communication unit 220.

The remote dialogue processing unit 215 may modify the minutes data generated by the above method, for example. The remote dialogue processing unit 215 may acquire, for example, the correction data input by the operation unit 340 of the worker terminal 300 via the communication unit 220. Then, the minutes data may be modified based on the modified data. The modified minutes data may be held in the storage unit 230, or may be transmitted to each worker terminal 300 via the communication unit 220.

The remote dialogue processing unit 215 may store and record each voice information or integrated voice information from each worker terminal 300 in the storage unit 230. That is, the server 200 may record the dialogue content of the remote dialogue.

During the remote dialogue implementation period, the remote dialogue processing unit 215 converts the face or body part of the virtual remote dialogue attendee HV3 located in the virtual dialogue room RV2 corresponding to the remote dialogue attendee HM3 into the virtual remote dialogue attendee HV3. It may be replaced with a real-time facial or body image of the corresponding remote dialogue attendee HM3. The face image or body image of the remote dialogue attendee HM3 may be a face image or body image included in the captured image obtained by imaging the remote dialogue attendee HM3, or 3D based on the captured image including the remote dialogue attendee HM3. It may be a face image or a body image included in the model. This increases the sense of presence during remote dialogue. Further, the remote dialogue processing unit 215 bases the face portion or body portion of the virtual worker HV on the face image or body image obtained by deforming the real-time face image or body image, or the real-time image during the implementation period of the remote dialogue. The 3D model may be a deformed face image or body image. This makes it possible to emphasize privacy protection during remote dialogue. In the deformation process, for example, a predetermined deformation is applied to the object. In the case of a face part, it is replaced with a face image, and in the case of a body part, it is replaced with a body image.

Further, during the remote dialogue implementation period, the remote dialogue processing unit 215 uses the face or body part of the virtual remote dialogue attendee HV3 as a face image or body other than the remote dialogue attendee HM3 corresponding to the virtual remote dialogue attendee HV3. It may be replaced with an image. The face image or body image other than the remote dialogue attendee HM3 may be a face image or body image included in any of the captured images, a face image or body image included in a 3D model based on the captured image, or communication. It may be a face image or a body image acquired from an external device via the unit 220. The face image or body image included in the face image or body image of the 3D model may be a two-dimensional image in which an arbitrary position of the 3D model is viewed from an arbitrary direction. In the case of a face part, it is replaced with a face image, and in the case of a body part, it is replaced with a body image.

The work information management unit 216 manages work information (attendance management) related to the work of each worker HM. The work information includes information such as the working hours worked by the worker HM and the work contents performed by the worker HM during the working hours. The work information management unit 216 recognizes the work content based on the movement information of each worker HM. The business content may include desk work work and various dialogue work by the in-house worker HM2, remote desk work work and various remote dialogue work by the remote worker HM1 and the like.

The work information management unit 216 may calculate the working hours of each worker HM based on the movement information of each worker HM. For example, the work information management unit 216 may calculate the time (seating time) in which each worker HM has been seated based on the seated state and the seated state determined by the movement information. Further, the work information management unit 216 may calculate the working hours of each worker HM regardless of the movement information of each worker HM. In this case, the work information management unit 216 may calculate the seating time at which each worker HM is determined to be seated. Then, the work information management unit 216 may calculate the working hours of each worker HM based on the seating time of each worker HM. In this case, the total sitting time may be simply used as the working time, or the working time may be calculated based on the sitting time and other factors (for example, the work content and work efficiency of each worker HM). The work information management unit 216 may store the work information in the storage unit 230. The work information may have a roster format.

The 3D model processing unit 217 acquires detection-related data from the communication unit 220 or the storage unit 230. The 3D model processing unit 217 may generate 3D point cloud data including the worker HM according to the 3D restoration technique based on the detection-related data. In this case, the 3D model processing unit 217 may generate 3D point cloud data based on a plurality of detected data (for example, captured images). The 3D model processing unit 217 may generate 3D point cloud data based on a plurality of detection data and its additional data. The 3D model processing unit 217 may generate 3D point cloud data in consideration of the imaging position by the camera 140, the imaging direction, the imaging time, and the like. The 3D point cloud data shows a set of three-dimensional coordinates of each position. A known technique may be used for the method of generating the 3D point cloud.

Further, the 3D model processing unit 217 extracts the features of the 3D point cloud data and performs various point cloud processing. Feature extraction of 3D point cloud data may be performed based on, for example, principal component analysis, multi-scale feature extraction method, or other method. The 3D model processing unit 217 estimates the shape indicated by the point cloud, associates the points with each other, recognizes parts having similar features in the point cloud, and registers (aligns) by extracting the features of the 3D point cloud data. , Segmentation (region extraction), object recognition, etc. may be performed.

The 3D model processing unit 217 generates a three-dimensional model (3D model) based on the 3D point cloud data. In the generation of the three-dimensional model, polygon generation, texture mapping, addition of color information, and the like may be performed, and a known technique may be used.

The 3D model processing unit 217 may sequentially acquire detection-related data, sequentially generate 3D point cloud data, and sequentially generate a 3D model. In this case, the server 200 can recognize the 3D point cloud data and the time series change of the 3D model. The 3D model processing unit 217 may generate motion information of the worker HM based on the 3D point cloud data or the 3D model.

The 3D model processing unit 217 uses the detection-related data from the detection device 100 instead of the detection-related data from the detection device 100, and the 3D point based on the image captured by the camera 380 of each worker terminal 300. Group data generation, 3D model generation, various point cloud processing, motion information generation, and the like may be performed.

FIG. 4 is a block diagram showing a configuration example of the worker terminal 300. The worker terminal 300 includes a control unit 310, a communication unit 320, a storage unit 330, an operation unit 340, a display unit 350, a microphone 360, a speaker 370, and a camera 380.

The control unit 310 realizes various functions by executing, for example, a program stored in the storage unit 330, including a processor. The processor may include an MPU, a CPU, a DSP, and the like. The control unit 310 comprehensively controls each unit of the worker terminal 300 and performs various processes. The control unit 310 cooperates with the server 200, for example, to generate or process an image displayed on the display unit 350. For example, when the operation unit 340 receives a click operation or a drag operation on the virtual object OV displayed in the virtual space area of the display unit 350, the control unit 310 executes the process associated with this operation. Good.

The communication unit 320 communicates various data and information. The communication method by the communication unit 320 may be a communication method of wired communication or wireless communication, and may include communication methods such as WAN, LAN, power line communication, infrared communication, and cellular communication for mobile phones. The communication unit 320 receives, for example, information on the virtual room RV, information on the virtual worker HV, motion information on the worker HM, detailed information on an object arranged in the virtual room, worker information, and the like from the server 200. ..

The data received by the communication unit 320 may be stored in the storage unit 330, read from the storage unit 330 in a timely manner, and used for various processes. Further, these data may be acquired via a storage medium or the like without going through the communication unit 320, stored in the storage unit 330, read from the storage unit 330 in a timely manner, and used for various processes.

The storage unit 330 includes a primary storage device (for example, RAM or ROM). The storage unit 330 may include a secondary storage device (for example, HDD or SSD) or a tertiary storage device (for example, an optical disk or SD card). The storage unit 330 may include other storage devices.

The operation unit 340 may include a keyboard, a mouse, keys, buttons, a touch panel, a touch pad, and the like. The operation unit 340 accepts various operations. The various operations may include a touch operation, a drag operation, a left-click operation, a right-click operation, and the like. For example, when the operation unit 340 accepts a right-click operation at the display position of the virtual worker HV located in the virtual room RV, the control unit 310 can request a remote dialogue. That is, instead of requesting a dialogue by hitting the shoulder of the worker HM in the real space, the worker terminal 300 can request a remote dialogue by operating in the virtual space.

The display unit 350 may include a liquid crystal display device, an organic EL device, or other display device. The display unit 350 displays various data and information. For example, the display unit 350 displays a virtual room RV in which the room RS in the real space is reproduced in the virtual space. The display of the virtual room RV may be a real-time display or a non-real-time display of the virtual room RV. For example, the display unit 350 displays detailed information about any virtual object OV arranged in the virtual room RV. The display unit 350 displays information (normal work information) for the worker terminal 300 to perform normal work. The normal work information may include, for example, information such as a desktop screen of a PC or tablet as a worker terminal 300, a home screen, a screen displayed during execution of various applications of the worker terminal 300, and the like.

The display unit 350 may have a plurality of display areas. The plurality of display areas include a virtual space area D1 for displaying information on the virtual space, a specific area D2 for displaying detailed information of an object, a normal work area D3 for displaying normal work information, and the like. In the virtual space area D1, information constituting the company such as an organization chart of the company, various virtual room RVs, and the like may be displayed. In the specific area D2, for example, the contents of the virtual material OV1 as the virtual object OV (for example, detailed information of the document held in the electronic document file) and the real space room RS and RR corresponding to the virtual object OV exist. Information about the object O to be used (for example, the name of the object, other information about the object) and the like are displayed. By displaying the detailed information of the virtual object OV in the specific area D2, the display unit 350 can secure a wider display space than displaying at the position of the virtual object OV in the virtual space area D1, for example, a virtual material as a virtual object OV. The contents of OV1 can be enlarged and displayed. Further, in the specific area D2, as detailed information of the virtual object OV, an image (for example, an enlarged image) in which the object O in the real space corresponding to the virtual object OV is captured may be simply displayed.

The microphone 360 collects sounds around the worker terminal 300 and generates voice information. The sound around the worker terminal 300 includes, for example, the voice generated by the worker HM located around the microphone 360 during a remote dialogue. The communication unit 320 transmits the voice information obtained by the microphone 360 to the server 200.

The speaker 370 outputs various sounds. For example, the speaker 370 acquires and outputs the voice and the like spoken by each worker HM during the remote dialogue via the communication unit 220.

The camera 380 images the periphery of the worker terminal 300. The imaging range of the camera 380 may include a worker HM using the worker terminal 300. The image captured by the camera 380 may be held in the storage unit 330, or may be transmitted to the server 200 by the communication unit 320. The image captured by the camera 380 may be treated in the same manner as the image captured by the detection device 100, and is used, for example, to generate motion information of each worker HM, or corresponds to a room RS or a worker HM. It may be used to generate a 3D model.

The worker terminal 300 may or may not include a camera 380. When each worker terminal 300 includes a camera 380, the worker terminal 300 may have the function of the detection device 100 without the detection device 100.

The control unit 310 includes an organization display processing unit 314, a remote dialogue processing unit 315, and a display control unit 317 as constituent units that realize various functions.

The organization display processing unit 314 processes the display of information about the organization in the company where each worker HM works. For example, the organization display processing unit 314 causes the display unit 350 to display the organization chart. The organization display processing unit 314 designates one or more organizations from a plurality of organizations in the company via the operation unit 340, and displays the virtual office RV1 corresponding to the room RS in which the designated organization is arranged. When there are a plurality of virtual offices RV1 to be displayed, the organization display processing unit 314 may sequentially display any one of the virtual offices RV1 or display a plurality of virtual offices RV1 to be displayed at the same time. May be good.

The organization display processing unit 314 acquires the information of the virtual office RV1, the information of the virtual worker HV, the association information, and the like via the communication unit 220. The organization display processing unit 314 arranges a virtual chair CV in the virtual office RV1 and displays it. The associated virtual worker HV can be seated in the virtual chair CV.

The remote dialogue processing unit 315 performs processing related to the remote dialogue.

The remote dialogue processing unit 315 detects the start of the remote dialogue. For example, the remote dialogue processing unit 315 stores the start time of the remote dialogue as schedule information in the storage unit 330, clocks the current time, and starts the remote dialogue when the current time becomes the start time of the remote dialogue. Is detected. Further, the remote dialogue processing unit 315 may detect the start of the remote dialogue by receiving the operation to start the remote dialogue via the operation unit 340. In these cases, the remote dialogue processing unit 315 transmits the start information of the remote dialogue indicating that the remote dialogue has started to the server 200 via the communication unit 320. The remote dialogue start information may include information such as the type of remote dialogue (eg, remote meeting, remote interview), start time, dialogue theme (eg, agenda), and the like.

As a specific example, it is assumed that the worker A has a remote dialogue with the worker B. In this case, the remote dialogue processing unit 315 of the worker terminal 300 of the worker A refers to the virtual worker HV corresponding to the worker B who is located in the virtual room RV and desires remote dialogue via the operation unit 340. Receive a click operation and specify a remote dialogue from various processes. Based on the designation of the remote dialogue, the request for the remote dialogue is notified to the worker terminal 300 of the worker B via the communication unit 320 of the worker terminal 300 and the communication unit 220 of the server 200. In the worker terminal 300 of the worker B, the remote dialogue processing unit 315 receives a request for the remote dialogue, and the start time of the remote dialogue (for example, from now on, XX hours YY minutes (XX, YY)) is received via the operation unit 340. Is an arbitrary number) or after tomorrow. ”The start time of the remote dialogue is set to the worker terminal 300 of the worker A via the communication unit 320 of the worker terminal 300 and the communication unit 220 of the server 200. The notified start time may be stored in the storage unit 330 as the schedule information of the remote dialogue. At the notified remote dialogue start time, the remote dialogue processing unit 315 starts the remote dialogue. To detect.

The remote dialogue processing unit 315 detects the end of the remote dialogue. For example, the remote dialogue processing unit 315 stores the storage unit 330 with the end time of the remote dialogue as schedule information, clocks the current time, and ends the remote dialogue when the current time becomes the end time of the remote dialogue. Is detected. Further, the remote dialogue processing unit 315 may detect the end of the remote dialogue by receiving the end operation of the remote dialogue via the operation unit 340. In these cases, the remote dialogue processing unit 315 transmits the end information of the remote dialogue indicating that the remote dialogue has ended to the server 200 via the communication unit 320.

The remote dialogue processing unit 315 acquires the information of the virtual dialogue room RV2, the information of the virtual remote dialogue attendee HV3, the association information, and the like according to the type of the remote dialogue via the communication unit 220. The remote dialogue processing unit 315 arranges and displays the virtual chair CV in the virtual dialogue room RV2. The associated virtual remote dialogue attendee HV3 can be seated in the virtual chair CV.

When the remote dialogue processing unit 315 detects the start of the remote dialogue, the remote dialogue processing unit 315 may be in a state in which voice input by the microphone 360 is possible and a state in which voice output by the speaker 370 is possible. That is, even if the voice input for the remote dialogue is not set to be possible via the operation unit 340, the voice output for the remote dialogue is set to be possible via the operation unit 340. Even without it, voice input and voice output can be performed automatically based on the start of remote dialogue. In this case, the worker terminal 300 can reduce the time and effort of the remote dialogue attendee HM3, and can prevent the microphone 360 and the speaker 370 from being disabled due to forgetting to set them. Making the voice input possible includes turning on the power of the microphone 360, enabling the voice input function of the microphone 360, raising the signal input level by the microphone 360, and the like. Making the audio output possible includes turning on the power of the speaker 370, enabling the audio output function of the speaker 370, raising the signal output level by the speaker 370, and the like.

When the remote dialogue processing unit 315 detects the end of the remote dialogue, the remote dialogue processing unit 315 may be in a state in which voice input by the microphone 360 is disabled and a state in which voice output by the speaker 370 is disabled. That is, even if the voice input for the remote dialogue is not set to be disabled via the operation unit 340, the voice output for the remote dialogue is set to be disabled via the operation unit 340. Without it, voice input and voice output can be automatically disabled based on the end of the remote dialogue. In this case, the worker terminal 300 can reduce the time and effort of the remote dialogue attendee HM3, and can prevent the microphone 360 and the speaker 370 from being disabled due to forgetting to set them. Disabling voice input may include turning off the power of the microphone 360, disabling the voice input function of the microphone 360, lowering the signal input level by the microphone 360, and the like. Disabling the audio output may include turning off the power of the speaker 370, disabling the audio output function of the speaker 370, lowering the signal output level by the speaker 370, and the like.

The remote dialogue processing unit 315 may acquire worker information from the communication unit 320 or the storage unit 330 and display it on the display unit 350 in association with the corresponding virtual remote dialogue attendee HV3. In this case, the remote dialogue processing unit 315 may display the virtual remote dialogue attendee HV3 with a name tag on which worker information is described.

When the remote consultation processing unit 316 detects the start of remote consultation as a remote dialogue, the remote consultation processing unit 316 causes the display unit 350 to display a chat area for chatting with the remote consultation partner. The display of the chat area is performed on the worker terminals 300 of both the remote consultation source and the remote consultation destination. As a result, the worker terminal 300 of the remote consultation source can chat with the worker terminal 300 of the remote consultation partner.

The remote consultation may be started even during another remote dialogue (for example, a remote meeting, a remote interview, or a remote interview). In this case, the remote dialogue processing unit 315 may request remote consultation as a remote dialogue as described above by accepting an operation for any virtual remote dialogue attendee HV3 in the virtual dialogue room RV2, for example.

When the remote dialogue is a remote consultation, the remote consultation processing unit 316 may at least display the chat area so that the chat can be performed, and other processing may be omitted. For example, the processes of time management for remote consultation, ON / OFF of the microphone 360 and the speaker 370, and creation of minutes data may be omitted.

The display control unit 317 controls the display of data and information on the display unit 350. For example, the display control unit 317 controls which display area on the display unit 350 to display according to the contents of data and information. The display control unit 317 may control the virtual room RV, the virtual worker HV, the virtual object OV, etc., which are examples of the information in the virtual space, to be displayed in the virtual space area D1. The display control unit 317 may control to display the detailed information of the designated virtual object OV in the specific area D2. The display control unit 317 may control the data and information related to the normal work to be displayed in the normal work area D3.

The display control unit 317 corresponds to each worker HM and displays each virtual worker HV that reflects the movement of each worker HM. That is, in the display of the virtual room RV, the display control unit 317 reflects the movement of each worker HM located in each room RS, RR in the real space on each virtual worker HV located in the corresponding virtual room RV. To display. For example, when the predetermined worker HM is seated, the display control unit 317 causes the defined virtual chair CV to display that the virtual worker HV corresponding to the predetermined worker HM is seated. Further, when the predetermined worker HM is away from the seat, the display control unit 317 displays from the designated virtual chair CV that the virtual worker HV corresponding to the predetermined worker HM is away from the seat. .. Further, for example, when at least a part of the body of the predetermined worker HM (for example, face, mouth, hands, feet) is moved, the display control unit 317 is the body of the virtual worker HV corresponding to the predetermined worker HM. The movement is reflected and displayed so that the same part of is moved. Further, when the predetermined worker HM is suspended from work, the display control unit 317 may display that the virtual worker HV corresponding to the predetermined worker HM is away from the desk.

Further, the display control unit 317 may display identifiable whether the worker HM corresponding to the virtual worker HV to be displayed is the remote worker HM1 or the in-house worker HM2. In this case, the display control unit 317 displays the virtual worker HV based on whether the worker HM corresponding to the virtual worker HV to be displayed is the remote worker HM1 or the in-house worker HM2. May be decided. That is, the display control unit 317 may display the virtual worker HV corresponding to the remote worker HM1 and the virtual worker HV corresponding to the in-house worker HM2 in different display modes. Different display modes mean that the display patterns of the virtual worker HV (for example, lighting, blinking, and blinking patterns) are different, and that the line patterns for drawing the virtual worker HV (for example, solid line, dotted line, and broken line) are different. It may include that the transmittance when displaying the virtual worker HV is different, and the like. Further, the display control unit 317 uses the remote worker HM1 or the in-house worker HM2 for the virtual worker HV so that it can be identified whether it is the remote worker HM1 or the in-house worker HM2. It may be displayed with a balloon stating whether or not it exists.

The display control unit 317 displays the virtual object OV arranged in the virtual room RV. The virtual object OV to be displayed may indicate, for example, a physically existing object O arranged in the rooms RS and RR in the real space corresponding to the virtual room RV, the worker terminal 300, and the work. It may be an object corresponding to data held in an external server capable of communicating with the personal terminal 300. When the display control unit 317 specifies the virtual object OV arranged in the virtual room RV via the operation unit 340, the display control unit 317 causes the display unit 350 to display detailed information about the virtual object OV. Detailed information about the virtual object OV is displayed in the specific area D2.

FIG. 5A is a diagram showing a display example of the virtual space area D1 when the worker HM is seated. FIG. 5B is a diagram showing a display example of the virtual space area D1 when the worker HM is away from his / her seat. In FIGS. 5A and 5B, the remote worker HM1 (for example, a teleworker) is working remotely (for example, working from home) in a remote place (for example, a room at home).

In FIG. 5A, a chair C is seated in a remote room RR, and remote desk work is performed using a worker terminal 300 arranged on the desk DK. In this case, since the movement of the remote worker HM1 is reflected, the display unit 350 uses the virtual worker HV1 corresponding to the remote worker HM1 for the virtual worker HV1 in the virtual room RV (for example, the virtual office RV1). It is displayed while seated in the virtual chair CV of. Further, in the virtual desk DKV, a virtual terminal 300V corresponding to the worker terminal 300 is arranged.

In FIG. 5B, the seat is away from the chair C in the remote room RR. In this case, since the movement of the remote worker HM1 is reflected in the display unit 350, the virtual worker HV1 corresponding to the remote worker HM1 is virtualized in the virtual room RV (for example, the virtual office room RV1) in the virtual room RV. It is displayed in a state of being away from the virtual chair CV for the worker HV1. FIG. 5B illustrates a state in which the virtual worker HV1 is absent in the virtual room RV, but a state in which the virtual worker HV1 is located in the virtual room RV but is not seated in the virtual chair CV is displayed. May be good.

In addition, in FIG. 5A and FIG. 5B, the movement of at least a part of the body of the remote worker HM1 may be reflected and displayed by the movement of the body of the virtual worker HV corresponding to the remote worker HM1.

FIG. 6 is a sequence diagram showing an operation example related to the generation of motion information of the worker HM by the remote work support system 5.

In the detection device 100, the camera 140 sequentially images the worker HM located in the imaging range of the camera 140 (S11). The communication unit 120 sequentially transmits the captured image of the worker HM to the server 200 (S12). The processing of S11 and S12 is performed for each detection device 100.

In the server 200, the communication unit 220 sequentially receives images of each worker HM from each detection device 100 (S21). The motion processing unit 212 analyzes the motion of each worker HM based on the image of each worker HM, and generates motion information (S22). The generated motion information may be stored in the storage unit 230. The movement information is reflected in the display of the virtual worker HV in the virtual room RV and the work information.

7A and 7B are sequence diagrams showing an operation example related to the display of the virtual office RV1 corresponding to the organization by the remote work support system 5. The processing of the worker terminal 300 is carried out for each worker terminal 300.

In the worker terminal 300, the organization display processing unit 314 sets the organization display mode when the organization display mode is selected via the operation unit 340 (S31). The organization display mode is a mode for displaying the state of the organization of the company in the virtual space, and for example, the state in the virtual office RV1 corresponding to the organization is shown. The organization display processing unit 314 requests the server 200 for information on the organization chart SZ of the company in which the worker HM works via the communication unit 320 (S32).

In the server 200, the organization display processing unit 214 receives the request information of the organization chart SZ from at least one worker terminal 300 via the communication unit 220 (S41). The organization display processing unit 214 acquires the information of the organization diagram SZ from, for example, the storage unit 230, and transmits the information of the organization diagram SZ to the worker terminal 300 that has transmitted the request information of the organization diagram SZ (S41).

In the worker terminal 300, the organization display processing unit 314 receives the information of the organization chart SZ from the server 200 via the communication unit 320 (S33). The organization display processing unit 314 displays the information of the organization chart SZ in one of the display areas (for example, the virtual space area D1) via the display unit 350 (S33). The worker HM selects a desired organization while looking at the display on the display unit 350. The organization display processing unit 314 accepts an organization selection operation via the operation unit 340, and designates one or more arbitrary organizations to which the virtual office RV1 is displayed (S34). The organization display processing unit 314 requests information on the virtual office room RV1 corresponding to the room RS (office room) to which the designated organization belongs (S35), for example, via the communication unit 320.

In the server 200, the organization display processing unit 214 receives the request information of the virtual office RV1 from at least one worker terminal 300 via the communication unit 220 (S42). The organization display processing unit 214 transmits the information of the virtual office RV1 to the worker terminal 300 that has transmitted the request information of the virtual office RV1 via the communication unit 220 (S42).

In the worker terminal 300, the organization display processing unit 314 receives the information of the virtual office RV1 from the server 200 via the communication unit 320 (S36). The organization display processing unit 314 displays the information of the virtual office RV1 in the virtual space area D1 via the display unit 350 (S36). The organization display processing unit 314 identifies each worker HM belonging to the organization designated in S34 by referring to the information in the organization chart SZ, for example (S37). For example, the information in the organization chart SZ may include identification information of each worker HM belonging to each organization. The organization display processing unit 314 requests the information of each virtual worker HV corresponding to each identified worker HM via the communication unit 220 (S38).

In the server 200, the organization display processing unit 214 receives the request information of the information of each virtual worker HV from at least one worker terminal 300 via the communication unit 220 (S43). The organization display processing unit 214 transmits the information of each virtual worker HV reflecting the movement of each corresponding worker HM requested to the worker terminal 300 that has transmitted the request information via the communication unit 220. (S43).

In the worker terminal 300, the organization display processing unit 314 receives the information of the virtual worker HV reflecting the movement of each worker HM identified in S37 from the server 200 via the communication unit 320 (S39). ). The organization display processing unit 314 displays each virtual worker HV reflecting the movement of each worker HM in the virtual office RV1 in the virtual space area D1 via the display unit 350 (S39).

The display control unit 317 acquires the designated information that specifies the virtual object OV arranged in the virtual office RV1 via the operation unit 340 (S51). The virtual object OV may be an object O existing in the room RS or room RR in the real space corresponding to the virtual room RV, or an object indicating arbitrary data or a file held in the worker terminal 300. The display control unit 317 acquires the detailed information of the designated virtual object OV (S52). The detailed information of the virtual object OV may be held in the storage unit 330, for example, or may be acquired from an external device (for example, a server 200 or another device) via the communication unit 320. The display control unit 317 displays the acquired detailed information of the virtual object OV in the specific area D2 of the display unit 350 (S52).

In addition, in FIG. 7A and FIG. 7B, it was illustrated that the processing of S51 to S52 is performed last, but the present invention is not limited to this. For example, the processes of S51 to S52 may be performed in parallel with the other processes of FIGS. 7A and 7B.

According to the display of the virtual office RV1 corresponding to such an organization, the remote work support system 5 can perform virtual work in the same virtual office RV1 regardless of whether it is a remote worker HM1 or an in-house worker HM2. The person HV can be displayed as if he / she is working, and the state of work can be visualized with a sense of reality.

FIG. 8 is a diagram showing a display example including the virtual office RV1 corresponding to the organization. In FIG. 8, it is illustrated that two organizations are designated from the organizations shown in the organization chart SZ. The virtual space area D1, the specific area D2, and the normal work area D3 are displayed on the display unit 350. In the virtual space area D1, the organization diagram SZ, the virtual office RV11 corresponding to the designated first organization, and the virtual office RV12 corresponding to the designated second organization are displayed. In FIG. 8, the virtual worker HV may or may not be seated in the virtual chairs CV arranged in the virtual offices RV11 and RV12. Whether or not the user is seated in the virtual chair CV corresponds to whether or not each worker HM belonging to this organization is seated in each room RS or RR in the real space.

Detailed information about the virtual object OV is displayed in the specific area D2. In FIG. 8, as the virtual object OV whose detailed information is displayed in the specific area D2, for example, the virtual worker HV0 is designated via the operation unit 340. In this case, detailed information about the virtual worker HV0 is displayed in the specific area D2. This detailed information includes, for example, an employee number, a name, a department name, a work status (for example, while seated, absent, desk work, or a meeting) that identifies a worker HM0 corresponding to a virtual worker HV0, and during in-house work. Information on whether or not they are working remotely (working from home) may be included.

As an example, the desktop screen of the worker terminal 300 displaying the normal work area D3 is displayed in the normal work area D3. Icons corresponding to various applications are displayed on the desktop screen.

9A and 9B are sequence diagrams showing an operation example at the start of a remote dialogue by the remote work support system 5. The processing of the worker terminal 300 is carried out for each worker terminal 300.

In the worker terminal 300, the remote dialogue processing unit 315 detects the start of the remote dialogue (S61). The remote dialogue processing unit 315 transmits the start information of the remote dialogue to the server 200 via the communication unit 320 (S62). The remote dialogue start information may include remote dialogue identification information, remote dialogue type information, remote dialogue start time information, and the like.

In the server 200, the remote dialogue processing unit 215 acquires the start information of the remote dialogue from at least one worker terminal 300 via the communication unit 220, and detects the start of the remote dialogue (S71). The remote dialogue processing unit 215 obtains information from the virtual dialogue room RV2 (for example, a remote conference room, a remote interview room, a remote interview room) from, for example, the storage unit 230, based on the type of the remote dialogue included in the remote dialogue start information. get. Further, the remote dialogue processing unit 215 acquires the information of each remote dialogue attendee HM3 who attends the remote dialogue from, for example, the storage unit 230, based on the identification information of the remote dialogue. The remote dialogue processing unit 215 reflects the information of the virtual dialogue room RV2 and the movement of each remote dialogue attendee HM3 on each worker terminal 300 corresponding to each remote dialogue attendee HM3 via the communication unit 220. Information on each virtual remote dialogue attendee HV3 is transmitted (S71). Further, the remote dialogue processing unit 215 starts measuring the remote dialogue time for which the remote dialogue identified by the identification information continues (S72).

In the worker terminal 300, the remote dialogue processing unit 315 acquires the information of the virtual dialogue room RV2 and the information of each virtual remote dialogue attendee HV3 via the communication unit 320 (S63). Further, the remote dialogue processing unit 315 associates each virtual chair CV arranged in the virtual dialogue room RV2 with each virtual remote dialogue attendee HV3 corresponding to each remote dialogue attendee HM3 via the communication unit 320. Get information and. The remote dialogue processing unit 315 arranges and displays the virtual remote dialogue attendee HV3 in each virtual chair CV in the virtual dialogue room RV2 based on the association information (S64). In this case, the remote dialogue processing unit 315 may display the virtual dialogue room RV2 in the virtual space area D1 at the same time as the start of the remote dialogue, and display the remote dialogue attendees HM3 so as to automatically gather. Further, the remote dialogue processing unit 315 displays each virtual remote dialogue attendee HV3 reflecting the movement of each remote dialogue attendee HM3 (S65).

The remote dialogue processing unit 315 is in a state where voice input by the microphone 360 is possible (microphone: on) and is in a state where voice output by the speaker 370 is possible (speaker: on) (S66). The remote dialogue processing unit 315 sequentially transmits the voice information input to the microphone 360 to the server 200 via the communication unit 320 (S67). Further, the remote dialogue processing unit 315 may sequentially transmit the images captured by the camera 380 to the server 200 via the communication unit 320. This image can be the original data for generating the motion information of each remote dialogue attendee HM3 by the server 200.

In the server 200, the remote dialogue processing unit 215 sequentially receives at least the voice information transmitted in S67 via the communication unit 220 (S73). The remote dialogue processing unit 215 transmits voice information to each worker terminal 300 corresponding to each of the other remote dialogue attendees HM3 via the communication unit 220 (S73). As a result, the voice emitted by each remote dialogue attendee HM3 can be shared by each worker terminal 300 corresponding to each remote dialogue attendee HM3.

Further, in the worker terminal 300, the display control unit 317 acquires the designated information that specifies the virtual material OV1 as the virtual object OV arranged in the virtual dialogue room RV2 via the operation unit 340 (S68). The virtual material OV1 may be a material existing in the dialogue room in the real space corresponding to the virtual dialogue room RV2, or an object indicating arbitrary data or a file held in the worker terminal 300. The display control unit 317 acquires the detailed information of the designated virtual material OV1. This detailed information may be, for example, data in the contents of an electronic file. The detailed information of the virtual material may be stored in the storage unit 330, for example, or may be acquired from an external device (for example, a server 200 or another device) via the communication unit 320. The display control unit 317 displays the detailed information of the acquired virtual material in the specific area D2 of the display unit 350 (S69).

In addition, in FIG. 9A and FIG. 9B, it was illustrated that the processing of S68 to S69 is performed as the final processing by the worker terminal 300, but the present invention is not limited to this. For example, the processes of S68 to S69 may be performed in parallel with the other processes of FIGS. 9A and 9B.

According to the operation at the time of starting such a remote dialogue, in the remote work support system 5, when the remote dialogue is started, the remote dialogue attendee HM3 is seated at his / her own seat, and the virtual remote dialogue attendee HV3 is virtual dialogue. Meet in room RV2. Then, regardless of whether the remote dialogue attendee HM3 is the remote worker HM1 or the in-house worker HM2, it can be displayed as if the virtual remote dialogue attendee HV3 is working in the same virtual dialogue room RV2. You can visualize the state of work with.

FIG. 10 is a sequence diagram showing an operation example at the end of the remote dialogue by the remote work support system 5. The processing of the worker terminal 300 is carried out for each worker terminal 300.

In the worker terminal 300, the remote dialogue processing unit 315 detects the end of the remote dialogue (S81). The remote dialogue processing unit 315 transmits the end information of the remote dialogue to the server 200 via the communication unit 320 (S82). The remote dialogue end information may include remote dialogue identification information, remote dialogue end time information, and the like.

In the server 200, the remote dialogue processing unit 215 acquires the end information of the remote dialogue from at least one worker terminal 300 via the communication unit 220, and detects the end of the remote dialogue. The remote dialogue processing unit 215 acquires the end time of the remote dialogue included in the end information of the remote dialogue, and calculates the time when the remote dialogue is executed (remote dialogue time) based on the start time and the end time of the remote dialogue. (S91). The remote dialogue processing unit 215 generates minutes data based on the voice information acquired from each worker terminal 300 corresponding to each remote dialogue attendee HM3 via the communication unit 220 (S92). The remote dialogue processing unit 215 causes the storage unit 230 to hold the generated minutes data (S93). The remote dialogue processing unit 215 transmits the minutes data to each worker terminal 300 corresponding to each remote dialogue attendee HM3 via the communication unit 320, and holds the minutes data in the storage unit 330 of each worker terminal 300. You may.

According to the operation at the end of such a remote dialogue, the remote work support system 5 can store the results of the remote dialogue based on the end of the remote dialogue. The results of the remote dialogue may be stored in the storage unit 230, for example, and may include information such as minutes data indicating the contents of the remote dialogue, the remote dialogue time, and the like. The minutes data may be transmitted to each worker terminal 300 via the communication unit 220 and shared by each remote dialogue attendee HM3.

FIG. 11A is a diagram showing a display example showing the state of the remote conference.

In FIG. 11A, the virtual conference room RV21 as the virtual dialogue room RV2 is displayed in the virtual space area D1. In the virtual conference room RV21, virtual remote conference attendees HV31, HV32, and HV33 corresponding to the remote conference attendees HM31, HM32, and HM33 who are attending the remote conference are displayed. In FIG. 11A, the virtual conference room RV21 has a virtual whiteboard and a display installed inside the virtual conference room RV21, and has a layout that makes it easy to present and explain using a white boat and a display. The sense of presence is increasing. In the scene of FIG. 11A, the virtual remote conference attendee HV31 explains using the common material OV11 as the virtual material OV1, and the virtual remote conference attendees HV32 and HV33 listen to the explanation by the virtual remote conference attendee HV31. ing. In addition, personal material OV12 as each virtual material OV1 is arranged at hand of the virtual remote conference attendees HV32 and HV33.

The detailed information of the personal material OV12 can be displayed only on the worker terminal 300 corresponding to the worker HM that manages this material, and can be displayed, and the other worker terminal 300 corresponding to the other worker HM can be displayed. Is not allowed to be displayed and cannot be displayed. Therefore, the worker terminal 300 corresponding to the virtual remote conference attendee HV32 can display the detailed information of the personal document OV12 managed by the worker terminal 300 and in the hand of the virtual remote conference attendee HV32, and can display the detailed information of the virtual remote conference attendee HV32. The worker terminal 300 corresponding to the attendees HV31 and HV33 cannot display the detailed information of the personal document OV12 that is not managed by the worker terminal 300. Further, the worker terminal 300 corresponding to the virtual remote conference attendee HV33 can display the detailed information of the personal document OV12 managed by the worker terminal 300 and in the hand of the virtual remote conference attendee HV33, and can display the detailed information of the virtual remote conference attendee HV33. The worker terminal 300 corresponding to the attendees HV31 and HV32 cannot display the detailed information of the personal document OV12 that is not managed by the worker terminal 300.

On the other hand, the common material OV11 can be displayed by each worker terminal 300 corresponding to each virtual remote conference attendee HV31. Therefore, the virtual remote conference attendees HV31 to HV33 can be confirmed by displaying the common material OV11 on each worker terminal 300 corresponding to the virtual remote conference attendees HV31 to HV33.

FIG. 11B is a diagram showing an example of the virtual material OV1 displayed in the specific area D2. Here, the common material OV11 is displayed in the specific area D2. For example, in each worker terminal 300, the remote dialogue processing unit 315 may display the common material OV11 in the specific area D2 of the display unit 350 by designating the common material OV11 via the operation unit 340. In each worker terminal 300, the remote dialogue processing unit 315 may display the personal material OV12 in the specific area D2 of the display unit 350 by designating the personal material OV12 via the operation unit 340.

Further, the operation unit 340 of the worker terminal 300 drags and drops (that is, drags and drops) the common material OV11 or the personal material OV12 from the virtual space area D1 to the specific area D2, or copies and pastes (that is, copies). & Paste) is accepted. In this case, the remote dialogue processing unit 315 may display the detailed information of the common material OV11 or the personal material OV12 in the specific area D2.

Further, it is assumed that the operation unit 340 of the worker terminal 300 accepts an operation of dragging and dropping the common material OV11 or the personal material OV12 from the normal work area D3 to the specific area D2, or an operation of copying and pasting. In this case, the remote dialogue processing unit 315 may display the detailed information of the common material OV11 or the personal material OV12 in the specific area D2. In this case, each worker terminal 300 can easily display the detailed information of the electronic file held by the own terminal that is not arranged in the virtual room RV.

Further, the remote dialogue processing unit 315 of the worker terminal 300 may associate the electronic file held by the own terminal with the virtual material OV1. For example, it is assumed that the operation unit 340 accepts an operation of dragging and dropping an arbitrary electronic file displayed in the normal work area D3 to the virtual material OV1 displayed in the virtual space area D1, or an operation of copying and pasting. .. In this case, the remote dialogue processing unit 315 may associate this electronic file with the virtual material OV1. As a result, the worker terminal 300 can perform various processes (for example, display of detailed information) on the associated electronic file by performing an operation on the virtual material OV1.

Further, the remote dialogue processing unit 315 may change the page of the virtual material OV1 displayed in the specific area D2. For example, the page of the virtual material OV1 may be changed by operating the page of the virtual material OV1 displayed in the specific area D2 via the operation unit 340. Here, the remote dialogue processing unit 315 returns to the page before the change by receiving an operation of designating (for example, clicking) the virtual material OV1 arranged in the virtual room RV of the virtual space area D1 via the operation unit 340. You can (return). The operation of changing the page or returning the page may include, for example, a drag operation while continuously rotating the mouse wheel or left-clicking the mouse.

Further, the remote dialogue processing unit 315 may change the page of the common material OV11. For example, the remote dialogue processing unit 315 may change the page of the virtual material OV1 by receiving a page turning operation for the virtual material OV1 displayed in the virtual space area D1 via the operation unit 340. In this case, the remote dialogue processing unit 315 may transmit the page change information to the other worker terminal 300 via the communication unit 320 and the communication unit 220 of the server 200. As a result, the page change of the common material OV11 is shared among the worker terminals 300 corresponding to each remote dialogue attendee HM3. On the other hand, for example, the remote dialogue processing unit 315 may change the page of the virtual material OV1 by receiving a page turning operation for the virtual material OV1 displayed in the specific area D2 via the operation unit 340. In this case, the remote dialogue processing unit 315 does not share the page change of the virtual material OV1 among the worker terminals 300 corresponding to the remote dialogue attendee HM3. As a result, the worker terminal 300 can change the page of the virtual material OV1 only in the worker terminal 300. The page turning operation may be an operation of moving the mouse in the direction in which the page is to be turned in the state of being clicked with the mouse, an operation of moving the finger in the direction in which the page is to be turned in the state of being touched with the finger, and the like. ..

The remote dialogue processing unit 315 may associate the electronic file with the virtual material OV1, change the page of the virtual material OV1, change the page of the common material OV11, and the like by the organization display processing unit 314. In this case, the same processing may be performed not only for the virtual material OV1 and the common material OV11 arranged in the virtual dialogue room RV2, but also for the virtual material OV1 and the common material OV11 arranged in the virtual office RV1. ..

The remote meeting may include the remote worker HM1 participating in the meeting in the real space by the in-house worker HM2 via the communication line. In this case, the remote dialogue processing unit 315 reads the contents of the whiteboard written by the in-house worker HM2 in the real space via a scanner (not shown) or the like, and displays the details of the whiteboard in the specific area D2. May be good.

FIG. 12 is a diagram showing a display example showing the state of the remote interview. In FIG. 12, the virtual interview room RV22 as the virtual dialogue room RV2 is displayed in the virtual space area D1. In the virtual conference room RV21, virtual remote interview attendees HV36, HV37, HV38, and HV39 corresponding to the remote interview attendees HM36, HM37, HM38, and HM39 who are attending the remote interview are displayed. In FIG. 12, the virtual interview room RV22 has a layout in which the virtual remote interview attendees HV36 to HV39 face each other, so that the presence of the interview is increased. In the remote interview, the virtual material OV1 does not have to be shared. Therefore, the common material OV11 may not be displayed in the specific area D2.

FIG. 13 is a diagram showing a display example in which the virtual worker HV1 corresponding to the remote worker HM1 and the virtual worker HV2 corresponding to the in-house worker HM2 are shown in different display modes. In FIG. 13, a scene similar to that in FIG. 12 is assumed, but the display mode is different.

In FIG. 13, it is assumed that the remote interview attendees HM36 and HM39 are in-house workers HM2. It is assumed that the remote interview attendees HM37 and HM38 are remote workers HM1. In this case, the virtual remote interview attendees HV36 and HV39 corresponding to the remote interview attendees HM36 and HM39 are displayed in a display mode (for example, solid line) indicating the virtual worker HV2 corresponding to the in-house worker HM2. Further, the virtual remote interview attendees HV37 and HV38 corresponding to the remote interview attendees HM37 and HM38 are displayed in a display mode (for example, a broken line) indicating the virtual worker HV1 corresponding to the remote worker HM1.

In this way, the remote work support system 5 displays the virtual worker HV in different display modes for the remote worker HM1 and the in-house worker HM2, so that each worker HM maintains a high sense of presence. However, it can help other worker HMs identify whether they are working remotely or in-house.

In this way, the remote work support system 5 reproduces the room RS in the real space, which is a three-dimensional space, as the virtual room RV, and displays it in the virtual space area D1. Then, the virtual worker HV in the virtual space area D1 and the virtual object OV can be specified via the operation unit 340 as the UI. As a result, the worker terminal 300 can perform processing related to remote dialogue with the worker HM corresponding to the designated virtual worker HV, and processing related to display of detailed information of the designated virtual object OV. In addition, by faithfully reproducing the movement of the corresponding worker HM to the movement of the virtual worker HV, the worker HM can work in the company even if the worker HM is working remotely. You can feel the feeling. Further, as compared with the real space, the remote work support system 5 can easily move within the company by changing the virtual room RV according to a predetermined operation in the virtual space, and necessary work (for example, remote dialogue). Can be easily carried out. Further, the worker HM can easily recognize how the other worker HM is working even if he / she is the remote worker HM1.

In addition, the remote work support system 5 creates an easy-to-use and realistic virtual work environment for important work items (for example, desk work, meetings, interviews, interviews, consultations, attendance) in performing remote work. By reflecting the work in the space, it is possible to realize remote work that is easy to use and has a sense of presence. Therefore, the worker HM can carry out the work while knowing the work situation of the other worker HM. For example, a worker HM can smoothly carry out a remote meeting with a sense of presence. For example, the remote work support system 5 can immediately leave the minutes data of the remote dialogue. For example, a worker HM can effectively consult with another worker HM. For example, a worker HM can remotely and smoothly communicate with another worker HM without having to make a business trip.

Further, when the image displayed in the virtual space area D1 is an image based on a three-dimensional model (for example, a cross-sectional image), each worker terminal 300 moves, rotates, enlarges, reduces, etc. via the operation unit 340. By receiving the operation, it can be displayed in a desired state. Further, the remote work support system 5 can pick up the utterance of the remote worker HM1 by the microphone 360, and can link the mouth and facial expression of the remote worker HM1 with the mouth and facial expression of the corresponding virtual worker HV. it can. Further, the remote work support system 5 can collect the sound emitted by the remote worker HM1 or the sound generated around the remote worker HM1 by the microphone 360, and by using this sound as the sound of the virtual space area D1. Other worker HM can also be heard.

Further, the remote work support system 5 can obtain, for example, the following effects by the motion detection function of each worker HM, the image processing function, and the work detection function of each worker HM. For example, the remote work support system 5 reflects a real-time image of the face or body of each worker HM or an image of the face or body designated via the operation unit 340 on the face or body of each virtual worker HV. it can. Further, the remote work support system 5 can detect the movements of the head, body, limbs, and fingers of each worker HM by image analysis, and can reflect the movements in the corresponding virtual worker HV in real time. Further, the remote work support system 5 can display the corresponding virtual worker HV1 so as to disappear from the virtual room RV when the remote worker HM1 does not perform work, that is, when he / she is away from his / her seat. Further, the remote work support system 5 can calculate the working hours of the remote worker HM1 based on the image analysis, and can automatically input the start time and the end time of the work in the attendance table data.

As described above, the remote work support system 5 of the present embodiment supports remote work. The remote work support system 5 includes a server 200, one or more worker terminals 300, and one or more detection devices 100 that detect the state of one or more worker HM including the remote work HM1. .. The server 200 acquires the detection information (for example, detection data) detected by the detection device 100, and based on the detection information, generates each virtual worker HV corresponding to each worker HM and reflecting the movement of each worker HM. It is generated and the information of each virtual worker HV is transmitted to each worker terminal 300. Each worker terminal 300 acquires the information of each virtual worker HV from the server 200, acquires the information of the virtual room RV corresponding to the room RS in the company where each worker HM works, and is arranged in the virtual room RV. The information of the virtual object OV is acquired. Then, each worker terminal 300 has at least one of the virtual room RV, each virtual worker HV, and the virtual object OV in the first display area (for example, virtual) of the display unit 350 included in each worker terminal 300. When the operation unit 340 included in each worker terminal 300 detects an operation on the virtual worker HV or the virtual object OV displayed in the space area D1) and displayed in the first display area, the information corresponding to this operation is detected. Is displayed in the second display area (for example, the specific area D2) of the display unit 350.

As a result, each worker HM has a sense of presence as if he / she is working in the company even during remote work, and can intuitively recognize the state of each other worker HM. Further, the remote work support system 5 can display as if working in a virtual room RV that virtually shows the room RS in the company even if the remote worker HM1 exists. Therefore, each worker HM can easily intuitively understand the working state of the remote worker HM1. Further, each worker HM specifies a virtual object OV or a virtual worker HV existing in the virtual space via the operation unit 340, so that the information corresponding to this operation (for example, designated) is displayed at the same time as displaying the virtual space. (Detailed information of the virtual object OV and the virtual worker HV) can be displayed and confirmed in a second display area different from the first display area for displaying the virtual space. Therefore, the remote work support system 5 is easy to use and present by generating a digital work environment with a sense of presence for important work items for remote work and reflecting the digital work environment in the real space. It is possible to realize support for remote work with a feeling.

Further, the detection device 100 may include one or more cameras 140 (an example of the image pickup device) for capturing an image including the worker HM. The server 200 may sequentially acquire images as detection information from the camera 140. As a result, the remote work support system 5 can grasp the movement of each worker HM by using the time-series captured images.

Further, each worker terminal 300 acquires the information of the hierarchical organization to which each worker HM belongs, displays the information of the organization in the first display area, and among the hierarchical organizations via the operation unit 340. In the virtual room RV (for example, virtual office RV1) where any organization is specified and the designated organization is placed, each worker HM belonging to the organization is assigned to each worker HM based on the detection information of each worker HM belonging to the organization. Each corresponding virtual worker HV may be displayed.

As a result, the remote work support system 5 specifies a predetermined organization (for example, an internal department or section) from the entire organization information (for example, the entire company), and creates an office space (for example, an office) corresponding to that organization. Since it can be virtually reproduced, each worker HM who confirms the display can grasp the state of each room RS in which each organization is located in the virtual space without actually moving around the company, for example. In addition, since the movement of the worker HM belonging to the organization is also reflected and displayed, each worker HM who confirms the display can specifically confirm the state of each worker HM.

In addition, each worker HM may include a first worker. The server 200 may determine whether or not the first worker is away from the desk based on the detection information. The server 200 corresponds to the first worker when the first worker is away from the first virtual seat (for example, the virtual chair CV) for the first virtual worker in the virtual room RV. A first virtual worker who is away may be generated. When the first worker is seated, the server 200 corresponds to the first worker and is seated in the first virtual seat for the first virtual worker in the virtual room RV. You may generate a virtual worker for. The server 200 may transmit the information of the first virtual worker to each worker terminal 300.
Each worker terminal 300 may acquire the information of the first virtual worker from the server 200 and display the first virtual worker in the first display area.

As a result, the remote work support system 5 can recognize the movement of the first worker based on the obtained detection information, and can determine whether the first worker is absent or seated, and the first worker can be assigned. It can be reflected in the state of the corresponding first virtual worker. Therefore, each worker HM who has confirmed the display can intuitively grasp the state of the first worker corresponding to the first virtual worker.

The server 200 may calculate the seating time determined that the first worker is seated, and calculate the working time of the first worker based on the seating time of the first worker. As a result, the remote work support system 5 can automatically manage the attendance of each worker HM based on the movement of each worker HM (for example, whether he / she is seated or absent).

Further, each worker terminal 300 may display each virtual worker HV corresponding to each worker HM in a different display mode depending on whether or not each worker HM is a remote worker HM1.

As a result, each worker terminal 300 can visually recognize which worker HM is located inside the company and which worker HM is located outside the company, while giving a sense of presence of the company. Therefore, each worker HM can easily respond to the other worker HM according to the location of the other worker HM.

Further, the virtual object OV may include a virtual material OV1 that can be viewed by the worker HM. When the virtual material OV1 is designated via the operation unit 340, each worker terminal 300 may display the contents of the virtual material OV1 in the second display area. As a result, each worker HM can intuitively select the virtual material OV1 desired to be viewed and display the details, and can grasp the contents of the virtual material OV1.

Further, the virtual material OV1 may include a personal material OV12 that can be individually viewed by each worker HM and a common material OV11 that can be shared and viewed by each worker HM. Each worker terminal 300 may display the contents of the personal material OV12 or the common material OV11 designated via the operation unit 340 in the second display area. As a result, each worker HM can confirm the details of the personal material OV12 or the common material OV11 separately from the state in the virtual room RV.

Further, each worker terminal 300 detects the start of a remote attended by a plurality of worker HMs, and based on the detection of the start of the remote dialogue, the virtual dialogue room RV2 and the virtual dialogue room RV2 for performing the remote dialogue as the virtual room RV. , Each virtual remote dialogue attendee HV3 (an example of a virtual worker) corresponding to each remote dialogue attendee HM3 (an example of an attendee) attending a remote dialogue may be displayed in the first display area. As a result, the remote work support system 5 can display that each remote dialogue attendee HM3 moves to the virtual dialogue room RV2 with the start of the remote dialogue, can reproduce the scene of the dialogue, and enhance the sense of presence. ..

Further, each worker terminal 300 detects the start of a remote dialogue by an operation performed on the virtual worker HV displayed in the first display area via the operation unit 340, and has a type according to the operation. You may start a remote dialogue. As a result, the worker terminal 300 can start a specific remote dialogue (for example, a remote conference or a remote interview) by the operation of each worker.

Further, when the remote dialogue is a remote consultation, each worker terminal 300 may display a chat area for chatting with the remote consultation partner based on the detection of the start of the remote dialogue. When the remote dialogue is a remote dialogue other than the remote consultation, each worker terminal 300 may display the virtual dialogue room RV2 having a layout according to the type of the remote dialogue in the first display area. For example, the virtual conference room RV21 may have a layout in which a virtual desk DKV for a meeting and a virtual material OV1 are arranged. The virtual interview room RV22 has a format that emphasizes dialogue by the remote dialogue attendee HM3, and may have a layout in which the remote dialogue attendee HM3 surrounds the virtual desk DKV. The virtual interview room may be laid out so that the virtual desk DKV is not arranged so that the whole body can be easily seen.

As a result, the remote work support system 5 can secretly chat only between the requester and the requestee of the remote consultation. Therefore, the contents of the remote consultation are not known to the worker HM other than the remote consultation attendees, and the remote dialogue consultation attendees can consult with peace of mind. Further, the remote work support system 5 can easily change the virtual object OV and layout existing in the virtual dialogue room RV2 according to the type of remote dialogue such as remote conference, remote interview, and remote interview. Therefore, the remote work support system 5 can support each remote dialogue attendee HM3 to intuitively obtain necessary information in the virtual space as well as the dialogue in the real space.

Further, each worker terminal 300 corresponding to the remote dialogue attendee HM3 is provided with a microphone 360 (an example of a voice input unit) and a speaker 370 (an example of a voice output unit), and is based on the detection of the start of the remote dialogue. Therefore, the state may be such that the sound input by the microphone 360 is possible and the sound output by the speaker 370 is possible.

As a result, the remote work support system 5 cannot input / output voice due to an erroneous operation of switching on / off of the microphone 360 and the speaker 370 by the remote dialogue attendee HM3 or forgetting to switch, even though the remote dialogue is started. Can be suppressed.

Further, each worker terminal 300 corresponding to the remote dialogue attendee HM3 may transmit the voice information of the voice input by the microphone 360 to the server 200. The server 200 may acquire each voice information from each worker terminal 300, recognize each voice information by voice, convert it into each character information, summarize each character information, and generate minutes data.

As a result, the remote work support system 5 can easily generate the minutes data based on the remarks made by the remote dialogue attendee HM3 without the remote dialogue attendee HM3 performing any special operation. The minutes data can be generated by, for example, AI (artificial intelligence).

Further, the server 200 analyzes each voice information from each worker terminal 300, estimates the emotion of each remote dialogue attendee HM3, and generates minutes data based on the emotion of each remote dialogue attendee HM3. You can.

Thereby, the remote work support system 5 can estimate the emotion of the remote dialogue attendee HM3 based on the voice analysis result (for example, voice color) without using the image. For example, supplementary information indicating the emotions of the remote dialogue attendee HM3 (worker A seemed happy at the time of remark A, etc.) can be added to the minutes data, and the effectiveness of the minutes data can be improved.

Further, the detection device 100 may include one or more cameras 140 that capture an image including the remote dialogue attendee HM3. The server 200 acquires each image from each camera 140, analyzes each image, estimates the emotion of each remote dialogue attendee HM3, and generates minutes data based on the emotion of each remote dialogue attendee HM3. It's okay.

Thereby, the remote work support system 5 can estimate the emotion of the remote dialogue attendee HM3 based on the image analysis result (for example, the facial expression and the gesture of the remote dialogue attendee HM3 who is speaking). For example, supplementary information indicating the emotions of the remote dialogue attendee HM3 (worker A seemed happy at the time of remark A, etc.) can be added to the minutes data, and the effectiveness of the minutes data can be improved.

Further, each worker terminal 300 corresponding to the remote dialogue attendee HM3 may transmit the voice information of the voice input by the microphone 360 to the server 200. The server 200 acquires each voice information from each worker terminal 300, detects the self-introduction section during the remote dialogue, analyzes each voice information in the self-introduction section, and names each remote dialogue attendee HM3. It may be determined and the information of the name of each remote dialogue attendee HM3 may be transmitted to each worker terminal 300 corresponding to each remote dialogue attendee HM3. Each worker terminal 300 corresponding to each remote dialogue attendee HM3 acquires the information of the name of each remote dialogue attendee HM3 from the server 200, and becomes each virtual remote dialogue attendee HV3 corresponding to each remote dialogue attendee HM3. Correspondingly, information on the name of each remote dialogue attendee HM3 may be displayed.

As a result, in the remote work support system 5, the worker information (for example, name) of the remote dialogue attendee HM3 corresponding to each displayed virtual remote dialogue attendee HV3 is performed without the remote dialogue attendee HM3 performing any special operation. Can be displayed. Therefore, each remote dialogue attendee HM3 who has confirmed the display can easily grasp the name of the remote dialogue attendee HM3 who is participating in the remote dialogue, and can smoothly proceed with the remote dialogue.

Further, each worker terminal 300 is a remote generated based on an image captured by the camera 140 or the camera 380 of the face or body of the virtual remote dialogue attendee HV3 corresponding to the remote dialogue attendee HM3 during the remote dialogue. The virtual worker may be displayed by replacing the face or body of the dialogue attendee HM3 with a deformed face image or body image.

Thereby, for example, when the predetermined remote dialogue attendee HM3 does not want to show the image of the real face or body to another remote dialogue attendee HM3, the image of the face or body of the predetermined remote dialogue attendee HM3 is displayed. By substituting a deformed face or body image, a given remote dialogue attendee HM3 can rest assured. Therefore, the remote work support system 5 does not want to be aware of the demands of the predetermined remote dialogue attendee HM3 (for example, no need to straighten hair, no need to shave, no need to put on makeup, and a depressed facial expression. ) Can be satisfied.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modifications or modifications within the scope of the claims, which naturally belong to the technical scope of the present disclosure. Understood.

In the above embodiment, the remote work support system 5 includes the detection device 100 as an independent device, but the present invention is not limited to this. For example, the function of the detection device 100 may be provided in the server 200 or each worker terminal 300.

In the above embodiment, the remote work support system 5 includes the server 200 as an independent device, but the present invention is not limited to this. For example, each worker terminal 300 may have the function of the server 200.

In the above embodiment, a plurality of candidates for the virtual room RV to be displayed may be prepared. A plurality of candidates for the virtual room RV may be stored in the storage unit 230 as a plurality of templates. In this case, a plurality of candidates for the virtual room RV are transmitted to each worker terminal 300 via the communication units 220 and 320, and each worker terminal 300 selects the operation unit 340 from among the plurality of candidates for the virtual room RV. One desired virtual room RV may be specified via. Designating one candidate from a plurality of candidates for the virtual room RV is applicable to the selection of both the virtual office room RV1 and the virtual dialogue room RV2. A plurality of candidates for the virtual room RV may be prepared for each type of remote dialogue.

In the above embodiment, the various sensors included in the detection device 100 may include a sensor for measuring the physical condition (for example, a state of heat generation) of the worker HM such as thermography. In this case, in each worker terminal 300, the control unit 310 may display the image of the virtual worker HV on the display unit 350 by reflecting the state of the physical condition of the worker HM.

In the above embodiment, it has been explained that the movement information of each worker HM is sequentially reflected in the movement of each virtual worker HV, but the present invention is not limited to this. For example, the control unit 210 of the server 200 or the control unit 310 of each worker terminal 300 may be able to specify whether or not the movement information of each worker HM is reflected in the movement of each virtual worker HV. Further, it is not essential that the server 200 generates motion information of each worker HM. For example, the control unit 210 of the server 200 need only recognize the motion and perform various processes based on the recognized motion without generating motion information.

In the above embodiment, the virtual worker HV is exemplified as a 3D model (avatar) imitating the corresponding worker HM in the real space, but the present invention is not limited to this. For example, at least a part of the image of the worker HM captured by the camera 140 may be pasted on at least a part of the 3D model of the virtual worker HV. At least a part of the 3D model of the virtual worker HV and at least a part of the image of the worker HM are corresponding parts. For example, the face part or body part of the worker HM in the image may be attached to the face part or body part of the 3D model of the virtual worker HV. Even in this case, the movement of the worker HM is reflected in the movement of the virtual worker HV.

In the above embodiment, it is illustrated that the virtual object OV is a 3D model that imitates the corresponding object O in the real space, but the present invention is not limited to this. For example, at least a part of the image of the object O captured by the camera 140 may be attached to at least a part of the 3D model of the virtual object OV. At least a part of the 3D model of the virtual object OV and at least a part of the image of the object O are corresponding parts.

This disclosure is useful for a remote work support system, a remote work support method, etc., which have a sense of presence as if working in-house even during remote work and can intuitively recognize the state of each worker.

5 Remote work support system 100 Detection device 110 Control unit 120 Communication unit 130 Storage unit 140 Camera 160 Position acquisition unit 170 Direction acquisition unit 200 Server 210 Control unit 212 Motion processing unit 214 Organization display processing unit 215 Remote dialogue processing unit 220 Communication unit 230 Storage unit 310 Control unit 311 Data acquisition unit 314 Organization display processing unit 315 Remote dialogue processing unit 317 Display control unit 320 Communication unit 330 Storage unit 340 Operation unit 350 Display unit 360 Microphone 370 Speaker 380 Camera D1 Virtual space area D2 Specific area D3 Normal Work area C Chair CV Virtual chair DK Desk DKV Virtual desk HM Worker HM1 Remote worker HM2 In-house worker HV, HV1, Virtual worker HV3 Virtual remote dialogue attendee O Object OV Virtual object OV1 Virtual material OV11 Common material OV12 Personal material RR, RS Room RV Virtual Room RV1, RV11, RV12 Virtual Office RV2 Virtual Dialogue Room RV21 Virtual Conference Room RV22 Virtual Interview Room

Claims (18)

It is a remote work support system that supports remote work,
It comprises a server, one or more worker terminals, and one or more detection devices that detect the status of one or more workers, including remote workers.
The server
Acquire the detection information detected by the detection device,
Based on the detection information, each virtual worker corresponding to each worker and reflecting the movement of each worker is generated.
The information of each virtual worker is transmitted to each worker terminal, and the information is transmitted to each worker terminal.
Each worker terminal is
The information of each virtual worker is acquired from the server, and the information is obtained from the server.
Obtain the information of the virtual room corresponding to the room in the company where each worker works,
Acquire information on virtual objects placed in the virtual room,
At least one of the virtual room, each virtual worker, and the virtual object is displayed in the first display area of the display unit included in each worker terminal.
When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. Display in the display area ,
Each worker terminal is
Obtain information on the hierarchical organization to which each worker belongs,
The information of the tissue is displayed in the first display area, and the information is displayed.
Any organization among the hierarchical organizations is designated via the operation unit, and the organization is designated.
In the virtual room where the designated organization is arranged, each virtual worker corresponding to each worker belonging to the organization is displayed based on the detection information of each worker belonging to the organization.
Remote work support system. The detection device includes one or more image pickup devices that capture an image including the worker.
The server sequentially acquires the image as the detection information from the imaging device.
The remote work support system according to claim 1. Each of the above workers includes a first worker
The server
Based on the detection information, it is determined whether or not the first worker is away from the desk.
When the first worker is away, the first virtual seat corresponding to the first worker and away from the first virtual seat for the first virtual worker in the virtual room. Generate a virtual worker and
When the first worker is seated, the first virtual seat corresponding to the first worker and seated in the first virtual seat for the first virtual worker in the virtual room. Generate virtual workers in
The information of the first virtual worker is transmitted to each worker terminal, and the information is transmitted to each worker terminal.
Each worker terminal is
The information of the first virtual worker is acquired from the server, and the information is obtained from the server.
Displaying the first virtual worker in the first display area,
The remote work support system according to claim 1 or 2. The server
The seating time determined that the first worker is seated is calculated, and the seating time is calculated.
The working hours of the first worker are calculated based on the seating time of the first worker.
The remote work support system according to claim 3. Each worker terminal is
Each virtual worker corresponding to each worker is displayed in a different display mode depending on whether or not each worker is a remote worker.
The remote work support system according to any one of claims 1 to 4. The virtual object includes virtual materials that can be viewed by the worker.
When the virtual material is designated via the operation unit, each worker terminal displays the contents of the virtual material in the second display area.
The remote work support system according to any one of claims 1 to 5. The virtual material includes personal materials that can be individually viewed by each worker and common materials that can be shared and viewed by each worker.
Each worker terminal displays the contents of the personal material or the common material designated via the operation unit in the second display area.
The remote work support system according to claim 6. It is a remote work support system that supports remote work,
It comprises a server, one or more worker terminals, and one or more detection devices that detect the status of one or more workers, including remote workers.
The server
Acquire the detection information detected by the detection device,
Based on the detection information, each virtual worker corresponding to each worker and reflecting the movement of each worker is generated.
The information of each virtual worker is transmitted to each worker terminal, and the information is transmitted to each worker terminal.
Each worker terminal is
The information of each virtual worker is acquired from the server, and the information is obtained from the server.
Obtain the information of the virtual room corresponding to the room in the company where each worker works,
Acquire information on virtual objects placed in the virtual room,
At least one of the virtual room, each virtual worker, and the virtual object is displayed in the first display area of the display unit included in each worker terminal.
When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. Display in the display area,
Each worker terminal is
Detects the start of a remote dialogue attended by multiple said workers and
Based on the detection of the start of the remote dialogue, the virtual dialogue room for performing the remote dialogue as the virtual room and the virtual workers corresponding to each attendee attending the remote dialogue are designated as the first. Display in the display area of 1,
Remote work support system. Each worker terminal is
The start of the remote dialogue is detected by the operation performed on the virtual worker displayed in the first display area via the operation unit, and the remote dialogue of the type corresponding to the operation is started.
The remote work support system according to claim 8. Each worker terminal is
When the remote dialogue is a remote consultation, a chat area for chatting with the remote consultation partner is displayed based on the detection of the start of the remote dialogue.
When the remote dialogue is a remote dialogue other than the remote consultation, the virtual dialogue room having a layout according to the type of the remote dialogue is displayed in the first display area.
The remote work support system according to claim 8 or 9. Each worker terminal corresponding to the attendee is
It is equipped with a voice input unit and a voice output unit.
Based on the detection of the start of the remote dialogue, the voice input unit is capable of voice input, and the voice output unit is capable of voice output.
The remote work support system according to any one of claims 8 to 10. Each worker terminal corresponding to the attendee is
Equipped with a voice input section
The voice information of the voice input by the voice input unit is transmitted to the server, and the voice information is transmitted to the server.
The server
Acquire each voice information from each worker terminal and
Voice recognition is performed for each of the above voice information and converted into each character information.
The minutes data is generated by summarizing each character information.
The remote work support system according to any one of claims 8 to 11. The server
The emotions of each attendee are estimated by analyzing each voice information from each worker terminal.
Generate the minutes data based on the emotions of each attendee.
The remote work support system according to claim 12. The detection device includes one or more image pickup devices that capture an image including the attendee.
The server
Each of the images is acquired from each of the imaging devices.
Each of the images is analyzed to estimate the emotions of each of the attendees.
Generate the minutes data based on the emotions of each attendee.
The remote work support system according to claim 12. Each worker terminal corresponding to the attendee is
Equipped with a voice input section
The voice information of the voice input by the voice input unit is transmitted to the server, and the voice information is transmitted to the server.
The server
Acquire each voice information from each worker terminal and
Detecting the section of self-introduction during the remote dialogue,
The names of each attendee are determined by analyzing each voice information in the self-introduction section.
Information on the name of each attendee is transmitted to each worker terminal corresponding to each attendee, and the information is transmitted.
Each worker terminal corresponding to each attendee is
The information of the name of each attendee is acquired from the server, and the information is obtained.
The information of the name of each attendee is displayed corresponding to each virtual worker corresponding to each attendee.
The remote work support system according to any one of claims 8 to 14. The detection device includes one or more image pickup devices that capture an image including the attendee.
Each worker terminal is
During the remote dialogue, the face or body of the virtual worker corresponding to the attendee is replaced with a deformed face or body image of the attendee generated based on the image. Show virtual workers,
The remote work support system according to any one of claims 8 to 15. It is a remote work support method that supports remote work.
Detects the status of one or more workers, including remote workers, and generates detection information
Based on the detection information, each virtual worker corresponding to each worker and reflecting the movement of each worker is generated.
Obtain the information of the virtual room corresponding to the room in the company where each worker works,
Acquire information on virtual objects placed in the virtual room,
At least one of the virtual room, the virtual worker, and the virtual object is displayed in the first display area of the display unit included in one or more worker terminals.
When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. Display in the display area ,
Obtain information on the hierarchical organization to which each worker belongs,
The information of the tissue is displayed in the first display area, and the information is displayed.
Any organization among the hierarchical organizations is designated via the operation unit, and the organization is designated.
In the virtual room where the designated organization is arranged, each virtual worker corresponding to each worker belonging to the organization is displayed based on the detection information of each worker belonging to the organization.
Remote work support method. It is a remote work support method that supports remote work.
Detects the status of one or more workers, including remote workers, and generates detection information
Based on the detection information, each virtual worker corresponding to each worker and reflecting the movement of each worker is generated.
Obtain the information of the virtual room corresponding to the room in the company where each worker works,
Acquire information on virtual objects placed in the virtual room,
At least one of the virtual room, the virtual worker, and the virtual object is displayed in the first display area of the display unit included in one or more worker terminals.
When the operation unit included in each worker terminal detects an operation on the virtual worker or the virtual object displayed in the first display area, the information corresponding to the operation is transmitted to the second display unit. Display in the display area,
Detects the start of a remote dialogue attended by multiple said workers and
Based on the detection of the start of the remote dialogue, the virtual dialogue room for performing the remote dialogue as the virtual room and the virtual workers corresponding to each attendee attending the remote dialogue are designated as the first. Display in the display area of 1,
Remote work support method.

Images