JP2018010612A - Virtual reality system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate settlement for a user who uses a VR headset and ensure security of the settlement.SOLUTION: A virtual reality system includes an information processing apparatus, a virtual reality headset, a biometric data acquisition apparatus, and a sensor. The information processing apparatus generates a virtual reality content. The virtual reality headset is worn on a user, to display a virtual reality content received biometric data acquisition apparatus acquires biometric data of the user. The sensor generates location information indicating a position of the biometric data acquisition apparatus and direction information indicating a direction of the biometric data acquisition apparatus, and transmits the location information and the direction information to the information processing apparatus. On receipt of a settlement instruction, the information processing apparatus adds an image to notify the user wearing the virtual reality headset about the location and direction of the biometric data acquisition apparatus, on the basis of the location information and direction information received from the sensor.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a virtual reality system.

  In recent years, systems related to virtual reality (hereinafter referred to as VR) have been actively developed. Generally, when a user experiences VR content, the user wears a goggle-shaped device (hereinafter referred to as a VR headset) on the head. The VR headset is provided with two displays corresponding to the left and right eyes, and each display displays an image with parallax for a user who has viewed the image to obtain a sense of depth (or a feeling of popping out). The

  In addition, the VR headset is equipped with a position sensor, acceleration sensor, gyro sensor, etc., and whenever the position or orientation of the user's head changes, the user's field of view moves in the direction in which the position or orientation of the user's head changes. As shown, the images displayed on the two displays are changed. Thereby, the user can obtain a feeling that the user is actually moving his / her head in the VR.

JP, 2006-506530, A JP2015-118556A Japanese Patent Laid-Open No. 2015-230236

  Usually, a user who experiences VR using a VR headset removes the VR headset and performs an operation for settlement when payment processing is required for purchase of paid content provided in the VR, for example. There is a need. The operation for settlement is, for example, input of a card number such as a credit card, input of a personal identification number, input of personal information, and the like. However, it is troublesome for the user to remove the VR headset from the head each time a payment is made, and to attach the VR headset to the head again after the payment. Further, removal of the VR headset impairs the immersive feeling of the user experiencing VR.

  Further, in a game where a VR headset is used, it is conceivable that one VR headset and a VR machine are shared by a plurality of people such as family members, and in this case, the security of payment can be ensured. desired.

  The present invention has been made in consideration of the above circumstances, and provides a VR system that facilitates settlement for a user who uses a VR headset and ensures the security of the settlement.

  A virtual reality system according to an aspect of the present invention includes an information processing device that generates virtual reality content, and a virtual reality headset that is attached to a user and displays the virtual reality content received from the information processing device to the user. Generating biometric data acquisition device for acquiring biometric data of the user, position information indicating the position of the biometric data acquisition device, and direction information indicating the direction of the biometric data acquisition device, and generating the position information and the direction And a sensor that transmits information to the information processing apparatus. When the information processing apparatus receives a settlement instruction, the information processing apparatus attaches the virtual reality headset to the position and the direction of the biometric data acquisition apparatus based on the position information and the direction information received from the sensor. A content generation unit that adds the first image to be recognized by the user to the virtual reality content.

  According to the present invention, it is possible to facilitate settlement for a user who uses a VR headset and to ensure the safety of settlement.

FIG. 1 is a block diagram illustrating an example of a configuration of a VR system according to the first embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the VR processing apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating an example of the configuration of the VR headset according to the first embodiment. FIG. 4 is a block diagram illustrating an example of the configuration of the controller according to the first embodiment. FIG. 5 is a block diagram illustrating an example of the configuration of the sensor device according to the first embodiment. FIG. 6 is a block diagram illustrating an example of the configuration of the server device according to the first embodiment. FIG. 7 is a block diagram illustrating an example of the configuration of the mobile terminal according to the first embodiment. FIG. 8 is a flowchart illustrating an example of payment processing using the VR system according to the first embodiment. FIG. 9 is a flowchart illustrating the biometric data matching process according to the first embodiment. FIG. 10 is a flowchart illustrating a biometric data registration process according to the first embodiment. FIG. 11 is a diagram illustrating a first example of a VR screen display at the time of biometric data registration or biometric data acquisition according to the first embodiment. FIG. 12 is a diagram illustrating a second example of the VR screen display at the time of biometric data registration or biometric data acquisition according to the first embodiment. FIG. 13 is a diagram illustrating a VR screen display when biometric data is selected according to the first embodiment. FIG. 14 is a diagram illustrating a VR screen display during content sharing according to the first embodiment. FIG. 15 is a diagram illustrating a VR screen display during character display according to the second embodiment. FIG. 16 is a flowchart illustrating an attribute determination process according to the second embodiment. FIG. 17 is a block diagram illustrating the configuration of an automatic diagnosis system according to the third embodiment. FIG. 18 is a flowchart illustrating the process of the automatic diagnosis system according to the third embodiment. FIG. 19 is a diagram illustrating a first example of a screen display in the automatic diagnosis system according to the third embodiment. FIG. 20 is a diagram illustrating a second example of screen display in the automatic diagnosis system according to the third embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, substantially or substantially the same functions and components are denoted by the same reference numerals, and description will be made only when necessary.

[First embodiment]
In the present embodiment, the position and direction of the biometric data acquisition device included in the VR system is detected, and an image that allows the user wearing the VR headset to recognize the position and direction of the biometric data acquisition device at the time of settlement is obtained. A VR system that is displayed on the display of the VR headset and allows the user to input biometric data while wearing the VR headset will be described.

  In this embodiment, the direction of the biometric data acquisition device may be a direction in which the biometric data acquisition device can acquire biometric data with high sensitivity and high accuracy, in other words, a direction in which the biometric data acquisition device performs sensing.

  In the present embodiment, the identification information is referred to as ID.

  In this embodiment, the biometric data is, for example, fingerprint data, vein data, artery data, palm data, retinal data, iris data, face data, blood vessel data, voice data, voiceprint data, ear data, or other data. It may be.

  In the present embodiment, the biological data acquisition device may be, for example, a biological sensor, a camera, a microphone, or the like for acquiring the biological data.

  In this embodiment, the case where the payment performed by the user is a card payment will be described. The card type may be, for example, a credit card, a debit card, a prepaid card, an electronic money card, a point card, a cash card, or another card for performing electronic payment. However, the present embodiment can be similarly applied when the medium or product used for settlement is not a card. The settlement in the present embodiment may be a settlement using virtual currency.

  In the present embodiment, the content includes, for example, image data, video data, software including at least one of a program and data, sound data, and the like. The content may be a game program, for example.

  FIG. 1 is a block diagram showing an example of the configuration of the VR system 1 according to the present embodiment.

  The VR system 1 includes a VR processing device 2, a human body wearing device HD, a sensor device 5, and server devices 6 and 12. The human body wearing device HD includes a VR headset 3 and a controller 4. The VR headset 3 and the controller 4 are devices that the user U wears or directly holds and operates. The VR system 1 can be controlled by the VR headset 3 and the controller 4. The VR system 1 may include the mobile terminal 7 of the user U, or may include the mobile terminal 8 of another user V who is not the user U. The mobile terminals 7 and 8 and the VR processing device 2 may be communicable via the network N, for example, or may be communicable without going through the network N. More specifically, the mobile terminals 7 and 8 and the VR processing device 2 may be communicable via a telephone line or the Internet, for example. For example, the portable terminals 7 and 8 and the VR processing device 2 may be communicable by wireless LAN communication, may be communicable by short-range wireless communication, or may be communicable by infrared communication.

  The VR headset 3 is a device that the user U wears on the head. The VR headset 3 includes a display, and the display displays the VR content generated by the VR processing device 2. The user U can experience VR by visually recognizing the VR content displayed on the display.

  The VR headset 3 includes, for example, a biometric data acquisition device 33 and a sensor unit 32.

  The biometric data acquisition device 33 acquires the biometric data of the user U, for example, in contact with the user U or without contact. More specifically, the biological data acquisition device 33 includes, for example, a biological sensor corresponding to fingerprint data, vein data, artery data, palm data, retinal data, iris data, or voiceprint data.

  The sensor unit 32 generates position information indicating the position of the VR headset 3 and direction information indicating the direction of the VR headset 3, and transmits the position information and direction information of the VR headset 3 to the VR processing device 2. . In addition, the sensor unit 32 corresponds to the biometric data acquisition device 33, generates position information indicating the position of the biometric data acquisition device 33 and direction information indicating the direction of the biometric data acquisition device 33, and generates biometric data. The position information and direction information of the acquisition device 33 may be transmitted to the VR processing device 2.

  The controller 4 is a device for the user U to operate the VR system 1 and receives an instruction from the user U. The controller 4 may be operated by being held in each of the left and right hands of the user U, or may be worn on either the left or right foot or other body part. In the following, it is assumed that the VR system 1 includes two controllers 4 and each controller 4 is operated by being held in the left and right hands of the user U.

  The controller 4 includes a biometric data acquisition device 43 and a sensor unit 42.

  The biometric data acquisition device 43 acquires the biometric data of the user U, for example, in contact with the user U or without contact. More specifically, the biometric data acquisition device 43 includes, for example, a biometric sensor corresponding to fingerprint data, vein data, artery data, palm data, or voiceprint data.

  The sensor unit 42 generates position information indicating the position of the controller 4 and direction information indicating the direction of the controller 4, and transmits the position information and direction information of the controller 4 to the VR processing device 2. In addition, the sensor unit 42 corresponds to the biometric data acquisition device 43, generates position information indicating the position of the biometric data acquisition device 43 and direction information indicating the direction of the biometric data acquisition device 43, and generates biometric data. The position information and direction information of the acquisition device 43 are transmitted to the VR processing device 2.

  The sensor device 5 is a device that assists the construction of the VR experienced by the user U. Specifically, for example, when a plurality of sensor devices 5 are connected to the VR processing device 2, the VR processing device 2 displays the position of the sensor device 5 on the display included in the VR headset 3, thereby U may indicate the range in which the VR headset 3 can be mounted and moved. Further, for example, the VR processing device 2 may measure the movement, posture, position of a specific part of the body, and the like using the sensors provided in the plurality of sensor devices 5. Further, for example, the sensor device 5 may give the user U a sense of reality by making a sound when the user U approaches. In the following, it is assumed that there are four sensor devices 5 included in the VR system 1.

  The sensor device 5 includes a biological data acquisition device 53 and a sensor unit 52.

  The biometric data acquisition device 53 acquires the biometric data of the user U without contacting the user U, for example. More specifically, the biometric data acquisition device 53 includes, for example, a biometric sensor and a camera corresponding to voiceprint data.

  The sensor unit 52 generates position information indicating the position of the sensor device 5 and direction information indicating the direction of the sensor device 5, and transmits the position information and direction information of the sensor device 5 to the VR processing device 2. In addition, the sensor unit 52 corresponds to the biometric data acquisition device 53, generates position information indicating the position of the biometric data acquisition device 53 and direction information indicating the direction of the biometric data acquisition device 53, and generates biometric data. The position information and direction information of the acquisition device 53 are transmitted to the VR processing device 2.

  The mobile terminal 7 can communicate with the VR processing device 2. The mobile terminal 7 may be, for example, a mobile phone such as a smartphone, a tablet terminal, or the like.

  The mobile terminal 7 may be used instead of the controller 4. That is, the user U may be able to operate the VR system 1 using the mobile terminal 7. The mobile terminal 7 includes, for example, a biometric data acquisition device 73 and a sensor unit 72, and may be used to input the biometric data of the user U similarly to the sensor device 5.

  The biometric data acquisition device 73 acquires the biometric data of the user U, for example, in contact with the user U or without contact. More specifically, the biometric data acquisition device 43 includes, for example, a biometric sensor corresponding to fingerprint data, vein data, artery data, palm data, or voiceprint data.

  The sensor unit 72 generates position information indicating the position of the mobile terminal 7 and direction information indicating the direction of the mobile terminal 7, and transmits the position information and direction information of the mobile terminal 7 to the VR processing device 2. In addition, the sensor unit 72 corresponds to the biometric data acquisition device 73, generates position information indicating the position of the biometric data acquisition device 73 and direction information indicating the direction of the biometric data acquisition device 73, and generates biometric data. The position information and direction information of the acquisition device 73 are transmitted to the VR processing device 2.

  The VR processing device 2 controls the entire VR system 1 or performs calculations necessary for the control. The VR processing device 2 can communicate with the VR headset 3, at least one controller 4, and at least one sensor device 5 in a wired or wireless manner.

  The VR processing device 2 generates VR content in response to the instruction received from the controller 4. The VR processing device 2 includes, for example, a biometric data acquisition device 25, a sensor unit 26, a content generation unit 222, a content transmission unit 223, a biometric data transmission unit 224, and a content sharing unit 225.

  The biometric data acquisition device 25 acquires the biometric data of the user U, for example, in contact with the user U or without contact. More specifically, the biometric data acquisition device 25 includes, for example, a biometric sensor corresponding to fingerprint data, vein data, arterial data, palm shape data, or voiceprint data.

  The sensor unit 26 corresponds to the biometric data acquisition device 25 and generates position information indicating the position of the biometric data acquisition device 25 and direction information indicating the direction of the biometric data acquisition device 25. In addition, the sensor unit 26 detects position information indicating the position of the VR headset 3 and direction information indicating the direction of the VR headset 3 by detecting light of a light mounted on the VR headset 3, for example. It may be generated.

  When the content generation unit 222 receives a settlement instruction from the controller 4, the position information and direction information of the biometric data acquisition devices 25, 33, 43, 53, 73 received from the sensor units 26, 32, 42, 52, 72 are received. The VR headset 3 is attached to the positions and directions of the biometric data acquisition devices 25, 33, 43, 53, 73 based on the position information and direction information of the user U received from the sensor units 26, 32, 52. The user U who recognizes the biometric data acquisition device 25, 33, 43, 53, 73, an image that prompts the user U wearing the VR headset 3 to add biometric data to the VR content.

  The content transmission unit 223 transmits the VR content to the VR headset 3.

  After the VR content including the image prompting the user U to input biometric data is displayed by the VR headset 3, the biometric data transmission unit 224 receives the user U from any of the biometric data acquisition devices 25, 33, 43, 53, and 73. The biometric data D1 is received, and the biometric data D1 is transmitted to the server device 6.

  The content sharing unit 225 can display the VR content displayed on the display unit 37 of the VR headset also on the mobile terminal 8 operated by another user V who is not the user U. That is, the content sharing unit 225 authenticates the mobile terminal 8 that can communicate with the VR processing device 2, and when the authentication with respect to the mobile terminal 8 is successful, permits the communication between the VR processing device 2 and the mobile terminal 8, The VR content is converted into content corresponding to the mobile terminal 8, and the converted content is transmitted to the mobile terminal 8. For example, the user V is a friend or family member of the user U. For example, the content transmitted to the mobile terminal 8 may be content obtained by converting VR content into content not corresponding to VR, or content obtained by converting VR content into content suitable for the browser of the mobile terminal 8.

  The mobile terminal 8 includes a display unit 75 and displays content received from the content sharing unit 225.

  The server device 6 is communicably connected to the VR processing device 2 via the network N. The server device 6 can refer to the database DB. The database DB stores biometric data D2 registered by the user U.

  The server device 6 includes a biometric authentication unit 62. The biometric authentication unit 62 collates the biometric data D1 received from the VR processing device 2 via the network N with the registered biometric data D2 stored in the database DB. When the collation is successful, the server device 6 transmits a settlement permission notification indicating that the settlement is permitted to the server device 12 via the network N.

  The server device 12 is communicably connected to the VR processing device 2 and the server device 6 via a network. The server device 12 may be integrated with the server device 6. The server device 12 includes a settlement processing unit 121. When the payment processing unit 121 receives a payment permission notification from the server device 6 via the network N, the payment processing unit 121 executes payment processing for the user U according to the payment instruction received from the VR processing device 2 via the network N.

  In the present embodiment, the server device 12 is, for example, an acquirer (card payment company) server device, an issuer (card issuing company), or a card brand server device.

  In the present embodiment, the content generation unit 222 adds an image or a character indicating that the content is shared to the VR content when the content sharing unit 225 transmits the content to the mobile terminal 8. The content transmission unit 223 transmits VR content including images or characters indicating that the content is shared to the VR headset 3. Thereby, the user U wearing the VR headset 3 can recognize that the user V is sharing the content.

  In addition, the content generation unit 222 determines the difference between the position information of one of the biometric data acquisition devices and the position information of the VR headset 3, the direction information of any one of the biometric data acquisition devices, and the direction information of the VR headset 3. An image that guides the user U to any one of the biometric data acquisition devices may be added to the VR content based on the difference between the VR content and the VR content.

  Further, the content generation unit 222 may determine the attribute of the character corresponding to the user U based on the biometric data D1 acquired by any of the biometric data acquisition devices, and generate VR content according to the attribute. . The content generation unit 222 receives the biological data D2 stored in the database DB via the server device 6 and the network N, determines the attribute of the character corresponding to the user U based on the received biological data D2, VR content may be generated according to the above.

  FIG. 2 is a block diagram illustrating an example of the configuration of the VR processing device 2 according to the present embodiment.

  The VR processing device 2 includes a connection unit 21, a processor 22, a storage unit 23, a communication unit 24, a biological data acquisition device 25, and a sensor unit 26. The VR processing device 2 may not include the biometric data acquisition device 25. The above-described processing units can communicate with each other via an internal bus IB.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The connection unit 21 connects the VR processing device 2 to the VR headset 3, the controller 4, and the sensor device 5.

  The processor 22 performs arithmetic processing of the VR system 1. The processor 22 includes a control unit 221, a content generation unit 222, a content transmission unit 223, a biometric data transmission unit 224, and a content sharing unit 225.

  The processor 22 may be, for example, a central processing unit (CPU), a micro-processing unit (MPU), or a digital signal processor (DSP).

  The control unit 221 controls the entire VR system 1. The control unit 221 transmits and receives commands, addresses, data, information, instructions, signals, and the like to and from the VR headset 3, the controller 4, and the sensor device 5 via the connection unit 21.

  The content generation unit 222 generates VR content to be displayed on the display included in the VR headset 3. The VR content generated by the content generation unit 222 is transmitted to the VR headset 3 by the content transmission unit 223 via the connection unit 21. The VR content may be transmitted to an external device such as the mobile terminals 7 and 8 via the communication unit 24.

  The storage unit 23 may be used as a main storage device, for example. The storage unit 23 follows the control from the processor 22. Data to be processed by the processor 22 is temporarily stored in the storage unit 23 based on the control of the processor 22.

  For example, the storage unit 23 stores the program 231 in a nonvolatile storage area. The processor 22 may implement functions as the control unit 221, the content generation unit 222, the content transmission unit 223, the biometric data transmission unit 224, and the content sharing unit 225 by executing the program 231.

  The storage unit 23 stores biometric data D1. The biometric data D1 is biometric data of the user U acquired using any of the biometric data acquisition devices 25, 33, 43, 53, and 73.

  The communication unit 24 transmits and receives commands, addresses, data, information, instructions, signals, and the like between the VR processing device 2 and the server devices 6 and 12 and the mobile terminals 7 and 8 via the network N, for example. The communication unit 24 may communicate with other devices that are not the server devices 6 and 12 and the mobile terminals 7 and 8 via the network N.

  The VR processing device 2 may be a general-purpose PC (personal computer), for example, or may be hardware dedicated to the VR system 1.

  FIG. 3 is a block diagram showing an example of the configuration of the VR headset 3 according to the present embodiment.

  The VR headset 3 includes a connection unit 31, a sensor unit 32, a biological data acquisition device 33, a control unit 34, a microphone 35, a speaker 36, and a display unit 37.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The connection unit 31 connects the VR headset 3 and the VR processing device 2.

  The sensor unit 32 includes an angular velocity sensor (gyro sensor) S1. The sensor unit 32 may be built in the biometric data acquisition device 33 or may be installed in the vicinity of the biometric data acquisition device 33.

  The angular velocity sensor S1 is a sensor that can detect a change in the rotation angle and orientation of an object including the sensor. In other words, the change in the position and the direction of the VR headset can be detected by the angular velocity sensor S1.

  Note that the sensors included in the sensor unit 32 are not limited to those described above, and may include, for example, a proximity sensor, an acceleration sensor, a position sensor, a magnetic sensor, and a luminance sensor.

  The control unit 34 controls each processing unit included in the VR headset 3. The control unit 34 transmits / receives commands, addresses, data, information, instructions, signals, and the like to / from the VR processing device 2 via the connection unit 31. More specifically, for example, the control unit 34 receives sensor data such as position information and direction information acquired by the sensor unit 32, biological data acquired by the biological data acquisition device 33, and audio data input to the microphone 35. The data is transmitted to the VR processing device 2 via the connection unit 31. For example, the control unit 34 receives audio data from the VR processing device 2 via the connection unit 31 and outputs the audio data to the speaker 36, and receives VR content and outputs it to the display unit 37.

  The microphone 35 inputs the voice of the user U to the VR system 1. When the VR headset 3 can acquire voiceprint biometric data, the microphone 35 may also serve as the biometric data acquisition device 33.

  The speaker 36 outputs the sound generated by the VR processing device 2 or the sound generated by the control unit 34. The speaker 36 may be a headphone, for example.

  The display unit 37 displays the VR content generated by the content generation unit 222 of the VR processing device 2. The display unit 37 includes two displays corresponding to the eyes of the user U. Hereinafter, the display unit 37 in a state where VR content is displayed is referred to as a VR screen.

  FIG. 4 is a block diagram illustrating an example of the configuration of the controller 4 according to the present embodiment.

  The controller 4 includes a connection unit 41, a sensor unit 42, a biological data acquisition device 43, a control unit 44, and an input unit 45.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The connection unit 41 connects the controller 4 and the VR processing device 2.

  The configuration of the sensor unit 42 is the same as that of the sensor unit 32.

  The control unit 44 controls each processing unit included in the controller 4. The control unit 44 transmits and receives commands, addresses, data, information, instructions, signals, and the like to and from the VR processing device 2 via the connection unit 41. More specifically, for example, the control unit 44 receives sensor data such as position information and direction information acquired by the sensor unit 42, biometric data acquired by the biometric data acquisition device 43, and data input to the input unit 45. The data is transmitted to the VR processing device 2 via the connection unit 41.

  The input unit 45 receives an operation of the user U. The input unit 45 may be, for example, a button, an analog stick, a rotary encoder, a touch panel, or various sensors.

  FIG. 5 is a block diagram illustrating an example of the configuration of the sensor device 5 according to the present embodiment.

  The sensor device 5 includes a connection unit 51, a sensor unit 52, a biological data acquisition device 53, a control unit 54, a microphone 55, and a speaker 56. Further, the sensor device 5 may include a camera 57.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The connection unit 51 connects the sensor device 5 and the VR processing device 2.

  The configuration of the sensor unit 52 is the same as that of the sensor unit 32.

  The control unit 54 controls each processing unit included in the sensor device 5. The control unit 54 transmits and receives commands, addresses, data, information, instructions, signals, and the like to and from the VR processing device 2 via the connection unit 51. The operation of the control unit 54 is equivalent to the operation of the control units 34 and 44.

  The microphone 55 inputs the voice of the user U to the VR system 1. Note that when the sensor device 5 can acquire voiceprint biometric data, the microphone 55 may also serve as the biometric data acquisition device 53.

  The speaker 56 outputs the sound generated by the VR processing device 2 or the sound generated by the control unit 34.

  The camera 57 captures a still image or a moving image. The camera 57 may be capable of capturing a two-dimensional image, or may be capable of acquiring depth information such as a stereo camera or an infrared camera. The control unit 54 performs image processing on a still image or a moving image captured by the camera 57, for example, a gesture of the user U that cannot be recognized by the sensor unit 52, or a leg, arm, or joint of the user U It may be possible to recognize the movement of a specific part such as an object or an object existing in a space where the user U can move.

  FIG. 6 is a block diagram illustrating an example of the configuration of the server device 6 according to the present embodiment.

  The server device 6 includes a communication unit 61, a biometric authentication unit 62, and a control unit 63.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The communication unit 61 can be connected to the network N and communicates with the VR processing device 2 via the network N. The communication unit 61 can communicate with the database DB.

  The database DB may be connected to the network N, and may be communicable with the server device 6 via the network N.

  The control unit 63 controls each processing unit included in the server device 6. The control unit 63 transmits / receives commands, addresses, data, information, instructions, signals, and the like between the VR processing device 2 and the database DB via the communication unit 61. For example, when the biometric authentication performed by the biometric authentication unit 62 is successful, the control unit 63 transmits a settlement permission notification to the effect that the settlement is permitted to the server device 12 via the network N.

  FIG. 7 is a block diagram illustrating an example of the configuration of the mobile terminals 7 and 8 according to the present embodiment.

  The portable terminal 7 includes a communication unit 71, a sensor unit 72, a biological data acquisition device 73, a processor 74, and a display unit 75.

  The portable terminal 8 includes a communication unit 71, a sensor unit 72, a processor 74, a display unit 75, and a content sharing unit 76. Note that the mobile terminal 8 may include a biometric data acquisition device 73.

  A description of the processing unit already described with reference to FIG. 1 is omitted.

  The communication unit 71 is connectable to the network N and communicates with the VR processing device 2 via the network N. The communication unit 71 may be capable of directly communicating with the communication unit 24 of the VR processing device 2 by wire or wireless without going through the network N.

  The sensor unit 72 is the same as the sensor unit 32.

  The processor 74 performs calculation processing of the mobile terminal 7. The processor 74 includes a control unit 741.

  The control unit 741 controls each processing unit included in the mobile terminal 7. The control unit 741 transmits and receives commands, addresses, data, information, instructions, signals, and the like to and from the VR processing device 2 via the communication unit 71. The operation of the control unit 741 is equivalent to the operation of the control units 34, 44 and 54.

  The display unit 75 is, for example, a general mobile terminal display.

  Further, as described above, when the VR content is displayed on the display unit 75 of the mobile terminal 8 by the content sharing unit 225, the mobile terminal 8 is preferably authenticated (logged in) to the VR system 1. For login to the VR system 1, for example, a login ID and a password may be used. More specifically, for example, the VR processing device 2 displays a login screen on the display unit 75 or the display unit 37 of the VR headset 3 when the mobile terminal 8 accesses the VR processing device 2. The user U inputs a login ID and a password according to the instruction on the login screen.

  When the mobile terminal 8 includes the biometric data acquisition device 73, biometric authentication may be used for logging in to the VR system 1. In this case, the biometric authentication procedure is the same as the biometric authentication procedure (described later with reference to FIG. 9) performed during the settlement process.

  FIG. 8 is a flowchart illustrating an example of payment processing using the VR system 1 according to this embodiment.

  In step S801, the VR processing device 2 is activated. The VR headset 3, the controller 4, and the sensor device 5 connected to the VR processing device 2 are also turned on, and can be operated by the user U.

  In step S <b> 802, the VR content generated by the content generation unit 222 of the VR processing device 2 is transmitted to the VR headset 3 via the connection unit 21. The display unit 37 of the VR headset 3 displays the VR content received from the VR processing device 2. Further, the content generation unit 222 generates VR content that the user U can select and purchase a product according to the selection of the user U or automatically.

  In step S <b> 803, a product to be purchased by the user U is selected on the product selection screen displayed on the display unit 37. The product may be selected, for example, by the user U operating the controller 4. Further, when the purchase of the selected product is approved, the VR processing device 2 starts a settlement process. The consent to purchase the product may be performed, for example, by the user U selecting a purchase button displayed in the VR content.

  In step S804, the VR processing device 2 starts biometric data collation processing. Details of the biometric data collating process will be described later with reference to FIG.

  In step S805, the VR processing device 2 confirms the biometric data matching processing result obtained by the processing in step S804.

  When the biometric data matching process result is successful, the VR processing device 2 or the server device 6 transmits a settlement permission notification to the server device 12. In this case, in step S806, the server device 12 receives a settlement permission notification from the VR processing device 2, and the settlement processing unit 121 performs a settlement process. When the payment process is completed, the server device 6 transmits a payment completion notification to the VR processing device 2 via the communication unit 61. In step S807, the VR processing device 2 receives the payment completion notification, and the content generation unit 222 generates VR content including a display indicating that the payment is completed.

  If the biometric data matching process result is unsuccessful, the VR processing device 2 stops the settlement process. In this case, in step S808, the content generation unit 222 of the VR processing device 2 generates VR content including a display indicating that payment has failed.

  In addition, the display of the completion of payment or the payment failure may be, for example, a display using a message, an icon, a dialog box, or the like.

  In the present embodiment, the VR processing device 2 may perform authentication (login) of the user U at the time of activation in step S801. Biometric authentication may be used to log in to the VR system 1. In this case, the biometric authentication procedure is the same as the biometric authentication procedure (see FIG. 9) performed during the settlement process.

  FIG. 9 is a flowchart illustrating a biometric data matching process according to this embodiment.

  In step S <b> 901, the VR processing device 2 causes the user to select which biometric data acquisition device to use to acquire biometric data when there are a plurality of biometric data acquisition devices that can be used in the VR system 1. When the user U selects the biometric data acquisition device, the process proceeds to step S902.

  When there is only one biometric data acquisition device in the VR system 1 or when the VR processing device 2 does not allow the user to select a biometric data acquisition device, the process proceeds to step S905.

  In step S902, the VR processing device 2 detects the position information and direction information of the biometric data acquisition device designated by the user U in step S901. Specifically, when the biometric data acquisition device 33 is designated, the VR processing device 2 uses the VR headset 3 and the biometric data acquisition device 33 detected by the sensor unit 32 from the control unit 34 of the VR headset 3. Position information and direction information are received. Similarly, when the biometric data acquisition device 43 is designated, the VR processing device 2 receives the position information and direction information of the controller 4 and the biometric data acquisition device 43 detected by the sensor unit 42 from the control unit 44. Or similarly, when the biometric data acquisition device 53 is designated, the VR processing device 2 receives the position information and direction information of the sensor device 5 and the biometric data acquisition device 53 detected by the sensor unit 52 from the control unit 54. Receive. Or similarly, when the biometric data acquisition device 73 is designated, the VR processing device 2 receives position information and direction information of the mobile terminal 7 and the biometric data acquisition device 73 from the control unit 741. The VR processing device 2 temporarily stores the received position information and direction information in the storage unit 23.

  In step S903, the VR processing device 2 displays the position information and direction information of each device detected in step S902 in the VR screen. Specifically, the content generation unit 222 of the VR processing device 2 reads the position information and direction information of each device stored in step S902 from the storage unit 23. The content generation unit 222 generates new VR content by superimposing the image of each device based on the read position information and direction information of each device on the original VR content. An example of VR content generated by the above-described processing will be described later with reference to FIG.

  In step S904, the VR processing device 2 causes the user U to input biometric data to the designated biometric data acquisition device.

  In step S905, the VR processing device 2 searches for available biometric data acquisition devices. Specifically, the VR processing device 2 can use the biological data acquisition devices 33, 43, 53, and 73 included in each device to the VR headset 3, the controller 4, the sensor device 5, and the mobile terminal 7. Inquire whether or not.

  The control units 34, 44, 54, 74 confirm the states of the biometric data acquisition devices 33, 43, 53, 73 and transmit to the VR processing device 2 whether or not each biometric data acquisition device is available. .

  In step S906, the VR processing device 2 displays the position, orientation, and type of the available biometric data acquisition device obtained in step S905. The detection method and display method of the position and orientation of the biological data acquisition apparatus are the same as those described in step S902 and step S903.

  In step S907, the VR processing device 2 causes the user U to input biometric data to any of the biometric data acquisition devices displayed in step S906. In this case, for example, each biometric data processing device transmits biometric data D1 to the VR processing device 2 when biometric data input from the user U is completed. The VR processing device 2 receives the biometric data D1 and stores the biometric data D1 in the storage unit 23.

  The VR processing device 2 may store the received plurality of biometric data in the storage unit 23 when the biometric data is received from a plurality of biometric data acquisition devices, and the first or last received biometric data is valid. You may store in the memory | storage part 23 as simple data.

  In step S <b> 908, the VR processing device 2 transmits a biometric data collation command to the server device 6. Specifically, the biometric data transmission unit 224 reads the biometric data D1 from the storage unit 23, and transmits the biometric data collation command and the biometric data D1 to the server device 6.

  In step S909, the biometric authentication unit 62 of the server device 6 receives the biometric data D1 of the user U and a biometric data collation command. The server device 6 confirms whether or not the received biometric data D1 is registered in the database DB. If the biometric data of the user U is registered, the process proceeds to step S911. When the biometric data of the user U is not registered, the server device 6 notifies the VR processing device 2 that the biometric data is not registered. The VR processing device 2 receives the notification that the biometric data is not registered, and performs biometric data registration processing in step S910.

  The biometric data registration process may be performed independently of the payment process. For example, the biometric data registration process may be performed when the VR system 1 is activated for the first time. Details of the biometric data registration process will be described later with reference to FIG.

  In step S911, the biometric authentication unit 62 of the server device 6 performs a biometric data matching process. Specifically, the biometric authentication unit 62 reads the registered biometric data D2 of the user U registered in the database DB and compares it with the biometric data D1 received from the VR processing device 2. The biometric authentication unit 62 transmits to the VR processing device 2 that the verification result has succeeded or failed.

  FIG. 10 is a flowchart illustrating a biometric data registration process according to this embodiment.

  In step S1001, the VR processing device 2 displays the types of biometric data acquisition devices that can be registered. The search process of the biometric data acquisition device that can be registered is the same as the process in step S905.

  In step S1002, the VR processing device 2 displays the position, orientation, and type of the biometric data acquisition device that can be registered obtained in step S1001 on the VR screen. The detection method and display method of the position and orientation of the biological data acquisition apparatus are the same as those described in step S906.

  In step S1003, the VR processing device 2 causes the user U to select one of the biological data acquisition devices displayed on the VR screen in step S1002.

  When the biometric data acquisition device is specified in step S901, the same biometric data acquisition device specified in step S901 may be automatically specified. In this case, in step S1001 and step S1002, the biometric data acquisition device specified in step S901 is displayed on the VR screen, and the process in step S1003 may not be performed.

  In step S1004, the VR processing device 2 causes the user U to input biometric data to the designated biometric data acquisition device. The input biometric data is transmitted to the VR processing device 2.

  In step S1005, the VR processing device 2 receives the biometric data input to the biometric data acquisition device in step S1004. The biometric data transmission unit 224 of the VR processing device 2 transmits the biometric data and a biometric data registration instruction to the server device 6.

  In step S1006, the server device 6 registers the received biometric data in the database DB.

  FIG. 11 is a diagram showing a first example of VR screen display at the time of biometric data registration or biometric data acquisition according to the present embodiment. More specifically, FIG. 11 shows the VR screen V1 displayed on the display unit 37 in S906 of FIG. 9 and S1002 of FIG.

  The VR screen V1 displays a biosensor selection screen W1 showing biosensors that can be used (or registered). In FIG. 11, the biosensor selection screen W1 is displayed in a superimposed manner on the VR screen V1, but the display method is not limited to this. For example, the VR screen V1 may be switched, and the biosensor selection screen W1 may be displayed on the entire screen.

  The biometric sensor selection screen W1 displays an available biometric data acquisition device and a device including the available biometric data acquisition device. In the example of FIG. 11, the biosensor selection screen W <b> 1 is included in the VR headset 103, the iris sensor 103 </ b> A, the voice print sensor 103 </ b> B, the vein sensor 103 </ b> C, the controller 104, and the controller 104 included in the VR headset 103. The vein sensor 104A, the fingerprint sensor 104B, and the controller 105, the vein sensor 105A included in the controller 105, the fingerprint sensor 105B, the sensor device 106, and the voiceprint sensor 106A included in the sensor device 106 are displayed.

  In addition, as for the direction of the image of each apparatus and biosensor displayed on the biosensor selection screen W1, it is desirable to change in real time according to the movement of the user U who operates each apparatus. In addition, when two or more types of biosensors are included, it is desirable that the display is such that the user U can easily distinguish between the same type of biosensors and different types of biosensors, for example, by color or shape.

  The biosensor selection screen W2 shows a state in which the vein sensor 104A is selected by the user U among the biosensors displayed on the biosensor selection screen W1.

  The selection of the biometric sensor may be performed, for example, by the user U operating the input unit 45 of the controller 4. In addition, the selection of the biometric sensor may be performed, for example, when the user U gives an instruction by voice through the microphone 35 of the VR headset 3.

  A sensing possible area A is displayed around the selected vein sensor 104A. The sensing possible region indicates a range in which biological data can be acquired by bringing a part of the body of the user U close to the biological sensor.

  The user U inputs vein data, for example, by bringing his / her finger close to the range of the detectable area A of the displayed vein sensor 104A.

  For example, when the display of the controller 104 displayed on the biosensor selection screen W <b> 2 changes due to the user U changing the position and orientation of the controller 4, the display of the sensing area A also changes to the position and orientation of the controller 104. It changes together.

  Note that, when another biological sensor is selected, a sensing possible region corresponding to the selected biological sensor is displayed as described above.

  Note that the display mode of the sensing area is not limited to the mode shown in FIG. 11 as long as the area is displayed.

  FIG. 12 is a view showing a second example of the VR screen display at the time of biometric data registration or biometric data acquisition according to the present embodiment.

  The VR screen V2 shows a state in which the voiceprint sensor 106A is selected by the user U among the biosensors displayed on the biosensor selection screen W1 in FIG.

  Since the sensor device 106 including the voiceprint sensor 106A exists at a location away from the current position of the user U, the VR processing device 2 guides the user U to a position where the voiceprint sensor 106A reacts.

  First, the control unit 221 acquires the position of the user U. When the camera 57 of the sensor device 5 captures the user U, the control unit 54 or the control unit 221 may calculate the position and orientation of the user U in the VR system 1. Alternatively, the position and orientation detected by the sensor unit of the VR headset 3 attached to the user U may be set as the position and orientation of the user U.

  Next, the control unit 221 determines a direction for guiding the user U based on the acquired position and orientation of the user U and the position and orientation of the voiceprint sensor 106A. The direction in which the user U is guided is converted into data indicating the position and orientation in the VR system 1 and input to the content generation unit 222.

  Based on the data received from the control unit 221, the content generation unit 222 draws an arrow 201 indicating the direction in which the user U is guided in the VR content.

  Note that the method of guiding the user U is not limited to the above-described method. For example, the position of the voiceprint sensor 106A may be highlighted on the VR screen V2 by the content generation unit 222.

  The VR screen V3 shows a state where the position of the user U has approached the position where the voiceprint sensor 106A reacts.

  The control unit 221 notifies the user U that the voiceprint sensor 106A can be used when the distance between the user U and the voiceprint sensor 106A is less than a certain value. For example, when the content generation unit 222 receives a notification from the control unit 221 that the distance between the user U and the voiceprint sensor 106A has become a certain value or less, the content generation unit 222 may draw the mark E1 on the voiceprint sensor 106A of the VR content. The user U inputs voiceprint data to the voiceprint sensor 106A by visually recognizing the mark E1 in the VR content.

  The shape and display method of the mark E1 are not limited to those displayed in FIG. The method for notifying the user U that the voiceprint sensor 106A is available is not limited to the above. For example, the user U may be notified by voice through the speaker 36 of the VR headset 3.

  FIG. 13 is a diagram illustrating a VR screen display when biometric data is selected according to the present embodiment. More specifically, FIG. 13 shows the VR screen V4 displayed on the display unit 37 in S901 of FIG.

  The VR screen V4 displays a biometric data selection screen W3. In FIG. 13, the biometric data selection screen W3 is displayed in a superimposed manner on the VR screen V4, but the display method is not limited to this as with the biometric sensor selection screen W1.

  The type of biometric data to be acquired is displayed on the biometric data selection screen W3. In the example of FIG. 13, the biometric data selection screen W3 displays fingerprint data B1, face data B2, iris data B3, vein data B4, and voiceprint data B5 as selectable items.

  When one of the above items is selected by the user U, the fact that the item has been selected is displayed on the VR screen V4. For example, a mark E2 may be displayed around the selected item on the VR screen V4.

  The biometric data may be selected by the user U operating the input unit 45 of the controller 4. The biometric data may be selected, for example, when the user U gives an instruction by voice through the microphone 35 of the VR headset 3.

  Note that the shape and display method of the mark E2 are not limited to those illustrated in FIG.

  FIG. 14 is a diagram illustrating a VR screen display during content sharing according to the present embodiment. More specifically, the VR screen V5 displayed on the display unit 37 when the content sharing unit 225 transmits content to the mobile terminal 8 is shown.

  The VR screen V5 includes a screen display W4. The screen display W <b> 4 is an image or a character indicating that the content is shared with the mobile terminal 8.

  The screen display W4 can be displayed at any position on the VR screen V5 that can be recognized by the user. Further, the screen display W4 may be moved or deleted automatically or by an operation of the user U.

  In the present embodiment described above, the VR system 1 can display the position and orientation of the biosensor on the VR screen. Thereby, the user U can input biometric data without removing the VR headset 3 even when purchasing a product within the VR screen and obtaining biometric data for settlement.

  In this embodiment, since biometric data is used for payment, the security of payment is ensured.

  In the present embodiment, the mobile terminal 7 owned by the user U can be added to the VR system 1 instead of the controller 4, for example. Thereby, a system can be constructed with a normal virtual realization system and a mobile phone, and the cost can be reduced.

  In the present embodiment, the VR system 1 guides the user U to a position where the user U can input biometric data even when the biometric data acquisition apparatus and the user U are separated from each other. Thereby, the convenience of the user U who inputs biometric data further increases.

  In the present embodiment, VR content displayed on the display unit 37 of the VR headset 3 can be output to the display unit 75 of the mobile terminal 8 authenticated by the VR system 1. Thereby, when sharing the VR system 1 with a family, for example, the parent can grasp what kind of game the child wearing the VR headset 3 is playing using his / her mobile terminal 8, Protect from harmful content. In addition, VR content can be shared with those who do not wear the VR headset 3, and amusement can be improved.

[Second Embodiment]
In the present embodiment, a VR system that determines an attribute of a character (avatar) to appear in a VR screen using biometric data acquired from a user of the VR system will be described.

  The VR system used in the present embodiment is the same as the VR system 1 in the first embodiment.

  FIG. 15 is a diagram illustrating a VR screen display during character display according to the second embodiment.

  The VR screen V6 displays the character C corresponding to the user U and the attribute X of the character C. The attribute X need not always be displayed on the VR screen V6, and may be displayed only when the user U receives an instruction to display the attribute X.

  The attribute X includes the attribute of the character C. For example, the attribute X includes name P1, gender P2, age P3, physical strength P4, and personality P5. The attribute of the character C may include items other than these.

  The appearance and state of the character C image may change depending on the content of the attribute X item. For example, the appearance of the character C may be changed to match the sex P2 or the age P3, the posture of the character C may be changed by the physical strength P4, and the expression of the character C may be changed by the personality P5. Also good.

  Each item of the attribute X may be updated based on the result of analyzing the biometric data of the user U by the VR system 1 every time the user U performs biometric authentication.

  FIG. 16 is a flowchart illustrating an attribute determination process according to the second embodiment.

  In step S <b> 1601, the biometric data of the user U is input to an arbitrary biometric data acquisition device in the VR system 1. The timing at which the biometric data of the user U is input may be step S904 or step S907 in FIG. 9 or step S1004 in FIG. 10, or may be input at a timing other than these. The biometric data acquired by the biometric data acquisition device is transmitted to the VR processing device 2.

  In step S1602, the control unit 221 of the VR processing device 2 receives the biometric data input by the user U and performs analysis processing. For example, when the biometric data is voiceprint data, the control unit 221 determines the age and sex of the user U based on the pitch and quality of the voice. For example, when the biometric data is face data, the control unit 221 determines the personality of the user U by reading a facial expression. Note that the analysis processing performed by the control unit 221 is not limited to the above.

  In step S1603, the control unit 221 determines the attribute of the character C based on the analysis result of the biometric data obtained in step S1602. The type (item) of the attribute may be determined in advance by the user U, or may be automatically generated by the control unit 221 based on the analysis result.

  In step S1604, the control unit 221 performs character C generation processing or update processing based on the attribute obtained in step S1603. When the character C is generated for the first time, the image that is the basic state of the character C may be selected in advance by the user U. Alternatively, the control unit 221 may automatically generate the character C based on the attribute.

  In step S <b> 1605, the control unit 221 inputs character information and attribute information to the content generation unit 222. The content generation unit 222 generates VR content including display of the character C and the attribute X based on the input character information and attribute information. The VR content is transmitted to the VR headset 3 by the content transmission unit 223 and displayed on the display unit 37.

  In the present embodiment described above, the character C in which the attribute X included in the biological data is automatically reflected can be displayed on the VR screen using the biological data acquired from the user U. Thereby, it is possible to generate a character reflecting the personality of the user U, and it becomes easy to identify an individual on the network through the VR. In addition, amusement can be enhanced by reflecting the personality of the user U in the character.

[Third embodiment]
In recent years, a system for automatically diagnosing a disease based on input examination data using artificial intelligence (AI) has been developed. However, in order to use such a diagnostic system, collection of medical examination data is a burden on the user, and thus a technique for improving user convenience is desired.

  In the present embodiment, biological data that a user of the VR system routinely inputs to the VR system is accumulated, the accumulated biological data is input to the diagnostic system, and the result is received, so that the user can use the diagnostic system. We propose a diagnostic system that can obtain simple diagnosis results of diseases without being conscious of collecting necessary data. In addition, user convenience can be further enhanced by proposing hospitals and departments that can be used by the user based on the diagnosis result.

  FIG. 17 is a block diagram illustrating the configuration of an automatic diagnosis system according to the third embodiment.

  The automatic diagnosis system 11 includes a VR system 1A, a diagnosis server device 9, and a reservation system 10.

  The VR processing device 2 included in the VR system 1 </ b> A includes a storage unit 23. The storage unit 23 includes biological data D1 and position information 233.

  The biometric data D1 stores the biometric data input by the user U to any of the biometric data acquisition devices 25, 33, 43, 53, and 73.

  The position information 233 is used, for example, for the diagnosis server device 9 to extract a hospital where the user U can consult.

  The location information 233 may be, for example, an address registered in advance in the VR system 1 by the user U. When the VR processing device 2 or the mobile terminal 7 includes a GPS (Global Positioning System), a screen display T1 described later is displayed. It may be a position indicated by the GPS when displaying. The method for acquiring the position information 233 is not limited to the above.

  The remaining configuration of the VR system 1A according to the present embodiment is the same as the configuration of the VR system 1 according to the first embodiment.

  The diagnostic server device 9 includes a storage unit 91, a control unit 92, and a communication unit 93.

  The storage unit 91 includes hospital information 911.

  The hospital information 911 is information that associates, for example, a disease name, a hospital and a department that can handle the disease name, and the position of the hospital.

  The control unit 92 controls the entire diagnostic server device 9. For example, the control unit 92 executes a diagnosis process based on the biometric data of the user U, and obtains a disease name as a diagnosis result. Further, the control unit 92 generates a list of hospitals and departments that can handle disease names based on the hospital information 911, the diagnosis result, and the position information 233 received from the VR processing device 2, and the list is displayed as VR. Send to processing device.

  The communication unit 93 transmits / receives commands, addresses, data, information, instructions, signals, and the like to / from the VR processing device 2 via the network N, for example.

  The reservation system 10 is a general hospital reservation system capable of accepting reservations for medical treatment dates and times from patients and the like. The reservation system 10 is realized by, for example, a program that operates on a server device provided in a hospital.

  FIG. 18 is a flowchart illustrating the process of the automatic diagnosis system 11 according to this embodiment.

  In step S1801, the VR processing device 2 is activated. The VR headset 3, the controller 4, and the sensor device 5 connected to the VR processing device 2 are also turned on, and can be operated by the user U.

  In step S <b> 1802, the VR processing device 2 collects biometric data used at the time of login when biometric data input is required when logging in to the VR system 1 </ b> A. Specifically, the VR processing device 2 includes the biological data acquisition device that receives the biological data input to any of the biological data acquisition devices 25, 33, 43, 53, and 73 when the user U logs in. It is received from any of the headset 3, the controller 4, the sensor device 5, and the mobile terminal 7 and stored in the storage unit 23.

  Note that the user U may log in to the VR system 1 </ b> A with respect to the VR processing device 2, or the user U may perform with respect to the mobile terminal 7.

  In step S1803, the VR processing device 2 collects biometric data used at the time of payment every time payment processing is performed in the VR system 1A. The biometric data collection method is the same as that in step S1802.

  In step S <b> 1804, the VR processing device 2 confirms whether the amount and type of the biological data D <b> 1 stored in the storage unit 23 are sufficient to be input to the diagnostic server device 9. When the VR processing device 2 determines that the amount and type of the biological data D1 are not sufficient, the VR processing device 2 returns to step S1802 and continues collecting the biological data. If the VR processing apparatus 2 determines that the amount and quality of the biometric data D1 are sufficient, the process proceeds to step S1805.

  In step S <b> 1805, the VR processing device 2 acquires the position information 233 of the VR processing device and stores it in the storage unit 23. Then, the VR processing device 2 transmits the biological data D1 and the position information 233 to the diagnostic server device 9.

  In step S1806, the diagnostic server device 9 executes a diagnostic process based on the biological data D1 received from the VR processing device 2 and obtains a diagnostic result. The diagnosis result may be a disease name, for example. The diagnosis process may be performed using AI (Artificial Intelligence), or may be performed using any other algorithm.

  In step S <b> 1807, the diagnostic server device 9 searches and extracts hospitals in a range that can be used by the user U based on the position information 233 received from the VR processing device 2. Further, the diagnosis server device 9 determines hospitals and departments that the user U can receive based on the diagnosis result, the extracted hospital, and the hospital information 911.

  In step S1808, the diagnostic server device 9 transmits the diagnosis result obtained in steps S1806 and S1807, and data including the hospital and department that the user U can consult to the VR processing device 2.

  In step S <b> 1809, the VR processing device 2 stores information such as a diagnosis result, a hospital, and a department on the display unit 37 of the VR headset 3 or the display unit 75 of the mobile terminal 7 based on the data received from the diagnostic server device 9. indicate. Details of the screens displayed on the display units 37 and 75 will be described later with reference to FIG.

  In step S <b> 1810, the VR processing device 2 causes the user U to specify the hospital and the medical department displayed on the display units 37 and 75. Data indicating the hospital and department designated by the user U is transmitted to the reservation system 10.

  In step S <b> 1811, the reservation system 10 searches for the reservation possible date and time of the hospital and the department designated in step S <b> 1810 and transmits them to the VR processing device 2.

  In step S1812, the VR processing device 2 displays the reservation available date and time received from the reservation system 10 on the display units 37 and 75, and causes the user U to specify the reservation date and time. The reservation date and time designated by the user U is transmitted to the reservation system 10.

  In step S1813, the reservation system 10 transmits the reservation result to the VR processing device 2 and completes the reservation.

  In step S1814, the VR processing device 2 notifies the user that the reservation has been completed. Specifically, for example, a reservation completion message is displayed on the display units 37 and 75.

  Note that the VR processing device 2 may transmit the diagnosis result to the reserved hospital after completion of the reservation, for example.

  FIG. 19 is a diagram showing a first example of screen display in the automatic diagnosis system 11 according to the present embodiment. Specifically, FIG. 19 shows a list of diagnosis results displayed on the display units 37 and 75 in step S1809 of FIG.

  The screen display T1 includes a disease name Q1, a hospital Q2, a medical department Q3, and an address Q4.

  The disease name Q1 indicates a disease name based on the received diagnosis result.

  Hospital Q2 indicates the name of a hospital where a diagnosis relating to disease name Q1 can be received.

  The medical department Q3 indicates a medical department that the user can receive regarding the disease name Q1 in the hospital Q2. It is desirable that the display of each medical department included in the medical department Q3 can be selected by the user U. The control unit 221 or the control unit 741 displays a reservation screen for the medical department selected by the user U. The selected medical department is highlighted as shown by display E3, for example.

  The address Q4 is an address indicating the location of information of the hospital Q2 and the medical department Q3. The address may be a URL (Uniform Resource Locator), for example.

  The items included in the screen display T1 are not limited to those described above. The screen display T1 may be displayed in a format other than the table format. The display of each address included in the address Q4 may be selectable by the user U. In this case, the control unit 221 or the control unit 741 may display a web page or the like associated with the address Q4 selected by the user U.

  FIG. 20 is a diagram illustrating a second example of screen display in the automatic diagnosis system 11 according to the present embodiment. Specifically, FIG. 20 shows a reservation screen displayed on the display units 37 and 75 in step S1812 of FIG. FIG. 20 shows a screen displayed when “Otolaryngology at Hospital A” is selected in the screen display T1 of FIG. 19 (see display E3).

  The screen display T2 includes a date selection display E4. The date selection display E4 may be, for example, a calendar display or a list display.

  In the date selection display E4, it is desirable that a date that can be reserved and a date that cannot be reserved are clearly indicated. When a date that can be reserved is selected by the user U, the control unit 221 or the control unit 741 may further display a time zone selection display E5. The user U further completes the designation of the reservation date and time by selecting a time slot that can be reserved. The time zone selection display E5 may be, for example, a pop-up display, a list display, or other display formats.

  In the present embodiment, the screen displays T1 and T2 may be displayed by an application installed in the VR processing device 2 or the mobile terminal 7. In this case, the process of searching for and displaying the hospital Q2 and the medical department Q3 and the screen display process may be performed by the application using the processor 22 and the processor 74.

  In the present embodiment described above, the automatic diagnosis system 11 collects biometric data input to the VR system 1A when the user U logs in to the VR system 1A or makes a payment, and transmits the collected biometric data to the diagnosis server device 9. To do. As a result, the user U can acquire the diagnosis result without intentionally collecting data to be input to the diagnostic server device 9, and the convenience for the user is enhanced. The automatic diagnosis system 11 selects an optimal hospital and department based on the diagnosis result, makes a visit reservation, and transmits the diagnosis result obtained from the diagnosis server device 9. This not only increases the convenience for the user, but also contributes to the prevention of medical errors and the reduction of medical hours.

  In the present embodiment, the automatic diagnosis system 11 is applicable not only to diagnosis of the user U's disease name but also to all types of diagnosis. For example, when the automatic diagnosis system 11 is applied to a health check, the automatic diagnosis system 11 associates a test item and a detailed inspection organization in the contents of the screen display T1 instead of the disease name Q1 and the clinical department Q3, and It may be presented.

  In the present embodiment, the automatic diagnosis system 11 may include another information processing apparatus or system capable of acquiring the biological data D1 and acquiring or storing the position information 233, instead of the VR system 1. .

  The above embodiments are illustrative and are not intended to limit the scope of the invention. The above embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. The above-described embodiments and modifications thereof are included in the invention described in the scope of claims and the equivalents thereof, as long as they are included in the scope and spirit of the invention.

  DESCRIPTION OF SYMBOLS 1 ... VR system, 2 ... VR processing device, 3 ... VR headset, 4 ... Controller, 5 ... Sensor device, 6, 12 ... Server device, 7, 8 ... Portable terminal, 25, 33, 43, 53, 73 ... Biometric data acquisition device, 26, 32, 42, 52, 72 ... sensor unit, 62 ... biometric authentication unit, 75 ... display unit, 121 ... settlement processing unit, 222 ... content generation unit, 223 ... content transmission unit, 224 ... biometrics Data transmission unit, 225 ... content sharing unit, D1, D2 ... biological data, DB ... database, HD ... human body wearing device, N ... network, U, V ... user.

Claims (3)

An information processing device for generating virtual reality content;
A virtual reality headset that is worn by a user and displays the virtual reality content received from the information processing device to the user;
A biometric data acquisition device for acquiring biometric data of the user;
A sensor that generates position information indicating a position of the biometric data acquisition apparatus and direction information indicating a direction of the biometric data acquisition apparatus, and transmits the position information and the direction information to the information processing apparatus;
Comprising
The information processing apparatus includes:
When the payment instruction is received, the position and the direction of the biometric data acquisition device are recognized by the user wearing the virtual reality headset based on the position information and the direction information received from the sensor. A content generation unit that adds the first image to be added to the virtual reality content;
Virtual reality system. The biometric data acquisition device is a camera, and the biometric data is fingerprint data,
The virtual reality system of claim 1. The content generation unit adds, to the virtual reality content, a second image that prompts the user wearing the virtual reality headset to input biometric data by the biometric data acquisition device.
The virtual reality system according to claim 1 or 2.

Images