WO2023223476A1

WO2023223476A1 - Information processing system and information processing method

Info

Publication number: WO2023223476A1
Application number: PCT/JP2022/020725
Authority: WO
Inventors: 英弟謝; 彦鵬張; 夢▲せん▼ 許; 雨佳劉; 道久井口
Original assignee: 株式会社Vrc
Priority date: 2022-05-18
Filing date: 2022-05-18
Publication date: 2023-11-23
Also published as: WO2023224083A1; JPWO2023223476A1

Abstract

This information processing system is provided with: a 3D data acquiring means for acquiring 3D data that show a 3D model of a living body; a biological data acquiring means for acquiring biological data that show biological information acquired from the living body; a storing means for storing the 3D data and the biological data in a database while correlating the 3D data and the biological data with each other; a receiving means for receiving a request including an identifier for the 3D model; an extracting means for extracting biological data corresponding to the identifier included in the request from the database; and a transmitting means for transmitting the extracted biological data to a transmission source of the request.

Description

Information processing system and information processing method

The present invention relates to a technology for processing 3D data representing a 3D model.

Techniques that utilize 3D data of objects are known. For example, Patent Document 1 discloses a technique for storing and utilizing 3D data representing a 3D model of a user in a database.

Patent No. 6791530

In the technology described in Patent Document 1, assuming various applications of 3D data, there is room for improvement in the data recorded in the database.

In contrast, the present invention provides 3D data that can be used in a variety of applications.

One aspect of the present disclosure includes: a 3D data acquisition unit that acquires 3D data representing a 3D model of a living body; a biological data acquisition unit that acquires biological data representing biological information acquired from the living body; a storage means for storing data in a database in association with each other; a reception means for receiving a request including an identifier of a 3D model; an extraction means for extracting biometric data corresponding to the identifier included in the request from the database; and transmitting means for transmitting biometric data obtained from the request to the source of the request.

The biological information may be information acquired at the same time as the image for generating the 3D model.

The biometric data may indicate a spatial distribution of the biometric information in the living body.

The biological information may include temperature distribution in the biological body.

The biological information may include the skin color of the living body.

The biological information may include medical information about the living body.

The biometric data may indicate a change in the biometric information over time.

The biometric data may include time information corresponding to the time when the biometric information was acquired.

The 3D data may indicate changes in the 3D model over time.

The biological information may indicate center-of-gravity sway of the biological body.

Another aspect of the present disclosure includes a step of acquiring 3D data indicating a 3D model of a living body, a step of acquiring biometric data indicating biometric information acquired from the living body, and associating the 3D data and the biometric data. a step of receiving a request including an identifier of the 3D model; a step of extracting biometric data corresponding to the identifier included in the request from the database; and adding the extracted biometric data to the request. An information processing method is provided, which includes the step of transmitting information to a transmission source.

According to the present invention, it is possible to use 3D data in a variety of applications.

FIG. 1 is a diagram showing an overview of a 3D data system 1 according to an embodiment. 1 is a diagram illustrating a functional configuration of a 3D data system 1. FIG. FIG. 2 is a diagram illustrating a hardware configuration of a 3D scanner 20. FIG. 1 is a diagram illustrating a hardware configuration of a server 10. FIG. FIG. 3 is a diagram illustrating a hardware configuration of a user terminal 30. FIG. A sequence chart illustrating the operation of the 3D data system 1. 3 is a diagram illustrating data recorded in a database 111. FIG.

DESCRIPTION OF SYMBOLS 1... 3D data system, 10... Server, 11... Storage means, 12... Acquisition means, 13... Acquisition means, 14... Storage control means, 15... Acceptance means, 16... Extraction means, 17... Transmission means, 19... Control means , 20... 3D scanner, 21... Storage means, 22... Photographing means, 23... Measuring means, 24... Measuring means, 25... Generating means, 26... Transmitting means, 29... Control means, 30... User terminal, 31... Storage means , 32...Requesting means, 33...Receiving means, 39...Controlling means, 101...CPU, 102...Memory, 103...Storage, 104...Communication IF, 201...Frame, 202...Lighting, 203...Camera, 204...Depth sensor , 205...Thermography camera, 206...Computer, 207...Display, 301...CPU, 302...Memory, 303...Storage, 304...Communication IF, 305...Display, 306...Input device

1. Configuration FIG. 1 is a diagram showing an overview of a 3D data system 1 according to an embodiment. The 3D data system 1 is an information processing system that provides 3D data services. In the 3D data system 1, 3D data and biological information are stored in association with each other. 3D data refers to data representing a 3D model. A 3D model is a model that represents the three-dimensional shape of an object in a virtual three-dimensional space. In this example, the object is a living organism, such as a human being. The biological information is information regarding the living body of the object, and includes, for example, at least one of body composition, skin color, skin unevenness, temperature, blood pressure, pulse rate, and arterial blood oxygen saturation. The biological information may include at least one of spatial distribution and temporal changes on the body surface or inside the body of the target object.

The 3D data system 1 includes a server 10, a 3D scanner 20, and a user terminal 30. The server 10 is a server that provides 3D data services. The 3D scanner 20 photographs an object and generates 3D data. In this example, 3D scanner 20 further measures biometric data of the object. The 3D scanner 20 transmits the generated 3D data and the measured biometric data to the server 10. The server 10 stores 3D data and biometric data in association with each other. The server 10, 3D scanner 20, and user terminal 30 are connected via a network 90. Network 90 is a computer network such as the Internet.

FIG. 2 is a diagram illustrating the functional configuration of the 3D data system 1. The 3D data system 1 includes a storage means 11, an acquisition means 12, an acquisition means 13, a storage control means 14, an acceptance means 15, an extraction means 16, a transmission means 17, a control means 19, a storage means 21, an imaging means 22, and a measurement means 23. , a measuring means 24, a generating means 25, a transmitting means 26, a controlling means 29, a storing means 31, a requesting means 32, a receiving means 33, and a controlling means 39. Of these, the storage means 11, the acquisition means 12, the acquisition means 13, the storage control means 14, the reception means 15, the extraction means 16, the transmission means 17, and the control means 19 are stored in the server 10. The means 23, the measuring means 24, the generating means 25, the transmitting means 26, and the controlling means 29 are installed in the 3D scanner 20, and the storing means 31, the requesting means 32, the receiving means 33, and the controlling means 39 are installed in the user terminal 30. Ru.

In the server 10, the storage means 11 stores various data. In this example, the data stored by the storage means 11 includes a database 111. The database 111 is a database that records 3D data and biometric information. The acquisition means 12 acquires 3D data representing a 3D model of the object. The acquisition means 13 acquires biometric data indicating biometric information of the object. The storage control means 14 records the 3D data and the biometric data in the database 111 in association with each other. The accepting means 15 accepts requests from the user terminal 30, for example. This request includes identification information of the 3D model. The extraction means 16 extracts biometric data corresponding to the identification information included in the request from the database 111. The transmitting means 17 transmits the extracted biometric data to the source of the request. The control means 19 performs various controls.

In the 3D scanner 20, the storage means 21 stores various data. The photographing means 22 photographs the object. Here, "photographing" a target object means acquiring a two-dimensional image of the target object. The measuring means 23 measures the distance from the photographing means 22 to the surface of the object. By measuring the distance, information regarding the three-dimensional shape of the object can be obtained. The measuring means 24 measures biological information of the object. The generating means 25 generates 3D data of the object using the image photographed by the photographing means 22 and the distance measured by the measuring means 23. The transmitting means 26 transmits the 3D data and biometric data to the server 10. The control means 29 performs various controls.

In the user terminal 30, the storage means 31 stores various data. The request means 32 sends a request to the server 10. This request requests the transmission of 3D data and biometric data. The receiving means 33 receives the 3D data and biometric data transmitted from the server 10 in response to this request. The control means 39 performs various controls.

FIG. 3 is a diagram illustrating the hardware configuration of the 3D scanner 20. The 3D scanner 20 has a frame 201, a light 202, a camera 203, a depth sensor (or distance sensor) 204, a thermography camera 205, a computer 206, and a display 207. The frame 201 is a structure for forming a photography room (or photography space). The frame 201 has a basic skeleton in the shape of, for example, a rectangular parallelepiped, a cube, a pentagonal prism, a hexagonal prism, or a similar shape. The interior of these three-dimensional objects will become the photography room. Lighting 202 illuminates the inside of the photographing room. The camera 203 photographs the object in the photographing room (that is, acquires a two-dimensional image or photograph of the object). Depth sensor 204 measures the distance to the surface of the object. More specifically, depth sensor 204 measures the spatial distribution of distances to the surface of the object. One camera 203 and one depth sensor 204 constitute a sensor unit. The sensor unit is fixed to the frame 201, for example. In one example, 3D scanner 20 includes multiple sensor units. The plurality of sensor units are arranged so as to photograph the object from different directions. The thermography camera 205 is a device that measures thermography, that is, heat distribution, of an object. Thermographic camera 205 may be included in the sensor unit. That is, the 3D scanner 20 may have the same number of thermography cameras 205 as the cameras 203 at approximately the same positions as the cameras 203. Alternatively, the thermography camera 205 may be provided independently of the sensor unit. That is, the position of thermography camera 205 may be different from the position of camera 203, or the number thereof may be different. Computer 206 processes the images taken by camera 203, the distance measured by depth sensor 204, and the heat distribution measured by thermography camera 205. Specifically, computer 206 generates 3D data representing a 3D model of the object. Computer 206 transmits the generated 3D data to server 10. The computer 206 is, for example, a general-purpose computer having a processor, memory, storage, and communication IF, and is equipped with a function of generating 3D data by executing a program. Display 207 is controlled by computer 206 and presents information to a user (eg, a human object).

The correspondence relationship with the functional configuration of FIG. 2 will be explained. The camera 203 is an example of the photographing means 22. Depth sensor 204 is an example of measuring means 23. A thermographic camera 205 is an example of the measuring means 24. That is, a thermograph measured by the thermography camera 205 is an example of biological information. The computer 206 stores in its storage a program for generating 3D data (hereinafter referred to as "3D data generation program"). In a state where the processor is executing the 3D data generation program, at least one of the memory and the storage is an example of the storage means 21, the processor is an example of the generation means 25 and the control means 29, and the communication IF is an example of the transmission means 26. This is an example.

FIG. 4 is a diagram illustrating the hardware configuration of the server 10. The server 10 includes a CPU (Central Processing Unit) 101, a memory 102, a storage 103, and a communication IF 104. The CPU 101 is a processor that performs various calculations according to programs. The memory 102 is a main storage device that functions as a work area when the CPU 101 executes a program, and includes, for example, RAM (Random Access Memory). The storage 103 is an auxiliary storage device that stores various data and programs, and includes, for example, an SSD (Solid State Drive) or an HDD (Hard Disc Drive). The communication IF 104 is a device that communicates with other devices according to a predetermined communication standard (eg, Ethernet (registered trademark)), and includes, for example, a NIC (Network Interface Card).

In this example, the programs stored in the storage 103 include a program for causing a computer to function as a server in the 3D data system 1 (hereinafter referred to as a "server program"). In a state where the CPU 101 is executing the server program, at least one of the memory 102 and the storage 103 is an example of the storage means 11, and the CPU 101 is the acquisition means 12, the acquisition means 13, the storage control means 14, the reception means 15, and the extraction means. 16 and the control means 19, and the communication IF 104 is an example of the transmission means 17.

FIG. 5 is a diagram illustrating the hardware configuration of the user terminal 30. The user terminal 30 is a computer including a CPU 301, a memory 302, a storage 303, a communication IF 304, a display 305, and an input device 306, and is, for example, a smartphone or a personal computer. The CPU 301 is a processor that performs various calculations according to programs. The memory 302 is a main storage device that functions as a work area when the CPU 301 executes a program, and includes, for example, a RAM. The storage 303 is an auxiliary storage device that stores various data and programs, and includes, for example, an SSD or an HDD. The communication IF 304 is a device that communicates with other devices according to a predetermined communication standard (for example, WiFi (registered trademark)), and includes, for example, a wireless chip. The display 305 is a device that displays information, and includes, for example, an organic EL display. Input device 306 is a device for inputting information to user terminal 30, and includes, for example, a touch screen, a keyboard, or a pointing device.

In this example, the programs stored in the storage 303 include a program for causing a computer to function as a client in the 3D data system 1 (hereinafter referred to as a "client program"). In a state where the CPU 301 is executing a client program, at least one of the memory 302 and the storage 303 is an example of the storage means 31, the CPU 101 is an example of the requesting means 32 and the control means 39, and the communication IF 104 is an example of the receiving means 33. This is an example.

2. Operation FIG. 6 is a sequence chart illustrating the operation of the 3D data system 1. In this example, the 3D scanner 20 is installed in a commercial facility, a sports gym, a hospital, etc., and the user can use the 3D scanner 20 for a fee or for free.

In step S1, the 3D scanner 20 photographs the user as a target object. Specifically, for example, it is as follows. A user who wants to generate 3D data enters the imaging room. The 3D scanner 20 has a human sensor (not shown) in the imaging room, and when it is detected that a person has entered the imaging room, the computer 206 displays a screen that guides imaging to generate 3D data. It is displayed on the display 207. This screen includes, for example, guidance on where the user should stand, and guidance on the pose or posture the user should take during shooting. Further, this screen may include a UI object that prompts the user to input the user ID. The user ID is identification information that identifies a user in the 3D data system 1. The user enters the user ID from the touch screen or keyboard. Alternatively, an image code indicating the user ID may be displayed on a terminal device (for example, a smartphone, not shown) owned by the user, and the computer 206 may read the image code with a camera (not shown) and accept the input of the user ID. good. When the user is ready, the computer 206 displays a countdown until shooting on the display 207. The computer 206 controls the camera 203 to photograph the user in the photographing room in accordance with this countdown. The computer 206 controls the depth sensor 204 to measure the distance to the object at the same timing as the camera 203 photographs the object. The computer 206 stores image data indicating an image (a still image in this example) captured by the camera 203 and distance data indicating the distance measured by the depth sensor 204 in a storage.

In step S2, the 3D scanner 20 measures biological information of the object. In this example, a thermograph of the object is measured as biological information. Specifically, for example, it is as follows. The computer 206 controls the thermography camera 205 to measure the thermograph of the object at the same timing (ie, approximately at the same time) as the object is photographed by the camera 203. The computer 206 stores thermograph data (an example of biological data) indicating a thermograph measured by the thermography camera 205 in a storage.

In step S3, the 3D scanner 20 generates 3D data of the object. Specifically, for example, it is as follows. The computer 206 generates a 3D model of the object from the image data and distance data according to the 3D data generation program. Specifically, the computer 206 generates a 3D model by pasting an image indicated by the image data as a texture onto a three-dimensional shape calculated from the distance data. Further, in this example, the biometric data indicates the surface distribution of biometric information (i.e., surface temperature) on the object. The computer 206 pastes the thermograph onto the three-dimensional shape of the object (an element body to which no image texture is pasted) in the same way as pasting a photographed image of the object onto a three-dimensional shape. That is, the computer 206 creates a 3D model in which an image of the object (visible light image, i.e., color distribution) is pasted onto the object's element body, and a 3D model in which the temperature distribution of the object is pasted onto the object's element body. Generate data corresponding to two 3D models of the model. The 3D data representing these two 3D models may be generated as separate data files, or may be included in a single data file. This 3D data includes a user ID and a time stamp. The timestamp is an example of time information indicating the date and time when the 3D model was generated. This 3D data may further include identification information of the 3D scanner 20.

In step S4, the 3D scanner 20 transmits the generated 3D data to the server 10. The server 10 to which the 3D data is transmitted is defined in the 3D data generation program. In step S5, the server 10 records the 3D data received from the 3D scanner 20 in the database 111.

FIG. 7 is a diagram illustrating data recorded in the database 111. Database 111 has multiple records. Each record includes one or more 3D data for one user. Each record includes one or more subrecords. A timestamp T[i] is associated with the subrecord. Each sub-record includes 3D data DD[i], biometric data LL[i], and attribute information. Attribute information is information indicating attributes of 3D data or biometric data. Here, as attributes, identification information ID [S, i] of the 3D scanner 20 that took the image and identification information ID [L, i] of the type of biometric data included in the 3D data are used. As will be described later, in the 3D data system 1, 3D data and biometric data are not limited to being acquired at the same time, so one of the 3D data DD and biometric data LL in one subrecord may be empty. A user generates 3D data at any timing. For example, a certain user generates new 3D data once a year. Another user generates new 3D data once a month. Still another user generates new 3D data irregularly. In this way, the database 111 records a plurality of 3D data DD and biometric data LL generated at different times for each of a plurality of users.

Note that the attribute information of 3D data is not limited to the above example. The attribute information of the 3D data includes, for example, the location of the 3D scanner 20 that generated the 3D data (address such as city, town, village name, etc.), the type of facility where the 3D scanner 20 is installed (sports gym, hospital, etc.), and the information for generating the 3D model. The image may include information indicating at least one type of user's behavior (exercising, eating, etc.) immediately before photographing the image. The 3D scanner 20 collects this information and sends it to the server 10.

Refer to FIG. 6 again. The processes in steps S1 to S5 are processes for recording the 3D data and biometric data generated by the 3D scanner 20 in the database 111. Hereinafter, a process for using the 3D data and biometric data recorded in the database 111 from the user terminal 30 will be described. Here, this process will be explained following steps S1 to S5, but the process of recording 3D data and biometric data in the database 111 and the process of using the 3D data and biometric data recorded in the database 111 are independent of each other. It will be done.

In step S6, the user terminal 30 transmits a 3D data transmission request to the server 10. Specifically, for example, it is as follows. The user starts a client program on the user terminal 30. The user terminal 30 transmits a 3D data transmission request to the server 10 according to the client program. This transmission request includes information that specifies the 3D data, for example, a user ID. Note that this user ID may be the user ID of the user operating the user terminal 30, or may be another user ID. That is, the user of the user terminal 30 may request his or her own 3D data or may request 3D data of another person.

Upon receiving a 3D data transmission request from the user terminal 30, the server 10 extracts attribute information of the requested 3D data from the database 111. The server 10 transmits a list of (at least part of) the extracted attribute information to the user terminal 30. The user terminal 30 displays the list sent from the server 10 as well as a UI object that prompts the user to select 3D data and biometric data. The user selects desired 3D data and biometric data. Upon accepting the user's selection, the user terminal 30 transmits a request to send the selected 3D data and biometric data to the server 10. This transmission request includes information specifying the selected 3D data and biometric data.

Upon receiving a request to send specific 3D data from the user terminal 30, the server 10 extracts the requested 3D data and biometric data from the database 111 (step S7). The server 10 transmits the extracted 3D data and biometric data to the user terminal 30, which is the source of the request, as a response to the transmission request (step S8).

Upon receiving the 3D data and biometric data from the server 10, the user terminal 30 performs processing using the received 3D data and biometric data (step S9). Processing using 3D data and biometric data includes, for example, displaying a 3D model and biometric information. In this way, the user of the user terminal 30 can visually confirm the 3D model and biometric information.

3. Usage Example Although the basic operation of the 3D data system 1 has been described so far, various modifications can be made to the system configuration, device configuration, type of biometric information, relationship between 3D data and biometric data, and the like. Hereinafter, some specific usage examples of the 3D data system 1 will be explained.

3-1. Use in health care or medicine (1) Periodic measurement The 3D data system 1 can be used in the field of health care or medicine. The user who is the object of the 3D model is a person to be managed or a patient. The 3D scanner 20 is installed at a health care center or hospital. Thermographs are measured as biological information. The user periodically (for example, once a year) generates a new 3D model and measures new biological information according to instructions from a staff member or a doctor. These data are recorded in the database 111.

The user terminal 30 is a terminal operated by a staff member or a doctor. The staff or doctor inputs the user ID of the person to be managed or the patient into the user terminal 30. The user terminal 30 acquires 3D data and biometric data of the person to be managed or the patient from the server 10 . The 3D data and biometric data that the user terminal 30 acquires here are 3D data and biometric data from the most recent predetermined period (for example, the most recent three years). The user terminal 30 displays a plurality of 3D models photographed at different times in such a way that changes over time can be seen. In one example, the user terminal 30 displays these multiple 3D models side by side along the time axis. At this time, the user terminal 30 may emphasize and display changes over time (for example, the waist circumference becomes thicker year by year, the back becomes rounder year by year). Display emphasizing time-series changes refers to, for example, displaying a portion where the change is greater than a standard in an appearance (color, size, or decoration) that is different from the others.

The user terminal 30 can switch between displaying a normal 3D model and a 3D model to which biometric information (for example, a thermograph) is attached using a UI object such as a button. Since this thermograph is pasted on the 3D model (its body), an image at a specified magnification is displayed as seen from a viewpoint that is changed according to instructions from the user (staff or doctor). The 3D model with biometric information pasted thereon is also displayed in a way that allows you to see its changes over time, just like a normal 3D model. At this time, the user terminal 30 may emphasize and display changes over time (for example, parts where the temperature becomes lower or higher year by year). Alternatively, when displaying a 3D model to which biometric information has been pasted, parts of a normal 3D model with large changes may be highlighted and displayed. Furthermore, when displaying a normal 3D model, parts where the biometric information changes greatly may be highlighted and displayed.

(2) Cooperation with other measuring devices Biometric information is not limited to what is measured by the 3D scanner 20. In addition to or in place of the biometric information measured by the 3D scanner 20, biometric information measured by a measuring device other than the 3D scanner 20 may be used. The other measuring devices mentioned here are, for example, devices such as body composition monitors, weight scales, blood pressure monitors, pulse meters, or pulse oximeters, and the biological information measured includes body resuscitation, weight, blood pressure, pulse rate, etc. , or medical information such as arterial oxygen saturation. Body composition refers to the composition of the elements (fat, bones, and lean soft tissue) that make up the body. This measuring device continuously, periodically, or temporarily connects with a terminal device (for example, a smartphone or a PC) owned by the user (here, the user is the user who is the object of the 3D model). and transmits biometric data indicating the measured biometric information to the terminal device. The terminal device stores biometric data received from the measuring device. A time stamp is attached to this biometric data. This terminal device continuously, periodically, or temporarily transmits the biometric data and user ID to the server 10 via a network such as the Internet. Upon receiving the biometric data from the terminal device, the server 10 records the received biometric data in a record corresponding to the received user ID in the database 111.

In this example, the time when the 3D model was generated (that is, the time when the user was photographed) and the time when the biometric information was acquired do not necessarily match. Therefore, the biometric data obtained from the measuring device is recorded as a single biometric data, not as a set with 3D data generated at a specific time. Note that if the time when the biometric information was acquired and the time when the 3D data was generated meet a predetermined condition (for example, the time difference between the two is less than or equal to a threshold), the server 10 converts the biometric data into the 3D data. It may also be recorded in the database 111 as a set. The server 10 transmits this biometric data (obtained by the measuring device) in response to a request from the user terminal 30.

Unlike the thermograph example, the biometric information shown by this biometric data does not indicate a spatial distribution, so it is not pasted onto the 3D model body. Even if they were pasted together, the spatial distribution is uniform, so when viewed as a single 3D model, the only difference would be the base body and color. However, when displaying multiple 3D models that were generated at different times side by side on the user terminal 30, for example, if different colors are given to the bodies of the 3D models according to blood pressure values, it is possible to visually see the time-series changes in blood pressure values. can be recognized visually.

Alternatively, on the user terminal 30, an image may be displayed in which biometric information corresponding to the period specified by a UI object (for example, a slide bar) specifying the period is pasted onto a single 3D model body. In this example, when the slide bar is moved, the color of the 3D model changes to the color corresponding to the biological information at that time. At this time, the 3D model also changes to a shape corresponding to that period. Thereby, the user can visually correlate and confirm the temporal change in the shape of the 3D model (that is, the temporal change in the body structure or posture) and the temporal change in the biological information.

Furthermore, the biometric information may be displayed on the user terminal 30 in a form other than being pasted onto the 3D model. For example, when a plurality of 3D models generated at different times are displayed side by side, a graph showing changes in biological information over time may be displayed above the 3D models.

(3) Use of skin color (or complexion), skin texture, etc. Assuming a situation in which 3D models are generated regularly at a health care center or hospital, it is possible to periodically take images using the same device. is assumed. Photographing with the same 3D scanner 20 each time means that the illumination (or lighting) conditions can be made almost the same each time. This means that skin color (or complexion) (in a medical sense) can be recorded under the same conditions. For example, changes in skin condition caused by changes in physical condition, such as jaundice, eczema, a pale face, or a dark face, can be recorded under almost the same lighting conditions. When generating a 3D model, if you instruct the user to take a picture with the tongue out (or facial expression), the color of the tongue can be recorded.

It is possible to measure the skin texture of a specific part of an object (for example, the face) with higher precision than other parts (for example, the body) at the same time as shooting to generate a 3D model, or separately from the shooting. good. For example, controlled light sources with adjustable orientation, height, and intensity are used for this measurement. Using a controlled light source, for example, light is irradiated onto the skin from various angles and reflected and scattered light is measured. According to such measurements, the texture of the skin, that is, minute irregularities (pimples or wrinkles) can be measured. Information on this fine unevenness is recorded as, for example, a normal map (or normal map). In this case, this normal map is an example of biological information. Highly accurate skin texture is measured using, for example, a lighting device different from that used for photographing to generate a 3D model. That is, the 3D scanner 20 has two systems of illumination, one for photographing to generate three models, and the other for measurement to record the detailed texture of the skin as biometric information. May be used.

(4) Acquisition of biological information by 3D scanner 20 The 3D scanner 20 may measure multiple types of biological information from the object. In this case, the biological information measured by the 3D scanner 20 includes information that has a spatial distribution (such as a thermograph) and information that does not have a spatial distribution (such as blood pressure, pulse rate, and arterial oxygen saturation). It may include both. For example, a chair is provided in the photographing room of the 3D scanner 20, and after photographing for 3D model generation while standing in a place other than the chair, biometric information is collected while the user is sitting in the chair. May be measured. This chair is equipped with, for example, a body composition monitor, weight scale, blood pressure monitor, pulse monitor, and pulse oximeter. Alternatively, this chair may be provided with an X-ray device or an ultrasonic examination device to take an X-ray image or an echo image (these are examples of biological information having a spatial distribution). Alternatively, the chair may have the ability to draw blood and perform blood tests. In this case, blood test results are an example of biological information. In this way, biological information may be measured by either a non-invasive measuring device or an invasive measuring device. Additionally or alternatively, the chair may have a robotic arm for palpating the user. This robot arm presses a predetermined part of the user (for example, the abdomen) and outputs the reaction force from that part as biological information. Note that although this device is expressed as a chair for convenience here, this device is not limited to a chair type, and may have any shape.

(5) Expansion of biological information Biological information is not limited to thermographs. In addition to or instead of information measured by a device such as a thermography camera or a blood pressure monitor, the biological information may include findings by a doctor (for example, "suspected hypertension" or "suspected diabetes"). Since the biological information includes the doctor's findings, the doctor's findings and the 3D model of the patient at the time corresponding to the findings are recorded in the database 111.

Alternatively, the biometric information may include information indicating the emotions of the user who is the object of the 3D model (such as "happy," "sad," "disappointed," or "feels good," etc.). The user's own emotions are input, for example, by the user making a selection from among the options presented by the 3D scanner 20 when photographing for 3D model generation. Since the biometric information includes the user's emotions, the user's emotions are recorded in the database 111 along with other biometric information and the 3D model of the patient at the corresponding time.

Alternatively, the biological information may be information indicating the sway of the center of gravity when the user, who is the object of the 3D model, assumes a predetermined posture (for example, an upright posture). In this case, the 3D scanner 20 has a center of gravity oscillation meter. A center of gravity sway meter is a device that measures the sway of the center of gravity of a person standing on it. When measuring the center of gravity sway, the locus of the center of gravity position is first measured for a predetermined period of time (for example, 60 seconds) with the eyes open and then for a predetermined period of time with the eyes closed. The measurement results include the total trajectory distance of the center of gravity sway, the sway area, the deviation of the center of gravity sway, the Romberg ratio, or the power spectrum.

3-2. Application in Fitness The 3D data system 1 can be used in the field of fitness. The user who is the object of the 3D model is a member of a fitness club or a sports gym. The 3D scanner 20 is installed in a fitness club or sports gym. Thermographs are measured as biological information. The user generates a new 3D model and measures new biological information before and after exercise according to instructions from the staff. These data are recorded in the database 111.

By recording biological information before and after exercise, it is possible to identify, for example, parts of the body that generate heat after exercise, that is, parts of the body where metabolism is improved. That is, the staff of the fitness club or sports gym can judge from this biological information whether the exercise performed by the user is applying an appropriate load to the target muscles. Furthermore, by checking the shape of the 3D model, that is, changes in body shape over time, the staff can determine whether the exercise is having an effect.

4. Modifications The present invention is not limited to the above-described embodiments, and various modifications are possible. Some modified examples will be explained below. Two or more of the items described below may be applied in combination.

(1) Analysis of biological information The server 10 may have a function as an analysis means for analyzing biological information. In the embodiment described above, imaging for 3D model generation and thermograph measurement are performed almost simultaneously. Therefore, it is considered that there is a correlation between the skin condition (color and unevenness) of the object obtained during photography for 3D model generation and the thermograph obtained at the same time. The analysis means analyzes the correlation between the skin condition and the thermograph using a technique such as correlation analysis, regression analysis, or machine learning. This correlation may be analyzed for one specific user, or may be analyzed for a user group including a plurality of users. The server 10 stores information indicating the correlation between the skin condition and the thermograph (for example, a relational expression or a machine learning model between the two) in the storage means 11.

Alternatively, the analysis means may analyze the correlation between changes over time in the biological information and changes over time in the 3D model. Through this analysis, for example, in a group of users whose biological information (specifically, for example, a doctor's findings) indicates that they have metabolic syndrome from a certain time, a predetermined period ( For example, an analysis result can be obtained that shows that changes in a predetermined region in the 3D model have a predetermined trend (for example, waist circumference increased by x% or more) in the last three years. The server 10 stores information indicating this analysis result in the storage means 11.

By using this information (information indicating correlation or information indicating analysis results), the server 10 can be used as a prediction means to predict what the health condition of a certain user is likely to be in the future based on changes over time of the 3D model up to the present. It can have the following functions. In the example of FIG. 2, the control means 19 is an example of this analysis means and prediction means.

(2) Target In the above-described embodiment, an example in which the target is a human has been described, but the target may be an animal other than a human, such as a pet or livestock.

(3) Data Recorded in the Database In the example embodiment, the database 111 records data indicating changes in the 3D model over time, as well as spatial distribution and changes over time of biological information. However, the database 111 may not include data indicating changes in the 3D model over time for a certain user. Further, the database 111 may record only one of the spatial distribution and temporal change of biological information. For example, the database 111 may record, for a certain user, only 3D data showing a 3D model at a single time and biometric data showing the spatial distribution of the user's biometric information (without information on changes over time). .

(4) Others The hardware configuration of the 3D data system 1 is not limited to that illustrated in the embodiment. The 3D data system 1 may have any hardware configuration as long as it can realize the required functions. For example, a plurality of physical devices may work together to function as the server 10. The server 10 may be a physical server or a virtual server (including a so-called cloud server). Further, the correspondence between functional elements and hardware is not limited to that illustrated in the embodiment. For example, at least some of the functions described as being implemented in the 3D scanner 20 or the user terminal 30 in the embodiment may be implemented in the server 10. For example, the generating means 25 may be implemented in the server 10. In this case, the 3D scanner 20 transmits to the server 10 a 3D model generation request that includes the image data captured by the camera 203, the distance measured by the depth sensor 204, and the thermograph measured by the thermography camera 205. Furthermore, the configuration of the 3D scanner 20 is not limited to that illustrated in the embodiment, and may include a smartphone that uses a smartphone instead of a camera, a fixed camera that photographs a rotating object, and a fixed camera that captures images of the surroundings of a fixed object. Any type of 3D scanner may be used, such as one that takes images with a rotating camera.

The sequence chart shown in FIG. 6 merely shows an example of processing, and the operation of the 3D data system 1 is not limited thereto. Some of the illustrated processes may be omitted, the order may be changed, or new processes may be added. In particular, in the embodiment, when transmitting 3D data and biometric data from the server 10 to the user terminal 30, first a list of data recorded in the database 111 is transmitted to the user terminal 30, and the data selected by the user terminal 30 is An example has been described in which data is sent from the server 10 to the user terminal 30. However, the procedure for transmitting 3D data and biometric data from the server 10 to the user terminal 30 is not limited to this. The application program of the user terminal 30 or the server program of the server 10 may automatically select the 3D data and biometric data to be transmitted to the user terminal 30. Alternatively, the server 10 may transmit only one of the 3D data and the biometric data to the user terminal 30. Furthermore, the method of specifying the 3D data and biometric data transmitted from the server 10 to the user terminal 30 is not limited to using a user ID. 3D data and biometric data having the selected attribute value at the server 10 or the user terminal 30 may be extracted from the database 111 and transmitted to the user terminal 30.

The application program executed by the CPU 101 may be provided by downloading via a network such as the Internet, or may be provided in a state recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. may be done.

Claims

3D data acquisition means for acquiring 3D data representing a 3D model of a living body;
biometric data acquisition means for acquiring biometric data indicating biometric information acquired from the living body;
storage means for storing the 3D data and the biometric data in a database in association with each other;
Acceptance means for accepting a request including an identifier of a 3D model;
Extracting means for extracting biometric data corresponding to the identifier included in the request from the database;
and transmitting means for transmitting the extracted biometric data to a source of the request.
The information processing system according to claim 1, wherein the biological information is information acquired simultaneously with an image for generating the 3D model.
The information processing system according to claim 1, wherein the biometric data indicates a spatial distribution of the biometric information in the living body.
The information processing system according to claim 3, wherein the biological information includes temperature distribution in the biological body.
The information processing system according to claim 3, wherein the biological information includes skin color of the living body.
The information processing system according to claim 3, wherein the biological information includes medical information about the living body.
The information processing system according to claim 1, wherein the biometric data indicates a change in the biometric information over time.
The information processing system according to claim 7, wherein the biometric data includes time information corresponding to a time when the biometric information was acquired.
The information processing system according to claim 7, wherein the 3D data indicates a change over time of the 3D model.
The information processing system according to claim 1, wherein the biological information indicates center of gravity sway of the biological body.
obtaining 3D data representing a 3D model of the living body;
acquiring biometric data indicating biometric information obtained from the living body;
storing the 3D data and the biometric data in a database in association with each other;
accepting a request including an identifier of the 3D model;
extracting biometric data corresponding to the identifier included in the request from the database;
An information processing method comprising: transmitting the extracted biometric data to a transmission source in response to the request.