JP6150281B2 - Server system and program - Google Patents

Description

  The present invention relates to a server system and a program.

The deep-sea exploration ship "Chikyu" owned by the applicant is excavating the seabed for the purpose of elucidating global environmental changes, the internal structure of the earth, and the biosphere in the crust. A columnar formation sample excavated in the seabed is called a drilled core sample (also simply called “core sample” or “core”). On the “CHIKYU” ship, X-ray CT (Computed) is one of the devices that analyze drill core samples.
Tomography) equipped with a scanner. By converting the drill core sample into digital data using this apparatus, the inside of the core sample can be visualized three-dimensionally and stored as a digital archive before the physical properties and structure of the core sample change.

  By conducting an investigation related to the inside of the core in advance using X-ray CT scan data (hereinafter referred to as “core data”) of the drilled core sample, the efficiency and effect of the analysis using the actual core sample can be enhanced. . In order to promote the use of core data, core data can be viewed and interactive images can be accessed by placing the core data on a server on the network and allowing researchers to access the server using a client terminal. It is possible to do.

  As prior arts related to the present invention, for example, there are techniques described in Patent Documents 1 to 5 as follows.

Japanese Patent No. 4337987 Japanese Patent No. 4646273 JP 2008-284291 A JP 2006-314626 A JP 2002-329190 A

However, the analysis using the core data has the following problems. The first problem is the size of the core data. The core sample is divided into a plurality of sections with a predetermined length (1.5 m), and core data is generated for each section. The core data is a three-dimensional image file created based on DICOM (Digital Imaging and Communication in Medicine), which is a standard format for medical images, and the data capacity of the core data is 400 megabytes per core section. In order to download a large amount of data of such a size from the server and perform three-dimensional drawing, a high-speed processor and a large-capacity memory are required. In other words, it is required to use an information processing apparatus (computer) equipped with high-spec hardware as a client terminal. In addition, a broadband communication line is required for smooth core data download.

  The second problem is a viewer problem for browsing a three-dimensional image based on core data. As described above, the core data is a DICOM image file. As software-only software (viewer) that reads a core data file (DICOM file) in DICOM format and displays a three-dimensional image, for example, there is OsiriX (Ozairix), which is open software specialized for referencing DICOM images.

  However, since OsiriX is software specialized for drawing DICOM images for medical use, drawing for CT values (X-ray absorption rate) corresponding to biological tissues such as bones, muscles, and ligaments is possible. Coloring a substance having a CT value different from that of a living tissue such as a core sample, that is, an optimal drawing setting for observation of the core sample could not be performed. DICOM defines a medical image format and a communication protocol between medical communication devices, and is a specification that can be directly applied to a general client-server system (for example, HTTP (HyperText Transfer Protocol) communication). Not equipped.

  Furthermore, as a third problem, it is assumed that the server is constructed so as to be able to provide core data to concerned parties including researchers all over the world through a network. For this reason, core data provision requests are concentrated on the server, or a certain viewer repeatedly requests several core data provisions within a certain period of time for comparison between core samples. Is assumed. If there is no mechanism to quickly generate the image data requested by the viewer on the server side and send it to the viewer in response to the above request, the appropriate survey environment for core data is provided to the viewer. I couldn't.

  The above-mentioned first to third problems are that the use range of X-ray CT scan data is specialized for medical use, and it is not assumed that X-ray CT scan data is distributed to an unspecified majority person via a network. Because of that. This problem is a problem raised against widely distributing images based on X-ray CT scan data as well as core data.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a technique capable of efficiently supplying an image of an object obtained by an X-ray CT scan to a client.

One aspect of the present invention is a search module that searches for an object to be browsed in response to a search request from a client received through a network,
In response to an image generation request including identification information of an object searched by the search module, a three-dimensional model can be generated from a plurality of slice images obtained by X-ray CT scan of an object corresponding to the identification information. A plurality of renderers capable of generating a two-dimensional image to be transmitted from the three-dimensional model to the client through the network;
A memory for temporarily storing a three-dimensional model generated by each renderer;
A storage unit that stores the correspondence between the identification information of the object and the renderer,
When a new image generation request is received, the new image generation request is supplied to a renderer corresponding to the object identification information included in the new image generation request based on the correspondence relationship. The server system generates a two-dimensional image corresponding to a new image generation request using a three-dimensional model temporarily stored in the memory.

  Another aspect of the present invention may be a program that causes a computer to execute processing in the server system or a computer-readable recording medium that records the program.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which can supply the image of the object obtained by X-ray CT scan to a client efficiently can be provided.

FIG. 1 is a diagram illustrating a configuration example of a core image distribution system according to the present invention. FIG. 2 is an explanatory diagram of search processing and image generation processing. FIG. 3A is a table showing parameters transferred to the search API and their meanings, and FIG. 3B is a table showing reception parameters and return values of the search API. FIG. 3C shows a specific example of parameters and return values delivered to the search API. 4A shows parameters of a query (image generation request (request)) transmitted from the client to the renderer API, and FIG. 4B shows a JSON format transmitted to the client as a response (response) to the query. The parameters are shown. FIG. 5 is an explanatory diagram of load distribution by the round robin method. FIG. 6 shows metadata used for image generation. FIG. 7 is a diagram for explaining a rendering flow in the drawing process. FIG. 8 is a diagram for explaining the positional relationship between the core and the viewpoint. FIG. 9 shows an example of rotation using viewpoint determination based on a quota anion. FIG. 10 shows an example of a function indicating RGB values. FIG. 11A is an explanatory diagram when the CT value changes linearly with respect to a color gradation (heat map), and FIG. 11B shows these points according to two points designated by the user. An example will be shown in which CT values are colored by a function that stores cubic splines in between. FIG. 12 shows an example of a function of the α value. FIG. 13 shows an example in which longitudinal cutting is applied. FIG. 14 shows an example of a Web page for searching for a core image, browsing the core image, and performing various operations on the core image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The configuration of the embodiment is an exemplification, and the present invention is not limited to the configuration of the embodiment.

〔System configuration〕
FIG. 1 is a diagram illustrating a configuration example of a core image distribution system according to the present invention. In the figure, the core image distribution system responds to a request received from a client terminal 10 (hereinafter referred to as “client”) 10 that requests provision of a core two-dimensional image (hereinafter referred to as “core image”). And a server system (hereinafter simply referred to as a “server”) 20 for generating and transmitting a two-dimensional image.

  The client 10 is, for example, a personal computer (PC), a tablet terminal, or a smartphone. The client 10 includes a client application that provides a user interface for browsing the core image.

For example, the client application is provided as one of Web applications provided in a normal PC. Alternatively, it is provided as an application that operates on a platform (for example, Android) included in a smartphone or a tablet terminal. The client application includes a renderer API (Application Programming Interface) 212 provided in the server 20 and a search A
By using the PI 211, core information and images can be acquired.

  The server 20 is connected to the client 10 via a network (for example, the Internet). The server 20 is, for example, a single computer (information processing apparatus) or a plurality of computers having a function of generating core image data in response to a request from the client 10 and transmitting it to the client 10 placed on the cloud. It is an aggregate.

〔server〕
As shown in FIG. 1, the server 20 roughly includes a gateway 21 that controls an interface function with the client 10, a rendering engine 22 that controls a core image generation function requested by the client 10, and a core search database ( (Search DB) 23 and a DICOM file server (DICOM server) 24.

A communication protocol between the client application included in the client 10 and the server 20 is formed as follows. For example, the communication method from the client application to the server 20 uses an HTTP GET method in HTTP (HyperText Transfer Protocol). In the HTTP GET method, "http: // URL? Parameter = value & parameter value"
One or more parameters can be sent to the server in the form Using such an HTTP GET method, it is possible to send a query (request) for search and core image generation to the server 20.

  Communication from the server 20 to the client application can be realized using, for example, JSON (JavaScript Object Notation), which is a lightweight data description language, or XML (Extensible Markup Language). That is, the client application receives the result (response) for the request in the JSON or XML format.

  The gateway 21 is a server that receives a request from the client 10 (an application included in the client 10). The gateway 20 includes, as an API 210 used by the client 10, a search API 211 that receives a request (search request) for searching for a core that the client 10 desires to browse, and a request (image generation request) for generating a core image. And a renderer API 212 that accepts.

  The gateway 21 further includes a search module 213 that performs a search process on the search DB 23 based on information obtained from the search API 211, and a renderer control module 214 that controls core image generation using information from the renderer API 212. . The renderer control module 214 receives core information corresponding to the request from the client 10 and operation information on the core image, and requests a renderer 221 included in the rendering engine 22 for core image generation processing (“request”, “request ”). However, the activation control of the renderer 221 is performed by the load balancer 215 included in the renderer control module 215. The load balancer 215 selects a node 220 having a low load from the plurality of nodes 220 forming the rendering engine 22, and requests a renderer 221 formed by the selected node to perform a core image generation process.

  The rendering engine 22 is formed of a plurality of nodes 220, and a renderer 221 that performs image generation operates on each node 220. Each renderer 221 can access the DICOM file server 24 and obtain core data (DICOM file) in response to a request from the renderer control module 214. Each node 220 has a local memory 222. The local memory 222 is used as a work area for core image generation processing using core data and a temporary storage area for the generated core image data. The local memory 222 is an example of “memory”.

  The functions of the gateway 21 described above are as follows: an information processing device (computer) including a CPU, a main storage device (main memory), an auxiliary storage device, and a communication interface circuit (communication I / F) (all not shown); For example, it is realized by loading the middleware (program) stored (installed) in the auxiliary storage device into the main storage device and executing it. As the information processing apparatus, for example, a PC or a dedicated or general-purpose server machine can be used.

The rendering engine 22 can be realized by using an information processing apparatus including a plurality of CPUs, a plurality of main storage devices, and one or more auxiliary storage devices. Each of the plurality of nodes 221 included in the rendering engine 220 is realized by one or a plurality of CPUs included in an information processing apparatus that implements the rendering engine 22 and a main storage device used by the one or more CPUs. The CPU or CPUs that implement the node 220 start / stop / delete (delete) the renderer 221 by executing a program stored (installed) in advance in the auxiliary storage device. The local memory 222 included in the node 220 is a predetermined storage area in a main storage device or an auxiliary storage device used by one or a plurality of CPUs that implement the node 220.

In addition, the plurality of nodes 220 included in the rendering engine 22 are formed from a single or plural GPUs (Graphical Processing Units) and a CPU that controls the operation of the single or plural GPUs, instead of the above-described configuration using the CPU. May be. At this time, a plurality of nodes may be allocated to a plurality of processors included in each GPU (a GPU cluster is formed).

<Search process>
Details of the core search process will be described below. The file name of the core data is determined to be created by a combination of “voyage number”, “excavation site information”, “hole number”, “core number”, “bit type”, and “section number”. The voyage number is the number of the voyage project, and the excavation site and hole number are identification information of the core excavation position. The core number is a serial number assigned to each core sample excavated. The bit type is an identification number of the drill that drilled the core sample. The section number is a serial number (core section identification information) assigned to one core sample at intervals of 1.5 m.

  In the search process, six parameters consisting of cruise number, excavation site number, hole number, core number, bit type and section number are selected in order, and the candidates for core data are gradually narrowed down and registered in the search DB 23. The target core data can be identified from the enormous (about 4000) core data.

  FIG. 2 is an explanatory diagram of search processing and image generation processing. The search API 221 receives a core search request (request) from the client 10 (client application) using the HTTP GET method (<1> in FIG. 2). The search API 221 delivers parameters included in the search request to the search module 213 (<2> in FIG. 2).

  The search module 213 generates a query (SQL sentence) based on the received parameters and makes an inquiry to the search DB 23 (<3> in FIG. 2). The search module 213 receives the search result from the search DB 23 and passes it to the search API 211 (<4> in FIG. 2). The search API 211 processes the search result into the JSON format, and transmits it as a return value to the client 10 (client application) (<5> in FIG. 2). The search DB 23 may be a DB stored in the auxiliary storage device of the information processing apparatus that implements the gateway 21, or may be an information processing apparatus (DB server) independent of the information processing apparatus.

The return value varies depending on the parameter transmitted from the client 10 to the search API 211. When a voyage number is transmitted (selected) as a parameter, a set of excavation site numbers having the selected voyage number is returned as a return value. When one excavation site number selected from the set of excavation site numbers is transferred from the client 10 to the search API 211 as a parameter, a set of hole numbers having the voyage number and the excavation site number becomes a return value. In this way, the hole number, core number, bit type, and section number are sequentially passed as parameters, so that one core sample (core section) can be finally specified. A unique ID (referred to as “core ID” or “core number”) for uniquely identifying the core section is allocated to the core section specified by the six parameters, and the search API 211 returns the core ID as a return value. To the client 10. The core ID is an example of “object identification information”.

  When the client 10 receives the core ID, a core image generation request (request) including the core ID is transmitted to the renderer API 212 using the HTTP GET method (<6> in FIG. 2). Thereby, the image generation process in the server 20 is started. Specifically, the request is supplied to the renderer 221 included in the rendering engine 22 via the renderer control module 214 (<7> in FIG. 2). The renderer 221 generates a two-dimensional image of the core according to the request. The drawing result of the two-dimensional image is transmitted to the client 10 via the renderer control module 214 and the renderer API 212 (<8> and <9> in FIG. 2). In this way, a two-dimensional image of the core corresponding to the core ID is generated by the rendering engine 22 and transmitted to the client 10 through the renderer API 212. In this way, the user of the client 10 can view an image of a desired core section.

  FIG. 3A is a table showing parameters delivered to the search API 211 and their meanings, and FIG. 3B is a table showing reception parameters and return values of the search API 211. FIG. 3C shows a specific example of parameters and return values delivered to the search API 211. According to FIGS. 3A to 3C, when a query that does not specify a parameter is first transmitted, a set of searchable navigation numbers (314, 315, 316, 319) is returned to the client 10 as a return value. Passed.

  For example, when the voyage number “315” is selected, a set of excavation site numbers (C0001, C0002) is passed to the client 10 as a return value. For example, when the excavation site number “C0001” is selected, a set of hole numbers (E, F, H) is passed to the client 10 as a return value. After that, when the hole number “E”, the core number “3”, the bit type “H”, and the section number “4” are selected, the core ID of the core section uniquely specified by the six parameters selected so far (Coreid) is passed to the client 10 as a return value.

<Renderer API and renderer control module>
Next, details of the renderer API 212 and the renderer control module 214 will be described. 4A shows parameters of a query (image generation request (request)) transmitted from the client 10 to the renderer API 212, and FIG. 4B is transmitted to the client 10 as a response (response) to the query. Indicates parameters in JSON format. A query including the parameter values shown in FIG. 4A is created in response to a user operation on the client 10.

As shown in FIG. 4A, the query includes the core ID (core number) provided to the client 10 by the search API 211. Further, in the query, the following parameters can be specified as the view information of the core image desired to be viewed. However, not all of the following information is always included in the query, and parameters according to user operations are included in the query.
・ Magnification ratio ・ Vertical cutting angle ・ Resolution ・ Rotation method ・ Rotation angle in xyz direction ・ Core vertical movement ・ Coloring method ・ Array of CT value and color relationship ・ Shows the excessive relationship between CT value Array of pairs, current camera coordinates, current camera up vector, mouse movement vector, rotation angle

On the other hand, as a response to the query, the following information shown in FIG. 4B is returned to the client 10 in the JSON format.
-URL (Uniform Resource Locator) of the generated core image
-CT value histogram array-Length per pixel-Current camera coordinates-Current camera up vector

  The renderer control module 214 receives the above-described query (request) from the renderer API 212, identifies the node 220 that generates the core image corresponding to the query, and gives a dedicated command to the identified node 220. A renderer 221 that performs image generation processing on the identified node 220 is activated. The renderer 221 resides in the node 220 and generates an image of the core sample (core section) specified by the core ID corresponding to the query (request).

  One renderer 220 is activated for one kind of core sample. For example, when image generation of three types of core samples is requested, the three renderers 220 are activated. For this reason, when a query for viewing a plurality of types of core samples is generated, a plurality of renderers 221 are activated on the node 220.

  The renderer 220 reads volume data (two-dimensional slice image data) from the DICOM file server 24, generates a three-dimensional model, and then generates a two-dimensional image (core image provided to the client). By such image processing, a high load is applied to the node. Therefore, the load is distributed to a plurality of nodes, and the concentration of the load on one node is avoided. For this reason, the renderer control module 214 includes a load balancer 215, and load distribution and load reduction of image processing are performed as follows.

  Basically, the load balancer 215 activates the renderers 221 in order for the plurality of nodes 220 by the so-called round robin method. FIG. 5 is an explanatory diagram of load distribution by the round robin method. As shown in FIG. 5, for example, when the rendering engine 22 includes 16 nodes 220, the load balancer 215 performs the operations from the first node (node 1) to the 16th node (node 16). In order, the renderer 221 is activated. When the next request (query) is received, the load balancer 215 gives the start command of the renderer 221 to the first node (node 1) again. In this way, the renderer activation from the node 1 to the node 16 is repeatedly performed.

  However, for example, when the renderer 221 of the node 3 shown in FIG. 5 is deleted by the renderer 221 deletion process (described later), the number of activations of the renderer 221 in the node 3 is one less than that of the other nodes 221. Become. For this reason, the load is not evenly distributed among the nodes. For this reason, the load balancer 215 monitors the number of activations of the renderer 221 in each node 220, and if a node 220 with a low load (the number of activations of the renderer 221) is found, the load balancer 215 has priority over the node 220 Renderer start command is given.

Incidentally, the renderer 221 activated by the renderer control module 214 (load balancer 215) resides in a query (request) standby state when image generation processing is not being performed. Even in this standby state, the renderer 221 occupies a part of the memory (local memory 222) of the node and also occupies a small amount of CPU resources. For this reason, if the number of activated renderers exceeds a certain number, the node may be in a high load state, which may cause delay or failure of image generation processing.

  Therefore, the renderer control module 214 monitors the state of each activated renderer 221 and deletes unnecessary renderers (end processing). For example, when there is a renderer 221 that has not received a request for a certain period of time, the renderer control module 214 gives a delete instruction for the renderer to the node 220.

For example, the renderer control module 214 starts a renderer monitoring program (script) periodically or periodically, and performs determination processing based on the following equation (1) for each renderer.
(Last accessed date / time)-(Script execution date / time)> T (1)

  When a difference (length of time) between the date and time when the last access was made (that is, the date and time when the request was received) and the date and time of script execution exceeds the threshold T, the renderer control module 214 An instruction to delete the renderer 221 is given to the node 220 corresponding to the renderer 221. In this way, the useless renderer 221 is deleted. The threshold T is, for example, 30 minutes, but a desired time such as several minutes to several tens of minutes, or one to several hours can be set.

In addition, the renderer control module 214 further monitors the total number of renderers 221, and when the total number exceeds the predetermined upper limit number N of renderers, the renderer 221 deletes at least one renderer 221. The judgment formula is given by the following formula (2).
(Total number of activated renderers)> N (2)

  When the above equation (2) is satisfied (the total number exceeds the threshold value N), the renderer control module 214 refers to the last access date and time (previously stored) of each renderer 221 and renders the renderer with the oldest last access date and time. A delete instruction for 221 is given to the corresponding node 220.

  The renderer 221 simply enters the request waiting state without performing the core image generation process. When the renderer 221 receives a request, the renderer 221 performs image generation processing (image drawing processing) according to the request. Transmission of a request from the renderer control module 214 to the renderer 221 is performed by socket communication. In addition, transmission of a delete command (end message) to the renderer 221 is also performed by socket communication.

In socket communication, a communication partner is selected by a combination of an IP (Internet Protocol) address and a port number. The plurality of nodes 220 included in the rendering engine 22 have different IP addresses. For this reason, by assigning different port numbers to the nodes in each node 220, it is possible to perform request transmission and end message transmission by accurate socket communication. Note that socket communication is performed by connection-type communication for confirming the normality of communication. For example, TCP is applied.

The activation status of the renderer 221 is recorded by the “activation log” 214A. The activation log 214A is a file that stores various types of information about the activated renderer 221. The activation log 214A is held and managed by the renderer control module 214. When the renderer is activated, the renderer control module 214 adds information related to the renderer 221 to be activated to the activation log 214A. When the renderer is deleted, the renderer control module 214 deletes information on the renderer 221 that is to be deleted. As information about the renderer 221, items of core ID (core number), node number, port number, and access date (time stamp) are recorded. The activation log 214A is stored in the main storage device or auxiliary storage device of the information processing apparatus that functions as the gateway 21.

  The core ID corresponding to the request is described as the core ID. As the core ID, the six parameters specifying the above-described core data may be described as a file path. Further, the node number where the renderer 221 is activated is described as the node number. Further, the port number designated when the renderer 221 is started is described as the port number. The latest date and time when the request is transmitted to the renderer 221 is described as the access date and time.

  Before starting the renderer 221, the renderer control module 214 refers to the core ID described in the start log 214A to check whether the renderer 221 corresponding to the core ID has already been started. If the activation log 214A does not have a core ID, it is determined that the renderer 221 corresponding to the core ID has not been activated, and the renderer control module 214 determines based on the node number reference result described in the activation log 214A. A start command (dedicated start command) for the renderer 221 is given to a certain node 220. At this time, the load balancer 215 determines the node 220 that activates the renderer 221 in consideration of the round robin method described above and the number of renderers activated in each node 220. Further, the renderer control module 214 assigns a port number to the renderer 221 to be activated, and describes the assigned port number in the activation log 214A.

  When the renderer 221 is activated, the renderer control module 214 transmits a core image generation request to the renderer 221 specified by the node number and port number described in the activation log 214A by socket communication. Each time a request is transmitted to a certain renderer 221, the last access date and time of the renderer in the activation log 214A is updated.

  In the activation log 214A, the last access date and time of each renderer is recorded. For this reason, the renderer control module 214 refers to the activation log 214A in periodic or periodic renderer monitoring, and issues a delete command (end message) for the renderer 221 whose difference (see equation (1)) exceeds the threshold T. can do. In addition, the renderer control module 214 can grasp the total number of renderers 221 that are activated from the description of the node number and port number in the activation log 214A. When the total number exceeds the threshold value N, the renderer control module 214 performs a deletion process on the renderer 221 having the oldest access date and time.

<Rendering engine>
Next, details of the rendering engine 22 (renderer 221) will be described. The renderer 221 obtains the DICOM file corresponding to the core ID in the request from the DICOM file server 24.

  The DICOM file server 24 is a DB server that stores image data (core data) for each core section stored in the DICOM format. One cross-sectional image obtained by the X-ray CT scan is called “slice (image)”, and a set of a plurality of slice images obtained for one core section is used as core data corresponding to one core ID. , Obtained from the DICOM file server 24 for 2D image generation.

A DICOM format image file is composed of metadata and image information. The metadata includes information about the shooting target. The image information includes information necessary for representing the photographed object as an image. Also, in the DICOM format image file, the pixel value of the image corresponds to the CT value. The CT value is a value expressing the degree of X-ray absorption. The CT value takes a value correlated with the density and the element number. Specifically, the higher the density and the element number, the higher the CT value, and the lower the density and the element symbol, the lower the CT value. Using the property that the pixel value of the image corresponds to the CT value. By assigning different colors to pixels having different CT values, an object corresponding to a certain CT value can be highlighted.

The rendering engine 22 generates a two-dimensional image that the user of the client 10 desires to view according to the following procedure.
1) Initialization processing 2) Reception of drawing parameters 3) Drawing processing 4) Transmission of drawing results

<< 1. Initialization processing >>
The renderer 221 acquires, from the DICOM file server 24, a plurality of DICOM files (slice images corresponding to the core ID) specified by the path (file path of the core ID) given as an argument at the time of activation. As described above, the DICOM file includes metadata and image information (image data).

FIG. 6 shows metadata used for image generation. “I _NUM ” shown in the table of FIG. 6 means what slice image the DICOM file corresponds to. The renderer 221 sorts a plurality of slice images based on the value of “I _NUM ” and creates volume data (a set of slice images).

  The created volume data is retained by the renderer 221 process until the renderer 221 is terminated (deleted). That is, the volume data is stored in the local memory 222 used by the renderer 221 for drawing. With such temporary storage of volume data, when a new request regarding the same core ID is received, generation of a two-dimensional image using the created volume data can be started. In this case, the response time to the client 10 can be shortened compared to the case of newly creating volume data.

<< 2. Receiving drawing parameters >>
The renderer 221 assigns a port number designated at the time of activation to a socket and waits for a connection (request transmission) from a communication partner (renderer control module 214). When the renderer 221 receives a request, the renderer 221 starts drawing based on information on viewpoint movement, image quality, and image storage destination included in the request.

<< 3. Drawing process >>
In the rendering process, the renderer 221 performs the following process in order to generate a two-dimensional image of the core.
(A) Determination of viewpoint (b) Compression of volume data (c) Color setting for CT value (d) α value setting for CT value (e) Longitudinal cutting

FIG. 7 is a diagram for explaining a rendering flow in the drawing process. First, the renderer 221 converts the volume data acquired in the initialization process (read into the local memory 222) into three-dimensional (3D) texture data. Subsequently, the renderer 221
Volume rendering is realized by performing texture mapping using 3D texture data on a plurality of rectangular polygons.

Specifically, the renderer 221 uses the metadata shown in FIG. 6 to calculate the length and width and the spacing of the rectangular polygons, and maintains the ratio of the height, width and height in the actual core section. The length of the sides of the rectangular polygon is the width P _W · S _W and the height P _H · S _H using the number of pixels and the pixel interval. The renderer 221 arranges a plurality of rectangular polygons so that the z-axis passes through the center of the rectangular polygon and the z-axis and the rectangular polygon are orthogonal to each other. At this time, the slice thickness T is applied as the interval between the rectangular polygons. The coordinates of the default (initial state) viewpoint are on the y-axis, and are directed from the + direction of the y-axis toward the origin. Note that the upward direction of the viewpoint coincides with the + direction of the z axis.

(A) Determination of viewpoint In order to view the core from a desired direction, two types of viewpoint calculation methods are employed. The first is the determination of the viewpoint by the Euler angle, and the second is the determination of the viewpoint by the quarteranion. Euler angle is a viewpoint calculation method by specifying the rotation angles of the x, y, and z axes. FIG. 8 is a diagram for explaining the positional relationship between the core and the viewpoint. Since the core is cylindrical, viewing of the side surface, top surface, and bottom surface is considered. By rotating along the axis of the core, the side surface of the core can be viewed. On the other hand, it is possible to view the bottom surface by rotating in the front-rear direction. For this reason, the default viewpoint is set from + on the y axis to the origin, and the rotation is applied in the order of z axis rotation (core axis rotation) and x axis (back and forth rotation).

Quartanion is a viewpoint calculation method by specifying a rotation axis vector and a rotation angle. In order to view the core, it is sufficient to consider the above-mentioned two axes. However, it is difficult for the user to determine where the viewpoint moves when the three axes are designated by adding the y axis. The solution is the determination of the viewpoint by the quarteranion, which assumes a rotation operation using a mouse drag on the image plane. The rotation axis vector a by this operation method is a straight line that is orthogonal to the current line of sight and the movement vector of the mouse and passes through the origin in the drawing space of the core. The unit vector from the current viewpoint position to the origin is set as e = (ex, ey, ez) ^T , the upward vector from the viewpoint u = (ux, uy, uz) ^T , and the mouse movement vector in the image plane as m = (mx, my, 0) ^T , where θ is the angle between the mouse movement vector and the x axis, the rotation axis vector is a = (ax, ay, az) ^T
Is represented by the following equation.

  A new viewpoint is determined by applying coordinate transformation about a to the current spatial coordinates. FIG. 9 shows an example of rotation using viewpoint determination based on a quota anion. The renderer 221 determines the viewpoint of the core image to be generated based on the parameters for determining the viewpoint as described above.

(B) Compression of Volume Data The size of the slice image stored in one DICOM file of core data is 512 pixels wide and 512 pixels high, and 16 bits per pixel. The number of DICOM files per core section is approximately 400-2500. For this reason, the volume data is large and has a data size exceeding 1 Gbyte. The volume data is compressed to shorten the drawing process time.

In this embodiment, the resolution of the volume data is prepared in five levels (data amount 1/1, 1/8, 1/64, 1/216, 1/512), and the resolution selected by the user according to the quality and response speed To generate an image. An average pixel method is used as a compression method. In the average pixel method, for example, when 3 * 3 * 3 voxels are compressed into 1 * 1 * 1 voxels, the average value of the original voxel values is used as a new voxel value. By applying the average pixel method, roughness can be suppressed as compared with a compression method in which voxels are simply thinned out.

(C) Setting of color and transmission for CT value A heat map is used for coloring the color corresponding to the CT value. The heat map is a map for mapping a low value to blue and a high value to red. For example, the gradation is blue → light blue → green → yellow → orange → red. The range of CT value can take the value of -1000-4000, for example. However, a substance having a value of 0 or less is a liquid or gas having a density lower than that of water, and is usually hardly contained in the core sample substance. For this reason, the range of CT values used for core image creation is 0 or more and 4000 or less, and RGB values are calculated using the CT values v (vmin ≦ v <vmax) as follows: And 4.6, and can be represented by a function as shown in FIG.

  Further, in the present embodiment, not only the coloring is linear with respect to the CT value, but also the user can change the coloring by a function using cubic spline interpolation. FIG. 11A is an explanatory diagram when the CT value changes linearly with respect to a color gradation (heat map), and FIG. 11B shows these points according to two points designated by the user. An example in which the CT values are colored by a function obtained by interpolating between them by a cubic spline is shown. The renderer 221 determines a color for the CT value of each pixel using information (included in the request) indicating a curve (function) designated by the user, and performs coloring for the pixel (voxel).

(D) Setting of α value for CT value The client application specifies a set of CT values and α values (transparency), and generates an α value function by linearly interpolating the point sequence. For example, by using a function of the α value as shown in FIG. 12, a voxel having a low CT value covering the surface of the core is transmitted, and the inner structure can be viewed. The renderer 221 determines the transparency of each pixel (voxel) using information (included in the request) indicating a function of the α value designated by the user.

(E) Longitudinal cutting Longitudinal cutting is realized by using, for example, an OpenGL clipping function. The clipping process does not draw an area that satisfies the following expression 4.7. Longitudinal cutting can be achieved by specifying a plane that is horizontal to the z axis and passes through the origin. The equation of the plane can be obtained by the following equation 4.8 using the angle θ. FIG. 13 shows an example in which longitudinal cutting is applied.

  As described above, the renderer 221 performs volume rendering on a plurality of slice images (volume data) corresponding to the core ID obtained from the DICOM file server 24, and generates three-dimensional model data. Furthermore, the renderer 221 performs viewpoint determination of the three-dimensional model, volume data compression, coloring for CT values, and setting of α values based on the above-described processes (a) to (d). Further, the renderer 221 performs the vertical direction longitudinal processing shown in (e) in response to the request. Then, the renderer 221 generates a two-dimensional color image according to the determined viewpoint, the color for the CT value, and the α value as a two-dimensional core image. The two-dimensional core image is generated, for example, by writing the two-dimensional image of the core section when the three-dimensional model is projected onto the projection plane based on the designated viewpoint (view) to a predetermined video memory (video RAM). Is done.

Then, the renderer 221 transmits a response (response) including the parameters shown in FIG. 4B to the designated transmission port (renderer control module 214) by socket communication. The response includes the URL (position information) of the two-dimensional core image data, the current viewpoint position vector, the upward direction vector of the viewpoint, the CT value histogram, and the length (mm) per pixel. Further, the response can include an OK or NG character string indicating whether or not the drawing has been completed normally. The histogram can be obtained by calculation when the volume data is compressed. 1 length s per pixel is calculated the length of the near clip plane of the view volume L, and the number of pixels per side of the generated image by 4.9 formula as W _W.
s = 2L / W _W · P _W · S _W (4.9)

  The renderer control module 214 passes the rendering result parameters to the renderer API 212, and the renderer API 212 transmits the rendering result to the client 10 in the JSON format.

  In the rendering process described above, the renderer 221 generates a three-dimensional model of the core section by volume rendering. The three-dimensional model data is temporarily stored in the local memory 222 until the end of the renderer 221 as with the volume data. On the other hand, when the renderer control module 214 receives a new request, the renderer control module 214 can identify the active renderer 221 corresponding to the request by referring to the activation log 214A. Requests can be sent. When the renderer 221 receives a new request, the renderer 221 uses the three-dimensional model data stored in the local memory 222 to perform the processes (a) to (e). As a result, access to the DICOM file server 24 and creation of volume data are omitted, so that two-dimensional core image data can be generated in a short time. That is, the response time can be shortened.

〔client〕
The client 10 includes, for example, a Web browser 11 that manages a viewer function as a client application. The Web browser 11 performs communication (for example, HTTP communication) to the server 20 based on HTTP GET, and transmits a provision request for core image data that the user of the client 10 desires to browse to the server 20. In addition, the Web browser 11 receives a file including core image data generated by the server 20 in response to a request for providing core image data, and displays a core image based on the file on a display. The file including the core image data is provided as a JSON (JavaScript Object Notation) file, for example.

  The client 10 includes a CPU, a main storage device, an auxiliary storage device, a communication I / F, an input device (keyboard, mouse (pointing device)), and a display device (none of which are shown). When the CPU loads and executes the Web browser 11 stored (installed) in the auxiliary storage device to the main storage device, the following functions of the Web browser 11 (client application) are realized.

  The client 10 accesses the Web site provided by the server 20 via the network, thereby performing a core image search, a core image browsing, and various operations on the core image as shown in FIG. Can be displayed on the display device.

  On the left side of the Web page 100, a user interface (UI) 101 for core search is laid out. In order from the top, there are provided columns for inputting or selecting six parameters including a voyage number, a drilling site number, a hole number, a core number, a bit type, and a section number. For example, when a button corresponding to a voyage number is pressed, a pull-down menu showing voyage number candidates is displayed. When any of the candidates is clicked, the clicked voyage number is entered and a search query including the voyage number is stored in the server. 20 is transmitted. As a result, excavation site candidates are displayed in a pull-down menu corresponding to the excavation site number in a selectable manner. In this way, by narrowing down the parameters after the excavation site number, the client 10 can receive the core ID (core number) desired to be browsed from the server 20.

  When the core ID is acquired, an image generation query (request) including the path of the core ID is transmitted from the Web browser 11 to the server 20. The server 20 generates a three-dimensional model by volume rendering from a plurality of DICOM files (slice images) corresponding to the core ID by the above-described method, and sets the default viewpoint, resolution, coloring, and α value for the three-dimensional model. Accordingly, a two-dimensional image is generated, and a response to the request is returned from the server to the client 10.

  The response includes the URL of the generated two-dimensional core image, and the Web browser 11 acquires the two-dimensional image data by automatic access to the URL, and the core image based on the two-dimensional image data is acquired. And displayed in a display area 102 provided in the center of the Web page 100.

  On the right side of the Web page 100, a UI for performing various settings for an image is provided. Each time the UI is operated, an image generation request (query) including parameters corresponding to the operation result is transmitted to the renderer API 212, and image generation corresponding to the request is performed. Then, the image generation result (two-dimensional image) is transmitted to the client 10, and the two-dimensional image reflecting the operation result is displayed in the display area 102.

Specifically, a resolution setting field 103 and a color map selection field 104 are provided on the upper right side. The user can specify the resolution and the color map using the input device and the setting column 103 and the selection column 104. Below the resolution setting field 103, a move button 104 for the core image in the display area 102 is provided, and the core image can be moved upward and downward in the display area 102. An input field 105 for setting the rotation angle of the core image is provided on the right side of the move button 104. By inputting the rotation angle in the input field 105, for example, the core image is rotated around the z axis. be able to.

  A slide bar 106 for controlling the enlargement / reduction of the core image is provided on the right side of the input field 105, and the size of the core image can be displayed at a desired size by moving the slide bar. it can. Further, an input field (UI) 107 for cutting in the vertical direction is provided at the lower stage of the slide bar 106. By specifying the cutting plane of the core using the input field 107, the core image that has been subjected to the vertical cutting process can be displayed in the display area 102.

  Further, a CT value and color map setting screen (UI) 108 is provided below the input field 107. On the setting screen, a histogram of CT values at each pixel (voxel) of the core image is displayed. The user can specify the CT value range to be colored by operating the setting screen 108. Furthermore, a function indicating a correlation between a color map (for example, a heat map) and a CT value can be designated on the setting screen 108. For example, the default alignment can be specified, or a function by cubic spline interpolation can be specified by specifying a plurality of points.

  A UI 109 for defining a relationship between transparency (transparency), that is, an α value and a CT value is provided at the lowermost stage on the right side of the Web page 100. By setting an α value (0 ≦ α ≦ 1), it is possible to browse the internal structure of the core (CT value distribution inside the core).

<Effect of embodiment>
According to the server 20 of the embodiment, two-dimensional core image data having a data capacity smaller than that of the DICOM file is transmitted to the client 10. As a result, a system (service) that does not depend on the performance of the terminal forming the client 10 and can be used even on a low-speed line can be provided to the user of the client 10.

  In particular, according to the server 20, the core ID and the renderer 221 are associated with each other on a one-to-one basis, and requests related to the core ID are supplied to the same renderer 221. Accordingly, when the user of the client 10 operates each UI indicated by reference numerals 103 to 109, a query (drawing request) corresponding to the operation result is sent to the server 20.

  In the server 20, as described above, the renderer 221 associated with the core ID in the activation log 214A generates a two-dimensional image (core image) using the three-dimensional model temporarily stored in the local memory 222. The core image is provided (distributed) to the client 10. By using such three-dimensional model data, the response time to the query can be shortened, and an appropriate survey environment using the core image for the user can be provided.

In FIG. 14, the Web browser-based UI has been described. However, for example, a tablet terminal-based viewer and UI may be provided to the user by the client 10. In this case, for example, a core data (core section) search screen, a core image display screen, a longitudinal section setting screen, a CT value coloring and α value setting screen are displayed on the display as individual screens. . Further, on the core image display screen (browsing screen), the core image may be enlarged or reduced in accordance with a finger operation (pinch-in, pinch-out operation) using a touch panel provided in the client 10. In addition, the core image may be rotated or moved in response to a finger operation on the touch panel on the browsing screen.

  In the embodiment, the core is exemplified as an example of the object to be X-ray CT scanned, but the target object is not limited to the core, and various objects including medical (biological tissue) have been described in the embodiment. The system can be used. For example, the object can include a rock sample or a sediment sample. The rock sample and the sediment sample include the above-described core. However, in the above description, the “core” is described as “a columnar geological sample excavated from the seabed”, but the core is a broad concept indicating a columnar geological sample, and the collection location is not limited to the seabed. Including those collected by excavating underground and riverbeds.

10 ... Client 11 ... Web browser (viewer)
20 ... Server 21 ... Gateway 22 ... Rendering engine 211 ... Search API
212 ... Renderer API
213 ... Search module (search means)
214 ... Renderer control module 214A ... Start log (storage unit)
215 ... Load balancer 220 ... Node 221 ... Renderer 222 ... Local memory (memory)

Claims (4)

A search module for searching for an object to be browsed in response to a search request from a client received via a network;
In response to an image generation request including identification information of an object searched by the search module, a three-dimensional model can be generated from a plurality of slice images obtained by X-ray CT scan of an object corresponding to the identification information. A plurality of renderers capable of generating a two-dimensional image to be transmitted from the three-dimensional model to the client through the network;
A memory for temporarily storing a three-dimensional model generated by each renderer;
A storage unit that stores the correspondence between the identification information of the object and the renderer,
When a new image generation request is received, the new image generation request is supplied to a renderer corresponding to the object identification information included in the new image generation request based on the correspondence relationship. A server system that generates a two-dimensional image corresponding to a new image generation request using a three-dimensional model temporarily stored in a memory. The server system according to claim 1, further comprising: a load balancer that determines a renderer for generating a three-dimensional model corresponding to the identification information of the object from the plurality of renderers. The server system according to claim 1, wherein the object includes a rock sample or a sediment sample. Searching for an object to be browsed in response to a search request from a client received over a network;
In response to an image generation request including identification information of the searched object, one of the plurality of renderers generates a three-dimensional model from a plurality of slice images obtained by X-ray CT scan of the object corresponding to the identification information. And generating a two-dimensional image to be transmitted from the three-dimensional model to the client through the network;
Temporarily storing the three-dimensional model generated by the renderer;
Storing the correspondence between the identification information of the object and the renderer;
When a new image generation request is received, a new image generation is performed using a three-dimensional model in which a renderer corresponding to the object identification information included in the new image generation request is temporarily stored based on the correspondence relationship. A program for causing a computer to execute the step of supplying the new image generation request to the renderer in order to generate a two-dimensional image corresponding to the request.

Images