JP2015225382A - Drawing system, drawing device, calculation device, drawing program, and drawing processor board - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a drawing system, a drawing device, a calculation device, a drawing program, and a drawing processor board for artificially synchronizing a high-load process for which real-time processing is difficult to a low-load process in a drawing process.SOLUTION: The drawing system comprises: calculation means for obtaining the amount of movement in a virtual space between a first high-speed drawing result obtained by a high-speed drawing process in which drawing is made at a time most recent to the drawing completion time of a first low-speed drawing result obtained by a low-speed drawing process, and a second high-speed drawing result obtained by a high-speed drawing process for drawing immediately after that; prediction means for obtaining, from the first low-speed drawing result and the amount of movement, a second low-speed drawing result, as a predicted low-speed drawing result, that is predicted to be obtained by the low-speed drawing process corresponding to the time most recent to the drawing completion time of the second high-speed drawing result; and display output means for outputting a first display image on the basis of the first low-speed drawing result and the first high-speed drawing result and then outputting a second display image on the basis of the predicted low-speed drawing result and the second high-speed drawing result.

Description

  The present invention relates to a drawing system for executing drawing means having a large amount of calculation in real time in computer graphics (CG).

  With the increase in performance of computing devices such as personal computers (PCs) and tablet terminals, the computation in the CG drawing means has increased in complexity and the amount of computation has become enormous. In recent years, an increasing number of systems are capable of performing high-quality server devices to perform drawing processing and transmitting images to a client device having a display device, so that high-quality images can be viewed even by non-high-performance devices.

  As described in Patent Document 1, in a system that transfers video drawn from a server device to a client device via a network, to provide high-quality CG video without depending on the drawing performance of the client device. There is also a method in which drawing processing is performed on the server device and a screen is provided to the client terminal.

JP 2013-99494 A

SCHERZER, D., YANG, L., MATTAUSCH, O., NEHAB, D., SANDER, PV, WIMMER, M., and EISEMANN, E., 2011, "A survey on temporal coherence methods in real-time rendering, "In Euro graphics State of the Art Reports. (3.2 Forward reprojection)

As in the above-mentioned Patent Document 1, by making a computing device with high processing capacity such as a server device bear a high load part of drawing processing, the computing load on the client device can be reduced, and drawing of the client terminal Although it becomes possible to draw a high-quality image without depending on performance, there is a drawing means that cannot be used depending on the drawing performance of the server device.
For example, drawing means with a large amount of calculation such as global illumination that accurately handle the reflection behavior of light in the virtual space can be calculated relatively quickly with a high-performance computing device. Real-time (real-time: Here, it is used as an expression indicating a drawing interval at which the viewer does not feel uncomfortable, for example, a drawing interval at which drawing is performed 30 to 60 times per second. )) Can not be done.

As described above, when drawing is performed using a drawing means having a high calculation load, if the server device performs processing with a high calculation load, the drawing interval of the server device catches up with the drawing interval required by the client device. As a result, the drawing update of the client device is delayed.
For example, as shown in FIG. 2, when the drawing timing at the server device is S100 to S300 and the drawing timing at the client device is C100 to C900, the drawing result is generated at S100 and C100 at the same time, and then until S200 and C500. In the meantime, in the client devices C200 to C400, since the drawing process is continued in the server device, the drawing result cannot be generated at the same drawing timing as the client device.
If the drawing interval of the client device is 30 FPS (FPS: the number of drawing operations performed per second) and the drawing interval of the server device is 10 FPS, one frame (frame: in this case, of the drawing intervals) (This also applies to the portion described later.) During drawing, the client device draws 3 frames, and a difference occurs. As a result, there is a problem that it is necessary to adjust the drawing interval of the client apparatus to 10 FPS because the necessary images are not prepared and to be adjusted to a slow drawing interval.

  An object of the present invention is to use a drawing means having a huge amount of calculation that cannot be drawn in real time as a high-load drawing process when the drawing process is divided into a high-load drawing process and a low-load drawing process. However, it is to provide a technique for displaying in real time in a pseudo manner.

  In order to solve the above problems, one aspect of the present invention is a drawing system that draws a screen in a virtual space, and includes drawing means that performs high-speed drawing processing and low-speed drawing processing, and low-speed drawing processing. The first high-speed drawing result obtained by the high-speed drawing process drawn at the time closest to the drawing completion time of the obtained first low-speed drawing result and the drawing immediately after the first high-speed drawing result are performed at high speed. The calculation means for obtaining the movement amount in the virtual space between the second high-speed drawing result obtained by the drawing process and the drawing completion time of the second high-speed drawing result closest from the first low-speed drawing result and the movement amount The second low-speed drawing result that will be obtained by the low-speed drawing process corresponding to the time of the time, and the prediction means for obtaining the predicted low-speed drawing result by the time, the first low-speed drawing result and the first The first table based on the high-speed drawing results After outputting the image, and a display output means for outputting a second display image based on the result of the prediction slow rendering result and a second high-speed drawing.

  Another aspect of the present invention is a drawing apparatus that constitutes the above-described drawing system, and includes one or more first calculation apparatuses that execute low-speed drawing processing, the first apparatus, and a network. And at least one or more second computing devices that are connected and execute high-speed drawing processing.

  Another aspect of the present invention is a computing device that constitutes the above-described drawing system, and includes one or more first drawing processors that execute low-speed drawing processing and one that executes high-speed drawing processing. At least the second drawing processor described above is mounted.

  Another aspect of the present invention is a drawing program that causes one computer to function or two or more computers to function in cooperation as the above-described drawing means, calculation means, prediction means, and display output means.

  Another aspect of the present invention is a drawing processor base in which the functions of the above-described drawing means, calculation means, prediction means, and display output means are integrated.

  According to the present invention, even if a drawing means having an enormous amount of calculation that cannot be drawn in real time can be used, it can be drawn in real time in a pseudo manner.

Explanatory drawing of the example of a structure of the drawing system in connection with embodiment Explanatory drawing about the subject when the drawing interval of the server apparatus (100) and client apparatus (200) shown in FIG. 1 differs. The flowchart explaining the detail of a process until one frame displayed on a display apparatus is drawn in the client apparatus (200) shown in FIG. The flowchart explaining the detail of a process until one frame displayed on a display apparatus is drawn in the server apparatus (100) shown in FIG. 2 is a diagram illustrating a method for matching the drawing interval of the server device (100) with the drawing interval of the client device (200) in order to solve the problem illustrated in FIG.

  Exemplary embodiments of the present invention are described below with reference to FIG. The embodiments listed below are not intended to identify essential features of the invention, nor are they intended to determine the scope of the claims.

  FIG. 1 is a system configuration diagram in which a drawing system according to an embodiment of the present invention is constructed by two computing devices. In this embodiment, a high-performance server device (100) and a client device (200) having a display device are connected via a network (300). The drawing processing of the image displayed on the display device is performed by the server device (100) and the client device (200), and the server device (200) among them performs the drawing processing with a high load, and the client device (200) The apparatus (200) performs processing with a low load so that drawing calculation can be performed in real time.

  The client device (200) can be an existing computing device such as a desktop PC, a notebook PC, a smartphone, or a tablet terminal. It is assumed that the client device (200) has a performance capable of drawing CG in accordance with standard specifications represented by OpenGL or the like which is a CG interface standard at a minimum. The high-performance computing server (100) may be an existing high-performance PC and server device (100) such as a workstation, a server cluster, a mainframe, and a cloud server. There may be two or more client devices (200) and server devices (100).

  In the above description, the processing form in the client-server type system configuration has been described. However, it is naturally possible to perform the same processing with only one client device without using the server device. For example, there is a method in which a plurality of drawing calculation chips such as a graphic board are mounted in the client device, and each drawing calculation chip shares a drawing process with a high load and a drawing process with a low load.

Hereinafter, exemplary details of the present invention will be described with reference to FIGS. 3 and 4. FIG. FIG. 3 and FIG. 4 respectively explain processing until one frame displayed on the display device is drawn in the client device (200) and the server device (100). The processing in FIGS. 3 and 4 is the scene displayed on the display device (scene: here means the entire virtual three-dimensional space displayed on the screen. The same applies to the parts described hereinafter. .) Is repeated until the change of arrangement and state is completed.
In addition, the Example given below is an example of the technique for performing drawing, and is not limited to this.

First, the processing of the client device (200) will be described. In F101, when the client device (200) is operated by the viewer and an operation for changing the scene displayed on the display device of the client device (200) occurs, the processing is started. At this time, the operation for changing the scene includes operations such as changing the brightness and position of the light source, changing the position of the object, and changing the camera position. At this time, when one or a plurality of operations are performed and it is necessary to change the displayed scene, the process moves to step F102.
In the client device (200), the display screen is repeatedly updated while waiting until an operation from the viewer is received.

  In F102, based on the operation information received in F101 as information for updating the scene displayed on the display device, the scene information is changed (such as moving the coordinates of the light source or moving the coordinates of the object). Update geometry information (light position, object position, etc.).

  In F103, in order to draw the changed scene, an image generation command is transmitted from the client device (200) to the server device (100). At this time, the scene update information of the light source / object / camera is transmitted together. (The transmitted information is received in step F201 of the server apparatus (100).)

  In F104, the behavior of multiple reflections of light in the scene transmitted from the server apparatus (100) through the network (300) is expressed by calculation, and light map texture data depicting the brightness of light hitting an object is received. .

  In F105, using the light map texture received in F104, the image that becomes the next frame of the screen displayed on the display device is drawn (object drawing, shadow calculation, etc.), and the display device screen is displayed. Update.

  In F106, the drawing process for one frame displayed on the display device is terminated. (If the change of the scene displayed on the display device has not been completed, the process returns to step F101 to start the drawing process for the next frame, and this process is repeated until the change of the scene is completed.)

Next, processing of the server device (100) will be described. In the server apparatus (100), a calculation process for generating a light map texture by a low calculation load prediction generation process (F200) for predicting and generating a light map texture used in the current frame and a high load multiple reflection calculation process. Two processes of a high-load drawing process (F220) are processed simultaneously.
Note that when the light map texture is generated in F222, if the reflection calculation of the entire scene is performed, the calculation becomes very large. Therefore, only the calculation in the range (screen space) displayed on the display device is performed. To do.

First, in F201, processing is started when the client device (200) generates a change in the scene displayed on the display device and receives the image generation command and the scene update information sent in step F103. To do.
In the server apparatus (100), the processes of F200 and F220 continue to wait until an image generation command is received from the client apparatus (200).

  In F202, a low-cost scene is generated from the scene update information received from the client device (200) (drawing is performed at a drawing interval equivalent to that of the client device (200) with minimum processing that does not calculate shadows or the like). (High-speed drawing processing). The image generated here is used in F203.

In F203, from the low-cost scene image generated in F202 and the low-cost scene image drawn in the previous frame, how much the moving amount is within the drawing range of the scene compared to the previous frame Calculate if there was.
Note that the amount of movement of the frame calculated here is used to predict the light map texture of the current frame in F203.

In F204, the light map texture to be drawn in the current frame is predicted from the movement amount of the frame calculated in F203 and the light map texture generated in F222 in the previous frame or generated in F204. Generate.
Regarding the drawing frame rate, the drawing process performed in the client apparatus (200) is a process that can perform drawing calculation in real time. Therefore, the drawing frame rate in the server apparatus (100) is compared with the frame rate in the client apparatus (200). A delay occurs. In order to match the drawing frame rate of the server apparatus (100) with the drawing frame rate of the client apparatus (200), as shown in FIG. 5, the client apparatus until the next frame is drawn by the server apparatus (100). It is necessary to generate a light map texture with a frame at the same timing as the frame drawn in (200).

In Non-Patent Document 1, the image drawn in the current frame is compared with the image drawn in the previous frame, the movement amount of the entire drawn scene is calculated, and the shadow information drawn in the previous frame, etc. There is a method of predicting and generating an image such as shadow information necessary for the current frame from an image. As shown in FIG. 5, as in Non-Patent Document 1, the server apparatus (100) renders a low-cost scene image at the rendering timing of L010 to L090, which is the same as the rendering timing of the client apparatus (200), and The amount of movement is calculated from the comparison between the low-cost scene image and the low-cost scene image of the previous frame, and during the period from S100 to S300 that cannot be drawn at the same timing as the drawing timing of the client device (200) A light map texture may be predicted and generated.
For example, when generating a light map texture to be drawn in the frame of S201 to match the drawing timing of C600 in FIG. 5, the low-cost scene image drawn in L050 is compared with the low-cost scene image drawn in L060, The amount of movement in the rendered low-cost scene is calculated, and the light and shadow that are considered to have moved at the timing of S201 according to the method of Non-Patent Document 1 are predicted from the light map texture drawn in S200. Generate a lightmap texture. Similarly, from the amount of movement in the scene of the low-cost scene image drawn in L060 and the low-cost scene image drawn in L070, the light in the frame of S202 is based on the light map texture generated in S201. Generate a map texture. By generating and predicting as described above from the rendering in S200 to the rendering in S300, the rendering timing of the server device (100) is simulated with the rendering timing of the client device (200). Can be matched.

  If the camera position has been moved in F101, information outside the range displayed on the display device may be required. In that case, when generating the light map texture predicted in F302, information outside the range of the light map texture drawn in the previous frame is necessary, and there is a possibility that the light map texture cannot be generated. For this reason, when the camera position is moved in F101, it is possible to cope by drawing and generating the range of the light map texture generated in F204 and the range of the light map texture generated in F302 in a range larger than the screen space.

  In F205, the light map texture generated in F204 or F222 is transmitted to the client apparatus (200). (The transmitted light map texture is received in step F104 of the server apparatus (100).)

  In F206, the drawing process of the light map texture used for lighting is finished by drawing for one frame displayed on the display device of the client device (200). (If the change in the scene displayed on the display device has not been completed and an image generation command has been sent, the light map texture drawing process is performed again from step F201, and this process is continued until the change in the scene is completed. repeat.)

  In F211, it is determined whether the lighting calculation is continued or completed in the process of F220. If the lighting calculation is in progress, the process moves to step F203. If the lighting calculation is completed, the process moves to step F223. To do.

  In F221, after the calculation result is transmitted in the process of F223, the lighting calculation of the next frame is started.

In F222, from the scene update information of the light source / object / camera received from the client device (200), the behavior of diffuse reflection and specular reflection of the multi-reflecting light affecting the scene is calculated and stored in the texture image. Then, a light map texture is generated (slow drawing process). Since the processing here is a high-load drawing process that does not end before the processing executed in one frame of the client device (200) ends, it matches the processing time of one frame in the client device (200) For the purpose, it is executed concurrently with the processing of F200. (Images of frames that cannot be generated by the process of F220 are generated by the process of F200, so that the drawing timing of the client apparatus (200) is matched.)
The light map texture generated here is not only transmitted to the client apparatus (200) but also used to generate a light map texture of the current frame in F204.

  In F223, after receiving the image generation command from F221, the calculation result generated in F222 is moved to the step of F205 and transmitted to the client apparatus (200). Thereafter, the lighting calculation is started again in step F221. (The light map texture drawing process is repeated from the step of F221 until the change of the scene displayed on the display device is completed. If there is no change of the scene and no image generation command is received, the processing of F220 Wait until you get out.)

  As described above, by combining the drawing means divided into a plurality of steps that can share the drawing process and the technique for predicting and generating the next drawn image, the drawing means that cannot be drawn in real time can be obtained. By using this, even a computing device that cannot perform high-load calculation processing can perform pseudo-real-time drawing. Note that some or all of the functions and processing contents described above can be executed by a program on one or more computers each having one or more processors, or on a circuit board including a drawing processor. It is also possible to accumulate. Further, the present invention can be appropriately applied not only to the generation of a light map texture in drawing a CG image but also to various arithmetic processes.

  The present invention is useful for a calculation device such as a personal computer (PC) or a tablet terminal that displays a CG image.

DESCRIPTION OF SYMBOLS 100 Server apparatus 200 Client apparatus 300 Network S100-S300 Server apparatus drawing timing C100-C900 Client apparatus drawing timing F101 Client apparatus processing F102 State update F103 Drawing command and update information transmission F104 Light map texture reception F105 Frame Drawing and update of display screen F106 End of processing of client device F200 Light map texture prediction generation processing of current frame in server device F201 Reception of drawing command and update information F202 Generation of low-cost scene F203 Calculation of moving amount of frame F204 Current Prediction / generation of lightmap texture for frame F205 Transmission of lightmap texture F206 Lightmap texture of current frame Completion of prediction generation processing F211 Completion of lighting calculation F220 Calculation processing of high-load multiple reflection in the server device F221 Start of lighting calculation of the next frame F222 Lighting calculation F223 Transmission of calculation results L010 to L090 Drawing of low-cost scene image Timing S101-103, S201-S203 Light map texture generation timing generated by prediction

Claims (5)

A drawing system for drawing a screen in a virtual space, wherein drawing means for executing high-speed drawing processing and low-speed drawing processing;
A first high-speed drawing result obtained by the high-speed drawing process drawn at a time closest to a drawing completion time of the first low-speed drawing result obtained by the low-speed drawing process, and the first high-speed drawing result Calculation means for obtaining a movement amount in the virtual space between the second high-speed drawing result obtained by the high-speed drawing processing for the next drawing;
From the first low-speed drawing result and the movement amount, the second low-speed drawing result that will be obtained by the low-speed drawing processing corresponding to the time closest to the drawing completion time of the second high-speed drawing result. Predicting means for predicting by the time as a predicted low-speed drawing result,
After outputting a first display image based on the first low-speed drawing result and the first high-speed drawing result, a second display is performed based on the predicted low-speed drawing result and the second high-speed drawing result. A drawing system comprising display output means for outputting an image. A drawing apparatus constituting the drawing system according to claim 1,
One or more first computing devices for executing the low-speed drawing process;
A drawing apparatus comprising at least one or more second computing devices connected to the first device via a network and executing the high-speed drawing processing. A computing device constituting the drawing system according to claim 1,
One or more first rendering processors that perform the slow rendering process;
A computing device including at least one or more second drawing processors that execute the high-speed drawing processing.   A drawing program that causes one computer to function or two or more computers to function in cooperation as the drawing means, calculation means, prediction means, and display output means according to claim 1.   A drawing processor base in which the functions of the drawing means, calculation means, prediction means, and display output means according to claim 1 are integrated.