JP2020067820A - Image output system, method, and program - Google Patents

Abstract

To provide a system, a method, and a program capable of outputting a high-quality (for example, high image quality) scene of a 3DCG virtual space being viewed.SOLUTION: An image output system comprises: a real-time rendering unit configured to perform real-time rendering based on rendering information; an acquisition unit configured to acquire the rendering information during the real-time rendering; an off-screen rendering unit configured to perform high-quality off-screen rendering based on at least the acquired rendering information; and an output unit configured to output an image having been high-quality off-screen rendered.SELECTED DRAWING: Figure 1

Description

  The present invention relates to image output.

  Conventionally, a technology capable of experiencing virtual reality (hereinafter also referred to as VR) by viewing a virtual three-dimensional space (hereinafter also referred to as virtual space) of three-dimensional computer graphics (hereinafter also referred to as 3DCG) Is being developed. Viewers can view the virtual space from various positions and angles. During such a VR experience, the viewer may desire to leave an image of the scene that was particularly impressive.

  For example, in Patent Document 1, an image captured by a virtual camera in a virtual game space is displayed on a display device. Then, when the player uses the input device to input a capture (shooting) instruction while playing the game, the game screen captures (shoots) the game screen when the capture instruction is input. After that, the captured image (screenshot) is viewed in the application of the game being played or in another application, transmitted by being attached to an e-mail, or uploaded to a web page (Patent Document 1). See paragraphs [0054] and [0058] of the book).

JP 2018-89269 JP

  However, just by screen-capturing the screen displayed on the display device as it is (also referred to as a screen shot), it cannot be said that the quality is sufficient as an image for high-resolution printing or viewing on the display device.

  Therefore, the present invention has been made in view of such a problem, and an object of the present invention is to output a scene in a virtual space of 3DCG being viewed with high quality (for example, high image quality). To provide a system, a method and a program.

  A first aspect of the present invention for achieving such an object is a system. The system includes a real-time rendering unit that performs real-time rendering based on rendering information, an acquisition unit that acquires the rendering information during the real-time rendering, and a high-quality off-screen rendering based on at least the acquired rendering information. And an output unit that outputs the high-quality off-screen rendered image.

  A second aspect of the invention is a method. The method comprises: real-time rendering based on rendering information; obtaining the rendering information during the real-time rendering; and high quality off-screen rendering based at least on the obtained rendering information. Outputting the high quality off-screen rendered image.

A third aspect of the present invention is a program. This program is a program that causes a computer to execute the method of the second aspect.

  According to the present invention, high quality off-screen rendering can be performed based on the rendering information acquired during real-time rendering.

FIG. 1 is an overall schematic diagram of an image output system according to an embodiment of the present invention. It is a functional block diagram of the image output device concerning one embodiment of the present invention. 6 is a flowchart showing an example of a processing flow in the image output device according to the embodiment of the present invention.

  First, terms used in this specification will be described.

-"Source data" is the original data for rendering. For example, the source data is data in a virtual space where objects and light sources are arranged. For example, the source data includes information such as the shape and material of the object and the number of light sources.

-"Rendering (also called drawing)" means information on the position and orientation of a viewpoint (also called a virtual camera) in the virtual space, information on an object in the virtual space, and information on a light source in the virtual space, Based on, it means to create an image from source data. That is, the image captured by the virtual camera arranged in the virtual space is rendered.

"Real-time rendering" is rendering at a processing speed that can update each frame (1/30 seconds, 1/60 seconds, etc.) according to the refresh rate of the output destination such as the screen. For example, if the processing speed is slow due to the large number of objects to be rendered, tuning such as lowering the quality of objects to be rendered, reducing the number of objects, or simplifying the lighting calculation is possible. Done.

-"Off-screen rendering" means rendering off-screen (behind the screen) separately from normal rendering drawn on the screen.

-"High quality off-screen rendering" does not have to be real-time (that is, it may take a few seconds), but images of higher quality (for example, higher image quality) than real-time rendering can be made off-screen (behind the screen). ) To render. Note that a high-quality image is not limited to an image of higher image quality than real-time rendering, and for example, an image based on a light source different from real-time rendering, an image based on an object different from real-time rendering, or a virtual image different from real-time rendering. An image based on the orientation of the camera, an image based on the angle of view of a virtual camera different from real-time rendering, and the like are included.

  The image output system 100 according to the embodiment of the present invention described below can perform real-time rendering based on rendering information and high-quality off-screen rendering based on the rendering information.

The "rendering information" is information necessary for real-time rendering and high-quality off-screen rendering, and is information indicating what viewpoint, object, and light source should be used for rendering. For example, the rendering information includes information on the position and orientation of the viewpoint in the virtual space, information on the object in the virtual space (for example, the position and orientation at which the object is arranged), and information on the light source in the virtual space (for example, The position and orientation of the light source, such as the intensity and color of the light).

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is an overall schematic diagram of an image output system 100 according to an embodiment of the present invention. The image output system 100 can include an image output device 101, a real-time rendering output device 102, an off-screen rendering output device 103, and a requesting device 104. Each will be described below.

  The image output device 101 performs real-time rendering based on the rendering information, performs high-quality off-screen rendering based on the rendering information acquired during the real-time rendering, and outputs the high-quality off-screen rendered image. It is a device that does. The image output apparatus 101 will be described in detail later with reference to FIG.

  The real-time rendering output device 102 is a device that outputs an image that is real-time rendered by the image output device 101. For example, the real-time rendering output device 102 is a display device such as a huge screen, display, or monitor that can be viewed by one or more viewers at the same time. Further, for example, the real-time rendering output device 102 is a display unit built in a computer such as an HMD (head mounted display), a tablet, a smartphone, a personal computer, a game machine, or a display device connected to them.

  The off-screen rendering output device 103 is a device that outputs an image that has been subjected to high-quality off-screen rendering by the image output device 101. For example, the off-screen rendering output device 103 is a printer. Further, for example, the off-screen rendering output device 103 is an HMD (head mounted display). Further, for example, the off-screen rendering output device 103 is a display unit built in a computer such as a tablet, a smartphone, a personal computer, or a game machine, or a display device connected to them.

  The request device 104 is a device that requests the image output device 101 to output an image that is being rendered in real time at that time with high quality (for example, high image quality). For example, the requesting device 104 is a computer such as a tablet. The viewer can request the output of the image being viewed with high quality (for example, high image quality) on the application software installed in the requesting device 104 while viewing the image that is being rendered in real time. The image output device 101 may have the function of the request device 104.

  Although the image output device 101, the real-time rendering output device 102, the off-screen rendering output device 103, and the request device 104 are described as separate devices in this specification, the functions of a plurality of devices are implemented on one device. You may do it. Further, some or all of the functions of each device in this specification may be implemented on another device.

  FIG. 2 is a functional block diagram of the image output apparatus 101 according to the embodiment of the present invention. The image output device 101 can include a real-time rendering unit 201, a request receiving unit 202, an acquisition unit 203, an off-screen rendering unit 204, and an output unit 205. Further, the image output device 101 is a program for operating the real-time rendering unit 201, the request receiving unit 202, the acquisition unit 203, the off-screen rendering unit 204, and the output unit 205, or a program for executing a processing flow described below. , Including a storage medium storing. For example, the image output device 101 is one or more computers including a processor and a memory. A computer executes a program for operating the real-time rendering unit 201, the request receiving unit 202, the acquisition unit 203, the off-screen rendering unit 204, and the output unit 205, or a program for executing a processing flow described below. .

  The real-time rendering unit 201 can perform real-time rendering based on the source data and the rendering information. Specifically, the real-time rendering unit 201 can receive at least a part of the rendering information from a device (not shown) for instructing the rendering information. In addition, the real-time rendering unit 201 tunes based on the rendering information including the received information so that rendering can be performed at a processing speed capable of updating each frame according to the refresh rate of the real-time rendering output device 102 (hereinafter, Real-time rendering / tuning) can be performed for rendering. For example, the real-time rendering unit 201 receives information on the position and orientation of the viewpoint in the virtual space from the device for instructing the rendering information. Then, the real-time rendering unit 201 receives the source data, the received viewpoint position and orientation information, the object information in the virtual space (for example, the position and orientation in which the object is arranged), and the light source information in the virtual space ( For example, the rendering is performed by performing real-time rendering / tuning based on the position and orientation of the light source, the intensity and color of light, and the like. Then, the pixel information is recorded in a buffer for the real-time rendering output device 102 (the buffer capacity depends on the resolution of the real-time rendering output device 102. For example, 1920 × 3840 × 3 (RGB) Byte). The image thus real-time rendered by the real-time rendering unit 201 is output to the real-time rendering output device 102 described above.

  The request receiving unit 202 can receive, from the requesting device 104, request data for requesting output of an image currently being rendered in real time with high quality (for example, high image quality). Further, upon receiving the request data, the request receiving unit 202 can cause the acquisition unit 203 to acquire the rendering information at that time. The request receiving unit 202 can receive one or more request data. Further, the request receiving unit 202 can receive request data from one or more requesting devices 104.

  The image output apparatus 101 may acquire the rendering information when the request receiving unit 202 receives the request data from the request apparatus 104, or at a predetermined time (for example, when a predetermined time is reached). In this case, the rendering information may be acquired when the position of the viewpoint in the virtual space moves to a predetermined place (that is, the request receiving unit 202 does not need to receive the request data from the request device 104). Absent).

  Returning to FIG. The acquisition unit 203 can acquire rendering information while the real-time rendering unit 201 is performing real-time rendering. Specifically, the acquisition unit 203 determines that the position of the viewpoint in the virtual space is predetermined when the request receiving unit 202 receives the request data or when the request data is predetermined (for example, when a predetermined time comes. When moving to a place), the real-time rendering unit 201 can acquire the rendering information used for the real-time rendering at that time. In addition, the acquisition unit 203 can store the acquired rendering information in a memory (not shown) so that the off-screen rendering unit 204 can use it. As described above, the rendering information is, for example, information on the position and orientation of the viewpoint in the virtual space at that time, information on the object in the virtual space (for example, the position and orientation in which the object is arranged), and the light source in the virtual space. Information (for example, the position and direction in which the light source is arranged, the light intensity and color, etc.).

  The off-screen rendering unit 204 can perform high-quality off-screen rendering based on the source data and the rendering information acquired by the acquisition unit 203. Specifically, the off-screen rendering unit 204 does not have to be in real time (that is, may take several seconds) based on the rendering information acquired by the acquisition unit 203, but has a higher quality than real-time rendering (for example, A high-quality image can be tuned (hereinafter also referred to as off-screen rendering / tuning) so that the image can be rendered off-screen before rendering. Then, the pixel information is recorded in the buffer for the off-screen rendering output device 103 (the buffer capacity is tens of thousands × tens of thousands × 3 (RGB) Byte or other ultra-high resolution). The off-screen rendering unit 204 can also store the high-quality off-screen rendered image in a memory (not shown) so that the output unit 205 can use the image. The image thus high-quality off-screen rendered by the off-screen rendering unit 204 is not displayed on the real-time rendering output device 102 but is stored in the memory (not shown).

  Note that the off-screen rendering unit 204 can synthesize the rendered images after rendering a plurality of times. Therefore, the off-screen rendering unit 204 can create an image that exceeds the upper limit of the rendering resolution of the video card of the image output device 101.

  Here, the relationship between the real-time rendering tuning and the off-screen rendering tuning will be described. Real-time rendering tuning and off-screen rendering tuning are both tunings for rendering, but they are different tunings. For example, an image rendered using real-time rendering tuning and an image rendered using off-screen rendering tuning have different resolutions and formats. Off-screen rendering tuning can achieve high quality lighting and high quality anti-aliasing, for example. Instead of tuning, different algorithms may be used for real-time rendering and high-quality off-screen rendering.

  Next, the relationship between rendering using real-time rendering tuning (that is, real-time rendering), rendering using off-screen rendering tuning (that is, high-quality off-screen rendering), and rendering information will be described. The image output apparatus 101 can perform real-time rendering based on the rendering information, and can perform high-quality off-screen rendering based on the rendering information (that is, the rendering information that is exactly the same as the rendering information used in the real-time rendering). . Alternatively, the image output device 101 performs real-time rendering based on the rendering information, and uses the rendering information and the rendering information other than the rendering information (that is, not only the rendering information used in the real-time rendering but also the rendering information used in the real-time rendering. High quality off-screen rendering (including rendering information that was not done) (eg, light sources and objects that were not used in real-time rendering can be used).

  For example, in high quality off-screen rendering, the off-screen rendering unit 204 changes rendering information in real-time rendering in response to an instruction from a device (not shown) for instructing a change in rendering information, High quality off-screen rendering can be performed based on the changed rendering information. Changes in rendering information are, for example, changes in light sources (for example, changes in time zones such as day and night changes), changes in objects (for example, addition of objects, high-quality models (for example, models with a large number of polygons). , Using models with high-resolution textures), changing the orientation of the viewpoint (virtual camera), changing the angle of view of the virtual camera, etc. For example, high-quality off-screen rendering of a virtual space at night, which was daytime during real-time rendering, can be performed. Also, for example, objects that were not displayed during real-time rendering can be printed (or displayed) in high quality off-screen rendering. Further, for example, high quality off-screen rendering can be performed with a different viewpoint (virtual camera) orientation from that during real-time rendering. Further, for example, high quality off-screen rendering can be performed with a virtual camera view angle different from that during real-time rendering.

  Returning to FIG. The output unit 205 can output the image that has been subjected to the high quality off-screen rendering by the off-screen rendering unit 204. Specifically, the output unit 205 can cause the off-screen rendering output device 103 to output the image that the off-screen rendering unit 204 has performed high-quality off-screen rendering.

  Note that high quality off-screen rendering and output may occur at any time. For example, the off-screen rendering unit 204 may perform high-quality off-screen rendering while the viewer is experiencing the VR by the real-time rendering or after the VR experience by the real-time rendering is completed. Further, for example, the output unit 205 can output while the viewer is experiencing the VR by the real-time rendering or after the experience of the VR by the real-time rendering is completed.

  Various output forms of high quality off-screen rendered images are described below.

<Print>
An embodiment for outputting a high quality off-screen rendered image as a printed matter will be described. The output unit 205 can cause the off-screen rendering output device 103 (a printer in this embodiment) to print the image that has been subjected to the high-quality off-screen rendering by the off-screen rendering unit 204 on paper or the like.

<Image data>
An embodiment in which a high quality off-screen rendered image is output as data will be described. The output unit 205 can transmit high-quality off-screen rendered image data to a computer such as a tablet, smartphone, personal computer, or game console. Then, the off-screen rendering output device 103 built in or connected to these computers can display the high-quality off-screen rendered image.

  As another embodiment for outputting a high-quality off-screen rendered image as data, rather than transmitting the data of the image itself, data of an identifier used to acquire the image (for example, QR code (registered trademark)). ) Can be sent.

  In both the case of <print> and the case of <image data> described above, in the virtual space, the virtual camera has an angle of view different from the angle of view when the real-time rendering unit 201 is performing real-time rendering. Can be taken for high quality off-screen rendering. For example, the off-screen rendering unit 204 does not use the angle of view when the real-time rendering unit 201 is performing real-time rendering, but captures the inside of the virtual space with a virtual camera that has a 360-degree angle of view in all directions of up, down, left, and right. Quality off-screen rendering is possible. Then, the output unit 205 can transmit the data of the high quality off-screen rendered image to the HMD (head mounted display). Therefore, a person wearing an HMD (head-mounted display) has an image based on a field of view different from that of the image rendered in real time (for example, an omnidirectional image including an image in a virtual space in a direction different from that of the image rendered in real time). ) Can be viewed.

  Whether in the case of <print> or <image data> described above, characters or illustrations are superimposed on the high-quality off-screen rendered image via an arbitrary computer (not shown). The image can be processed, such as cropping a high quality off-screen rendered image.

  In addition, in both the case of <print> and the case of <image data>, the position and orientation of the viewpoint in the virtual space in which the acquisition unit 203 has acquired the rendering information are rendered in real time. It can be made explicit in the existing virtual space.

  FIG. 3 is a flowchart showing an example of a processing flow in the image output apparatus 101 according to the embodiment of the present invention.

  In step 301 (S301), the image output apparatus 101 continues to perform real-time rendering based on the rendering information. Specifically, the image output device 101 receives at least a part of the rendering information from a device (not shown) for instructing the rendering information. After that, the image output apparatus 101 performs real-time rendering / tuning based on the rendering information including the received information and performs rendering.

  In step 302 (S302), the image output apparatus 101 outputs request data for requesting the high-quality (for example, high-quality) output of the image currently being rendered in real time from the requesting apparatus 104 during the real-time rendering in S301. To receive. The image output device 101 can also receive an instruction such as a change regarding a light source, a change regarding an object, a change in the direction of a viewpoint (virtual camera), a change in the angle of view of a virtual camera in high quality off-screen rendering. Note that S302 may be omitted.

  In step 303 (S303), the image output apparatus 101 acquires rendering information during the real-time rendering in S301. Specifically, the image output apparatus 101 receives the request data in S302, or at a predetermined time (for example, when a predetermined time is reached or when the user moves to a predetermined space), At that time, the rendering information used for the real-time rendering is acquired.

  In step 304 (S304), the image output apparatus 101 performs high-quality off-screen rendering based on the rendering information acquired in S303. Specifically, the image output device 101 performs off-screen rendering / tuning based on the rendering information acquired in S303 and performs rendering.

  In step 305 (S305), the image output apparatus 101 outputs the image subjected to the high quality off-screen rendering in S304. Specifically, the image output apparatus 101 causes the off-screen rendering output apparatus 103 to output the image that has been subjected to the high-quality off-screen rendering in S304.

  Note that S304 and S305 may be performed subsequent to S303, may be performed after the real-time rendering of S301 is completed, or may be performed at any time.

  As described above, in real-time rendering, high-quality (for example, high-quality) images in the virtual space can be subjected to high-quality off-screen rendering and output as printed matter while performing real-time rendering.

  So far, the embodiments of the present invention have been described, but the above-described embodiments are merely examples, and the present invention is not limited to the above-described embodiments, and may be carried out in various different forms within the scope of the technical idea thereof. It goes without saying that it is good.

  Moreover, the scope of the present invention is not limited to the exemplary embodiments shown and described, but also includes all embodiments providing equivalent effects to those intended by the present invention. Furthermore, the scope of the invention is not limited to the combination of inventive features defined by each claim, but can be defined by any desired combination of specific features of each disclosed feature. .

100 image output system 101 image output device 102 real-time rendering output device 103 off-screen rendering output device 104 requesting device 201 real-time rendering unit 202 request receiving unit 203 acquisition unit 204 off-screen rendering unit 205 output unit

Claims (8)

A real-time rendering unit that performs real-time rendering based on rendering information,
An acquisition unit that acquires the rendering information during the real-time rendering,
An off-screen rendering unit that performs high-quality off-screen rendering based on at least the acquired rendering information,
An image output system, comprising: an output unit that outputs the high-quality off-screen rendered image.   The off-screen rendering unit performs high-quality off-screen rendering based on only the acquired rendering information, or high-quality off-screen based on the acquired rendering information and rendering information other than the acquired rendering information. The image output system according to claim 1, wherein rendering is performed.   The outputting comprises at least one of printing the high quality off-screen rendered image and outputting data of the high quality off-screen rendered image. The image output system according to Item 1 or 2.   The image according to any one of claims 1 to 3, wherein the high-quality off-screen rendered image includes an image based on a field of view different from real-time rendering when the rendering information is acquired. Output system.   The off-screen rendering unit modifies the acquired rendering information in response to an instruction to modify the rendering information, and performs the high quality off-screen rendering based on the modified rendering information. The image output system according to any one of claims 1 to 4.   The image output system according to claim 1, wherein the acquisition unit acquires the rendering information in response to receiving the request data from the request device. Real-time rendering based on the rendering information,
Obtaining the rendering information during the real-time rendering,
High quality off-screen rendering based at least on the obtained rendering information;
Outputting the high quality off-screen rendered image.   A program for causing a computer to execute the method according to claim 7.

Images