JP7176348B2 - Image output system, method and program - Google Patents

Description

The present invention relates to image output.

2. Description of the Related Art Conventionally, technology that allows users to experience virtual reality (hereinafter also referred to as VR) by viewing a virtual three-dimensional space (hereinafter also referred to as virtual space) of three-dimensional computer graphics (hereinafter also referred to as 3DCG). is being developed. Viewers can view the virtual space from various positions and angles. During such a VR experience, the viewer may wish to leave an image of a particularly impressive scene.

For example, in Patent Document 1, an image captured by a virtual camera in a virtual game space is displayed on a display device. When the player uses the input device to input a capture (shooting) instruction while playing the game, the game device captures (shoots) the game screen at the time when the capture instruction was input. After that, the captured image (screenshot) is viewed in the application of the game being played or in another application, sent as an attachment to an e-mail, or uploaded to a web page (description of Patent Document 1). (see paragraphs [0054] and [0058]).

Japanese Patent Application Laid-Open No. 2018-89269

However, simply capturing a screen displayed on a display device as it is (also referred to as a screen shot) does not provide sufficient quality as an image for high-resolution printing or viewing on a display device.

Therefore, the present invention has been made in view of such problems, and its object is to be able to output a 3DCG virtual space scene being viewed with high quality (for example, high image quality). An object is to provide a system, a method, and a program.

A first aspect of the present invention to achieve these objectives is a system. This system includes a real-time rendering unit that performs real-time rendering based on rendering information, an acquisition unit that acquires the rendering information during the real-time rendering, and a high-quality off-screen rendering based on at least the acquired rendering information. and an output unit for outputting the high-quality off-screen rendered image.

A second aspect of the invention is a method. The method comprises the steps of: real-time rendering based on rendering information; obtaining said rendering information during said real-time rendering; and performing high quality off-screen rendering based at least on said obtained rendering information. , and outputting the high quality off-screen rendered image.

A third aspect of the present invention is a program. This program is a program that causes a computer to execute the method of the second aspect.

According to the present invention, high-quality off-screen rendering can be performed based on rendering information obtained during real-time rendering.

1 is an overall schematic diagram of an image output system according to an embodiment of the present invention; FIG. 1 is a functional block diagram of an image output device according to one embodiment of the present invention; FIG. 4 is a flow chart showing an example of a processing flow in the image output device according to one embodiment of the present invention;

First, terms used in this specification will be explained.

- "Source data" is the original data for rendering. For example, the source data is virtual space data in which objects and light sources are arranged. For example, the source data includes information on the shape and material of the object, the number of light sources, and the like.

・"Rendering (also referred to as drawing)" refers to information on the position and orientation of a viewpoint (also referred to as a virtual camera) in a virtual space, information on objects in the virtual space, information on a light source in the virtual space, The creation of an image from source data based on That is, an image captured by a virtual camera placed in the virtual space is rendered.

"Real-time rendering" means rendering at a processing speed that can update each frame (1/30 second, 1/60 second, etc.) in accordance with the refresh rate of the output destination such as the screen. For example, if the processing speed is slowed down due to the large number of objects to be rendered, tuning can be done by lowering the quality of the objects to be rendered, reducing the number of objects, or simplifying lighting calculations. done.

・"Off-screen rendering" means rendering off-screen (behind the screen) in addition to normal rendering on the screen.

・"High-quality off-screen rendering" means that the image is rendered off-screen (on the back of the screen), although it may not be real-time (that is, it may take several seconds), but it is of higher quality (e.g., high image quality) than real-time rendering. ) to render. Note that high-quality images are not limited to images with higher image quality than real-time rendering. This includes images based on the direction of the camera, images based on the angle of view of a virtual camera that differs from real-time rendering, and the like.

An image output system 100 according to an embodiment of the present invention, described below, is capable of real-time rendering based on rendering information and high quality off-screen rendering based on the rendering information.

- "Rendering information" is information necessary for real-time rendering and high-quality off-screen rendering, and is information indicating what kind of viewpoint, object, and light source should be used for rendering. For example, the rendering information includes information on the position and orientation of the viewpoint in the virtual space, information on the objects in the virtual space (for example, the position and orientation where the objects are arranged), and information on the light source in the virtual space (for example, position and orientation of the light source, intensity and color of the light, etc.).

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an overall schematic diagram of an image output system 100 according to one embodiment of the present invention. Image output system 100 may include image output device 101 , real-time rendering output device 102 , off-screen rendering output device 103 , and request device 104 . Each of these will be described below.

The image output device 101 performs real-time rendering based on the rendering information, high-quality off-screen rendering based on the rendering information acquired during real-time rendering, and outputs the high-quality off-screen rendered image. It is a device that The image output device 101 will be described in detail later with reference to FIG.

A real-time rendering output device 102 is a device that outputs an image that has been rendered in real time by the image output device 101 . For example, the real-time rendering output device 102 is a display device such as a large screen, display, monitor, etc. that can be viewed by one or more viewers simultaneously. Also, for example, the real-time rendering output device 102 is a display unit built in a computer such as an HMD (head-mounted display), tablet, smartphone, personal computer, or game machine, or a display device connected thereto.

The off-screen rendering output device 103 is a device that outputs an image that has been high-quality off-screen rendered by the image output device 101 . For example, off-screen rendering output device 103 is a printer. Also, for example, the off-screen rendering output device 103 is an HMD (head mounted display). Also, for example, the off-screen rendering output device 103 is a display unit built in a computer such as a tablet, a smartphone, a personal computer, or a game machine, or a display device connected thereto.

The request device 104 is a device that requests the image output device 101 to output an image that is being rendered in real time at that time with high quality (for example, high image quality). For example, requesting device 104 is a computer, such as a tablet. While viewing an image being rendered in real time, the viewer can request high-quality (eg, high-quality) output of the image being viewed on application software installed in the requesting device 104 . Note that the image output device 101 may have the function of the requesting device 104 .

In this specification, the image output device 101, the real-time rendering output device 102, the off-screen rendering output device 103, and the requesting device 104 are described as separate devices. You may do so. Also, part or all of the functions of each device described herein may be implemented on another device.

FIG. 2 is a functional block diagram of the image output device 101 according to one embodiment of the invention. The image output device 101 can include a real-time rendering unit 201 , a request receiving unit 202 , an obtaining unit 203 , an off-screen rendering unit 204 and an output unit 205 . The image output device 101 also includes a program for operating the real-time rendering unit 201, the request reception unit 202, the acquisition unit 203, the off-screen rendering unit 204, and the output unit 205, or a program for executing a processing flow described later. , including a storage medium storing . For example, image output device 101 is one or more computers including processors and memory. Also, the computer executes a program for operating the real-time rendering unit 201, the request receiving unit 202, the acquiring unit 203, the off-screen rendering unit 204, and the output unit 205, or a program for executing a processing flow described later. .

The real-time rendering unit 201 can perform real-time rendering based on source data and rendering information. Specifically, the real-time rendering unit 201 can receive at least part of the rendering information from a device (not shown) for indicating rendering information. In addition, the real-time rendering unit 201 is tuned so that rendering can be performed at a processing speed at which each frame can be updated in accordance with the refresh rate of the real-time rendering output device 102 based on the rendering information including the received information (hereinafter referred to as (also called real-time rendering tuning) can be performed for rendering. For example, the real-time rendering unit 201 receives information about the position and orientation of the viewpoint in the virtual space from a device for indicating rendering information. Then, the real-time rendering unit 201 receives the source data, the received viewpoint position and orientation information, the object information in the virtual space (for example, the position and orientation where the object is arranged), and the light source information in the virtual space ( For example, the position and orientation of the light source, the intensity and color of the light, etc.) are used to perform real-time rendering tuning. Then, pixel information is recorded in a buffer for the real-time rendering output device 102 (the buffer capacity depends on the resolution of the real-time rendering output device 102. For example, 1920×3840×3 (RGB) bytes). The image thus real-time rendered by the real-time rendering unit 201 is output to the real-time rendering output device 102 described above.

The request receiver 202 can receive request data from the requesting device 104 requesting that the image currently being rendered in real time be output in high quality (eg, high quality). Further, upon receiving the request data, the request reception unit 202 can cause the acquisition unit 203 to acquire the rendering information at that time. The request receiver 202 can receive one or more requested data. The request receiver 202 can also receive request data from one or more request devices 104 .

The image output apparatus 101 may acquire the rendering information when the request reception unit 202 receives the request data from the request apparatus 104, or may acquire the rendering information at a predetermined time (for example, at a predetermined time). Rendering information may be acquired when the position of the viewpoint in the virtual space moves to a predetermined location (that is, the request receiving unit 202 does not need to receive the request data from the requesting device 104). do not have).

Return to FIG. The acquisition unit 203 can acquire rendering information while the real-time rendering unit 201 is performing real-time rendering. Specifically, when the request receiving unit 202 receives the request data, or at a predetermined time (for example, at a predetermined time), the obtaining unit 203 detects that the position of the viewpoint in the virtual space is at a predetermined time. Rendering information used for real-time rendering by the real-time rendering unit 201 at that time can be acquired. The acquisition unit 203 can also store the acquired rendering information in a memory (not shown) so that the off-screen rendering unit 204 can use it. As described above, the rendering information includes, for example, information on the position and orientation of the viewpoint in the virtual space at that time, information on the object in the virtual space (for example, the position and orientation where the object is arranged), and the light source in the virtual space. (for example, the position and direction of the light source, the intensity and color of the light, etc.).

The offscreen rendering unit 204 can perform high quality offscreen rendering based on the source data and the rendering information acquired by the acquisition unit 203 . Specifically, based on the rendering information acquired by the acquisition unit 203, the off-screen rendering unit 204 does not have to be real-time (that is, it may take several seconds), but has higher quality than real-time rendering (for example, Rendering can be performed by performing tuning (hereinafter also referred to as off-screen rendering tuning) so that an image with high image quality can be rendered off-screen. Then, the pixel information is recorded in a buffer for the off-screen rendering output device 103 (buffer capacity: ultra-high resolution such as tens of thousands x tens of thousands x 3 (RGB) bytes). The off-screen rendering unit 204 can also store the high-quality off-screen rendered image in memory (not shown) for use by the output unit 205 . The high-quality off-screen rendered image by the off-screen rendering unit 204 in this way is not displayed on the real-time rendering output device 102, but is stored in memory (not shown).

Note that the off-screen rendering unit 204 can combine the rendered images after rendering multiple times. Therefore, the off-screen rendering unit 204 can create an image that exceeds the upper limit of the rendering resolution of the video card of the image output device 101 .

Here, the relationship between real-time rendering tuning and off-screen rendering tuning will be described. Both real-time rendering tuning and off-screen rendering tuning are tunings for rendering, but they are different tunings. For example, an image rendered using real-time rendering tuning and an image rendered using off-screen rendering tuning have different resolutions and formats. Offscreen rendering tuning can, for example, achieve better lighting and better anti-aliasing. Instead of tuning, different algorithms may be used for real-time rendering and high-quality off-screen rendering.

Next, the relationship between rendering using real-time rendering tuning (ie, real-time rendering), rendering using off-screen rendering tuning (ie, high-quality off-screen rendering), and rendering information will be described. The image output device 101 can perform real-time rendering based on the rendering information and high-quality off-screen rendering based on that rendering information (i.e., the exact same rendering information that was used for real-time rendering). . Alternatively, the image output device 101 performs real-time rendering based on the rendering information, and the rendering information and rendering information other than the rendering information (that is, not only the rendering information used in real-time rendering but also the rendering information used in real-time rendering). high-quality off-screen rendering (eg, light sources and objects that were not used in real-time rendering can be used).

For example, in high-quality off-screen rendering, the off-screen rendering unit 204 modifies the rendering information in real-time rendering in response to an instruction from a device (not shown) for instructing modification of the rendering information to: High quality off-screen rendering can be performed based on the modified rendering information. Changes in rendering information include, for example, changes in light sources (for example, changes in time zones such as day and night changes), changes in objects (for example, addition of objects, high-quality models (for example, models with a large number of polygons) , models using high-resolution textures), changing the orientation of the viewpoint (virtual camera), changing the angle of view of the virtual camera, and so on. For example, a high quality off-screen rendering of a virtual space at night, which was daytime during real-time rendering, can be performed. Also, for example, objects that were not displayed during real-time rendering may be printed (or displayed) in high-quality off-screen rendering. Also, for example, high-quality off-screen rendering can be performed with a viewpoint (virtual camera) orientation different from that during real-time rendering. Also, for example, high-quality off-screen rendering can be performed at a virtual camera angle of view different from that during real-time rendering.

Return to FIG. The output unit 205 can output an image that has been high-quality off-screen rendered by the off-screen rendering unit 204 . Specifically, the output unit 205 can cause the off-screen rendering output device 103 to output an image that has been high-quality off-screen rendered by the off-screen rendering unit 204 .

Note that high quality off-screen rendering and output may occur at any time. For example, the off-screen rendering unit 204 can perform high-quality off-screen rendering while the viewer is experiencing real-time rendered VR or after the viewer has finished experiencing real-time rendered VR. Also, for example, the output unit 205 can output while the viewer is experiencing VR by real-time rendering or after experiencing VR by real-time rendering.

Various output forms of high-quality off-screen rendered images are described below.

<Print>
Embodiments for outputting high-quality off-screen rendered images as printed matter are described. The output unit 205 can cause the off-screen rendering output device 103 (a printer in this embodiment) to print on paper or the like the image that has been rendered with high quality off-screen rendering by the off-screen rendering unit 204 .

<Image data>
Embodiments that output high quality off-screen rendered images as data are described. The output unit 205 can transmit high-quality off-screen rendered image data to computers such as tablets, smartphones, personal computers, and game consoles. An off-screen rendering output device 103 built into or connected to those computers can then display high quality off-screen rendered images.

Another embodiment for outputting a high-quality off-screen rendered image as data is to send the data of the identifier (e.g., QR code) used to retrieve the image rather than sending the data of the image itself. ) can be sent.

In the case of <printing> and <image data> described above, a virtual camera with a different angle of view than the real-time rendering by the real-time rendering unit 201 is used in the virtual space. can be captured to enable high-quality off-screen rendering. For example, the off-screen rendering unit 204 shoots the inside of the virtual space with a virtual camera having an angle of view of 360 degrees in all directions, instead of the angle of view when the real-time rendering unit 201 performs real-time rendering. Can be rendered off-screen for quality. Then, the output unit 205 can transmit the data of the high-quality off-screen rendered image to the HMD (head mounted display). For this reason, a person wearing an HMD (head-mounted display) can view an image based on a field of view different from the real-time rendered image (for example, an omnidirectional image including an image in a virtual space with a different orientation from the real-time rendered image). ) can be viewed.

Whether in the case of <printing> or <image data> above, text and illustrations are superimposed on high-quality off-screen rendered images via any computer (not shown). or crop high-quality off-screen rendered images.

Also, in both the above <print> and <image data>, the position and direction of the viewpoint in the virtual space from which the acquisition unit 203 has acquired the rendering information are obtained by real-time rendering. You can make it explicit in the virtual space where you are.

FIG. 3 is a flowchart showing an example of a processing flow in the image output device 101 according to one embodiment of the invention.

At step 301 (S301), the image output device 101 continues real-time rendering based on the rendering information. Specifically, the image output device 101 receives at least part of the rendering information from a device (not shown) for designating the rendering information. After that, the image output device 101 performs real-time rendering tuning and rendering based on the rendering information including the received information.

In step 302 (S302), the image output device 101, during real-time rendering in S301, receives request data from the requesting device 104 requesting that the image being real-time rendered at that time be output in high quality (for example, high image quality). receive. The image output device 101 can also receive instructions such as changes in light sources, changes in objects, changes in viewpoint (virtual camera) direction, changes in angle of view of the virtual camera, and the like in high-quality off-screen rendering. Note that S302 may be omitted.

At step 303 (S303), the image output device 101 acquires rendering information during real-time rendering at S301. Specifically, when the requested data is received in S302, or at a predetermined time (for example, at a predetermined time or when moving to a predetermined space), the image output apparatus 101 Get the rendering information used for real-time rendering at that time.

At step 304 (S304), the image output device 101 performs high-quality off-screen rendering based on the rendering information acquired at S303. Specifically, the image output apparatus 101 performs off-screen rendering tuning based on the rendering information acquired in S303 for rendering.

At step 305 (S305), the image output device 101 outputs the image rendered in high quality off-screen at S304. Specifically, the image output device 101 causes the off-screen rendering output device 103 to output the high-quality off-screen rendered image in S304.

Note that S304 and S305 may be performed following S303, may be performed after real-time rendering in S301 is completed, or may be performed at any time.

In this way, in real-time rendering, a high-quality (for example, high-quality) image in a virtual space can be rendered in real time and output as printed matter after high-quality off-screen rendering.

Although the embodiment of the present invention has been described so far, the above embodiment is merely an example, and the present invention is not limited to the above embodiment, and can be implemented in various forms within the scope of the technical idea. It goes without saying that it is good.

Moreover, the scope of the invention is not limited to the illustrated and described exemplary embodiments, but includes all embodiments that achieve equivalent effects for which the invention is intended. Furthermore, the scope of the invention is not limited to the combination of inventive features defined by each claim, but may be defined by any desired combination of the particular features of each of the disclosed features. .

100 image output system 101 image output device 102 real-time rendering output device 103 off-screen rendering output device 104 request device 201 real-time rendering unit 202 request reception unit 203 acquisition unit 204 off-screen rendering unit 205 output unit

Claims (7)

a real-time rendering unit that performs real-time rendering based on the rendering information;
an acquisition unit that acquires the rendering information during the real-time rendering;
an off-screen rendering unit that performs high-quality off-screen rendering based at least on the obtained rendering information;
an output unit that outputs the high-quality off-screen rendered image ,
The image output system wherein the high quality off-screen rendered image includes an image based on a different field of view than the real-time rendering at the time the rendering information was obtained. a real-time rendering unit that performs real-time rendering based on the rendering information;
an acquisition unit that acquires the rendering information during the real-time rendering;
an off-screen rendering unit that performs high-quality off-screen rendering based at least on the obtained rendering information;
an output unit that outputs the high quality off-screen rendered image;
with
The off-screen rendering unit modifies the obtained rendering information in response to an instruction to modify rendering information to render the high-quality off-screen rendered image based on the modified rendering information. output system. 4. The outputting step includes at least one of printing the high quality offscreen rendered image and outputting data of the high quality offscreen rendered image. Item 3. An image output system according to Item 1 or 2 . 4. The image output system according to claim 1, wherein the obtaining unit obtains the rendering information in response to receiving request data from a requesting device. real-time rendering based on the rendering information;
obtaining the rendering information during the real-time rendering;
high quality off-screen rendering based at least on the obtained rendering information;
and outputting the high quality off-screen rendered image;
The method wherein the high quality off-screen rendered image includes an image based on a different field of view than the real-time rendering when the rendering information was obtained. real-time rendering based on the rendering information;
obtaining the rendering information during the real-time rendering;
high quality off-screen rendering based at least on the obtained rendering information;
outputting the high quality off-screen rendered image;
including
In the step of rendering high quality off-screen, modifying the obtained rendering information in response to an instruction to modify rendering information, and rendering the high quality off-screen based on the modified rendering information. how to render. A program for causing a computer to execute the method according to claim 5 or 6 .

Images