WO2024189704A1 - ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム - Google Patents

ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム Download PDF

Info

Publication number
WO2024189704A1
WO2024189704A1 PCT/JP2023/009428 JP2023009428W WO2024189704A1 WO 2024189704 A1 WO2024189704 A1 WO 2024189704A1 JP 2023009428 W JP2023009428 W JP 2023009428W WO 2024189704 A1 WO2024189704 A1 WO 2024189704A1
Authority
WO
WIPO (PCT)
Prior art keywords
game
image
game image
display device
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/009428
Other languages
English (en)
French (fr)
Japanese (ja)
Other versions
WO2024189704A9 (ja
Inventor
大樹 瀬戸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sega Corp
Original Assignee
Sega Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sega Corp filed Critical Sega Corp
Priority to JP2025506260A priority Critical patent/JPWO2024189704A1/ja
Priority to PCT/JP2023/009428 priority patent/WO2024189704A1/ja
Publication of WO2024189704A1 publication Critical patent/WO2024189704A1/ja
Publication of WO2024189704A9 publication Critical patent/WO2024189704A9/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering

Definitions

  • the present invention relates to a technology for reducing the delay between image generation and display.
  • the cloud gaming environment consists of a server and a client.
  • a console and a monitor are connected to the client.
  • the game program is controlled by the server.
  • game images generated by the server are transferred to the client and then to the monitor.
  • the server generates game images at regular intervals.
  • the monitor displays the game images generated by the server at regular intervals.
  • This latency includes the time it takes to send the game image from the server to the client, and the time it takes to transmit the game image from the client to the monitor.
  • the server processing will not be able to keep up with the monitor's display speed.
  • the present invention was completed based on the recognition of the above problems, and its main purpose is to provide a technology for continuously displaying game images on a monitor when the monitor's frame rate is increased to reduce display delays.
  • a game system includes a game server, a game terminal, and a display device.
  • the game server and the game terminal are connected via a communication network, and the game server includes a game control unit that controls the progress of the game in accordance with operation signals from the game terminal, an image generation unit that generates game images at a first period, and an image transmission unit that transmits the game images to the game terminal.
  • the game terminal includes an input unit that accepts operations from a player regarding the game, an operation transmission unit that transmits an operation signal indicating the content of the operation from the player to the game server, an image receiving unit that receives game images from the game server, and an image transmission unit that transmits the received game images to a display device.
  • the display device includes a display control unit that displays the game image at a second cycle that is shorter than the first cycle.
  • An image transmission section of the game terminal transmits the new game image to the display device when a new game image is acquired from the game server before the display device reaches the timing for displaying the next game image, and transmits a substitute game image different from the new game image to the display device when the new game image is not acquired from the game server when the display device reaches the timing for displaying the next game image.
  • a game terminal in one aspect of the present invention is connected to a game server via a communications network and comprises an input unit that accepts operations from a player regarding the game, an operation transmitting unit that transmits an operation signal indicating the content of the operation from the player to the game server, an image receiving unit that receives a game image generated based on the operation signal from the game server, and an image transmitting unit that transmits the received game image to a display device.
  • the image transmission unit transmits the new game image to the display device when a new game image is acquired from the game server before the timing for the display device to display the next game image, and transmits a substitute game image different from the new game image to the display device when a new game image is not acquired from the game server before the timing for the display device to display the next game image.
  • a display adjustment device in one aspect of the present invention includes an image receiving unit that receives a game image from a game executing entity, and an image transmitting unit that transmits the received game image to a display device.
  • an image transmission unit transmits the new game image to the display device when a new game image is acquired from the game execution entity before the display device reaches the timing for displaying the next game image, and transmits a substitute game image different from the new game image to the display device when a new game image is not acquired from the game execution entity before the display device reaches the timing for displaying the next game image.
  • the game system of the present invention makes it easier for the monitor to continue to stably display game images, even when a monitor with a faster frame rate for image display is introduced.
  • FIG. 1 is a diagram showing the overall configuration of a typical game system in a local environment.
  • 1 is a time chart showing a typical game image transfer process in a game system in a local environment.
  • FIG. 1 is an overall configuration diagram of a typical remote environment game system.
  • 11 is a time chart showing a typical game image transfer process in a game system in a remote environment.
  • FIG. 2 is a functional block diagram of a server according to the present embodiment.
  • 2 is a functional block diagram of a game terminal according to the present embodiment.
  • FIG. FIG. 2 is a functional block diagram of a monitor according to the embodiment.
  • 5 is a time chart showing a process of transferring a game image in the present embodiment.
  • 13 is a time chart showing a process of transferring a game image in a modified example.
  • FIG. 1 is a diagram showing the overall configuration of a typical local environment game system.
  • a personal computer 101 is connected to a monitor 102 and an input device 103.
  • the local game system 100 is not a server-client type, but a standalone type game system in which a game is realized by the personal computer 101 alone.
  • the input device 103 is assumed to be a keyboard, a mouse, a joystick, or the like.
  • the game program is controlled by the personal computer 101.
  • the personal computer 101 is equipped with a CPU (Central Processing Unit) that executes the game program.
  • the personal computer 101 is also equipped with a GPU (Graphics Processing Unit), which is a processor that generates game images.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • this input operation is converted into an operation signal and sent to the personal computer 101.
  • the personal computer 101 calculates the placement of objects in the game according to the operation signal. Objects are assumed to be three-dimensional structures such as characters and buildings.
  • the GPU in the personal computer 101 generates a game image corresponding to the results obtained by this calculation process. The game image is transferred to the monitor 102 and then displayed.
  • FIG. 2 is a time chart showing a typical game image transfer process in a local environment game system. Specifically, the process is shown in which a game image generated by a GPU in a personal computer 101 is transferred to a monitor 102 and is then displayed on the monitor 102 .
  • a processing time slot has a rendering time (RT) and a wait time (WT).
  • RT rendering time
  • WT wait time
  • the GPU renders (generates) a game image.
  • the rendering time is not constant, but it is guaranteed to be completed within the processing timeslot.
  • the remaining time from the completion of rendering to the completion of the processing timeslot is the wait time.
  • the GPU waits to render a new game image.
  • processing time slot PTS0 is from time t0 to t1
  • processing time slot PTS1 is from time t1 to t2
  • processing time slot PTS2 is from time t2 to t3.
  • the display rate R is 60 Hz
  • one processing time slot corresponds to approximately 17 milliseconds
  • the rendering time is 17 milliseconds or less.
  • the GPU generates a game image for each processing time slot.
  • the Nth game image generated since the start of the game is referred to as "game image (N)."
  • processing time slot PTS0 for generating game image (N) is completed, the GPU generates the next game image (N+1) in the next processing time slot PTS1.
  • the rendering time for a game image (N) is denoted as a "rendering time RT(N).”
  • the wait time after the rendering time RT(N) is completed is denoted as a "wait time WT(N).”
  • the GPU generates a game image (N) during rendering time RT(N) from time t0 to t4 within processing timeslot PTS0.
  • wait time WT(N) from time t4 to time t1 when the next processing timeslot PTS1 starts the GPU waits to generate a new game image (N+1).
  • the start time t1 of the next processing timeslot PTS1 arrives, the game image (N) is transferred to the monitor 102.
  • the monitor delay time refers to the time required for display preparation, from when the personal computer 101 sends a control signal to the monitor 102 instructing it to display a game image, until the monitor 102 applies a drive voltage to all pixels and emits light.
  • the control signal will be described later.
  • the monitor delay time required to prepare for displaying the game image (N) is represented as "monitor delay time MD(N)".
  • the game image (N) is transferred to the monitor 102 from time t1 to time t5.
  • the transferred game image (N) passes through a monitor delay time MD(N) from time t5 to time t6, and is then displayed on the monitor 102 at time t6.
  • the GPU After completing processing timeslot PTS0, the GPU generates a new game image (N+1) in processing timeslot PTS1.
  • the rendering time RT(N+1) starts at time t1, which is before time t6 when the previous game image (N) is displayed on the monitor 102. That is, the GPU starts generating the next game image (N+1) before the display of the previously generated game image (N) begins.
  • processing timeslot PTS1 completes at time t2
  • the new game image (N+1) is transferred to the monitor 102.
  • the game image (N+1) is displayed on the monitor 102 at time t7, which is the start of processing timeslot PTS2.
  • the delay time until game image (N) is displayed on monitor 102 is the time from time t0 when the GPU starts generating game image (N) to time t6 when monitor 102 displays game image (N).
  • This delay time includes monitor delay time MD(N). The same applies to game image (N+1).
  • monitor 102 displays the game image after the delay time has elapsed.
  • FIG. 3 is a diagram showing the overall configuration of a typical remote game system.
  • the server 201 is connected to the game terminal 300 via the communication network 202.
  • the game terminal 300 is specifically assumed to be a personal computer, a home game machine, or an arcade game machine.
  • the communication network 202 is assumed to be the Internet or a dedicated line.
  • the game terminal 300 is connected to a monitor 400 and an input device 500.
  • the server 201 controls the game program.
  • the server 201 is equipped with a CPU that executes the game program.
  • the server 201 is also equipped with a GPU for generating game images.
  • the server 201 is equipped with an encoder that performs encoding before transmitting the game image via the communication network 202.
  • encoding refers to compressing the game image and dividing it into packets.
  • the game terminal 300 is equipped with a decoder that decodes the encoded game image (hereinafter referred to as "encoded image") transferred via the communication network 202.
  • decoding refers to restoring the game image from the encoded image.
  • the game terminal 300 transmits an operation signal relating to this input operation to the server 201 via the communication network 202.
  • the server 201 calculates the placement of objects in the game, etc., in accordance with the operation signal. Based on the results obtained from this calculation process, the server 201 generates a game image.
  • the game image is encoded and then transmitted to the game terminal 300.
  • the game terminal 300 decodes the encoded image sent from the server 201.
  • the decoded game image is transmitted from the game terminal 300 to the monitor 400, and then displayed.
  • FIG. 4 is a time chart showing a typical game image transfer process in a game system in a remote environment.
  • a game image generated by the server 201 is converted into an encoded image.
  • the encoded image is then transmitted to the game terminal 300 and decoded by a decoder.
  • the decoded game image is transmitted to the monitor 400 and then displayed.
  • Fig. 4 unlike the local system in Fig. 2, the game image needs to be transferred via the communication network 202, so overhead (accompanying work) of encoding, network transmission, and decoding is added.
  • the time allotted for displaying one game image is called the display time slot (DTS).
  • the monitor 400 displays the game image that is ready to be displayed at the start of the display time slot.
  • the generation rate of the GPU and the display rate of the monitor 400 are both R (times/second). Also, in FIG. 4, the period from time t4 to t5 is display time slot DTS0, and the period from time t5 to t6 is display time slot DTS1.
  • the GPU generates a game image (N) in the rendering time RT(N) from time t0 to t7 within the processing time slot PTS0.
  • the encoder completes the generation of the game image (N) at time t7, it starts encoding the game image (N) without waiting for the completion of the wait time WT(N).
  • the processing time required for encoding is called the encoding time (EC).
  • the encoding time of the game image (N) is expressed as the "encoding time EC(N)".
  • the game image (N) is converted into an encoded image (N) through the encoding time EC(N) from time t7 to t8.
  • the encoded image (N) is transmitted to the game terminal 300 through the transmission time from time t8 to t9.
  • the encoding time varies depending on the load of the encoding process.
  • the transmission time is not constant either. Note that encoding is performed when compressing the volume of the game image (N) to a small size, or when converting the game image (N) into a format for transmission over the communication network 202. However, if the communication speed is fast, the transmission time from time t8 to t9 will be short. Therefore, in encoding, these processes may not be performed, or only the format of the game image (N) may be converted.
  • the encoded image (N) is decoded by the game terminal 300.
  • the time required for decoding is referred to as the decode time (DC).
  • the decode time for game image (N) is expressed as "decode time DC (N)."
  • the encoded image (N) is decoded after the decode time DC (N) from time t9 to t10. At time t10 when the decoding is completed, the game terminal 300 completes acquisition of the game image (N).
  • the decode time also varies depending on the game image.
  • the game terminal 300 transmits the game image (N) to the monitor 400 from time t10 when acquisition of the game image (N) is completed.
  • the transmission time of the game image (N) is the period shown as time t10 to t11.
  • the game image (N) is displayed on the monitor 400 from time t4, after a monitor delay time MD(N) from time t11.
  • the game image (N) continues to be displayed on the monitor 400 during the display time slot DTS0.
  • the server 201 generates the next game image (N+1) in processing time slot PTS1.
  • the game image (N+1) also goes through encoding time EC(N+1), transmission time, decoding time DC(N+1) and monitor delay time MD(N+1) before being displayed on the monitor 400 at start time t5 of the next display time slot DTS1.
  • the new game image (N+1) continues to be displayed on the monitor 400 during display time slot DTS1.
  • the monitor 400 displays the game images that were ready to be displayed before the start of that display time slot.
  • the next display time slot begins and the next game image is displayed on the monitor 400.
  • the gaming terminal 300 prepares new game images at a constant frequency of R (times/second) and causes the monitor 400 to display the prepared game images.
  • the delay time of the game image (N) is the time between time t0 when the generation of the game image (N) begins and time t4 when the monitor 400 displays the game image (N).
  • This delay time includes the monitor delay time MD(N), as well as the encoding time EC(N), transmission time, and decoding time DC(N).
  • the monitor delay time can be reduced.
  • the generation rate of the server 201 cannot be increased and only the display rate of the monitor 400 is increased, the generation speed of the game images will not be able to keep up with the display speed of the monitor 400.
  • the display rate of the monitor 400 is increased, the interval between the display time slots DTS of the monitor 400 will be shortened. In other words, the time for which the monitor 400 displays one game image will be shortened.
  • the display rate is three times the generation rate (3R).
  • the display time slot DTS0' is shortened to the period from time t4 to t12, which is one third the length of the display time slot DTS0.
  • next display time slot DTS1' is shortened to the period from time t12 to time t13.
  • the monitor 400 will not be able to prepare a new game image (N+1) by the start of the next display time slot DTS1'.
  • the monitor 400 will not be able to display anything.
  • FIG. 5 is a functional block diagram of the server 201 in this embodiment.
  • Each component of the server 201 is realized by hardware including computing units such as a CPU, a GPU, and various co-processors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, and software stored in the storage device and supplying processing instructions to the computing units.
  • the computer program may be composed of a device driver, an operating system, various application programs located in higher layers, and libraries that provide common functions to these programs.
  • Each block described below is not a hardware unit configuration, but a functional unit block. The same applies to the game terminal 300 described later.
  • the server 201 includes a data processing unit 210 and a communication unit 220 .
  • the data processing unit 210 executes various processes based on operation signals from the game terminal 300.
  • the communication unit 220 is responsible for communication with the game terminal 300 via the communication network 202.
  • the data processing unit 210 includes a processing command unit 211 , a game control unit 212 , an image generation unit 213 , and an image processing unit 214 .
  • the processing command unit 211 commands the game control unit 212 to execute processing using a game program based on an operation signal received by the receiving unit 221 from the game terminal 300.
  • the game control unit 212 performs various calculation processes based on the game program in response to a command from the processing command unit 211.
  • the image generation unit 213 generates a game image.
  • the image processing unit 214 encodes the game image generated by the image generation unit 213.
  • the image processing unit 214 functions as an "encoder.”
  • the processing command unit 211 commands the transmission unit 222 to transmit the encoded image generated by the image processing unit 214 to the game terminal 300.
  • the communication unit 220 includes a receiving unit 221 and a transmitting unit 222 .
  • the receiving unit 221 receives an operation signal from the game terminal 300.
  • the transmitting unit 222 transmits an encoded image to the game terminal 300.
  • the transmitting unit 222 functions as an "image transmitting unit.”
  • FIG. 6 is a functional block diagram of the game terminal 300 in this embodiment.
  • the gaming terminal 300 includes a user interface processing unit 310 , a data processing unit 320 , a communication unit 330 , and a data storage unit 340 .
  • the user interface processing unit 310 is responsible for output to the monitor 400 and input from the input device 500.
  • the data processing unit 320 is responsible for various processes related to the progress of the game.
  • the communication unit 330 is responsible for communication processing with the server 201, the monitor 400, and the input device 500 via the communication network 202 and a cable.
  • the communication unit 330 can also directly communicate with the monitor 400 and the input device 500 by short-range wireless communication such as Wi-Fi (registered trademark).
  • the data storage unit 340 receives processing related to game images performed in the data processing unit 320 and temporarily stores the game images.
  • the user interface processing unit 310 includes an input unit 311 and an output unit 312 .
  • the input unit 311 accepts operational inputs from the player via the input device 500.
  • the output unit 312 transmits digital signals such as control signals via the communication unit 330 and causes the monitor 400 to output game images and sounds during the progress of the game.
  • the data processing unit 320 includes a processing command unit 321 and an image restoration unit 322 .
  • the processing command unit 321 executes commands related to the transmission of operation signals, the decoding of encoded images, the temporary storage of game images, and the output of game images.
  • the processing command unit 321 also acquires information such as the resolution and display rate at which the monitor 400 can display, and the audio format that can be output, from the communication unit 330.
  • the processing command unit 321 determines the resolution, display rate, and audio format of the game images to be output to the monitor 400 based on the information acquired from the communication unit 330.
  • the image restoration unit 322 decodes the encoded images.
  • the image restoration unit 322 functions as a "decoder.”
  • the communication unit 330 includes a receiving unit 331 and a transmitting unit 332 .
  • the receiving unit 331 receives encoded images from the server 201.
  • the receiving unit 331 also receives operation signals from the input device 500.
  • the receiving unit 331 receives information from the monitor 400, such as the resolution and display rate that the monitor 400 can display, and the audio format that the monitor 400 can output.
  • the receiving unit 331 functions as an "image receiving unit.”
  • the transmitting unit 332 transmits operation signals to the server 201 and transmits game images to the monitor 400.
  • the transmitting unit 332 functions as an "operation transmitting unit” and an "image transmitting unit.”
  • an operation input by the player is accepted by the input unit 311.
  • the process command unit 321 commands the transmission unit 332 to transmit an operation signal indicating the operation content to the server 201.
  • the transmission unit 332 transmits the operation signal to the server 201 in accordance with the command from the process command unit 321.
  • the server 201 transmits the encoded image to the game terminal 300.
  • the receiver 331 of the game terminal 300 receives the encoded image.
  • the process command unit 321 of the game terminal 300 commands the image restoration unit 322 to decode the encoded image.
  • the image restoration unit 322 receives the command from the process command unit 321, it decodes the encoded image.
  • the process command unit 321 commands the output unit 312 to generate a control signal from the decoded game image.
  • the control signal is a collective term for data indicating coloring designation for each pixel of the monitor 400 and a signal indicating the timing of image display (pixel emission).
  • the process command unit 321 also stores the decoded game image in the data storage unit 340.
  • the output unit 312 that has received the command generates a control signal from the decoded game image and transfers it to the transmitter 332.
  • the transmitter 332 transmits the control signal transferred from the output unit 312 to the monitor 400.
  • FIG. 7 is a functional block diagram of the monitor 400 in this embodiment.
  • the monitor 400 includes a data processing unit 410 and a communication unit 420 .
  • the data processing unit 410 is responsible for processing to display game images on the screen.
  • the communication unit 420 is responsible for communication with the game terminal 300.
  • the data processing unit 410 includes a display control unit 411 .
  • the display control unit 411 receives the control signal received by the receiving unit 421 , it completes the display preparation and then displays the game image on the monitor 400 .
  • the communication unit 420 includes a receiving unit 421 .
  • the receiving unit 421 receives a control signal obtained by converting a game image from the game terminal 300 via a cable or the like.
  • FIG. 8 is a time chart showing the process of transferring a game image in this embodiment.
  • a game image generated by the image generation unit 213 in the server 201 is converted into an encoded image by the image processing unit 214.
  • the encoded image is transmitted to the game terminal 300 and decoded by the image restoration unit 322.
  • the decoded game image is then transmitted to the monitor 400 and displayed.
  • the generation rate of the image generation unit 213 is set to R (times/second), while the display rate of the monitor 400 is set to 4R (four times R) (times/second).
  • R times/second
  • 4R four times R
  • four display time slots are provided for one processing time slot.
  • the image generation unit 213 of the server 201 generates a game image (N) in the rendering time RT(N) from time t0 to t7 during the processing time slot PTS0.
  • the image processing unit 214 encodes the game image (N) without waiting for the completion of the wait time WT(N).
  • the game image (N) is converted into an encoded image (N) through an encoding time EC(N) from time t7 to t8.
  • the encoded image (N) is transmitted to the game terminal 300 through a transmission time from time t8 to t9.
  • the encoded image (N) is decoded by the image restoration unit 322 of the game terminal 300.
  • the encoded image (N) is decoded after a decoding time DC(N) from time t9 to time t10.
  • the processing command unit 321 acquires the game image (N).
  • the game image (N) is transmitted to the monitor 400 from time t10 to time t11.
  • the processing command unit 321 also temporarily stores the game image (N) acquired at time t10 in the data storage unit 340.
  • the game image (N) is displayed on the monitor 400 in the display time slot DTS0 starting at time t12, after a monitor delay time MD(N) from time t11 to time t12.
  • the game image (N) is displayed on the monitor 400 during the display time slot DTS0.
  • the monitor delay time MD(N) in FIG. 8 is shorter than that in FIG. 4 due to the faster display rate of the monitor 400. Therefore, the delay time of the game image (N) is shorter than that in FIG. 4.
  • the generation rate of the image generation unit 213 is R (times/second), while the display rate of the monitor 400 is set to 4R (four times R) (times/second).
  • the monitor 400 After the image generation unit 213 starts generating a game image (N), the monitor 400 must display four game images (N) before generating a new game image (N+1).
  • the monitor 400 displays the game image (N) every time in the four display time slots DTS0, DTS1, DTS2, and DTS3.
  • the processing command unit 321 commands the output unit 312 to display the game image refers to the process of "The processing command unit 321 commands the output unit 312 to generate a control signal from the game image, the output unit 312 generates the control signal from the game image, and the transmission unit 332 transmits the control signal to the monitor 400.”
  • the image restoration unit 322 has not completed decoding the encoded image (N+1) by time t13 when the display time slot DTS0 is completed.
  • the processing command unit 321 has not acquired a new game image (N+1). Therefore, the processing command unit 321 commands the output unit 312 to display the game image (N) once more, a little after time t12. In other words, the processing command unit 321 commands the output unit 312 to display the game image (N:2).
  • the processing command unit 321 reads out the game image (N) saved in the data storage unit 340, and commands the output unit 312 to redisplay the game image (N) as the game image (N:2).
  • the receiving unit 421 of the monitor 400 receives a control signal corresponding to the game image (N:2) transmitted by the game terminal 300.
  • the display control unit 411 prepares to display the game image (N:2) based on this control signal.
  • the display control unit 411 When the display control unit 411 has completed preparations for displaying the game image (N:2), it displays the game image (N:2) on the monitor 400 during the display time slot DTS1 starting at time t13.
  • the monitor delay time MD (N:2) is completed by time t13. That is, the game image (N:1) is displayed in the display time slot DTS0, and the same game image (N:2) is also displayed in the next display time slot DTS1.
  • the processing command unit 321 reads out the game image (N) again from the data storage unit 340 a little after time t13.
  • the processing command unit 321 commands the output unit 312 to display the game image (N:3).
  • the receiving unit 421 of the monitor 400 receives a control signal corresponding to the game image (N:3) transmitted by the game terminal 300.
  • the display control unit 411 prepares to display the game image (N:3).
  • preparations for displaying the game image (N:3) are complete, and the game image (N:3) is displayed on the monitor 400 during the display time slot DTS2.
  • the game image (N) is displayed three times at this stage. To the player, the game image (N) appears to be displayed continuously.
  • the processing command unit 321 reads out the game image (N) again from the data storage unit 340 a little after time t14 in case the next game image (N+1) cannot be prepared for display by time t15.
  • the processing command unit 321 commands the output unit 312 to display game image (N:4).
  • the receiver 421 of the monitor 400 receives a control signal corresponding to the game image (N:4) transmitted by the game terminal 300.
  • the display controller 411 prepares to display the game image (N:4).
  • preparations for displaying the game image (N:4) are completed, and the game image (N:4) is displayed on the monitor 400 during the display time slot DTS3.
  • the game terminal 300 receives the encoded image (N+1) during display time slot DTS1 (times t13 to t14). Furthermore, at time t16 during display time slot DTS3 (times t15 to t17), the image restoration unit 322 completes the decoding of the encoded image (N+1). In other words, the game image (N+1) will not be able to be displayed in time for display in display time slot DTS3, but it will be able to be displayed in time for the next display time slot DTS4. After the start of display time slot DTS3 for the game image (N:4), the process command unit 321 commands the output unit 312 to display the new game image (N+1).
  • the receiving unit 421 of the monitor 400 receives a control signal corresponding to the new game image (N+1) transmitted by the game terminal 300.
  • the display control unit 411 prepares to display the game image (N+1:1) based on this control signal.
  • preparations for displaying the game image (N+1:1) are completed, and the game image (N+1:1) is displayed on the monitor 400 during the display time slot DTS4. That is, the game image (N) is displayed four times at this stage, and then the game image (N+1) is displayed.
  • the display control unit 411 of the monitor 400 sequentially displays game images that are ready to be displayed on the monitor 400 at the start of each display time slot.
  • the processing command unit 321 of the game terminal 300 commands the output unit 312 to read out the game images stored in the data storage unit 340 a specific number of times, depending on the ratio between the generation rate and the display rate, and display them in each display time slot.
  • the ratio between the generation rate and the display rate is 1:4.
  • four display time slots arrive during one processing time slot. That is, one game image is displayed four times. Therefore, the processing command unit 321 commands the output unit 312 to read out the game images stored in the data storage unit 340 four times, and display them for each display time slot.
  • the processing command unit 321 may command the output unit 312 to display an additional game image (N:5).
  • the processing command unit 321 obtains the game image (N) from the data storage unit 340 in advance, slightly after time t15.
  • the processing command unit 321 may command the output unit 312 to display the game image (N:5) in the display time slot DTS4.
  • the game image (N) is additionally displayed in the next display time slot, thereby preventing interruption in the continuous display of the game images.
  • the image restoration unit 322 completes the decode time DC(N+1) early.
  • the decode time DC(N+1) is completed during the display time slot DTS1.
  • the processing command unit 321 acquires a new game image (N+1) before the completion time t14 of the display time slot DTS1.
  • the display control unit 411 has not started preparing to display the game image (N:4) at the time of completion of the decode time DC(N+1).
  • the processing command unit 321 may instruct the monitor 400 to display the game image (N+1:1) instead of the game image (N:4).
  • the game image (N+1:1) is displayed on the monitor 400 instead of the game image (N:4).
  • the game image (N) scheduled for display in the next display time slot can be changed to the new game image (N+1), allowing the new game image (N+1) to be displayed earlier.
  • the game system according to this embodiment has been described above.
  • This embodiment assumes a situation in which the display rate of the monitor is faster than the generation rate of the server in a game system in a remote environment.
  • the speed at which the server generates game images cannot keep up with the speed at which the monitor displays the game images.
  • the server may not provide the game images or the monitor may not be ready to display them in time for the monitor to display the next game image, so that the game image may not be displayed on the monitor.
  • one game image is transferred multiple times from the game terminal to the monitor so that the game image can be displayed in each display time slot. Therefore, the monitor is always ready to display the next game image at the start of each display time slot. This allows the monitor to continue displaying game images without interruption.
  • delay time can be alleviated by introducing a monitor with a high-speed display rate.
  • delay time mainly consists of encoder time, transmission time, decoder time, and monitor delay time.
  • the encoder time, transmission time, and decoder time are not constant.
  • none of these times can be directly adjusted by the player.
  • the monitor delay time can be shortened by increasing the display rate of the monitor 400. This can be easily achieved if the player introduces a monitor that can increase the display rate. From the above, it is assumed that the game system of this embodiment allows the player to play the game without feeling the stress of display delay.
  • the present invention is not limited to the above-described embodiments and modifications, and can be embodied by modifying the components without departing from the spirit of the invention.
  • Various inventions may be formed by appropriately combining multiple components disclosed in the above-described embodiments and modifications.
  • some components may be deleted from all the components shown in the above-described embodiments and modifications.
  • the remote game system 200 has been described as having a function for adjusting the game image display inside the game terminal 300.
  • this image display adjustment function may be implemented as hardware or software.
  • this display adjustment device When implemented as hardware, it may be constructed as a display adjustment device.
  • This display adjustment device may be mounted on the game terminal 300 and the server 201.
  • this display adjustment device may be constructed as an independent device such as a relay device.
  • the relay device may be connected between the server 201 and the communication network 202.
  • the relay device may be connected between the communication network 202 and the game terminal 300.
  • This display adjustment program When implemented as software, it may be constructed as a display adjustment program.
  • This display adjustment program may be installed in the game terminal 300, the monitor 400, and the relay device described above.
  • the configuration of the game system does not need to depend on the remote environment.
  • the personal computer 101 may be equipped with a display adjustment program.
  • the game program executes calculation processing.
  • the GPU generates a game image corresponding to the result of this processing.
  • the display adjustment program performs display adjustment of the game image according to this embodiment.
  • the monitor 102 may display the game image adjusted by the display adjustment program.
  • the difference between the generation rate and the display rate may be adjusted by the display adjustment device or display adjustment program of the personal computer 101.
  • the monitor 400 may adjust this difference.
  • the monitor 400 has a data storage unit for storing game images.
  • the process command unit 321 of the game terminal 300 commands the monitor 400 to display the game image.
  • the monitor 400 displays a game image (N:1) in display time slot DTS0.
  • the monitor 400 may store the game image (N) in the data storage unit.
  • the processing command unit 321 is unable to command the monitor 400 to display a new game image (N+1) in time for the start of the next display time slot DTS1.
  • the monitor 400 may display the game image (N) in the data storage unit as game image (N:2) during display time slot DTS1.
  • the process command unit 321 commands the monitor 400 to display a new game image (N+1) in time for the start of the next display time slot DTS1.
  • the monitor 400 may display the game image (N+1:1) in the display time slot DTS1.
  • the monitor 400 overwrites and saves the new game image (N+1) in the data storage unit.
  • the monitor 400 may display the game image (N+1) in the data storage unit 340 as the image (N+1:2).
  • the monitor 400 may repeatedly display the game images stored in the data storage unit until the new game image is ready to be displayed.
  • the image generation unit 213 of the server 201 generates game image (NA), game image (NB), game image (NC), and game image (ND) which have the same drawing content as game image (N).
  • the server 201 transmits these four images to the game terminal 300. In this case, four game images are transmitted sequentially per processing time slot.
  • the processing command unit 321 transmitting game image (NA), game image (NB), game image (NC), and game image (ND) to the monitor 400 in succession, the monitor 400 can display game images at a display rate of 4R (times/second).
  • the image generation unit 213 may duplicate one game image (NA) to duplicate game image (NB), game image (NC), and game image (ND). Alternatively, the image generation unit 213 may independently generate each of the four game images (NA) to (ND). For example, assume that the image generation unit 213 is provided within the server 201 with four independent GPUs, GPU (A), GPU (B), GPU (C), and GPU (D). In this case, within one processing time slot, each GPU may independently generate game image (N). The game images independently generated by the four GPUs may be identical, but do not have to be completely identical.
  • the processing command unit 321 of the game terminal 300 may generate the next game image to be transmitted to the monitor 400 (hereinafter referred to as a "predicted image") from previously acquired game images and the currently acquired game image.
  • the processing command unit 321 acquires a game image (N). At this time, the processing command unit 321 generates a predicted image (N+1) to be transmitted next to the monitor 400 from a past game image (N-1) stored in the data storage unit 340 and the currently acquired game image (N). A method for predicting a future image from consecutive past images can be achieved by applying known technology.
  • the processing command unit 321 commands the output unit 312 to display the predicted image (N+1). After transmitting the game image (N) to the monitor 400, the processing command unit 321 may repeatedly command the output unit 312 to display the predicted image (N+1). Assume that time has passed and the processing command unit 321 has acquired an actual new game image (N+1).
  • the processing command unit 321 may command the output unit 312 to display the new game image (N+1).
  • this prediction function may be implemented by the monitor in a monitor adjustment method. After the new game image (N+1), the next new game image (N+2) may be displayed. If the next new game image (N+2) cannot be prepared for display in time, a predicted image (N+2) generated from the game image (N) and the new game image (N+1) may be displayed.
  • the monitor 400 will always continue to display a game image with the same drawing content during each display time slot DTS.
  • the monitor 400 may display the next game image midway through the display time slot DTS.
  • Fig. 9 is a time chart showing an enlarged transfer process of the image restoration unit 322, the display control unit 411, and the display on the monitor 400 in Fig. 8 in a modified example.
  • the modified example shown in Fig. 9 shows a flow in which a new game image (N+1) is displayed on the monitor 400 from the middle of the display time slot DTS3.
  • illustration of the monitor delay times MD(N:2), MD(N:3), and MD(N:4) is omitted.
  • the decode time DC(N+1) is completed at time t19 during the display time slot DTS2.
  • the process command unit 321 immediately commands the output unit 312 to display a new game image (N+1).
  • the monitor 400 After the monitor delay time MD(N+1:1), the monitor 400 becomes ready to display the game image (N+1:1) at time t20 during the display time slot DTS3.
  • the monitor 400 may display the game image (N+1:1) from time t20 before time t17 without waiting for the start time t17 of the next display time slot DTS4.
  • the game image (N:4) is displayed on the monitor 400 from time t15 to t20, which is the first half of the display time slot DTS3.
  • the game image (N+1:1) is displayed from time t20 to t17, which is the remaining time of the display time slot DTS3.
  • the monitor 400 continues to display the game image (N+1:2).
  • the game image is displayed on the monitor 400 from the point in time when it is ready to be displayed, without waiting for the start of the next display time slot.
  • the delay time can be further reduced compared to when the display adjustment of the game image according to this embodiment is implemented.
  • This modified example may be applied to video distribution services and the like.
  • it may be used in the distribution of video of live sports broadcasts.
  • the thrill is in witnessing decisive moments that determine the outcome of a match, such as the moment a point is scored or a move is executed, in real time.
  • it is better to be able to display images regardless of the interval of the display time slot.
  • the generation rate is set to R (times/second) and the display rate is set to 4R (four times R) (times/second).
  • the ratio between the generation rate and the display rate does not have to be limited to 1:4.
  • the ratio between the generation rate and the display rate may be set to 1:8.
  • the ratio between the generation rate and the display rate does not have to be limited to an integer ratio.
  • the ratio between the generation rate and the display rate may be set to 1:3.5.
  • the acceptable range for delay times is within 100 milliseconds. It is also said that even general players feel stressed when the delay time reaches 200 milliseconds. For example, assume that a monitor 400 with a display rate of 60 Hz is installed in the remote game system 200. In this case, it is said that a monitor delay time of 8 milliseconds is required. 8 milliseconds accounts for approximately 8% of the aforementioned acceptable range of delay times for the professional player. Therefore, for this player, even a small reduction in the monitor delay time can greatly change the comfort of playing the game.
  • the present invention can reduce the delay time by increasing the display rate of the monitor 400 and continuously transmitting game images to the monitor 400.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
PCT/JP2023/009428 2023-03-10 2023-03-10 ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム Ceased WO2024189704A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2025506260A JPWO2024189704A1 (https=) 2023-03-10 2023-03-10
PCT/JP2023/009428 WO2024189704A1 (ja) 2023-03-10 2023-03-10 ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/009428 WO2024189704A1 (ja) 2023-03-10 2023-03-10 ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム

Publications (2)

Publication Number Publication Date
WO2024189704A1 true WO2024189704A1 (ja) 2024-09-19
WO2024189704A9 WO2024189704A9 (ja) 2025-01-16

Family

ID=92754621

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/009428 Ceased WO2024189704A1 (ja) 2023-03-10 2023-03-10 ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム

Country Status (2)

Country Link
JP (1) JPWO2024189704A1 (https=)
WO (1) WO2024189704A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016009936A (ja) * 2014-06-23 2016-01-18 トヨタ自動車株式会社 車両周辺監視装置
US20210160530A1 (en) * 2019-11-27 2021-05-27 Sony Interactive Entertainment Inc. Systems and methods for decoding and displaying image frames
JP2021158508A (ja) * 2020-03-26 2021-10-07 株式会社ソニー・インタラクティブエンタテインメント 画像処理装置および画像処理方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016009936A (ja) * 2014-06-23 2016-01-18 トヨタ自動車株式会社 車両周辺監視装置
US20210160530A1 (en) * 2019-11-27 2021-05-27 Sony Interactive Entertainment Inc. Systems and methods for decoding and displaying image frames
JP2021158508A (ja) * 2020-03-26 2021-10-07 株式会社ソニー・インタラクティブエンタテインメント 画像処理装置および画像処理方法

Also Published As

Publication number Publication date
WO2024189704A9 (ja) 2025-01-16
JPWO2024189704A1 (https=) 2024-09-19

Similar Documents

Publication Publication Date Title
JP7436644B2 (ja) レイテンシとビデオ品質の間のトレードオフをクラウドゲーミングアプリケーションにおいて改善するためのエンコーダ調整
JP7637299B2 (ja) クラウドゲーミング方法、コンピュータ可読媒体、およびコンピュータシステム
JP7494203B2 (ja) ユーザ入力デバイスの待ち時間調整
EP2603848B1 (en) Cloning or extending a computer desktop on a wireless display surface
US20170007923A1 (en) System and Method for Improving the Graphics Performance of Hosted Applications
US8170701B1 (en) Methods and apparatus of running game and rendering game audio remotely over broadband network
US10075525B2 (en) Output data providing server and output data providing method
US20160303479A1 (en) System and method for improving the graphics performance of hosted applications
EP4282500A1 (en) Multiplayer video game systems and methods
US20140213353A1 (en) Apparatus and method for providing streaming-based game images
CN114786024B (zh) 直播间游戏同步方法、系统、装置、设备及存储介质
JPWO2021067317A5 (https=)
US11562715B2 (en) Graphics processing systems
WO2024189704A1 (ja) ゲームシステム、ゲーム端末、表示調整装置およびコンピュータプログラム
US12170801B2 (en) Server-side frame render timing delay to reduce client-side frame present delay
US20250330553A1 (en) System and method for frame rate control and priority frame processing for 3d rendering
US20250387704A1 (en) Information processing system, storage medium, and information processing method
GB2644123A (en) Multiplayer gaming system and method
JP2026000840A (ja) 情報処理システム、ホスト装置、プログラム、および、コンピュータに実装された方法
JP2026000841A (ja) 情報処理システム、プログラム、および、コンピュータに実装された方法
CN121967738A (zh) 低延时视频流-减少帧缓冲区
EP4397038A1 (en) A multi-try encoding operation for streaming applications
WO2013040261A1 (en) System and method for improving the graphics performance of hosted applications

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23927328

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2025506260

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23927328

Country of ref document: EP

Kind code of ref document: A1