JP7441735B2 - Distribution server and image distribution method - Google Patents

Description

The present invention relates to a technique for sharing images among multiple users.

Patent Document 1 discloses an image of a game being played by a host user in an environment where the host user's information processing device and the guest user's information processing device are connected via P2P (Peer to Peer) without going through a server. An image sharing system for sharing images with guest users is disclosed. In this image sharing system, there are two sharing modes: a guest user viewing game images (Share Screen), a sharing mode where a guest user plays the game on behalf of the host user (Hand over my controller), and a guest user new to the game. A shared mode (Hand over another controller) is provided in which a host user and a guest user participate as a player and play the game together.

JP 2017-35298 Publication

Game network services serve as communication tools, and are expected to enable more active communication by allowing multiple users participating in the same chat room to share game images, for example. Therefore, it is preferable to implement a mechanism that allows a user playing a game to easily share the game image being played with other users. It is expected that smooth communication can be supported not only in games but also in network services that connect multiple information processing devices, such as conference systems, by creating a mechanism that allows images to be easily shared.

Therefore, an object of the present invention is to provide a useful technique for sharing images.

In order to solve the above problems, a distribution server according to an aspect of the present invention is a distribution server that distributes image data, and includes an image acquisition unit that acquires image data from a first terminal device, and a distribution server that distributes the acquired image data. a conversion unit that transcodes image data with a resolution lower than the resolution of the image data; and a distribution unit that distributes image data with a resolution corresponding to a connection state between the second terminal device and the second terminal device. Be prepared.

Another aspect of the present invention is an image distribution method, which includes the steps of acquiring image data from a first terminal device, and transcoding the acquired image data into image data having a lower resolution than the resolution of the image data. and a step of distributing, to the second terminal device, image data having a resolution according to a connection state between the second terminal device and the second terminal device.

Note that arbitrary combinations of the above-mentioned components and expressions of the present invention converted between methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of the present invention.

According to the present invention, a useful technique for sharing images can be provided.

1 is a diagram showing an image sharing system according to an embodiment of the present invention. 1 is a diagram showing a hardware configuration of an information processing device. FIG. 2 is a diagram showing functional blocks of an information processing device that transmits game images. It is a figure showing an example of a game screen. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. It is a diagram showing functional blocks of a distribution server. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. FIG. 2 is a diagram showing functional blocks of an information processing device that receives a screen-shared game image. It is a figure showing an example of a game screen displayed on an output device. FIG. 3 is a diagram showing an example of a message displayed on an output device. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. FIG. 6 is a diagram illustrating a display example in a full-page display format. FIG. 3 is a diagram illustrating a display example in a picture-in-picture display format. FIG. 3 is a diagram illustrating a display example in a screen split format. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. FIG. 3 is a diagram showing an example of a message displayed on an output device. FIG. 3 is a diagram showing an example of a system image superimposed on a game image. FIG. 3 is a diagram showing an example of a system image displayed on an output device. FIG. 3 is a diagram showing a game image played by a user.

FIG. 1 shows an image sharing system 1 according to an embodiment of the present invention. The image sharing system 1 includes a plurality of information processing devices 10a, 10b, 10c, and 10d (hereinafter referred to as "information processing devices 10" unless otherwise distinguished), a management server 5, and a distribution server 9. , these are connected via a network 3 such as the Internet or a LAN (Local Area Network).

The information processing devices 10a, 10b, 10c, and 10d are terminal devices operated by users, and each has an output device 4a, 4b, 4c, and 4d (hereinafter referred to as "output device 4" unless otherwise distinguished). Connecting. The output device 4 may be a television that has a display that outputs images and a speaker that outputs sound, or may be a head-mounted display. The output device 4 may be connected to the information processing device 10 with a wired cable or wirelessly.

The access point (hereinafter referred to as "AP") 8 has the functions of a wireless access point and a router, and the information processing device 10 connects to the AP 8 via wireless or wired, and connects to the management server 5 on the network 3. It is communicatively connected to the distribution server 9 and other information processing devices 10 .

The information processing device 10 is connected wirelessly or wired to an input device 6 operated by a user, and the input device 6 outputs operation information indicating the result of the user's operation to the information processing device 10. When the information processing device 10 receives the operation information from the input device 6, it reflects it in the processing of system software and application software, and outputs the processing result from the output device 4. In the image sharing system 1, the information processing device 10 is a game device that executes a game, and the input device 6 may be a device such as a game controller that supplies user operation information to the information processing device 10. The input device 6 may include a plurality of input units such as a plurality of push-type operation buttons, an analog stick that can input an analog quantity, and a rotary button.

The auxiliary storage device 2 is a storage such as an HDD (hard disk drive) or an SSD (solid state drive), and may be a built-in storage device, or an external storage device connected to the information processing device 10 via a USB (Universal Serial Bus) or the like. It may also be a storage device. A camera 7, which is an imaging device, is provided near the output device 4 and images the space around the output device 4.

In the image sharing system 1, users A, B, C, and D are room members who are in the same chat room, and the users can text chat with each other, and if they have a headset, they can use voice communication. You can chat. The chat room is created and managed by the management server 5, and upon receiving chat data (text data and/or voice data) sent from each user, the management server 5 forwards it to other users in the room. . For example, the management server 5 transfers chat data sent from user A to other users B, C, and D in the room.

In the chat room, users can share images of the game they are playing with other users. Although an upper limit may be set on the number of participants who can enter a chat room, there may be no limit to the number of participants. Note that the chat room is an example of a virtual room or group where online users gather, and may be another type of room or group.

The management server 5 is maintained and managed by the operator of the image sharing system 1, and provides network services including chat services to users of the image sharing system 1. The management server 5 manages network accounts that identify users, and users sign in to network services using the network accounts. By signing in to a network service, a user can enter a chat room and communicate with other room members. Note that by signing in, the user can also save, for example, game save data in the management server 5.

The distribution server 9 is maintained and managed by the operator of the image sharing system 1, and provides users participating in the chat room with a service of distributing streaming data of game images being played by other users. Although the streaming data naturally includes game sounds, the description of the distribution of the game sounds will be omitted below, and the distribution of the game images will be mainly described.

FIG. 2 shows the hardware configuration of the information processing device 10. The information processing device 10 includes a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, and a wired communication The system includes a module 40, a subsystem 50, and a main system 60.

The main system 60 includes a main CPU (Central Processing Unit), a memory serving as a main storage device, a memory controller, a GPU (Graphics Processing Unit), and the like. The GPU is mainly used for arithmetic processing of game programs. These functions may be configured as a system-on-chip and formed on one chip. The main CPU has a function of executing a game program recorded in the auxiliary storage device 2 or the ROM medium 44.

The subsystem 50 includes a sub CPU, a memory serving as a main storage device, a memory controller, and the like, but does not include a GPU and does not have the function of executing a game program. The number of circuit gates of the sub CPU is smaller than the number of circuit gates of the main CPU, and the operating power consumption of the sub CPU is smaller than the operating power consumption of the main CPU.

The main power button 20 is an input unit through which a user inputs an operation, is provided on the front of the casing of the information processing device 10, and turns on or off the power supply to the main system 60 of the information processing device 10. be manipulated for. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off.

The system controller 24 detects depression of the main power button 20 by the user. When the main power button 20 is pressed while the main power is in the OFF state, the system controller 24 acquires the pressing operation as an "on instruction"; on the other hand, when the main power is in the ON state, the main power When the button 20 is pressed, the system controller 24 acquires the pressing operation as an "off instruction".

The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24, subsystem 50, and main system 60. The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that transfers information between devices like a south bridge. As illustrated, devices such as a system controller 24, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60 are connected to the device controller 30. Ru. The device controller 30 absorbs differences in electrical characteristics and data transfer speeds of each device, and controls data transfer timing.

The media drive 32 is a drive device that loads and drives a ROM medium 44 in which application software such as games and license information are recorded, and reads programs, data, etc. from the ROM medium 44 . The ROM medium 44 may be a read-only recording medium such as an optical disk, a magneto-optical disk, or a Blu-ray disk.

The USB module 34 is a module that is connected to an external device using a USB cable. The USB module 34 may be connected to the auxiliary storage device 2 and the camera 7 using a USB cable. The flash memory 36 is an auxiliary storage device that constitutes internal storage. The wireless communication module 38 wirelessly communicates with, for example, the input device 6 using a communication protocol such as the Bluetooth (registered trademark) protocol or the IEEE802.11 protocol. The wired communication module 40 performs wired communication with external devices and connects to the network 3 via the AP 8.

In the image sharing system 1, members participating in the chat room can use three sharing modes to share game images. Hereinafter, a user who distributes game images will be referred to as a "host" or "host user," and a user who receives game images will be referred to as a "guest" or "guest user."

<Sharing mode 1>
Sharing mode 1 is a mode in which a guest user views a host user's game image, and is called "screen sharing." In screen sharing, a host user's game image is shared with a guest user via the distribution server 9. That is, the host user's information processing device 10 transmits the game image to the distribution server 9, and the guest user's information processing device 10 receives the game image from the distribution server 9. The guest user can view the host user's game images, but cannot operate the game.

<Sharing mode 2>
Sharing mode 2 is a mode in which a guest user plays a game in place of the host user while viewing the host user's game image, and is called "assist play." In assisted play, the host user's game image is shared with the guest user without going through the distribution server 9. In other words, the host user's information processing device 10 and the guest user's information processing device 10 are connected via P2P, and game images are shared. Since the host user hands over the right to operate his own game to the guest user, the host user cannot operate the game, and only the guest user can operate the game.

<Sharing mode 3>
Sharing mode 3 is a mode in which a guest user views the game image of the host user, participates in the game as a new player, and plays the game together with the host user, and is called "cooperative play." In joint play, the host user's game image is shared with the guest user without going through the distribution server 9. In other words, the host user's information processing device 10 and the guest user's information processing device 10 are connected via P2P, and game images are shared. In joint play, the guest user also uses the host user's game resources, the host user participates in the game as player 1, the guest user participates in the game as player 2, and the host user and guest user can operate the game together.

In this manner, in sharing modes 2 and 3, the host user's information processing device 10 and the guest user's information processing device 10 are connected via P2P to share game images, and at least the guest user has the right to operate the game. In the sharing modes 2 and 3, the guest user can play the game, so the sharing modes 2 and 3 are sometimes collectively referred to as "share play."

One user becomes a host user by distributing an image of a game that he or she is playing, and becomes a guest user by viewing an image of a game that someone else is playing. Therefore, one information processing device 10 is equipped with both a transmitting side configuration for becoming a host user and a receiving side configuration for becoming a guest user. For convenience of explanation, the information processing device 10a of user A has a transmitting side configuration for transmitting game images, and the information processing device 10c of user C has a receiving side configuration for receiving game images. A case will be described in which the transmitting side configuration and the receiving side configuration are both installed in one information processing device 10.

FIG. 3 shows functional blocks of an information processing device 10a that transmits images of a game that user A is playing. The information processing device 10a includes a processing section 100a, a communication section 102a, and a reception section 104a. The processing section 100a includes an execution section 110a, a system image generation section 120a, an image processing section 140a, a frame buffer 150a, and a shared processing section 160a.

The system image generation section 120a includes a notification generation section 122a and a room image generation section 124a. The frame buffer 150a includes a game buffer 152a for temporarily storing game image data and a system buffer 154a for temporarily storing system image data. The sharing processing section 160a includes a situation information transmission section 162a, a situation information acquisition section 164a, a transmission processing section 166a, an invitation transmission section 172a, and a connection processing section 174a. The transmission processing section 166a includes a first transmission processing section 168a and a second transmission processing section 170a.

In FIG. 3, each element described as a functional block that performs various processes can be configured with a circuit block, memory, or other LSI in terms of hardware, and can be configured with system software, memory, etc. in terms of software. This is realized by a game program loaded on the computer. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various ways using only hardware, only software, or a combination thereof, and are not limited to either.

The communication unit 102a receives operation information regarding user A's operation of the input unit of the input device 6. The communication unit 102a also receives chat data from other room members in the chat room from the management server 5, and further receives information indicating the status of the other room members.

The communication unit 102a transmits information indicating the user A's status to the management server 5. The communication unit 102a also transmits streaming data of game images and game sounds generated by the processing unit 100a to the distribution server 9 and/or other information processing device 10. In the following, streaming data for playing a game video may be simply referred to as a game image. The functional block of the communication unit 102a is expressed as a configuration having both the functions of the wireless communication module 38 and the wired communication module 40 shown in FIG.

The reception unit 104a is provided between the communication unit 102a and the processing unit 100a, and transmits data or information between the communication unit 102a and the processing unit 100a. Upon receiving operation information from the input device 6 via the communication section 102a, the reception section 104a supplies the operation information to a predetermined functional block in the processing section 100a.

The execution unit 110a executes a game program (hereinafter also simply referred to as a "game"). The functional block shown here as the execution unit 110a is realized by software such as system software and game software, and hardware such as a GPU. The execution unit 110a executes the game program to generate game image data and game sound data. Note that a game is an example of an application, and the execution unit 110a may execute applications other than games.

While the user A is playing the game, the execution unit 110a executes the game program and performs calculation processing to move the game character in the virtual space based on the operation information input by the user A to the input device 6. The GPU receives the results of arithmetic processing in the virtual space and generates game image data from a viewpoint position (virtual camera) in the virtual space.

FIG. 4 shows an example of a game screen displayed on user A's output device 4a. User A is playing a game titled "Special Soccer." When the execution unit 110a generates game image data and supplies it to the image processing unit 140a, the image processing unit 140a temporarily stores the game image data in a game buffer 152a, and generates a display image from the image data temporarily stored in the frame buffer 150a. is generated and provided to the output device 4a. As a result, the output device 4a outputs the game image. Note that the actual output device 4a also outputs the game sound generated by the execution unit 110a, and the user A operates the input device 6 to play the game while viewing the game image and game sound output from the output device 4a. Play the title "Special Soccer".

In the sharing processing unit 160a, the status information transmitting unit 162a transmits information indicating the status of user A to the management server 5. The information indicating the situation includes information indicating whether a game is being played, the title of the game if a game is being played, information indicating whether the microphone is on or off, and information regarding image sharing. The information regarding image sharing includes information when user A is in the position of a host user and information when user A is in the position of a guest user. Information when in the position of the host user includes information indicating that user A will start screen sharing (sharing mode 1), and information indicating that user A will start screen sharing (sharing mode 1), and information indicating that user A will start screen sharing (sharing mode 2) and room members for assist play (sharing mode 2) or collaborative play (sharing mode 3). Contains information indicating that the person is inviting. The information when the user is in the guest user's position includes information regarding the sharing mode of game images being distributed by another room member. Specifically, when another room member views an image distributed from the distribution server 9, the situation information transmitting unit 162a transmits information indicating that the image is to be viewed to the management server 5, and When accepting an invitation for assist play or joint play, information indicating acceptance of the invitation is transmitted to the management server 5.

The function of the status information transmitting unit 162a is also installed in the information processing devices 10b to 10d of users B to D, so each user's information processing device 10 transmits information indicating the status of each user to the management server 5. do. Preferably, when a change occurs in the situation, the information processing device 10 immediately transmits information indicating the change to the management server 5. The management server 5 acquires information indicating the status of each user from each information processing device 10, and manages the latest status of each user. The management server 5 transmits information indicating the status of each user to the information processing devices 10 belonging to the same chat group. The situation information acquisition unit 164a in the information processing device 10a acquires information indicating the situation of each user.

The information processing device 10 of the embodiment provides a mechanism that allows user A, who is playing a game, to easily share images. During game play, when user A briefly presses a predetermined button on the input device 6, the reception unit 104a accepts the button operation and supplies the operation information to the system image generation unit 120a. The system image generation unit 120a acquires the button operation information as a system image display request, and calls the situation information acquisition unit 164a.

The status information acquisition unit 164a acquires information indicating the status of each user from the management server 5, and provides the information to the system image generation unit 120a. In the system image generation section 120a, the room image generation section 124a generates system image data indicating the status of the chat room in which user A is participating, and supplies it to the image processing section 140a. The image processing unit 140a temporarily stores system image data in a system buffer 154a, generates a display image from the image data temporarily stored in the frame buffer 150a, and provides the display image to the output device 4a. Specifically, the image processing unit 140a generates a display image by combining the game image data temporarily stored in the game buffer 152a and the system image data temporarily stored in the system buffer 154a, and provides the display image to the output device 4a. As a result, the output device 4a outputs a display image in which the system image is superimposed on the game image.

FIG. 5 shows an example of a system image 200 superimposed on a game image. The room image generation unit 124a generates system image data based on the information indicating the status of each user acquired by the status information acquisition unit 164a. The system image 200 is provided with a member display field 202 that shows the status of the members of the chat room in which user A is participating. The room image generation unit 124a generates a member display field 202 including information about the members based on information indicating the status of a plurality of room members (users). The member display field 202 shown in FIG. 5 displays the user's icon, user name, the title of the game being played, and information indicating whether the microphone is on or off. may also include information indicating.

The sharing start button 204 is an operation element for user A to start screen sharing, which is sharing mode 1. When user A operates the sharing start button 204, the status information transmitting unit 162a transmits information indicating that user A starts screen sharing (sharing mode 1) to the management server 5 and the distribution server 9. 1 transmission processing unit 168a transmits to the distribution server 9 streaming data of the game image read out from the game buffer 152a by the image processing unit 140a. As described above, the streaming data also includes game sound data, and the first transmission processing unit 168a compresses the streaming data in a predetermined format and transmits it to the distribution server 9.

At this time, the image processing section 140a reads only the game image data temporarily stored in the game buffer 152a and provides it to the first transmission processing section 168a, and converts the system image data temporarily stored in the system buffer 154a into game image data. Do not synthesize. As a result, the image data to be distributed does not include system image data, and only game image data can be distributed.

The first transmission processing unit 168a determines the resolution of the game image data to be transmitted to the distribution server 9, depending on the quality of the connection between the communication unit 102a and the distribution server 9. That is, if the connection state is poor, the first transmission processing unit 168a decides to lower the resolution of the game image data. Note that the game image data is generated by the execution unit 110a at a frame rate of 60 fps (frames/second) or 30 fps, but the first transmission processing unit 168a may change the resolution if the connection state with the distribution server 9 is extremely poor. In addition to this, you can also lower the frame rate.

When game image data with a resolution of 1080p is temporarily stored in the game buffer 152a, if the connection state is good, the image processing section 140a supplies the game image data with a resolution of 1080p to the first transmission processing section 168a. However, if the connection state is not good, the image processing unit 140a needs to lower the resolution of the game image. The image processing unit 140a lowers the resolution to 720p if it is difficult to transmit a game image with a resolution of 1080p, lowers the resolution to 540p if it is difficult to transmit a game image with a resolution of 720p, and lowers the resolution if it is difficult to transmit a game image with a resolution of 540p. Lower it to 360p.

The first transmission processing unit 168a determines the quality of the connection between the communication unit 102a and the distribution server 9, and instructs the image processing unit 140a to request conversion of the game image to a resolution corresponding to the determined quality. . Thereby, the image processing section 140a reconstructs the game image data at the instructed resolution, and the first transmission processing section 168a transmits the reconstructed game image data to the distribution server 9. Note that the first transmission processing section 168a may constantly monitor the connection state and, when the connection state changes, instruct the image processing section 140a to set a resolution according to the change.

FIG. 6 shows functional blocks of the distribution server 9. The distribution server 9 includes a control section 300 and a communication section 302. The control unit 300 includes an image acquisition unit 310, a conversion unit 312, and a distribution unit 314. The distribution unit 314 holds member information of the chat room, and upon receiving a viewing request from a member in the same chat room, distributes the game image to the information processing device 10 of the member.

In FIG. 6, each element described as a functional block that performs various processes can be configured with a circuit block, memory, or other LSI in terms of hardware, and can be configured with system software, memory, etc. in terms of software. This is realized by a game program loaded on the computer. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various ways using only hardware, only software, or a combination thereof, and are not limited to either.

The image acquisition unit 310 acquires image data streamed from the information processing device 10a. The resolution of the image data is dynamically set according to the connection state between the information processing device 10a and the distribution server 9, and here, game image data with a resolution of 1080p and 60 fps is transmitted from the information processing device 10. It is assumed that there is The conversion unit 312 transcodes the acquired image data into image data of a plurality of transcodable resolutions. Specifically, the conversion unit 312 transcodes the acquired image data into image data having a lower resolution than the resolution of the image data.

The conversion unit 312 of the embodiment has a function of converting image data to a resolution of 720p, 540p, or 360p, and converts the resolution of the image data acquired by the image acquisition unit 310 to a lower resolution. Therefore, when the image acquisition unit 310 acquires 1080p image data, the conversion unit 312 converts the 1080p image data into 720p, 540p, or 360p image data.

Note that when the image acquisition unit 310 acquires 720p image data, the conversion unit 312 converts the 720p image data into 540p and 360p image data. When the image acquisition unit 310 acquires 540p image data, the conversion unit 312 converts the 540p image data into 360p image data. The conversion unit 312 executes this conversion process regardless of whether or not the image data is distributed, and waits for viewing requests from other members in the same chat room.

FIG. 7 shows an example of a system image 200 superimposed on user A's game image. The room image generation unit 124a generates system image data based on the information indicating the status of each user acquired by the status information acquisition unit 164a. Compared to the situation when the system image 200 shown in FIG. 5 is displayed, user A himself has started screen sharing in the chat room. Note that since the information processing device 10a side can recognize that user A has started screen sharing, the situation information acquisition unit 164a acquires information indicating the situation of user A as internal information of the sharing processing unit 160a. Good too.

The room image generation unit 124a generates a member display column 202 in which information about users who are sending images and information about users who are not sending images are included in different areas based on information indicating the status of a plurality of users. generate. In the example shown in FIG. 7, the system image 200 is provided with a host display area 206 that displays information about users who are sending images, and a non-host display area 208 that displays information about users who are not sending images. It will be done. The host display area 206 and the non-host display area 208 are distinguished and provided as different areas as shown in FIG. Thereby, user A can easily recognize members who are performing screen sharing and members who are not performing screen sharing. Note that since the display area of the system image 200 is limited, the host display area 206 is placed above the non-host display area 208 so that user A can preferentially view information in the host display area 206. preferable.

The viewing member display area 212 is an area for displaying room members who are viewing the game image screen-shared by user A. In the state shown in FIG. 7, none of the members has viewed user A's game image yet.

The share play start button 210 is an operation element for the user A to start a share play in the second or third sharing mode. The room image generation unit 124a includes in the system image 200 a share play start button 210 for performing an operation to invite a room member to the game that is being played and for which screen sharing is being performed. When user A operates the share play start button 210, a window for inviting room members other than himself to share play is displayed, and user A can select the room members to invite.

FIG. 8 shows an example of a system image 200 superimposed on user A's game image. Compared to the system image 200 shown in FIG. 7, information indicating the user D is placed in the host display area 206, and therefore it can be seen that the user D is performing screen sharing. By providing the system image 200 with the host display area 206 that displays information about users who are sending images and the non-host display area 208 that displays information about users who are not sending images, users can A can easily recognize members who are performing screen sharing and members who are not performing screen sharing.

FIG. 9 shows functional blocks of an information processing device 10c that receives a screen-shared game image. The information processing device 10c includes a processing section 100c, a communication section 102c, and a reception section 104c. The processing section 100c includes an execution section 110c, a system image generation section 120c, an image processing section 140c, a frame buffer 150c, and a shared processing section 160c.

The system image generation section 120c includes a notification generation section 122c and a room image generation section 124c. The frame buffer 150c includes a game buffer 152c that temporarily stores game image data and a system buffer 154c that temporarily stores system image data. The sharing processing section 160c includes a situation information transmission section 162c, a situation information acquisition section 164c, a request transmission section 180c, an acceptance transmission section 184c, and an image acquisition section 186c. The image acquisition section 186c includes a first image acquisition section 188c and a second image acquisition section 190c.

In FIG. 9, each element described as a functional block that performs various processes can be configured with a circuit block, memory, or other LSI in terms of hardware, and can be configured with system software, memory, etc. in terms of software. This is realized by a game program loaded on the computer. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various ways using only hardware, only software, or a combination thereof, and are not limited to either. Note that structures having the same names in FIG. 3 and FIG. 9 may be the same structure.

The communication unit 102c receives operation information regarding user C's operation of the input unit of the input device 6. The communication unit 102c also receives chat data from other room members in the chat room from the management server 5, and further receives information indicating the status of the other room members. The communication unit 102c also receives streaming data of game images from the distribution server 9 and/or another information processing device 10. The communication unit 102c also transmits information indicating the status of the user C to the management server 5. The functional block of the communication unit 102c is expressed as a configuration having both the functions of the wireless communication module 38 and the wired communication module 40 shown in FIG.

The reception unit 104c is provided between the communication unit 102c and the processing unit 100c, and transmits data or information between the communication unit 102c and the processing unit 100c. Upon receiving operation information from the input device 6 via the communication section 102c, the reception section 104c supplies the operation information to a predetermined functional block in the processing section 100c.

The execution unit 110c executes the game program. The functional block shown here as the execution unit 110c is realized by software such as system software and game software, and hardware such as a GPU. The execution unit 110c executes the game program to generate game image data and game sound data. Note that a game is an example of an application, and the execution unit 110c may execute applications other than games.

During the game play by the user C, the execution unit 110c executes the game program and performs calculation processing to move the game character in the virtual space based on the operation information input by the user C to the input device 6. The GPU receives the results of arithmetic processing in the virtual space and generates game image data from a viewpoint position (virtual camera) in the virtual space.

FIG. 10 shows an example of a game screen displayed on user C's output device 4c. User C is playing a game titled "Combat Field." When the execution unit 110c generates game image data and supplies it to the image processing unit 140c, the image processing unit 140c temporarily stores the game image data in the game buffer 152c and creates a display image from the image data temporarily stored in the frame buffer 150c. is generated and provided to the output device 4c. As a result, the output device 4c outputs the game image. Note that the actual output device 4c also outputs the game sound generated by the execution unit 110c, so the user C operates the input device 6 while viewing the game image and game sound output from the output device 4c. Play the game title "Combat Field".

In the sharing processing unit 160c, the status information transmitting unit 162c transmits information indicating the status of user C to the management server 5. The information indicating the situation includes information indicating whether a game is being played, the title of the game if a game is being played, information indicating whether the microphone is on or off, and information regarding image sharing. The information regarding image sharing includes information when user C is in the position of a host user and information when user C is in the position of a guest user. Information when in the position of the host user includes information indicating that user C starts screen sharing (sharing mode 1), and information indicating that user C will start screen sharing (sharing mode 1), and information indicating that user C will start screen sharing (sharing mode 2) and room members in collaborative play (sharing mode 3). Contains information indicating that the person is inviting. The information when the user is in the guest user's position includes information regarding the sharing mode of game images being distributed by another room member. Specifically, when another room member views an image distributed from the distribution server 9, the situation information transmitting unit 162c transmits information indicating that the image is to be viewed to the management server 5, and the other room member When accepting an invitation for assist play or joint play, information indicating acceptance of the invitation is transmitted to the management server 5.

Here, the operation of the information processing device 10c when a room member other than user C starts screen sharing will be described. Referring to FIG. 5, when user A operates the sharing start button 204, the status information transmitting unit 162a of the information processing device 10a transmits information indicating that the user A starts screen sharing (sharing mode 1). The first transmission processing unit 168a transmits to the distribution server 9 the streaming data of the game image read from the game buffer 152a by the image processing unit 140a. When user A starts screen sharing, the management server 5 allows user A to start screen sharing on the information processing devices 10b, 10c, and 10d of users B, C, and D who are other room members in the chat room. We will notify you that we have done so.

FIG. 11 shows an example of a message 220 displayed on user C's output device 4c. The notification generation unit 122c generates system image data including the message 220 based on the notification from the management server 5, and supplies it to the image processing unit 140c. The image processing unit 140c temporarily stores system image data in a system buffer 154c, generates a display image from the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. Specifically, the image processing unit 140c generates a display image by combining the game image data temporarily stored in the game buffer 152c and the system image data temporarily stored in the system buffer 154c, and provides the display image to the output device 4c. As a result, the output device 4c outputs a display image in which the system image is superimposed on the game image. By seeing the message 220, user C recognizes that user A has started screen sharing. Note that the notification generation unit 122c may notify user C that user A has started screen sharing through audio output.

When the user C presses a predetermined button on the input device 6 while the message 220 is being displayed, the request transmitting unit 180c transmits a viewing request including information identifying the user A to the management server 5. Note that the request transmitting unit 180c may transmit a viewing request including information identifying user A to the distribution server 9.

Note that the message 220 is displayed for only about 5 seconds, for example. After the message 220 disappears, the user C can send a request to view the game image distributed by the user A to the management server 5 or the distribution server 9 by displaying the system image. When the user C briefly presses a predetermined button on the input device 6, the reception unit 104c accepts the button operation and supplies the operation information to the system image generation unit 120c. The system image generation unit 120c acquires the button operation information as a system image display request, and calls the situation information acquisition unit 164c.

The status information acquisition unit 164c acquires information indicating the status of each user from the management server 5, and provides the information to the system image generation unit 120c. In the system image generation unit 120c, the room image generation unit 124c generates system image data indicating the status of the chat room in which user C is participating, and supplies it to the image processing unit 140c. The image processing unit 140c temporarily stores system image data in a system buffer 154c, generates a display image from the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. Specifically, the image processing unit 140c generates a display image by combining the game image data temporarily stored in the game buffer 152c and the system image data temporarily stored in the system buffer 154c, and provides the display image to the output device 4c. As a result, the output device 4c outputs a display image in which the system image is superimposed on the game image.

FIG. 12 shows an example of a system image 200 superimposed on a game image. The room image generation unit 124c generates system image data based on the information indicating the status of each user acquired by the status information acquisition unit 164c. The system image 200 is provided with a member display column 202 that shows the status of the members of the chat room in which user C participates. The room image generation unit 124c generates a member display column 202 including information about the members based on information indicating the status of a plurality of room members (users). The member display field 202 shown in FIG. 12 displays the user's icon, user name, the title of the game being played, and information indicating whether the microphone is on or off. may also include information indicating.

The room image generation unit 124c generates a member display column 202 in which information about users who are sending images and information about users who are not sending images are included in different areas based on information indicating the status of a plurality of users. generate. In the example shown in FIG. 12, the system image 200 is provided with a host display area 206 that displays information about users who are sending images, and a non-host display area 208 that displays information about users who are not sending images. It will be done. The host display area 206 and the non-host display area 208 are distinguished and provided as different areas as shown in FIG. Thereby, user C can easily recognize members who are performing screen sharing and members who are not performing screen sharing.

When user C selects the display field of user A using the input device 6, the reception unit 104c accepts an operation to select the user who is transmitting the image, and the request transmission unit 180c specifies the selected user. A viewing request including the information is sent to the management server 5. When the management server 5 receives the viewing request, it transmits the viewing request to the distribution server 9 to notify that the user C requests viewing of the user A's game image. Note that the request transmitting unit 180c may transmit the viewing request directly to the distribution server 9.

The first image acquisition unit 188c determines the resolution of the game image data received from the distribution server 9 depending on the quality of the connection between the communication unit 102c and the distribution server 9, and views the received image resolution. Include in request. That is, if the connection state is poor, the first image acquisition unit 188c determines to receive game image data of low resolution. In the distribution server 9, the conversion unit 312 generates image data of a plurality of resolutions, and the distribution unit 314 transmits image data of a resolution according to the connection state with the information processing device 10c to the information processing device 10c. To deliver.

Here, it is assumed that game image data with a resolution of 1080p is transmitted from the information processing device 10a of user A, and the conversion unit 312 generates game image data with resolutions of 720p, 540p, and 360p. If the connection state between the distribution server 9 and the information processing device 10c is good, the first image acquisition unit 188c requests game image data with a resolution of 1080p, and the distribution unit 314 transmits the game image data with a resolution of 1080p as information. It is distributed to the processing device 10c. However, if the connection state is not good, the distribution unit 314 distributes game image data with a resolution lower than 1080p.

Returning to FIG. 9, the first image acquisition unit 188c acquires game image data from the distribution server 9. In the information processing device 10c, a format for displaying game images to be screen shared is set. In the embodiment, three display formats are provided: (1) full-screen display format, (2) picture-in-picture display format, and (3) split-screen format. User C has set one of the display formats in advance.

When the image processing unit 140c receives image data from the first image acquisition unit 188c, the image processing unit 140c temporarily stores the image data in the system buffer 154c according to the set display format, and displays the image data from the temporarily stored image data in the frame buffer 150c. An image is generated and provided to the output device 4c.

FIG. 13 shows a display example in the full-page display format. When the full-screen display format is set, the image processing unit 140c temporarily stores the image data acquired by the first image acquisition unit 188c in the system buffer 154c, generates a display image from the temporarily stored image data, and The information is provided to the output device 4c. As a result, the output device 4c displays the game image distributed by user A in its entirety.

FIG. 14 shows a display example in the picture-in-picture display format. When the picture-in-picture display format is set, the image processing unit 140c reduces the image data acquired by the first image acquisition unit 188c and temporarily stores it in the system buffer 154c, and temporarily stores it in the game buffer 152a. A display image is generated by combining the game image data of user C and the game image data of user A temporarily stored in the system buffer 154c, and is provided to the output device 4c. Thereby, the output device 4c outputs a display image in which the display area 230 of the game image distributed by user A is superimposed on the game image being played by user C. Note that in the picture-in-picture display format, the position and size of the display area 230 may be freely set by the user C.

FIG. 15 shows an example of display in a screen split format. When the screen split format is set, the image processing unit 140c reduces the image data acquired by the first image acquisition unit 188c and temporarily stores it in the system buffer 154c, and uses the game image data generated by the execution unit 110c. It is reduced in size and temporarily stored in the game buffer 152c. The image processing unit 140c generates a display image by combining the game image data of user C temporarily stored in the game buffer 152a and the game image data of user A temporarily stored in the system buffer 154c, and outputs the display image to the output device 4c. provide. In the split screen format, the screen of the output device 4c is divided to display the game image being played by user C and the game image distributed from user A side by side. Note that in the split screen format, the size of the display area 232 may be freely set by the user C.

In the picture-in-picture display format or the split screen format, the image processing unit 140c displays the game image distributed from the distribution server 9 on a part of the display. By displaying the distributed image on a part of the display, user C can watch user A's play video while playing the title "Combat Field."

Note that in the picture-in-picture display format or the screen split format, the user A's game image is reduced and displayed on a part of the display, so the resolution of the image does not need to be high. Therefore, the request transmitter 180c may request transmission of game image data with a resolution lower than 1080p. In this case, the distribution unit 314 in the distribution server 9 distributes, to the information processing device 10c, image data having a resolution that corresponds to the manner in which it is displayed on the information processing device 10c. Therefore, communication resources can be used efficiently.

Returning to FIG. 3, when user A briefly presses a predetermined button on the input device 6 during game play, the reception unit 104a accepts the button operation and supplies the operation information to the system image generation unit 120a. The system image generation unit 120a acquires the button operation information as a system image display request, and calls the situation information acquisition unit 164a.

The status information acquisition unit 164a acquires information indicating the status of each user from the management server 5, and provides the information to the system image generation unit 120a. The room image generation unit 124a generates system image data indicating the status of the chat room in which user A is participating, and supplies it to the image processing unit 140a. The image processing unit 140a temporarily stores system image data in a system buffer 154a, generates a display image from the image data temporarily stored in the frame buffer 150a, and provides the display image to the output device 4a.

FIG. 16 shows an example of a system image 200 superimposed on user A's game image. Compared to the system image 200 shown in FIG. 8, information indicating that the user C is viewing the game image screen-shared by the user A is displayed in the viewing member display area 212. By looking at the viewing member display area 212, user A can recognize that there are members who are viewing the distributed game image.

As described above, the share play start button 210 is a button for user A to start share play in sharing mode 2 or sharing mode 3, and is an operation element for performing an operation to invite a member to the game being played. It is. When the user A operates the share play start button 210 and selects a member to play share play with, the invitation sending unit 172a sends information indicating that the selected member is invited to share play to the management server 5. do. The management server 5 notifies the selected member's information processing device 10 of the invitation to share play. In the example, user A invites user C to share play.

FIG. 17 shows an example of a message 222 displayed on user C's output device 4c. Note that the image processing unit 140c displays the game image distributed by user A in the display area 230 in a picture-in-picture format.

The notification generation unit 122c generates system image data including the message 222 based on the notification from the management server 5, and supplies it to the image processing unit 140c. The image processing unit 140c temporarily stores system image data in a system buffer 154c, generates a display image from the game image data and system image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. By seeing the message 222, user C recognizes that user A has invited him to share play. Note that the notification generation unit 122c may notify the user C that the user A has invited him to share play through audio output.

When the user C presses a predetermined button on the input device 6 while the message 222 is being displayed, the acceptance transmitter 184c sends information identifying the user C and information indicating acceptance of the invitation to the management server. Send to 5.

Note that the message 222 is displayed for only about 5 seconds, for example. After message 222 disappears, user C can accept the invitation from user A by displaying the system image. When the user C briefly presses a predetermined button on the input device 6, the reception unit 104c accepts the button operation and supplies the operation information to the system image generation unit 120c. The system image generation unit 120c acquires the button operation information as a system image display request, and calls the situation information acquisition unit 164c.

The status information acquisition unit 164c acquires information indicating the status of each user from the management server 5, and provides the information to the system image generation unit 120c. In the system image generation section 120c, the room image generation section 124c generates system image data indicating the status of the chat room in which user C is participating, and supplies it to the image processing section 140c. The image processing unit 140c temporarily stores system image data in a system buffer 154c, generates a display image from the game image data and system image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c.

FIG. 18 shows an example of a system image 200 displayed superimposed on user C's game image. Compared to the system image 200 shown in FIG. 12, since user C is invited to share play, information indicating that user A has invited user C to share play is displayed in the share play host display area 216. Displayed. By looking at the share play host display area 216, user C can recognize that user A has invited him to share play.

The share play participation button 214 is a button for user C to participate in share play, and is an operation element for user C to perform an operation to accept an invitation from user A. When user C operates the share play participation button 214, the acceptance transmitting unit 184c transmits information indicating that user A's invitation regarding game play is accepted to the management server 5. The management server 5 notifies the information processing device 10a of the user A of information indicating that the user C has accepted the invitation.

Returning to FIG. 3, when the reception unit 104a receives information indicating that the user C has accepted the invitation, the connection processing unit 174a performs processing for connecting to the information processing device 10c of the user C via P2P. do. When connected by P2P, the second transmission processing unit 170a transmits streaming data of the game image read from the game buffer 152a by the image processing unit 140a to the information processing device 10c connected by P2P. In other words, the second transmission processing section 170a
A game image for game play is transmitted to the information processing device 10c of the user C who has accepted the invitation, without going through the distribution server 9. The streaming data to be transmitted may be the same as the streaming data being transmitted to the distribution server 9. Note that if the connection state between the communication unit 102a and the distribution server 9 is different from the connection state between the communication unit 102a and the information processing device 10c, streaming data with a resolution corresponding to each connection state will be transmitted. good.

FIG. 19 shows an example of a system image displayed on user C's output device 4c. In the system image 200 shown in FIG. 18, when the user C operates the share play participation button 214, a process for connecting the information processing device 10c and the information processing device 10a via P2P is executed in the background. Therefore, the system image generation unit 120c generates a system image as shown in FIG. 19 to make the user C aware that the connection is being made until the game play starts.

Note that the image processing unit 140c continues to display the game image acquired from the distribution server 9 in the display area 230 until preparations for game play are completed. This allows user C to watch user A playing while waiting for the game to start. Note that the period until preparations for game play are completed means the period from when the P2P connection is completed until the second transmission processing unit 170a transmits the streaming data of the game image.

Note that even when the distributed image is displayed in a split-screen format other than the picture-in-picture display format, the image processing unit 140c similarly displays the game image obtained from the distribution server 9 until preparations for game play are completed. It may continue to be displayed in area 232. If the user C operates the share play participation button 214 while the distributed image is displayed on a part of the display in this way, the image processing unit 140c continues to display the distributed image to establish a connection. User C can view user A's distributed images even while on standby.

After the P2P connection is established, the second transmission processing unit 170a transmits the game image for game play to the information processing device 10c of the user C who accepted the invitation, and the second image acquisition unit in the information processing device 10c 190c acquires a game image for game play from the information processing device 10a without going through the distribution server 9.

FIG. 20 shows a game image played by user C. A message 224 is displayed superimposed on the game image, and user C recognizes that the shared play has started. The image processing unit 140c temporarily stores the image data acquired by the second image acquisition unit 190c in the system buffer 154c, generates a display image from the temporarily stored image data, and provides it to the output device 4c. As a result, the output device 4c displays the game image distributed by user A. User C performs share play while viewing the displayed game image.

The present invention has been described above based on examples. Those skilled in the art will understand that this example is merely an example, and that various modifications can be made to the combinations of these components and treatment processes, and that such modifications are also within the scope of the present invention. .

DESCRIPTION OF SYMBOLS 1... Image sharing system, 4... Output device, 5... Management server, 6... Input device, 9... Distribution server, 10a, 10b, 10c, 10d... Information processing device, 20... Main power button, 100a, 100c... Processing unit, 102a, 102c... Communication unit, 104a, 104c... Reception unit, 110a, 110c... Execution unit, 120a, 120c... System image generation unit, 122a, 122c... Notification generation unit, 124a, 124c... Room image generation unit, 140a, 140c... Image processing unit, 150a, 150c... Frame buffer, 152a, 152c... - Game buffer, 154a, 154c...System buffer, 160a, 160c...Sharing processing section, 162a, 162c...Situation information transmission section, 164a, 164c...Situation information acquisition section, 166a... - Transmission processing section, 168a... First transmission processing section, 170a... Second transmission processing section, 172a... Invitation transmission section, 174a... Connection processing section, 180c... Request transmission section, 182c ...Invitation reception unit, 184c...Acceptance transmission unit, 186c...Image acquisition unit, 188c...First image acquisition unit, 190c...Second image acquisition unit, 300...Control unit, 302... Communication department, 310... Image acquisition section, 312... Conversion section, 314... Distribution section.

Claims (4)

A distribution server that distributes image data,
an image acquisition unit that acquires image data from a first terminal device with a resolution according to a connection state between the first terminal device and the first terminal device ;
a conversion unit that transcodes the acquired image data into image data with a lower resolution than the resolution of the image data, regardless of whether it is distributed or not;
Waiting for a viewing request from another user participating in the same chat room as the first user of the first terminal device , and requesting the first user from the second user of the second terminal device participating in the same chat room. When receiving a viewing request including information specifying the content , image data with a resolution corresponding to the connection state with the second terminal device and the display mode on the second terminal device is delivered to the second terminal device. The distribution department and
A distribution server comprising: When the image acquisition unit acquires image data, the conversion unit transcodes the acquired image data into image data of a plurality of transcodable resolutions.
The distribution server according to claim 1 , characterized in that: acquiring image data from a first terminal device with a resolution according to a connection state between the first terminal device and the first terminal device ;
Transcoding the acquired image data into image data with a lower resolution than the resolution of the image data, regardless of whether it is distributed or not;
Waiting for a viewing request from another user participating in the same chat room as the first user of the first terminal device ;
When a viewing request including information identifying the first user is received from a second user of a second terminal device participating in the same chat room , the connection state with the second terminal device and the second terminal device delivering to the second terminal device image data with a resolution corresponding to the manner in which it is displayed ;
An image distribution method comprising: to the computer,
A function of acquiring image data from a first terminal device at a resolution according to a connection state between the first terminal device and the first terminal device ;
A function to transcode the acquired image data into image data with a lower resolution than the resolution of the image data, regardless of whether it is distributed or not;
a function of waiting for a viewing request from another user participating in the same chat room as the first user of the first terminal device ;
When a viewing request including information identifying the first user is received from a second user of a second terminal device participating in the same chat room , the connection state with the second terminal device and the second terminal device a function of distributing image data with a resolution according to a display mode to the second terminal device;
A program to make this happen.

Images