JP7333529B1 - Terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method - Google Patents

Abstract

An object of the present invention is to improve a user's desire to continue communicating with other users. A control program for a terminal device sends at least a first image including a first object representing a first user and a second image including a second object representing a second user to a first user's terminal device. If the first user's first voice is acquired, the first user's first voice is transmitted to the second user's terminal device, and information regarding the display of the first image is transmitted to the second user's terminal device. displays a first image including a first object that changes according to the action of the second user or the first sound, and when the second sound of the second user is received, outputs the second sound and displays the first image that changes according to the action of the second user or the second sound. Displaying a second image including a second object that changes in response to audio, and displaying a first image that is at least partially changed or at least partially changed in response to an instruction by the first user or an instruction by the second user. displaying the second image. [Selection diagram] Figure 3

Description

The embodiments disclosed in this specification relate to a terminal device control program, a terminal device, a terminal device control method, a server device control program, a server device, and a server device control method.

Conventionally, a plurality of users use terminal devices such as PCs (Personal Computers) to communicate with each other by transmitting various information such as an image representing each user and/or each user's voice to the terminal devices of other users. There is known an information system that realizes For example, Patent Literature 1 discloses information system technology in which each user uses a character object such as an avatar that acts as the user's alter ego in a virtual space to have a voice chat with another user's character object. there is

JP 2009-199198 A

However, in the conventional information system, since there is no variety in the output from the user's terminal device while the user is communicating with other users, there is a problem that the user gets tired of communicating with other users. Ta.

The disclosed terminal device control program, terminal device, terminal device control method, server device control program, server device, and server device control method improve motivation of users to continue communication with other users. make it possible to

The control program for the terminal device is a control program for the terminal device of the first user, and is configured to display a first image including a first object representing the first user and a first object different from the first user on the terminal device of the first user. When at least a second image including a second object representing two users is displayed, information regarding the display of the first image is transmitted to the terminal device of the second user, and a first voice of the first user is obtained, the first When the voice is transmitted to the second user's terminal device, the first image including the first object that changes according to the first user's action or the first voice is displayed, and the second user's second voice is received, outputting a second sound, displaying a second image including a second object that changes according to the second user's motion or the second sound, and responding to an instruction by the first user or an instruction by the second user, at least one displaying the partially modified first image or the at least partially modified second image.

Further, in the control program of the terminal device, in the display of the first image whose at least part has been changed, the terminal device of the first user displays the first candidate object among the one or more first candidate objects associated with the first user. Displaying a first image including the partially changed first object based on the first candidate object selected according to an instruction by one user is preferably performed.

Further, in the control program of the terminal device, the first object is a three-dimensional object arranged in the virtual space, and in displaying the first image including the first object, the first user's terminal device receives a predetermined image in the virtual space. displaying, as a first image, an image showing the virtual space seen from the viewpoint of the first user, and showing the virtual space seen from the predetermined viewpoint that automatically moves in the virtual space on the terminal device of the first user. displaying the image as the first image.

Further, it is preferable that the control program of the terminal device controls the movement of the predetermined viewpoint so as to approach the changed part of the first object.

Also, in the control program of the terminal device, it is preferable that the predetermined viewpoint is controlled to move around the first object while matching the line-of-sight direction from the predetermined viewpoint to the direction facing the first object.

In addition, in the control program of the terminal device, in the display of the first image whose at least part has been changed, the second user from among the selection group including one or more second candidate objects is displayed on the terminal device of the first user Displaying a first image including a third object corresponding to the second candidate object selected in accordance with the instruction is preferably performed.

Further, it is preferable that the selected group includes a specific second candidate object when a predetermined condition is satisfied in the control program of the terminal device.

Moreover, in the control program of the terminal device, it is preferable that the predetermined conditions include conditions regarding the display time of the first image and/or the display time of the second image.

Further, in the control program of the terminal device, based on the second candidate object selected from the selection group including one or more second candidate objects in response to the instruction by the first user, the terminal device of the first user , displaying a second image including a third object corresponding to the selected second candidate object, and transmitting information regarding the display of the third object to the terminal device of the second user; The conditions preferably include conditions regarding the number of times the second candidate object is selected by the first user and/or the second user.

Further, in the control program of the terminal device, in the display of the first image and the second image, displaying a plurality of second images respectively representing the plurality of second users on the terminal device of the first user. a first image and a plurality of displaying a plurality of second images representing each of the second users.

In addition, in the control program of the terminal device, the first user can instruct the selection of the third object, and the terminal device of the first user can partially change the object according to the first user's selection instruction of the third object. displaying a first image including the rendered first object.

Further, in the control program of the terminal device, an image showing progress of the game played by the first user and/or the second user is displayed on the terminal device of the first user in the display of the first image at least partially changed. is preferably displayed in place of at least part of the first image.

Further, in the control program of the terminal device, when the second candidate object selected in response to the instruction by the second user corresponds to the game, the first user and/or the second user play the game on the terminal device of the first user. It is preferable to display an image showing the progress of the game being played instead of at least part of the second image.

A control program for a server device is a control program for a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user, wherein a first object indicating the first user is receiving, from a first user's terminal device, information relating to display of a first image including a Information for displaying at least one image is transmitted to at least the terminal device of the second user, information for displaying at least the second image is transmitted to at least the terminal device of the first user When the voice is received, the first voice is transmitted to the second user's terminal device, and the information for displaying the first image including the first object that changes according to the first user's action or the first voice, Transmit to at least the second user's terminal device, and when receiving the second user's second voice, transmit the second voice to the first user's terminal device, and change according to the second user's action or the second voice information for displaying a second image including a second object to be displayed is transmitted to at least the first user's terminal device, and the first image, at least partially changed, is displayed in accordance with an instruction from the first user to at least the terminal device of the second user, and information for displaying the second image of which at least a part has been changed according to an instruction from the second user is transmitted to at least the terminal device of the first user send to, and have it run.

The terminal device is a terminal device of a first user and displays a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user. at least display and transmit information about display of the first image to the terminal device of the second user; if the first voice of the first user is acquired, transmit the first voice to the terminal device of the second user; displaying a first image including a first object that changes according to a user's motion or a first voice; outputting a second voice when a second user's second voice is received; 2 Displaying a second image including a second object that changes according to voice, and changing at least a portion of the first image or at least a portion of the first image according to an instruction from the first user or an instruction from the second user a processing unit for displaying the generated second image.

The server device is capable of communicating with the terminal device of the first user and the terminal device of a second user different from the first user, and displays a first image including a first object representing the first user. receiving information about display of a second image including a second object representing the second user from a terminal device of the first user; receiving information about displaying a second image including a second object representing the second user from a terminal device of the second user; information for displaying the second image is transmitted to at least the terminal device of the second user, information for displaying the second image is transmitted to at least the terminal device of the first user, and the first voice of the first user is received , transmitting the first sound to a second user's terminal device, and transmitting information for displaying a first image including a first object that changes according to the first user's action or the first sound to at least the second user's terminal device When the second voice of the second user is received by transmitting the second voice to the terminal device, the second voice is transmitted to the terminal device of the first user, and the second object that changes according to the second user's action or the second voice transmitting information for displaying a second image including, to at least a first user's terminal device, information for displaying the first image at least partially changed in accordance with an instruction from the first user; A process of transmitting to at least a terminal device of a second user, and transmitting to at least a terminal device of a first user information for displaying a second image that is at least partially changed according to an instruction from the second user. have a department.

A method for controlling a terminal device is a method for controlling a terminal device of a first user, wherein the terminal device of the first user controls a first image including a first object representing the first user and a first object different from the first user. When at least a second image including a second object representing two users is displayed, information regarding the display of the first image is transmitted to the terminal device of the second user, and a first voice of the first user is obtained, the first When the voice is transmitted to the second user's terminal device, the first image including the first object that changes according to the first user's action or the first voice is displayed, and the second user's second voice is received, outputting a second sound, displaying a second image including a second object that changes according to the second user's motion or the second sound, and responding to an instruction by the first user or an instruction by the second user, at least one displaying the partially modified first image or the at least partially modified second image.

A method of controlling a server device is a method of controlling a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user, wherein a first object indicating the first user is receiving, from a first user's terminal device, information relating to display of a first image including a transmitting information for displaying at least one image to at least a terminal device of a second user; transmitting information for displaying at least a second image to at least a terminal device of the first user; When the first sound is received, the first sound is transmitted to the second user's terminal device, and information for displaying the first image including the first object that changes according to the first user's action or the first sound is displayed. , at least to the second user's terminal device, and upon receiving the second user's second voice, transmit the second voice to the first user's terminal device, and according to the second user's action or the second voice Information for displaying a second image including a changing second object is transmitted to at least the first user's terminal device, and the first image, at least partially changed, is displayed according to an instruction from the first user. to at least the terminal device of the second user, and information for displaying the at least partially changed second image in accordance with an instruction from the second user is transmitted to at least the terminal device of the first user transmitting to the device.

The control program for the terminal device, the terminal device, the control method for the terminal device, the control program for the server device, the server device, and the control method for the server device can improve the motivation of the user to continue communication with other users. It becomes possible.

1 is a schematic diagram for explaining an example of an outline of an information system; FIG. It is a figure which shows an example of schematic structure of an information system. It is a figure which shows an example of schematic structure of a terminal device. It is a figure which shows an example of the data structure of various tables. It is a figure which shows an example of schematic structure of a server apparatus. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. It is a figure which shows an example of the operation sequence of an information system. It is a figure which shows an example of the screen displayed on the display apparatus of a terminal device. FIG. 4 is a schematic diagram for explaining the inside of a virtual space;

Various embodiments of the present invention will now be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and equivalents thereof.

(Overview of information system)
FIG. 1 is a schematic diagram for explaining an example of an outline of an information system. An information system includes a terminal device and a server device operated by each of a plurality of users. For example, the terminal device is an information processing device such as a multifunctional mobile phone (so-called "smartphone") owned by the user. For example, a server device is a computer for providing communication services between terminal devices via a communication network.

A terminal device stores a control program such as an application program, loads the control program into a memory in response to a user's start operation, and starts a communication service by executing instructions included in the loaded control program. When the communication service is started, the terminal device implements multiple functions by executing instructions contained in the control program.

For example, the terminal device of the user implements a generating function of generating output information including character moving image data including motion data based on various input data input by the user. For example, the input data is a plurality of pieces of imaging data acquired at predetermined sampling time intervals by an imaging device provided in the user's terminal device, and in this case, each piece of imaging data shows the user. Character moving image data is an example of information regarding display of a user output image including a character object representing a user. During communication services, output information is generated at predetermined intervals.

Further, for example, the terminal device of the user displays the user output image based on the character moving image data, and transmits the generated output information to the terminal devices of other users via the server device. It realizes an output function that transmits at predetermined time intervals. Also, for example, the terminal device of the user realizes a function of displaying other user's output image including a character object representing the other user based on the other user's output information transmitted from the server device at predetermined time intervals.

When the communication service is started, for example, a user output image (1) and another user output image (2) are displayed by the user's terminal device, as shown in FIG. 1(a). If no other user participates in the communication service when the communication service is started, only the user output image (1) is displayed by the user's terminal device. If the user is not participating in the communication service when the communication service is started, only the other user's output image (2) is displayed on the other user's terminal device.

User output image (1) includes a user character object that moves in accordance with user motion data included in character moving image data. Further, the other user's output image (2) includes the other user's character object that moves according to the other user's motion data included in the received other user's output information. In this way, the user's terminal device displays the user output image (1) including the user's character object that changes according to the user's motion, and includes the other user's character object that changes according to the other user's motion. Other user output image (2) is displayed.

Since both the user's terminal device and the other user's terminal device have a microphone, the user's terminal device acquires the voice uttered by the user, and the other user's terminal device acquires the voice uttered by the other user. Then, the terminal device of the user includes the acquired voice data of the user in the output information, and transmits the output information including the voice data of the user to the terminal device of the other user via the server device. Send. In addition, the terminal device of the user receives output information including voice data of the other user transmitted to the server device by the terminal device of the other user from the server device.

For example, when the user makes a sound, the user's motion data included in the character moving image data includes face motion data indicating the movement of the user's lips. In this case, the user's terminal device displays the user output image (1) including the user's character object whose lips move so as to substantially synchronize with the voice generated by the user. The other user's output information received by the user's terminal device may include the other user's voice data together with the other user's motion data. In this case, on the terminal device of the user, the other user's voice is output, and the other user's output image (2) including the other user's character object whose lips move so as to substantially synchronize with the other user's voice is displayed. will be

In addition to the functions described above, the terminal device of the user changes at least a part of the user output image (1) according to instructions from the user, and realizes the function of displaying the changed user output image (1). In addition, the terminal device of the user may change at least a part of the other user's output image (2) according to an instruction from the user, and realize a function of displaying the changed other user's output image (2).

For example, the user's terminal device changes at least part of the user's character object in accordance with the user's change instruction, thereby changing at least part of the user output image (1). As shown in FIG. 1B, for example, the user's terminal device changes a part of the user's character object (the item "hat" worn on the head) in accordance with the user's change instruction. (The item "Hat" is changed to the item "Ribbon"). For example, a user's change instruction is input by a selection operation in which the user selects a predetermined operation object (such as a button object). Alternatively, the change instruction by the user may be input by a selection operation in which the user selects one of a plurality of candidate objects each corresponding to a different item. In this case, the item of the user's character object included in the user output image (1) is changed to an item corresponding to one candidate object selected by the user. Alternatively, a user output image (1) including the user's character object to which an item corresponding to one candidate object selected by the user has been added may be displayed.

Further, for example, the terminal device of the user includes an additional object in the other user's output image (2) in response to the user's additional instruction, thereby changing at least a part of the other user's output image (2). For example, as shown in FIG. 1(c), the user's terminal device is placed around the other user's character object or at least one of the other user's characters in response to an additional instruction from the user. The additional object is included in the other user output image (2) so as to be superimposed on the part. The user's additional instruction is input by, for example, a selection operation in which the user selects a predetermined operation object (such as a button object). Further, the user's additional instruction may be input by a selection operation in which the user selects one of a plurality of candidate objects each corresponding to a different additional object. In this case, another user output image (2) including an additional object corresponding to one candidate object selected by the user is displayed.

Also, the terminal device of the user may change at least a part of the user output image (1) according to an instruction from another user, and realize a function of displaying the changed user output image (1). For example, in this function, the user's terminal device is arranged to surround the user's character object or superimpose on at least a part of the user's character in response to an additional instruction from another user. Include the object in the user output image (1). Further, the terminal device of the user may change at least part of the other-user output image (2) according to an instruction from the other user, and realize a function of displaying the changed other-user output image (2). . For example, in this function, the terminal device of the user changes a part of the other user's character object in response to a change instruction from the other user, and outputs the other user's output image (2) including the changed character object of the other user. indicate.

Note that the terminal device of the user may change the output of the terminal device of the user according to instructions from the user or instructions from other users. For example, the terminal device of the user may change the sound quality of the user's voice or the other user's voice for a predetermined period in accordance with the user's instruction or the other user's instruction. Also, the user's terminal device may operate the vibrating section of the user's terminal device for a predetermined period of time in response to an instruction from the user or an instruction from another user.

As described above with reference to FIG. 1, in the control program for the terminal device, the terminal device, the control method for the terminal device, the control program for the server device, the server device, and the control method for the server device, the user's instruction or another user's At least a part of the changed user output image or at least a part of the changed other user output image is displayed according to the instruction by . In this way, the control program for the terminal device, the terminal device, the control method for the terminal device, the control program for the server device, the server device, and the control method for the server device enable the output from the terminal device to be used by the user or others during communication services. By allowing the user to change it, it is possible to increase the user's willingness to continue using the communication.

In the example shown in FIG. 1, the user and other users are users belonging to a predetermined communication group among a plurality of users who can use the communication service. A predetermined communication group is a group or the like created by either a user or another user. In the example shown in FIG. 1, character objects of two users are displayed, but character objects of three or more users belonging to a predetermined communication group are displayed so that three or more users can communicate. may be configured.

Also, in the example shown in FIG. 1, the user and other users may be users having a predetermined relationship. For example, other users who have a predetermined relationship with the user are mutual followers of the user. Note that when a user follows another user and the other user also follows the user (the user and the other user follow each other), the user and the other user become mutual followers. Further, the other user having a predetermined relationship with the user may be another user who is in a mutual following relationship with the predetermined user who is a mutual follower of the user. In addition, other users having a predetermined relationship with the user may be other users who have a friend relationship with the user, or specific information (telephone number, e-mail address, predetermined ID (identification), etc.) is stored in the user's terminal device. It may be another user who is In addition, when the user's terminal device has a function of distributing the user output image including the user's character object to one or more other user's terminal devices via the server device, the other user who has a predetermined relationship with the user is another user who viewed the user output image distributed by the user. In this case, when a user and other users having a predetermined relationship communicate as shown in FIG. Alternatively, it may be temporarily created by the server device.

Note that the above description of FIG. 1 is merely a description for deepening the understanding of the content of the present invention. The present invention is embodied in particular in each of the embodiments described below, and may be embodied in various modifications without substantially departing from the principles of the invention. All such variations are included within the scope of the present invention and the disclosure herein.

(Information system 1)
FIG. 2 is a diagram showing an example of a schematic configuration of the information system 1. As shown in FIG. The information system 1 includes a terminal device 2 operated by each of a plurality of users, and a server device 3 . The terminal device 2 and the server device 3 are interconnected via a base station 4, a mobile communication network 5, a gateway 6, and a communication network such as the Internet 7, for example. Communication is performed between the terminal device 2 and the server device 3 based on a communication protocol such as the Hypertext Transfer Protocol (HTTP). Based on WebSocket, a connection is first established between the terminal device 2 and the server device 3 by HTTP communication, and then two-way communication is performed at a lower cost (less communication load and processing load) than HTTP communication. communication may take place. The communication method between the terminal device 2 and the server device 3 is not limited to the above-described method. may be used.

The terminal device 2 is an information processing device such as a smart phone. The terminal device 2 may be a mobile phone, a notebook PC, a tablet terminal, a tablet PC, a head mounted display (HMD), an electronic book reader, a wearable computer, or the like. The terminal device 2 may be a portable game machine, a game console, or the like. The terminal device 2 may be any information processing device as long as it can display the character objects of the user and other users and output the voices of the user and other users.

In the example shown in FIG. 2, one server device 3 is illustrated as a component of the information system 1, but the server device 3 may be a set of a plurality of physically separate server devices 3. . In this case, each of the plurality of server devices 3 may have the same function, or may have the functions of one server device 3 in a distributed manner.

(Terminal device 2)
FIG. 3 is a diagram showing an example of a schematic configuration of the terminal device 2. As shown in FIG. Terminal device 2 connects to server device 3 via base station 4 , mobile communication network 5 , gateway 6 , and a communication network such as the Internet 7 to communicate with server device 3 . The terminal device 2 generates character moving image data including motion data according to various data (imaging data, etc.) input by the user, and outputs output information including the generated character moving image data and/or user voice data to the server device 3. Send. In addition, the terminal device 2 receives the output information of the other user transmitted from the server device 3, displays the character object of the other user based on the received output information, and/or outputs the voice of the other user. . In order to realize such functions, the terminal device 2 includes a terminal communication I/F 21, a terminal storage device 22, a display device 23, an input device 24, an imaging device 25, a microphone 26, and a terminal processing device 27. and

The terminal communication I/F 21 is implemented as hardware, firmware, communication software such as a TCP/IP (Transmission Control Protocol/Internet Protocol) driver or PPP (Point-to-Point Protocol) driver, or a combination thereof. The terminal device 2 can transmit data to and receive data from other devices such as the server device 3 via the terminal communication I/F 21 .

The terminal storage device 22 is, for example, a semiconductor memory device such as ROM (Read Only Memory) or RAM (Random Access Memory). The terminal storage device 22 stores an operating system program, a driver program, a control program, data, and the like used for processing in the terminal processing device 27 . The driver programs stored in the terminal storage device 22 include an output device driver program that controls the display device 23, an input device driver program that controls the input device 24, and the like. The control programs stored in the terminal storage device 22 are, for example, application programs for realizing various functions related to communication services. Note that the control program may be transmitted from the server device 3 or another device.

The terminal storage device 22 stores, as data, identification information (eg, user ID (identification)) for uniquely identifying a user who operates the terminal device 2 . The terminal storage device 22 also stores background data, model data, and the like as data. The terminal storage device 22 also stores a user table T1, an object table T2, and a group table T3 as data. In addition, the terminal storage device 22 may temporarily store data related to predetermined processing.

The background data is asset data for constructing a virtual space in which the user's character object exists in the user output image. The background data includes data for drawing the background of the virtual space, data for drawing various objects included in the user output image, and data for drawing various background objects displayed in the user output image other than these. is included. The background data may include object position information indicating positions of various background objects in the virtual space.

The display device 23 is a liquid crystal display. The display device 23 may be an organic EL (Electro-Luminescence) display or the like. The display device 23 displays, on a display screen, a moving image corresponding to the moving image data and/or a still image corresponding to the still image data supplied from the terminal processing device 27 . In addition, the display device 23 may not be a component of the terminal device 2. In this case, the display device 23 may be an HMD display capable of communicating with the server device 3, or may communicate with the terminal device 2 via a wired or wireless connection. Alternatively, a projector for projection mapping, a retinal projection type projector, or the like may be used.

The input device 24 is a pointing device such as a touch panel. When the input device 24 is a touch panel, the input device 24 can detect various touch operations such as tapping, double tapping, and dragging by the user. The touch panel may include a capacitive proximity sensor and be configured to detect a user's non-contact operation. The input device 24 may be an input key or the like. The user can use the input device 24 to input characters, numbers and symbols, positions on the display screen of the display device 23, and the like. The input device 24 generates a signal corresponding to the operation when operated by the user. The input device 24 then supplies the generated signal to the terminal processing device 27 as a user's instruction.

The imaging device 25 is a camera having an imaging optical system, an imaging element, an image processing section, and the like. The imaging optical system is, for example, an optical lens, and forms an image of the light flux from the subject on the imaging surface of the imaging element. The imaging device is a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and outputs an image of a subject image formed on an imaging plane. The image processing unit creates moving image data in a predetermined file format for each predetermined period from images continuously generated by the imaging device, and outputs the moving image data as captured image data. Alternatively, the image processing unit creates still image data in a predetermined file format from the image generated by the imaging device, and outputs the still image data as imaging data.

The microphone 26 is a sound collecting device configured to acquire voice uttered by the user and convert it into voice data. The microphone 26 is configured to be able to acquire user's voice input, converts the user's voice input acquired by the microphone 26 into voice data, and outputs the voice data to the terminal processing device 27 .

The terminal processing device 27 is a processor that loads an operating system program, a driver program and a control program stored in the terminal storage device 22 into memory and executes instructions contained in the loaded programs. The terminal processing device 27 is, for example, an electronic circuit such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), or a combination of various electronic circuits. The terminal processing device 27 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), or an MCU (Micro Controller Unit). Although terminal processor 27 is illustrated as a single component in FIG. 3, terminal processor 27 may be a collection of multiple physically separate processors.

The terminal processing device 27 functions as a generation unit 271, a transmission unit 272, a reception unit 273, a display processing unit 274, and an audio output unit 275 by executing various commands included in the control program. Functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described later.

(Various tables)
FIG. 4 is a diagram showing an example of the data structure of the user table T1, object table T2, and group table T3 stored in the terminal storage device 22. As shown in FIG. At least one of the user table T1, object table T2, and group table T3 may be stored in the server storage device 32 of the server device 3. FIG. In this case, the terminal device 2 may receive the latest table stored in the server storage device 32 and store it in the terminal storage device 22 at a predetermined timing.

(User table T1)
FIG. 4(a) is an example of the data structure of a user table T1 that manages users participating in the communication service. In the user table T1, for each user, the user ID, user name, user's character object, user's owned object, used object, and the like are stored in association with each other. In the user table T1, the user's owned coins (information indicating the total amount of "coins" of virtual monetary value owned by the user) and/or user points etc. are stored in association with the user's user ID. good. A user ID is an example of identification data for uniquely identifying each user. The name is an example of data indicating the name of each user.

For example, a character object is a model ID for identifying model data for generating animation of the character object. The model data is stored in the terminal storage device 22 in association with the model ID. The model data may be three-dimensional model data for generating three-dimensional animation, or two-dimensional model data for generating two-dimensional model data. The model data includes, for example, rig data (so-called “skeleton data”) indicating the skeleton of the character object's face and parts other than the face, and surface data indicating the surface shape and texture of the character object. The model data may include a plurality of different model data. Each of the plurality of model data may have rig data different from each other, or may have the same rig data. Each of the plurality of model data may have surface data different from each other, or may have the same surface data.

A owned object is an object ID indicating an object owned by each user. Possessed objects are wearable objects that can be associated with specific parts of the character object, wallpaper objects that are placed behind the user's character object in the user output image, and the like. Details of the wearing object and the wallpaper object will be described later. A owned object is an object selected by a user's operation in a lottery game executed by the user's terminal device 2 or server device 3 . The owned object may also be an object owned by the user as a result of the user consuming "coins" of virtual monetary value in the purchase function of the information system 1 .

The used object is an object ID indicating an object used in the user output image being displayed on the terminal device 2 of the user. For example, when a user's change instruction is input during a communication service, the user's character object included in the user output image being displayed is equipped with the wearable object held as the user's owned object. In this case, the object ID of the wearable object worn by the user's character object is stored in the user table T1 in association with the user ID as the used object. Also, for example, when a user's change instruction is input during a communication service, a wallpaper object owned by the user as a owned object is arranged behind the user's character object in the user output image being displayed. In this case, the object ID of the wallpaper object arranged in the user output image is stored in the user table T1 in association with the user ID as the used object.

(Object table T2)
FIG. 4B is an example of the data structure of an object table T2 that manages objects selected as additional objects. An additional object is, for example, a gift object given by a user to another user. In the object table T2, the object ID, name, image information, rarity level, arrangement location, etc. of each additional object are stored in association with each other.

The object ID is an example of identification data for uniquely identifying each additional object. The image information is one or more still images corresponding to each additional object. The image information may be one or more moving images corresponding to each additional object. The rarity level is information indicating the rarity level of each additional object. The placement location is information indicating the location where the image information of each additional object is displayed. Note that when the display position and display range of the character object are determined, information indicating the position relative to the character object may be stored as the placement location.

Additional objects are classified into multiple types (categories). For example, additional objects include effect objects used as effect gifts, normal objects used as normal gifts, attachment objects used as attachment gifts, and message objects used as message gifts. . Additional objects may include wallpaper objects that are placed behind character objects in the user output image. Information indicating the type (category) of the additional object may be stored in the object table T2 in association with the object ID of the additional object. Image information and placement locations will be described below according to the types of additional objects.

An effect object is an object that affects the overall impression of a user output image and/or other user output images. The effect object is, for example, an object that imitates confetti. When the effect object is an object resembling confetti, an image representing a plurality of sheets of paper is stored as the image information.

Information indicating "space" is stored as the arrangement location of the effect object. For example, an object resembling confetti is displayed over the user output image and/or the other user output image. Note that the effect object may be displayed on the entire screen (communication screen, which will be described later) including the user output image and the other user output image according to the user's instruction. Also, the effect object may be displayed only in the output image selected by the user, out of the user output image and the other user output image. Also, the effect object may be displayed so as to overlap the character object included in the user output image and/or other user output images. Effect objects are displayed without being associated with specific parts of the character, unlike wearable objects, which will be described later. In this way, the terminal device 2 of the information system 1 displays the effect object in accordance with the user's instruction, so that the user output image and/or other user output images are displayed more than before the effect object is displayed. It can change the overall impression.

A typical object is, for example, an object that resembles a stuffed animal, a bouquet of flowers, an accessory, or an item suitable for a gift or present. Information indicating "space" is stored as the placement location of the normal object. For example, the normal object is associated with information about a predetermined moving route, and the normal object moving along the predetermined moving route within the user output image or other user output image is displayed. The information indicating the "space" stored as the placement location of the "normal object" may be classified into a plurality of types. A plurality of types of "space" are, for example, "central space" (region in the center of an image when a user output image or other user output image is divided into left and right), "left space" (user output image or other left-hand side area of the image when the user output image is divided into left and right thirds), and "right space" (right-side area of the image when the user output image or another user's output image is divided into left and right thirds). In addition, the multiple types of "space" include "central space" (the area in the center of the image when the user output image or other user output image is divided vertically into three), "upper space" (user output image or other The upper area of the image when the user output image is vertically divided into three) and the "lower space" (the area of the lower side of the image when the user output image or another user's output image is vertically divided into three). In this case, the movement route associated with the normal object is set within the type of "space" associated as the location of the normal object. For example, when information indicating the "right space" is stored in the normal object, the normal object moves within the area on the right side of the image when the user output image or other user output image is divided into left and right into three. Information about the route is associated. In addition, the normal object is displayed as a normal object that moves according to a predetermined movement rule from a position set for each normal object in the user output image or another user output image or a position automatically and randomly determined. may The predetermined movement rule is, for example, a movement rule that is set so that the object moves as if it were in free fall. Note that the normal object may be displayed so as to overlap with the character object. In this case, the normal object is displayed without being associated with a specific part of the character object, unlike the wearable object which will be described later. Also, the normal object may be displayed so as to bounce back at the timing of contact with the character object.

When the normal object is displayed overlapping with the character object, for example, the normal object may be displayed so as to overlap "a portion other than the 'head' including the 'face'" of the character object. In this case, the normal object is displayed so as not to overlap the "head" of the character object. Also, the normal object may be displayed so as to overlap with "a portion other than the 'upper body' including the 'face'" of the character object. In this case, the normal object is displayed so as not to overlap the "upper body" of the character object.

A wearable object is an object that is displayed in a user output image or another user output image in association with a specific part (wearing location) of a character object. For example, the specific parts include the front left side, front right side, rear left side, rear right side, center front side, center rear side, left eye, right eye, left ear, right ear, and the entire hair of the head of the character object. is. Also, the specific part may be the thumb, ring finger, wrist, elbow, shoulder, upper arm, whole hand, whole arm, or the like.

The wearable object associated with the specific part of the character object is displayed in the user output image or other user output image so as to be in contact with the specific part of the character object. Also, the wearable object associated with the specific part of the character object may be displayed in the user output image or other user output image so as to cover part or all of the specific part of the character object. The specific part may be identified by three-dimensional position information indicating the position in the three-dimensional coordinate space, or may be associated with position information in the three-dimensional coordinate space.

The image information of the wearable object includes, for example, accessories to be worn by the character object (headband, necklace, earrings, etc.), clothes (T-shirt, dress, etc.), costumes, and other items that can be worn by the character object. This is simulated image information.

As the place where the wearable object is placed, information indicating the place where the wearable object is associated with which part of the character object is stored. For example, when the wearable object is a "headband", information indicating the "head" of the character object is stored as the placement location of the wearable object. Further, when the wearable object is a "T-shirt", information indicating the "torso" of the character object is stored as the placement location of the wearable object.

Information indicating a plurality of attachment sites in the three-dimensional coordinate space may be stored as the placement locations of the attachment object. For example, when the wearable object is a "headband", information indicating two parts of the character object, "left rear head" and "right rear head", may be stored as placement locations of the wearable object. As a result, the wearable object imitating the "headband" is displayed so as to be worn on both the "rear left side of the head" and the "rear right side of the head" of the character object.

When a character object is equipped with a plurality of types of mountable objects having a common mounting position, the plurality of types of mountable objects are mounted on the character object at intervals of time. As a result, a plurality of types of wearable objects having a common wearable part are not worn on the character object at the same time. For example, when "head" is stored as the mounting location of the mountable object indicating "headband" and the mountable object indicating "cap", the mountable object indicating "headband" and the mountable object indicating "cap" are At the same time, it is not displayed on the head of the character object.

In the object table T2, the display time corresponding to the type of additional object may be stored in association with the object ID of the additional object. For example, the display time of the wearable object may be longer than the display time of the effect object and the display time of the normal object. For example, when "60 seconds" is stored as the display time of the wearable object, "5 seconds" may be stored as the display time of the effect object, and "10 seconds" may be stored as the display time of the normal object.

(Group table T3)
FIG. 4(c) shows data of a group table T3 for managing groups to which users belong (friend-related user groups, mutual follower groups with other users, groups arbitrarily created by users or other users, etc.). It is an example of structure. The group table T3 is individually provided for each user, and the group table T3 shown in FIG. 4(c) is the group table T3 of the user who operates the terminal device 2 storing the group table T3.

In the group table T3, for each group, the group ID of the group, the name of the group, the users belonging to the group, etc. are stored in association with each other. A group ID is an example of identification data for uniquely identifying each group. The name is an example of data indicating the name of each group. The belonging user is the user ID of each user belonging to the group.

Returning to FIG. 3, functions of the generation unit 271, the transmission unit 272, the reception unit 273, the display processing unit 274, and the audio output unit 275 will be described.

(Generating unit 271)
The generation unit 271 acquires the imaging data continuously output by the imaging device 25 from the imaging device 25 . For example, the imaging device 25 of the terminal device 2 held by the user is provided in the terminal device 2 so that the face of the user looking at the display screen of the display device 23 faces the shooting direction. The imaging device 25 continuously captures images of the user's face in the shooting direction, acquires captured data of the user's face, and outputs the captured data to the generation unit 271 of the terminal processing device 27 . The imaging device 25 acquires imaging data by imaging parts other than the user's face, such as the head, arms, hands (including fingers), chest, body, legs, or parts other than these. You can Also, the imaging device 25 may be a 3D camera capable of detecting the depth of a person's face.

The generation unit 271 generates face motion data, which is a digital representation of the movement of the user's face, from time to time based on the continuously acquired imaging data. Face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data generated by the generation unit 271 can digitally represent the movement of the user's face (change in facial expression) in chronological order.

The generation unit 271 generates a body motion that is a digital representation of the position and orientation of each part of the user (head, arms, hands (including fingers), chest, torso, legs, or other parts). The data may be generated along with the face motion data or separately from the face motion data.

Body motion data may be generated based on detection information from known motion sensors worn by the user. In this case, the terminal communication I/F 21 of the terminal device 2 has a predetermined communication circuit for wirelessly acquiring detection information from the motion sensor worn by the user. The generator 271 then generates body motion data based on the detection information acquired by the terminal communication I/F 21 . Body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data generated by the generation unit 271 can digitally represent the motion of the user's body in chronological order.

Generation of body motion data based on detection information from motion sensors worn by the user may be performed, for example, in a photography studio. In this case, the photography studio may be equipped with a base station, a tracking sensor, and a display. A base station is, for example, a multi-axis laser emitter. The motion sensor worn by the user may be, for example, a Vive Tracker provided by HTC CORPORATION, and the base station may be, for example, a base station provided by HTC CORPORATION.

Also, a supporter computer may be installed in a separate room adjacent to the photography studio. A display in the photography studio may be configured to display information received from the supporter computer. The server device 3 may be installed in the same room as the room where the supporter computer is installed. The room in which the supporter computer is installed and the photography studio may be separated by a glass window. In this case, the operator of the supporter computer can visually recognize the user. The supporter computer may be configured to be able to change the settings of various devices provided in the photography studio according to the operation of the supporter. The supporter computer can, for example, set the scanning interval by the base station, set the tracking sensor, and change various settings of various devices other than these. The operator may enter a message into the supporter computer, and the entered message may be displayed on the display of the photography studio.

The generation unit 271 generates character moving image data including face motion data and/or body motion data generated as needed, and outputs the generated character moving image data to the display processing unit 274 . Hereinafter, face motion data and body motion data may be collectively referred to as motion data. The generation unit 271 also generates output information including the generated character moving image data and the user ID stored in the terminal storage device 22 , and outputs the generated output information to the transmission unit 272 . Note that when the generating unit 271 acquires the user's voice data output from the microphone 26, the output information includes the generated character video data, the acquired voice data, and the user ID stored in the terminal storage device 22. and outputs the generated output information to the transmission unit 272 .

The generation unit 271 generates game image data showing a game screen of a game played by the user at any time during the progress of the game, and output information including the generated game image data and the user ID stored in the terminal storage device 22. and outputs the generated output information to the transmission unit 272 . The output information may include the generated character moving image data and/or the acquired audio data together with the generated game image data.

(transmission unit 272)
The transmission unit 272 transmits the output information output from the generation unit 271 to the server device 3 via the terminal communication I/F 21 together with the destination information. The destination information includes user IDs of one or more other users participating in the communication service with the user. Also, the user ID included in the output information may be used as the destination information. For example, the server device 3 stores the user IDs of one or more other users participating in a communication service with the user, and when the output information is acquired, the communication service with the user whose user ID is included in the output information. The user IDs of one or more other users participating in the program are specified, and the received output information is transmitted to the terminal devices 2 of the other users indicated by the specified user IDs.

In addition, the transmission unit 272 transmits information indicating various instructions input by the user by operating the input device 24 together with the user ID and destination information stored in the terminal storage device 22 via the terminal communication I/F 21. to the server device 3. Various instructions by the user include, for example, additional instructions. The additional instruction is transmitted to the server device 3 via the terminal communication I/F 21 together with the object ID and user ID of the additional object to be included in the other user's output image.

(Receiver 273)
The receiving unit 273 receives, via the terminal communication I/F 21 , output information of other users, information indicating various instructions of other users, and the like, which are transmitted from the server device 3 . The receiving section 273 outputs the received information to the terminal processing device 27 . The other user's output information transmitted from the server device 3 is generated by the generation unit 271 of the other user's terminal device 2 and transmitted to the server device 3 by the transmission unit 272 of the other user's terminal device 2 . The other user's output information includes the other user's character moving image data, the other user's voice data, the other user's user ID, and the like. The other user's output information may not include the other user's voice data, and the other user's output information may not include the other user's character moving image data.

(Display processing unit 274)
The display processing unit 274 displays an image drawn based on the motion data (character moving image data) generated by the generating unit 271 on the display device 23 as a user output image. The display processing unit 274 also displays an image drawn based on the motion data (character moving image data) included in the other user's output information received by the receiving unit 273 on the display device 23 as the other user's output image. An image rendering process based on motion data will be described below. In the image drawing process based on the motion data of the user (the user who operates the terminal device 2), the user ID stored in the terminal storage device 22 is used and included in the other user's output information received by the receiving unit 273. The user ID included in the output information is used in the drawing process of the image based on the motion data.

First, the display processing unit 274 extracts the model ID of the character object associated with the user ID and the object ID of the used object from the user table T1. Next, the display processing unit 274 reads from the terminal storage device 22 the model data associated with the extracted model ID, and the image information and arrangement location associated with the extracted object ID of the used object. Next, the display processing unit 274 generates an animation of the character object with the use object attached to the placement location based on the read model data, the image information of the use object, the placement location, and the motion data. When the motion data includes face motion data, the display processing unit 274 generates animation of the character object such that the facial expression of the character object changes based on the face motion data. Accordingly, the display processing unit 274 can generate an animation of the character object that moves in synchronization with the movement of the user's facial expression based on the read model data and face motion data. When the motion data includes face motion data and body motion data, the display processing unit 274 moves in synchronization with the user's facial expression and body motion based on the read model data, face motion data, and body motion data. Generate animations for character objects. Next, the display processing unit 274 draws an image including the generated animation (moving image) of the character object and the background image generated using the background data stored in the terminal storage device 22. Note that the display processing unit 274 may use, as the background image, a wallpaper object pre-associated with the communication group and stored in the terminal storage device 22 . In this case, the same wallpaper object may be applied as the background image for all members of the communication group. Then, the display processing unit 274 outputs drawing data representing the drawn image, and ends the image drawing processing based on the motion data. An image drawn based on the character object associated with the user ID stored in the terminal storage device 22 is the user's user output image. Also, an image drawn based on the character object associated with the user ID included in the received output information of the other user is the other user output image of the other user.

In addition, the display processing unit 274 may display, on the display device 23, a character object newly equipped with a wearable object owned by the user as a user owned object according to the user's change instruction. For example, when the wearable object instructed to be changed by the user is "headband", the display processing unit 274 sets the object ID of the wearable object ("headband") to the holding object associated with the user's user ID in the user table T1. Extract from object and store as used object. The display processing unit 274 extracts the image information associated with the object ID stored as the object to be used and the arrangement location from the object table T2. Next, the display processing unit 274, based on the extracted image information and arrangement location, displays the character object with the wearing object (“Katyusha”) attached to the head (placement location associated with the object ID of “Katyusha”). Generate a user output image containing Note that the display processing unit 274 may generate a user output image including a character object wearing a wearable object that moves along with the movement of the wearable part (arrangement location) of the character object.

In addition, the display processing unit 274 displays a user output image including an additional object in response to an additional object addition instruction from another user. The display processing unit 274 displays the other user's output image including the additional object in response to an additional instruction from the user to display the additional object in the other user's output image.

If the object ID included in the additional object addition instruction from the other user is the object ID of any effect object, the display processing unit 274 refers to the object table T2 and associates the object ID with the object ID included in the addition instruction. A still image or moving image (image information) of the obtained effect object is extracted from the terminal storage device 22, and a user output image including the extracted still image or moving image is generated. For example, when an instruction to add an effect object representing confetti or fireworks is given, the display processing unit 274 generates a user output image including a moving image of an effect object (not shown) imitating confetti or fireworks. When the object ID included in the addition instruction of the additional object from the user is the object ID of any effect object, the display processing unit 274 similarly refers to the object table T2 to obtain the object ID included in the addition instruction. A still image or moving image (image information) of the effect object associated with is extracted from the terminal storage device 22, and another user's output image including the extracted still image or moving image is generated.

When the object ID included in the addition instruction of the additional object from the other user is the object ID of the normal object, the display processing unit 274 refers to the object table T2 to obtain the normal object associated with the object ID of the normal object. Still images or moving images (image information) and locations of arrangement are extracted. Next, the display processing unit 274 generates a user output image including a still image or moving image of the normal object arranged at the position indicated by the arrangement position. Further, when the object ID included in the addition instruction of the additional object from the user is the object ID of the normal object, the display processing unit 274 similarly refers to the object table T2 to refer to the object ID of the normal object. In addition, the still image or moving image (image information) of the normal object and the placement location are extracted, and the other user's output image including the still image or moving image of the normal object placed at the position indicated by the placement location is generated.

The display processing unit 274 may generate a user output image and other user output images including a moving image of a normal object that moves within the display area of the user output image and other user output images. User output images and other user output images may be collectively referred to as output images hereinafter. For example, the display processing unit 274 may generate an output image including a moving image of a normal object falling from the top to the bottom of the output image. In this case, the normal object may be displayed within the display area of the output image while falling from the start of falling to the bottom of the output image, and disappear from the output image after falling to the bottom of the output image. It should be noted that the movement route of a normal object may be a left-to-right direction, a right-to-left direction, an upper-left to lower-left direction, or any other route in the output image. It may be a route along a trajectory, a helical trajectory, or some other trajectory.

When the object ID included in the additional object addition instruction from the other user is the object ID of the wearable object, the display processing unit 274 displays notification information regarding the wear instruction in the user output image. The display processing unit 274 refers to the object table T2 according to the user's mounting instruction, and extracts the image information and the placement location of the mounting object associated with the object ID included in the additional instruction. Next, the display processing unit 274 generates a user output image including the character object wearing the wearing object at the wearing part indicated by the placement position based on the extracted image information and the placement position.

In addition, the display processing unit 274 starts a game played by the user and displays the game screen of the game on the display device 23 according to the user's instruction. When the game is started, the display processing unit 274 displays on the display device 23 the game screen of the game progressing according to the game progress instruction by the user's game operation. End the display. The end condition of the game is predetermined according to the rules of the game. Alternatively, the game object operated by the user is in a predetermined state (for example, a plurality of pieces (game objects) are arranged in a square on the game field).

Further, when the output information of the other user transmitted from the server device 3 includes the game image data of the game played by the other user, the display processing unit 274 displays the game image of the game played by the other user on the display device 23. to display. Note that when the game image of the game played by the other user is displayed, the other user output image of the other user is not displayed, but the game image of the game played by the other user and the other user output image of the other user are displayed. may be displayed at the same time.

(Audio output unit 275)
When the other user's output information includes the other user's voice data, the voice output unit 275 causes the display processing unit 274 to display the other user's output image and output the other user's voice generated based on the voice data to the speaker ( (not shown) for audio output. As a result, the voice of the other user is output while the lips of the character object of the other user included in the output image of the other user are moving. You can see the user output image.

(Server device 3)
FIG. 5 is a diagram showing an example of a schematic configuration of the server device 3. As shown in FIG. A server device 3 that provides a communication service to each terminal device 2 of a plurality of users transmits output information and information indicating various instructions transmitted from the terminal device 2 of one user to another user different from the one user. is transmitted to the terminal device 2 of the For this purpose, the server device 3 includes a server communication I/F 31 , a server storage device 32 and a server processing device 33 . For example, the terminal device 2 to which information is sent from one user's terminal device 2 is the user's terminal device 2 indicated by the user ID included in the received destination information.

The server communication I/F 31 is implemented as hardware, firmware, communication software such as a TCP/IP driver or PPP driver, or a combination thereof. The server device 3 can transmit information to and receive information from other devices via the server communication I/F 31 .

The server storage device 32 is, for example, a semiconductor memory device such as ROM and RAM. The server storage device 32 may be, for example, a magnetic disk, an optical disk, or any other storage device capable of storing data. The server storage device 32 stores an operating system program, a driver program, an application program, data, and the like used for processing in the server processing device 33 . The server storage device 32 may store the user table T1, the object table T2, and the group table T3 as data.

The server processing device 33 is a processor that loads an operating system program, a driver program, and a control program stored in the server storage device 32 into memory and executes instructions contained in the loaded programs. The server processing device 33 is, for example, an electronic circuit such as a CPU, MPU, DSP, or GPU, or a combination of various electronic circuits. The server processing device 33 may be realized by integrated circuits such as ASIC, PLD, FPGA, and MCU. Although server processing unit 33 is illustrated as a single component in FIG. 5, server processing unit 33 may be a collection of a plurality of physically separate processors. The server processor 33 functions as a server receiver 331 and a server transmitter 332 by executing various commands included in the control program.

(Server receiving unit 331)
The server reception unit 331 receives, via the server communication I/F 31, output information and information indicating various instructions transmitted from the terminal device 2 of one of the users. The server reception unit 331 may receive destination information together with output information and information indicating various instructions transmitted from the terminal device 2 .

(Server transmission unit 332)
The server transmission unit 332 receives the output information and the information indicating various instructions received by the server reception unit 331 via the server communication I/F 31 and transmits the information to another user indicated by the user ID included in the destination information received at the same time. is transmitted to the terminal device 2 of the In addition, the server transmission unit 332 transmits information indicating various instructions received by the server reception unit 331 to one or more other users designated by the user who transmitted the information via the server communication I/F 31. may be transmitted to the terminal device 2 of

(Examples of various screens)
Examples of various screens displayed on the display device 23 of the user's terminal device 2 will be described below with reference to FIGS. 6 to 15. FIG. In FIGS. 6 to 15, a plurality of constituent elements denoted by the same reference numerals are constituent elements having mutually equivalent functions.

FIG. 6A is a diagram showing an example of a group creation screen 600 displayed on the display device 23 of the user's terminal device 2. FIG. A group creation screen 600 includes an other user display area 601 , a selection object 602 and a create button 603 . For example, on the group creation screen 600, a group creation object or the like of a home screen (not shown) displayed according to execution of a control program stored in the terminal storage device 22 is selected by the user's operation of the input device 24. displayed when

The other user display area 601 displays other user information indicating each of the other users having a predetermined relationship with the user. In the example shown in FIG. 6A, the display processing unit 274 of the user's terminal device 2 displays the thumbnail image of the character object of the other user and the name of the other user as other user information.

The selection object 602 is an operation object for selecting each other user indicated by each other user information. In the example shown in FIG. 6( a ), the display processing unit 274 of the user's terminal device 2 displays check box objects corresponding to each other user's information as selection objects 602 . For example, when the selection object 602 is selected by the user's operation of the input device 24, the selection object 602 is displayed including a "check" mark. In this case, the other user of the other user information corresponding to the selection object 602 displayed including the "check" mark is selected. Further, when the selection object 602 displayed including the "check" mark is selected by the user's operation of the input device 24, the selection object 602 is displayed without the "check" mark.
In this case, the other user of the other user information corresponding to the selection object 602 displayed without the "check" mark is not selected. Each selection object 602 is associated with the user ID of the other user associated with the corresponding other user information.

A create button 603 is a button object or the like for the user to create a new communication group. When the create button 603 is selected by the user operating the input device 24, a new group is created to which the user operating the terminal device 2 and the other users selected in the selection object 602 belong. For example, the display processing unit 274 of the user's terminal device 2 stores the user ID of the user stored in the terminal storage device 22 and the other user associated with the selection object 602 related to the other user selected in the selection object 602. and the user ID of Then, the display processing unit 274 associates the user ID of the specified user and the user IDs of other users with the newly created group ID as belonging users, and stores them in the group table T3. At this time, the display processing unit 274 may associate the name of the automatically created group with the newly created group ID, or may associate the name of the group input by the user. .

When a new group is created in response to selection of the creation button 603, the display processing unit 274 displays an information exchange screen 700 by users belonging to the new group. FIG. 7A shows an example of an information exchange screen 700 displayed on the display device 23 of the terminal device 2 of the user. The information exchange screen 700 includes a display area 701 for displaying information such as text or images input by each of the users belonging to the new group, and information (characters) indicating the belonging users who input the information displayed in the display area 701 . object thumbnail image, etc.) 702, an input object 703 for the user to enter text or an image to be displayed in the new display area 701, and a start object 704 for starting the communication service.

When the information exchange screen 700 is displayed, the transmission unit 272 of the user's terminal device 2 transmits the display instruction of the information exchange screen to the server device 3 via the terminal communication I/F 21 together with the information on the new group and the destination information. Send to The information about the new group is the group ID, name, and user IDs of the users belonging to the new group. The destination information includes the user IDs of each of the users belonging to the new group, excluding the user who operates the terminal device 2 . The server receiving unit 331 of the server device 3 receives, via the server communication I/F 31 , the instruction to display the information exchange screen, the information on the new group, and the destination information. Note that the server reception unit 331 may store the received destination information in the server storage device 32 . The server transmission unit 332 of the server device 3 transmits the display instruction of the information exchange screen and the information on the new group to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication I/F 31. Send. When receiving the information exchange screen display instruction transmitted from the user's terminal device 2 via the server device 3, the other user's terminal device 2 displays the information exchange screen 700 on which information can be exchanged by users belonging to a new group. can be displayed.

Returning to FIG. 7(a). For example, when new information is input to the input object 703 by the operation of one of the affiliated users, the display area 701 displays the new information on the information exchange screen 700 displayed by the terminal device 2 of the user who has input the information. is added. Also, the transmitting unit 272 of the terminal device 2 of the user who has input the information transmits the new input information to the terminal devices 2 of the affiliated users other than the user who has input the information via the server device 3 . As a result, a display area 701 for displaying new information is added to the information exchange screen 700 displayed on the terminal device 2 of another user who belongs to the user other than the user who has input the information.

A start object 704 is a button object or the like for starting a communication service in which each user belonging to a new group can participate. When the start object 704 is selected by the user's operation of the input device 24, a communication service start process is executed in which a group capable of exchanging information on the information exchange screen 700 is set as a communication group. An example of start processing in response to selection of the start object 704 by the user will be described below.

First, the display processing unit 274 of the terminal device 2 of the user displays a communication screen 810 (FIG. 8B) that can be displayed by the terminal device 2 of each user belonging to the communication group. Also, the display processing unit 274 stores the group ID of the communication group in the terminal storage device 22 as the group ID of the group that started the communication service. Further, the display processing unit 274 stores the user's user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. Note that if all other users belonging to the communication group have not yet participated in the communication service, only the user's user output information is displayed on the communication screen 810 .

Next, the transmission unit 272 of the user's terminal device 2 transmits the communication service start instruction to the terminal communication I/F 21 together with the user's user ID, communication group information and/or transmission destination information stored in the terminal storage device 22. to the server device 3 via the The information about the communication group is the group ID and name of the communication group, and the user IDs of the users belonging to the communication group. The destination information includes the user IDs of each of the users belonging to the communication group, excluding the user who operates the terminal device 2 . Note that if the destination information is stored in the server storage device 32, the destination information does not have to be transmitted. The server receiving unit 331 of the server device 3 receives the start instruction, the user ID, the information on the communication group, and the destination information via the server communication I/F 31 . The server transmission unit 332 of the server device 3 transmits the start instruction, the user ID, and information about the communication group to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication I/F 31. .

The receiving unit 237 of the other user's terminal device 2 receives from the server device 3 the start instruction, the user ID, and the communication group information transmitted from the user's terminal device 2 . The receiving unit 237 stores the group ID of the information about the communication group in the terminal storage device 22 as the group ID of the group that started the communication service. Furthermore, the display processing unit 274 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the other user's terminal device 2 displays a notification screen (not shown) based on the received information about the communication group. The display processing unit 274 displays a group selection screen 710 (FIG. 7B) for participating in the communication service on the display device 23 in response to the user's selection of a predetermined object in the notification screen. With the above, the start processing ends. Note that the display processing unit 274 of the other user's terminal device 2 may include a participation button 805, which will be described later, on the notification screen. This allows other users to immediately participate in the communication service without displaying the group selection screen 710 .

The start process may be executed when the creation button 603 of the group creation screen 600 is selected by the user operating the input device 24 . That is, in response to selection of the creation button 603 by the user, a communication service start process may be executed in which a new group created on the group creation screen 600 is set as a communication group. In this case, the information exchange screen 700 is not displayed.

Further, the start processing is not limited to the start processing of the communication service in which the new group created on the group creation screen 600 is set as the communication group. For example, a communication service start process may be executed in which an already created group is set as a communication group. For example, FIG. 6(b) shows an example of a group selection screen 610 for selecting a communication group created by any user. For example, on the group selection screen 610, a group selection object or the like of a home screen (not shown) displayed according to execution of a control program stored in the terminal storage device 22 is selected by the user's operation of the input device 24. displayed when

A group selection screen 610 displayed on the user's terminal device 2 includes a group display area 611 for selecting one of a plurality of communication groups. The group display area 611 displays group information indicating each communication group that has already been created and to which the user belongs. In the example shown in FIG. 6B, the display processing unit 274 of the user's terminal device 2 displays the thumbnail image of each user belonging to the communication group and the name of the group as group information. In the example shown in FIG. 6B, thumbnail images of other users among belonging users are displayed, but thumbnail images of all belonging users may be displayed. If the communication group includes only one user other than the user, the name of the other user is displayed instead of the name of the group, as shown in FIG. 6(b). good too.

When one group information included in the group display area 611 is selected by the user's operation of the input device 24, an information exchange screen 700 by users belonging to the communication group corresponding to the selected group information is displayed. The information exchange screen 700 includes information such as text or images input by each of the users belonging to the communication group. In this case as well, an instruction to display an information exchange screen and information about the communication group are sent to the terminal devices 2 of other users among the users belonging to the communication group via the server device 3. An information exchange screen 700 is also displayed.

Then, when the start object 704 is selected by the user on the information exchange screen 700, a communication service start process is executed by the communication group that exchanges information on the information exchange screen 700. FIG. When the start process is executed, the display processing unit 274 of the terminal device 2 of the user displays a communication screen 810 (FIG. 8B) that can be displayed by the terminal device 2 of each user belonging to the communication group. In addition, the display processing unit 274 of the other user's terminal device 2 displays a notification screen (not shown), and according to the other user's operation on the notification screen, a group selection screen 710 ( 7(b)) is displayed on the display device 23. FIG. Also in this case, the display processing unit 274 of the other user's terminal device 2 may include a participation button 805, which will be described later, on the notification screen.

Also in this case, the start process may be executed when one piece of group information is selected on the group selection screen 610 by the user's operation of the input device 24 . That is, in response to selection of one piece of group information by the user, communication service start processing by the communication group corresponding to the selected group information may be executed. In this case, the information exchange screen 700 is not displayed.

Next, a method for participating in a user who has not yet participated in a communication service that has already started will be described. FIG. 7B is a diagram showing an example of a group selection screen 710 displayed on the display device 23 of the user's terminal device 2. As shown in FIG. For example, group selection screen 710 is displayed in response to selection of a given object within the displayed notification screen. The group selection screen 710 is displayed when a group selection object or the like of the home screen displayed according to the execution of the control program stored in the terminal storage device 22 is selected by the user's operation of the input device 24. good too.

In the group display area 711 of the group selection screen 710, group information indicating the communication group that has started the communication service is displayed so as to be distinguishable from other group information. In the example shown in FIG. 7B, the group information displayed at the top indicates the communication group that has started the communication service. For example, information indicating the number of belonging users participating in the communication service is displayed superimposed on or adjacent to each thumbnail image of the users belonging to the communication group. Also, a mark image 712 indicating the communication service is displayed near the name of the communication group.

When the user operates the input device 24 to select, from among the group information contained in the group display area 711, the group information indicating the communication group that has started the communication service, the communication group corresponding to the selected group information is displayed. An information exchange screen 800 by affiliated users is displayed.

FIG. 8A is a diagram showing an example of an information exchange screen 800 displayed on the display device 23 of the user's terminal device 2. FIG. On the information exchange screen 800, display areas 801a to 801b for displaying information such as text or images input by each of the users belonging to the communication group that has started the communication service, and display areas 801a to 801b. Information (thumbnail images of character objects, etc.) 802a to 802b indicating belonging users who have input the information, an input object 803 for the user to input text or images to be displayed in a new display area 801, It includes a service display area 804 that shows the communication services that are available, and a participation button 805 for participating in communication services that have already started.

In the service display area 804 shown in FIG. 8A, thumbnail images of character objects of three users participating in the communication service are displayed. A display area 801b shown in FIG. 8A displays information indicating that the communication service has started. Participation processing is executed when the participation button 805 is selected by the user. An example of participation processing will be described below.

When the user selects the participation button 805 by operating the input device 24 , first, the display processing unit 274 of the user's terminal device 2 displays the user's user ID stored in the terminal storage device 22 as a service display area 804 related to the service display area 804 . It is stored in the terminal storage device 22 as the user ID of the user participating in the communication service. Next, the display processing unit 274 displays a communication screen 810 (FIG. 8B) including user output images of all users participating in the communication service.

Next, the transmission unit 272 of the user's terminal device 2 sends the communication service participation instruction to the terminal communication I/F 21 together with the user's user ID, communication group information and/or destination information stored in the terminal storage device 22. to the server device 3 via the The information about the communication group is the group ID of the communication group, the name of the communication group, and the user IDs of the users belonging to the communication group. The destination information includes the user IDs of each of the users belonging to the communication group, excluding the user who operates the terminal device 2 . The server reception unit 331 of the server device 3 receives the participation instruction, the user ID, the communication group information and/or the transmission destination information via the server communication I/F 31 . The server transmission unit 332 of the server device 3 transmits the participation instruction, the user ID, and information about the communication group to the terminal device 2 of the other user indicated by the user ID included in the destination information via the server communication I/F 31. .

The receiving unit 237 of the other user's terminal device 2 receives from the server device 3 the participation instruction, the user ID, and the communication group information transmitted from the user's terminal device 2 . The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. When the communication screen 810 is displayed on the terminal device, the display processing unit 274 of the other user's terminal device 2 displays the communication screen 810 including user output images of all users participating in the communication service. Participation processing is completed by the above.

The selection process may be executed when group information is selected on the group selection screen 710 by the user operating the input device 24 . That is, according to the selection of group information by the user, a process of participating in a communication service in which the communication group corresponding to the group information can participate may be executed. In this case, the information exchange screen 800 is not displayed.

Embodiments for initiating or joining a communication service are not limited to the examples given above. For example, other initiation processes may be performed that are different from initiation processes in response to user selection of initiation object 704 . Another start process is executed when the user selects a predetermined display object for designating the start of the communication service on a predetermined screen such as the home screen. Note that the predetermined display object may be displayed when a thumbnail image or the like of the user's character object is selected, or may be included in menu items on a predetermined screen. Other initiation processes may be automatically executed when predetermined initiation conditions are met. For example, the predetermined start condition is that the current time reaches a predetermined time, that a predetermined time has passed since the end of the previous communication service, or that the number of mutual followers of the user exceeds a predetermined number. be. An example of another start process will be described below.

First, the transmission unit 272 of the user's terminal device 2 transmits the communication service start instruction to the server device via the terminal communication I/F 21 together with the user's user ID and/or destination information stored in the terminal storage device 22. Send to 3. Also, the transmission unit 272 stores the user ID of the user in the terminal storage device 22 as the user ID of the user participating in the communication service. The destination information includes user IDs of one or more other users who have a predetermined relationship with the user. For example, the destination information may include user IDs of one or more other users who are mutual followers of the user. In addition, instead of or in addition to the user IDs of one or more other users who are mutual followers of the user, the destination information may include other users who have a mutual following relationship with a predetermined user who is a mutual follower of the user. An ID may be included.

The server reception unit 331 of the server device 3 receives the start instruction, user ID and/or destination information via the server communication I/F 31 . The server transmission unit 332 of the server device 3 transmits the start instruction, the user ID, and the destination information to the other user's terminal device 2 indicated by the user ID included in the destination information via the server communication I/F 31 . The receiving unit 237 of the other user's terminal device 2 receives from the server device 3 the start instruction, the user ID, and the transmission destination information transmitted from the user's terminal device 2 . The receiving unit 237 stores the transmitted user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. The display processing unit 274 of the other user's terminal device 2 displays a notification screen (not shown) on the display device 23 based on the received user ID. The notification screen includes information for prompting the user indicated by the received user ID to participate in the communication service, a participation button, and the like.

When the participation button in the notification screen is selected by another user's operation of the input device 24, the display processing unit 274 of the other user's terminal device 2 stores the other user's user ID stored in the terminal storage device 22 in the service. This is stored in the terminal storage device 22 as the user ID of the user participating in the communication service related to the display area 804 . Next, the display processing unit 274 displays a communication screen 810 (FIG. 8(b)) including user output images of all users participating in the communication service.

Next, the transmission unit 272 of the other user's terminal device 2 transmits the communication service participation instruction together with the other user's user ID and/or destination information stored in the terminal storage device 22 via the terminal communication I / F 21 Send to the server device 3 . For the destination information, the user ID of the other user of the terminal device 2 to which the destination information is to be transmitted is deleted from the user ID included in the destination information previously received by the receiving unit 237. , plus the user ID received by H.237. The server reception unit 331 of the server device 3 receives the participation instruction, user ID, and/or transmission destination information via the server communication I/F 31 . The server transmission unit 332 of the server device 3 transmits the participation instruction, the user ID, and the destination information to the other user's terminal device 2 indicated by the user ID included in the destination information via the server communication I/F 31. .

The receiving unit 237 of the terminal device 2 of the user to which the participation instruction is transmitted receives from the server device 3 the participation instruction, the user ID, and the transmission destination information transmitted from the user's terminal device 2 . The receiving unit 237 stores the received user ID in the terminal storage device 22 as the user ID of the user participating in the communication service. When the communication screen 810 is displayed on the terminal device 2, the display processing unit 274 of the terminal device 2 displays the communication screen 810 including user output images of all users participating in the communication service. A series of participation processing is completed by the above.

It should be noted that the embodiment of starting or participating in a communication service with other users having a predetermined relationship with the user is not limited to the above examples. For example, when the user selects a predetermined display object for specifying the start of the communication service, a selection screen including information indicating other users who are mutual followers of one or more users may be displayed. . In this case, when information indicating any other user is selected by the user, another start process may be executed based on a start instruction for starting communication with the selected other user. Furthermore, in this case, an information exchange screen with the selected other user is displayed, and by selecting a start button in the information exchange screen, another start process based on a start instruction to start communication with the other user is started. may be performed.

In a conventional information system, a user could not use a simple interface to initiate communication and/or to instruct other users having a predetermined relationship with the user, such as mutual followers of the user. In the information system 1 disclosed in the present embodiment, by executing the above-described other start processing, the user interface related to communication start and/or participation instruction is improved, and the communication between the server device 3 and the terminal device 2 It is possible to reduce the communication load.

FIG. 8B is a diagram showing an example of a communication screen 810 displayed on the display device 23 of the terminal device 2. As shown in FIG. The communication screen 810 is a communication screen of a communication group including the user who operates the terminal device 2 as a belonging user. Hereinafter, the user who operates the terminal device 2 will be referred to as user A, and the users other than user A participating in the communication service corresponding to the communication screen 810 will be referred to as user B1, user B2, user B3, . User B1, user B2, user B3, . . . may be collectively referred to as user B.

Communication screen 810 includes at least user output image 812a including user A's character object 811a. In the example shown in FIG. 8B, there are three users (user B1, user B2, and user B3) participating in the communication service other than user A. That is, the communication screen 810 includes user output images 812b1, 812b2, and 812b3 including character objects 811b1, 811b2, and 811b3 of user B1, user B2, and user B3, respectively. Hereinafter, the character objects 811b1, 811b2, and 811b3 may be collectively referred to as the character object 811b, and the user output images 812b1, 812b2, and 812b3 may be collectively referred to as the user output image 812b. Note that the user B is another user for the user A, and the user output image 812b of the user B displayed on the terminal device 2 of the user A is the other user output image.

When there is no user participating in the communication service other than user A, only the user output image 812a is included in the communication screen 810, and each time user B participates in the communication service, the user output image 812b of user B is newly displayed. is included in the communication screen 810. There is no upper limit to the number of users who can participate in the communication service. 812b1, 812b2, 812b3, . . . 812b9 are included in the communication screen 810.

The character object 811a is generated by the display processing unit 274 based on the motion data of the user A generated by the generation unit 271 and the character object and the use object associated with the user ID of the user A in the user table T1. This is an animation (moving image) of a character object of user A. FIG. The motion data of user A generated by the generation unit 271 is the same as the motion data included in the output information transmitted to the terminal device 2 of user B. FIG. User A's output information includes user A's voice data together with user A's motion data. When user A's face motion data is included in the user A's motion data generated by the generation unit 271, a user output image 812a including a character object 811a whose facial expression changes in synchronization with the facial expression of user A is displayed. be. For example, when user A is speaking, a user output image 812a including a character object 811a with moving lips is displayed. Further, when user A's body motion data is included in the user A's motion data generated by the generation unit 271, the user output including the character object 811a whose corresponding part moves in synchronization with the user A's body motion is generated. Image 812a is displayed.

Note that the transmission unit 272 of the terminal device 2 of the user A transmits the output information of the user A together with the transmission destination information (the user ID of each user B (user B1 to user B3) included in the communication group) to the terminal communication I/ It is transmitted to the server device 3 via F21. Next, the server receiving unit 331 of the server device 3 receives the output information of the user A transmitted from the terminal device 2 of the user A via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 refers to the received user ID of user B (user B1 to user B3), and transmits the output information of user A to user B (user B1 to the terminal device 2 of user B3). When the receiving unit 273 of the terminal device 2 of user B receives the output information via the terminal communication I/F 21, the display processing unit 274 of the terminal device 2 of user B receives the user ID included in the output information. displays a user output image 812a including a character object 811a based on user A's motion data indicated by .

The character object 811b is generated by the display processing unit 274 based on the motion data included in the output information of the user B received by the receiving unit 273 and the character object associated with the user ID of the user B in the user table T1. 10 is an animation (moving image) of a character object of user B that has been created. When user B's motion data received by the receiving unit 273 includes user B's face motion data, a user output image 812b including a character object 811b whose facial expression changes in synchronization with the movement of user B's facial expression is displayed. be. For example, when user B utters a voice, a user output image 812b including a character object 811b with moving lips is displayed. Further, when user B's body motion data is included in the user B's motion data generated by the generation unit 271, the user output including the character object 811b whose corresponding part moves in synchronization with the user B's body motion is generated. Image 812b is displayed. When user B's output information received by the receiving unit 273 includes user B's voice data, the user output image 812b is displayed by the display processing unit 274, and the user output image 812b is generated by the voice output unit 275 based on the voice data. User B's voice is output as voice.

By displaying the communication screen 810 including the user output image 812a and the user output image 812b in this way, the user A can listen to the voice of the user B while looking at the character object 811b of the user B who is uttering a voice. can hear User A's character object and voice data are output by user B's terminal device 2 . As a result, user A and user B participating in the communication service can communicate with each other via character objects.

Communication screen 810 includes an add button 813 and a game button 814 . In the example shown in FIG. 8B, the add button 813 and the game button 814 are arranged within the display area of the user output image 812a, but they may be displayed outside the display area of the user output image 812a. Also, the add button 813 and the game button 814 may be controlled by the display processing unit 274 so as to be displayed for a predetermined period of time when predetermined display conditions are satisfied.

The add button 813 is an object image such as a button object for instructing display of the selection screen 1200 for selecting an additional object to be included in the communication screen 810 . When the user A operates the input device 24 to select the add button 813, a selection screen 1200 is displayed. Details of the selection screen 1200 will be described later.

A game button 814 is an object image such as a button object for instructing the start of a game. When the user A operates the input device 24 to select the game button 814 , the input device 24 inputs a game start instruction to the display processing unit 274 . When the terminal processing device 27 acquires a game start instruction input from the input device 24 , the terminal processing device 27 executes various commands included in the control program (game program) stored in the terminal storage device 22 . Thereby, the display processing unit 274 of the terminal processing device 27 realizes the function of starting the game and displaying the game display object 1500 (FIG. 15(a)) including the game screen of the started game on the display device 23 . Details of the game display object 1500 including the game screen will be described later.

When the user A wants to equip the character object 811a with a new attachment object, or wants to change the attachment object attached to the character object 811a to a new attachment object, the user A selects the character object 811a. conduct. When the user A operates the input device 24 to select the character object 811a, the display of the communication screen 810 ends, and the communication screen 900 (FIG. 9A) is displayed. The communication screen 900 may be displayed when the user A operates the input device 24 to select the user output image 812a.

FIG. 9A is a diagram showing another example of the communication screen 900 displayed on the display device 23 of the terminal device 2. FIG. The communication screen 900 is displayed when the character object 811a or the user output image 812a is selected by the user A operating the input device 24 included in the communication screen 810 (FIG. 8B).

On the communication screen 900, a user output image 812a including the user A's character object 811a is displayed on the entire surface. Also, a user output image 901 including character objects 811b1, 811b2, and 811b3 of user B1, user B2, and user B3 is displayed superimposed on the user output image 812a. Note that the user output image 901 may not be displayed on the communication screen 900 . In this case, the voice data of each of user B1, user B2, and user B3 may be output.

Further, communication screen 900 includes add button 813 and game button 814 as well as change button 902 and screen exit button 903 . Note that the communication screen 900 may not include the add button 813 and the game button 814 .

The change button 902 is an object image such as a button object for instructing display of the selection screen 910 (FIG. 9B). When the user A operates the input device 24 to select the change button 902, a selection screen 910 for selecting a new wearable object to be worn by the character object 811a is displayed. Details of the selection screen 910 will be described later.

A screen termination button 903 is an object image such as a button object for instructing termination of display of the communication screen 900 . When the screen end button 903 is selected by user A's operation of the input device 24, the display of the communication screen 900 is terminated and the communication screen 810 (FIG. 8(b)) is displayed.

FIG. 9B is a diagram showing an example of a selection screen 910 displayed on the display device 23 of the terminal device 2. As shown in FIG. On the selection screen 910, a user output image 812a including the character object 811a of the user A is displayed on the entire surface, similar to the communication screen 900 (FIG. 9A). The selection screen 910 also includes a placement candidate object 911 and a screen exit button 912 .

As shown in the object table T2, the placement location where each mounting object is placed is one of a plurality of predetermined placement locations. In the example shown in FIG. 9B, a selection screen 910 includes a plurality of placement candidate objects 911 each indicating a plurality of placement locations. Note that the plurality of placement candidate objects 911 may correspond to a plurality of categories (types) to which each mountable object belongs. The number of placement candidate objects 911 displayed on the selection screen 910 is not limited to five. The display processing unit 274 of the terminal device 2 may display the placement candidate objects 911 in a scrollable manner when part of the image showing the plurality of placement candidate objects 911 cannot be included in the selection screen 910 . In this case, in response to a swipe operation by user A (for example, an operation in which the finger moves from the bottom to the top of the screen while the finger is in contact with the screen), the placement candidate objects 911 that are not currently displayed on the selection screen 910 are displayed on the selection screen. 910 is scrolled.

First, the user A selects an arrangement position of an attachment object to be attached to the character object 811a. That is, when one of the plurality of placement candidate objects 911 is selected by user A's operation of the input device 24, the wearable object of user A associated with the placement candidate corresponding to the selected placement candidate object 911 is selected. A selection screen 1000 for is displayed.

A screen termination button 912 is an object image such as a button object for instructing termination of display of the selection screen 910 . When the screen end button 912 is selected by the user A operating the input device 24, the display of the selection screen 910 is terminated, and the communication screen 810 (FIG. 8B) is displayed.

FIG. 10A is a diagram showing an example of a selection screen 1000 displayed on the display device 23 of the terminal device 2. FIG. On the selection screen 1000, a user output image 812a including the character object 811a of the user A is displayed on the entire surface, like the communication screen 900 (FIG. 9A). The selection screen 1000 also includes an attachment candidate object 1001 that is a candidate for an attachment object to be attached to the character object 811a, and an end screen button 1002. FIG.

The mounting candidate object 1001 is a selection object corresponding to the object ID of the owned object of user A associated with the placement candidate corresponding to the placement candidate object 911 selected on the selection screen 910 (FIG. 9B). In the example shown in FIG. 10A, the selection screen 1000 includes three mounting candidate objects 1001 respectively corresponding to the object IDs of the three possessed objects. The number of mounting candidate objects 1001 displayed on the selection screen 1000 is not limited to three. The display processing unit 274 of the terminal device 2 may display the mounting candidate objects 1001 in a scrollable manner if part of the image showing the plurality of mounting candidate objects 1001 cannot be included in the selection screen 1000 . In this case, in response to a swipe operation by user A (for example, an operation in which a finger moves from the bottom to the top of the screen while the finger is in contact with the screen), the attachment candidate object 1001 that is not currently displayed on the selection screen 1000 is displayed on the selection screen. 1000 is displayed by scrolling.

When user A operates the input device 24 to select one of the plurality of mounting candidate objects 1001 , the input device 24 inputs a change instruction to the terminal processing device 27 . The input device 24 inputs the object ID corresponding to the selected mounting candidate object 1001 to the terminal processing device 27 together with the change instruction. An example of change processing according to a change instruction will be described below.

First, when the display processing unit 274 acquires the change instruction and the object ID input from the input device 24, the acquired object ID is stored in the user table T1 in association with the user ID of the user A as an object to be used. Note that if the user table T1 stores the object ID of another use object at the same mounting position as the mount object of the acquired object ID, the display processing unit 274 stores the object ID of the other use object in the user table T1. The object ID is deleted from T1, and the acquired object ID is stored in the user table T1 in association with the user ID of user A as a used object. The display processing unit 274 displays the character object 1011a of the user A based on the motion data of the user A generated by the generation unit 271 and the character object and the use object associated with the user ID of the user A in the user table T1. A user output image 1012a including animation (moving image) is generated, and a selection screen 1010 (FIG. 10B) including the user output image 1012a is displayed. As a result, a user output image 1012a including the character object 1011a wearing the new wearable object is displayed.

Next, when the transmission unit 272 acquires the change instruction and the object ID input from the input device 24, the transmission unit 272 transmits information indicating the change instruction to the user ID of the user A stored in the terminal storage device 22 and the acquired object ID. ID and transmission destination information (user ID of user B (user B1 to user B3)) are transmitted to server apparatus 3 via terminal communication I/F 21 . Next, the server receiving unit 331 of the server device 3 receives the information indicating the change instruction, the user A's user ID, the object ID, and the transmission destination information via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 transmits the information indicating the change instruction, the user ID of the user A, and the object ID via the server communication I/F 31 to the terminals of the users B (users B1 to B3). Send to device 2. When the receiving unit 273 of the terminal device 2 of the user B receives the information indicating the change instruction, the user ID of the user A, and the object ID via the terminal communication I/F 21, the received object ID is used as the object to be used. It is stored in the user table T1 in association with the user A's user ID. The display processing unit 274 of the terminal device 2 of the user B displays the character of the user A based on the received motion data of the user A and the character object and the use object associated with the user ID of the user A in the user table T1. A user output image 1012a including an animation (moving image) of the object 1011a is generated, and a communication screen 1110 (FIG. 11B) including the user output image 1012a is displayed. As a result, the user output image 1012a including the character object 1011a wearing the new wearable object is displayed on the terminal device 2 of the user B as well. With the above, the change processing ends.

A screen termination button 1002 is an object image such as a button object for instructing termination of display of the selection screen 1000 . When the screen end button 1002 is selected by user A's operation of the input device 24, the display of the selection screen 1000 is terminated and the communication screen 810 (FIG. 8(b)) is displayed. Note that when one of the plurality of mounting candidate objects 1001 is selected by user A's operation of the input device 24, the change processing described above is executed, the display of the selection screen 1000 ends, and a new mounting object is mounted. A communication screen 1110 (FIG. 11(b)) including a user output image 1012a including a character object 1011a may be displayed.

FIG. 10B is a diagram showing an example of a selection screen 1010 displayed on the display device 23 of the terminal device 2. As shown in FIG. On the selection screen 1010, a user output image 1012a including the character object 1011a of the user A is displayed on the entire surface, similarly to the selection screen 1000 (FIG. 10(a)). Further, the selection screen 1010 includes, similarly to the selection screen 1000, an attachment candidate object 1013, which is a candidate for an attachment object to be attached to the character object 1011a, and a screen end button 1014.

The character object 1011a is the character object of the user after the change instruction is input on the selection screen 1000 (FIG. 10(a)). The wearing candidate object 1013 is a selection object corresponding to the object ID of the owned object of the user A associated with the same placement position as the wearing candidate object 1001 on the selection screen 1000 . The character object 1011a is equipped with an equipable object corresponding to one of the equipable candidate objects 1001 on the selection screen 1000 (FIG. 10A). A mounting candidate object corresponding to the object is not included in the mounting candidate object 1013 . Note that the wearing candidate object corresponding to the wearing object attached to the character object 1011 a may be included in the wearing candidate object 1013 . In this case, the wearing candidate object corresponding to the worn object may be grayed out so that the user A cannot select it.

A screen end button 1014 is an object image such as a button object for instructing to end the display of the selection screen 1010 . When the screen end button 1014 is selected by user A's operation of the input device 24, the display of the selection screen 1010 ends, and a communication screen 1110 ( FIG. 11(b)) is displayed. Note that an OK button (not shown) may be displayed on the selection screen 1010 . When the enter button is selected by user A's operation of the input device 24, the display of the selection screen 1010 ends, and a communication screen 1110 (FIG. 11) including a user output image 1012a including a character object 1011a equipped with a new attachment object is displayed. (b)) may be displayed. In this case, when the screen exit button 1014 is selected by the user A operating the input device 24, the display of the selection screen 1010 is terminated, and the communication screen 810 ( FIG. 8(b)) may be displayed.

The screen displayed when the change button 902 of the communication screen 900 is selected by the user A operating the input device 24 is not limited to the selection screen 910, and the selection screen 1100 may be displayed. FIG. 11A shows an example of a selection screen 1100 displayed on the display device 23 of the terminal device 2. FIG.

A selection screen 1100 shown in FIG. 11A is displayed superimposed on the communication screen 900 . The selection screen 1100 may be displayed after the display of the communication screen 900 is terminated.

The selection screen 1100 includes a plurality of placement candidate objects 1101 respectively indicating a plurality of placement locations, and mounting candidate objects 1102 that are candidates for mounting objects to be mounted on the character object 811a. When one of the plurality of placement candidate objects 1101 is selected by User A's operation of the input device 24, the object ID of the owned object of User A associated with the placement candidate corresponding to the selected placement candidate object 1101 corresponds to the object ID. The selection screen 1100 includes a mounting candidate object 1102 to be mounted. When one of the plurality of mounting candidate objects 1102 is selected by user A's operation of the input device 24, the input device 24 inputs a change instruction to the terminal processing device 27, the above-described change processing is executed, and display processing is performed. The unit 274 displays a communication screen 1110 (FIG. 11(b)) including a user output image 1012a including a character object 1011a wearing a new wearable object.

FIG. 11B is a diagram showing another example of the communication screen 1110 displayed on the display device 23 of the terminal device 2. As shown in FIG. The communication screen 1110 includes at least a user output image 1012a including a character object 1011a of user A wearing a new wearable object.

In addition, the communication screen 1110 includes character objects 811b1, 811b2, and 811b3 of users B1, B2, and B3 who are participating in the communication service, similarly to the communication screen 810 (FIG. 8B). User output images 812b1, 812b2, and 812b3 are included.

Note that when a change instruction input by user B participating in the communication service is input to user B's terminal device 2, change processing corresponding to the change instruction input by user B is performed on user B's terminal device. 2 and user A's terminal device 2 . As a result, the user output image including the character object of user B equipped with the new wearable object is displayed on user B's terminal device 2 and user A's terminal device 2 .

As described with reference to FIGS. 8B to 11B, at least part of the character object of user A is changed in response to a change instruction by user A, or Accordingly, at least a portion of user B's character object is changed. As a result, users A and B can be prevented from becoming bored during communication via character objects, and user A's willingness to continue communication with user B can be enhanced.

In addition, in conventional information systems, after a user has started communicating with another user through a character object, the user must interrupt the communication service in order to change the output image containing the character object. Ta. Then, after the communication service is interrupted, it is necessary to activate a function for changing the appearance of the character object and/or the image such as the background in the output image including the character object. After using the change function to change the character object and/or the output image, the user has to go through a complicated procedure of participating in the communication service again. In contrast, the information system 1 of the present embodiment enables the user's character object to be changed without interrupting communication via the character object and resetting the user's character object. As a result, in the information system 1 of the present embodiment, the user interface for changing the user's character object during communication does not become complicated, so it is possible to improve usability compared to conventional information systems. In addition, in the information system 1 of the present embodiment, the user's character object can be changed without interrupting communication, so the number of communications and the amount of communications between the server device 3 and the terminal device 2 can be reduced. Thus, in the information system 1 of this embodiment, it is possible to improve communication between the server device 3 and the terminal device 2 .

Returning to FIG. 8B, when user A wants to change at least one user B's user output image 812b by including an additional object in at least one user B's user output image 812b, user A clicks the add button 813 is selected. When the user A operates the input device 24 to select the add button 813, a selection screen 1200 is displayed.

FIG. 12A shows an example of a selection screen 1200 displayed on the display device 23 of the terminal device 2. FIG. A selection screen 1200 shown in FIG. 12A is displayed superimposed on the communication screen 810 . The selection screen 1200 may be displayed after the display of the communication screen 810 is terminated.

In the example shown in FIG. 12( a ), the display area of the selection screen 1200 includes images and names indicating eight types of addition candidate objects. The number of additional candidate objects included in the display area of selection screen 1200 is not limited to eight. The display processing unit 274 of the terminal device 2 displays the images showing the additional candidate objects in a scrollable manner when part of the images showing the plurality of additional candidate objects cannot be included in the display area of the selection screen 1200. good. In this case, in response to a swipe operation by user A (for example, an operation in which a finger moves from right to left while being in contact with the screen), an addition candidate object that is not currently included in the display area of the selection screen 1200 is displayed. The displayed image is scrolled and appears within the display area of the selection screen 1200 .

Each of the additional candidate objects indicates an additional object, and the image indicating the additional candidate object is, for example, an icon image or thumbnail image indicating a gift object. In the selection screen 1200, the image and/or name of each additional candidate object may be classified and displayed according to the type of additional object (free gift, paid gift, accessory, support goods, appeal, variety, etc.). . Note that the additional object belonging to the paid gift is an additional object that can be purchased by the user by consuming "coins" of virtual monetary value. An additional object belonging to a free gift is an additional object that can be obtained by consuming points that can be obtained by watching distribution instead of consuming monetary value. Note that the additional object may be an object owned by the user A by playing a predetermined mission executed on the terminal device 2 and clearing the predetermined mission.

When user A operates the input device 24 to select one of the additional candidate objects, the input device 24 inputs an additional instruction to the terminal processing device 27 . Before or after the selection of additional candidate objects, when user A operates the input device 24 to select one of the user output images 812b of user B who wants to include an additional object, the selected user output image 812b The corresponding user ID of user B is identified. The input device 24 inputs the object ID corresponding to the selected addition candidate object and the specified user ID to the terminal processing device 27 together with the addition instruction. When none of the user output images 812b of user B is selected, the input device 24 inputs an object ID corresponding to the selected addition candidate object to the terminal processing device 27 together with an addition instruction. In this case, all user B's user output images 812b will include the additional object. The selection of the user output image 812b of the user B who wants to include the additional object may be the selection of the character object 811b included in the user output image 812b. Also, two or more user output images 812b or character objects 811b of user B may be selected. An example of additional processing in response to an additional instruction when the user output image 812b1 of the user B1 is selected will be described below.

First, when the display processing unit 274 acquires the additional instruction and the object ID input from the input device 24, the display processing unit 274 refers to the object table T2 to refer to the still image or moving image (image) of the additional object associated with the acquired object ID. information) from the terminal storage device 22 and generate a user output image containing the extracted still or moving image. Note that when the user ID of the user B1 is acquired together with the additional instruction, the user output image including the extracted still image or moving image of the additional object is the user output image corresponding to the acquired user ID of the user B1. Note that if the user IDs of any of the users B1 to B3 have not been obtained along with the additional instruction, the user output images including the extracted still images or moving images are for all users (the user output image of the user A and the user output images of the users B1 to B3). User B3) is a user output image corresponding to each.

Next, when the transmitting unit 272 acquires the additional instruction and the object ID input from the input device 24, the transmission unit 272 transmits information indicating the additional instruction to the user ID of the user A stored in the terminal storage device 22 and the acquired object ID. ID and destination information (each user ID of user B (users B1 to B3)) are transmitted to server device 3 via terminal communication I/F 21 . Next, the server receiving unit 331 of the server device 3 receives the information indicating the addition instruction, the user ID of the user A, and the object ID via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 transmits the information indicating the addition instruction, the user ID of the user A, and the object ID to all the users B (users B1 to B3) via the server communication I/F 31. is transmitted to the terminal device 2 of the Then, when the receiving unit 273 of the terminal device 2 of the user B receives the information indicating the change instruction, the user ID of the user A, and the object ID via the terminal communication I/F 21, the terminal device 2 of the user B displays The processing unit 274 refers to the object table T2, extracts a still image or moving image (image information) of the additional object associated with the received object ID from the terminal storage device 22, and extracts the extracted still image or moving image of the additional object. Generating user output images that include moving images. Note that when the transmission unit 272 of the terminal device 2 of the user A transmits the user ID of the user B1 together with the addition instruction, the user ID of the user B1 is also transmitted by the server device 3 . When the terminal device 2 of the user B receives the user ID of the user B1 together with the addition instruction, the user output image including the additional object displayed on the terminal device 2 of the user B is the user output image corresponding to the received user ID of the user B1. Output image only. If user B's terminal device 2 has not acquired user B's user ID together with an additional instruction, the user output image including the additional object displayed on user B's terminal device 2 is available to all users (user A and user B1 to user B3). With the above, the addition process ends.

FIG. 12(b) is a diagram showing another example of the communication screen 1210 displayed on the display device 23 of the terminal device 2 after the user A gives the additional instruction. FIG. 12(b) shows a communication screen 1210 displayed on the display device 23 of the terminal device 2 after the addition instruction when the user ID of the user B1 is received together with the addition instruction. As shown in FIG. 12B, the user output image including the additional object 1211 is only the user output image 812b1 corresponding to the user ID of user B1.

FIG. 13(a) is a diagram showing another example of the communication screen 1300 displayed on the display device 23 of the terminal device 2 after the user A has given an additional instruction. FIG. 13A shows a communication screen 1300 displayed on the display device 23 of the terminal device 2 after the addition instruction when the user ID of the user B is not received together with the addition instruction. As shown in FIG. 13(a), the user output images including the additional object 1301 are user output image 812a of user A and user output images 812b1 to 812b3 corresponding to all user B user IDs. The user output image including the additional object 1301 after the user A instructed addition may be the user output images 812b1 to 812b3 corresponding to all the user IDs of the user B (the user output image 812a of the user A is not included). can also be used).

In the examples shown in FIGS. 12(a) to 13(a), an additional object is automatically added to the user output image of one or more users in response to the user's additional instruction. The additional object may be configured to be added to the user output image of one or a plurality of users later when there is an approval instruction from a user other than the user who gave the addition instruction.

FIG. 13(b) is a diagram showing another example of the communication screen 1310 displayed on the display device 23 of the terminal device 2 after the addition instruction. In the example shown in FIG. 13B, the additional instruction is input by operating the input device 24 of the terminal device 2 by the user B1. The additional instruction in the example shown in FIG. 13(b) is to select one of the additional candidate objects indicating the mounted object, and to select the user output image 812a of user A as the user output image to include the mounted object. is input by the input device 24 of the terminal device 2 of the user B1.

The terminal device 2 of the user A who has received the addition instruction together with the object ID of the mountable object corresponding to the selected addition candidate object displays a user output image 612a including the addition object 1311 as shown in FIG. 13(b). . The additional object 1311 displays information indicating that an additional object has been given by the user B1 who has given the additional instruction. The additional object 1311 is not displayed on the terminal devices 2 of the users B1 to B3. Note that the additional object 1311 may be displayed on the terminal devices 2 of the users B1 to B3.

When user A performs a predetermined operation (for example, a tap operation on user output image 812a, a swipe operation in a predetermined direction, etc.) on communication screen 1310 on which user output image 812a including additional object 1311 is displayed, user A The input device 24 of the terminal device 2 inputs the approval instruction to the terminal processing device 27 together with the object ID of the mounted object transmitted from the terminal device 2 of the user B1. An example of acceptance processing in response to an acceptance instruction will be described below.

First, when the display processing unit 274 acquires the acceptance instruction and the object ID input from the input device 24, the display processing unit 274 refers to the object table T2 and displays the information for the user wearing the additional object (wearing object) associated with the acquired object ID. Generate a user output image containing A's character object. Then, the display processing unit 274 displays the communication screen 1400 including the user output image 812a including the character object 811a of the user A wearing the additional object (wearing object) 1401. FIG.

Next, when acquiring the consent instruction and the object ID input from the input device 24, the transmission unit 272 transmits information indicating the consent instruction to the user ID of the user A stored in the terminal storage device 22 and the acquired object ID. ID and transmission destination information (user ID of user B (user B1 to user B3)) are transmitted to server apparatus 3 via terminal communication I/F 21 . Next, the server receiving unit 331 of the server device 3 receives the information indicating the acceptance instruction, the user ID of the user A, the object ID, and the destination information via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 transmits the information indicating the acceptance instruction, the user ID of the user A, and the object ID to each of the users B (users B1 to B3) via the server communication I/F 31. is transmitted to the terminal device 2 of the Then, the receiving unit 273 of the terminal device 2 of the user B receives the information indicating the acceptance instruction, the user ID of the user A, and the object ID via the terminal communication I/F 21 . Next, the display processing unit 274 of the terminal device 2 of the user B refers to the object table T2 to include the character object 811a of the user A wearing the additional object (wearable object) 1401 associated with the received object ID. A user output image 812a is generated. Then, the display processing unit 274 displays the communication screen 1400 including the user output image 812a including the character object 811a of the user A wearing the additional object (wearing object) 1401. FIG. With the above, the approval process is completed.

Note that the character objects changed by the additional object (wearable object) related to the additional instruction by the user B1 are all users other than the user B1 (user A, user B2, and user B3 in the example shown in FIG. 13B). character object. In this case, user output images 812a, 812b2 and 812b3 including the additional object 1311 are displayed on the terminal devices 2 of all users other than the user B1. When user A, user B2, and user B3 input approval instructions for additional object 1311, user output image 812a including character object 811a of user A wearing additional object (wearing object) 1401, additional object (wearing object) ) 1401 and a user output image 812b3 including a character object 811b3 of the user B3 wearing the additional object (wearing object) 1401 are displayed. In this case, the additional object (wearable object) 1401 may be attached to the character object 811b1 of the user B1 who has given the additional instruction.

FIG. 14(a) is a diagram showing another example of the communication screen 1400 displayed on the display device 23 of the terminal device 2 after the acceptance instruction. As shown in FIG. 14A, a communication screen 1400 displays a user output image 812a including a character object 811a to which an additional object (wearable object) 1401 is worn.

When a user's character object is newly equipped with a wearable object, it may be possible for other users to evaluate it. FIG. 14B is a diagram showing another example of the communication screen 1410 displayed on the display device 23 of the terminal device 2 of the user A when the wearable object 1411 is newly worn on the character object 811b2 of the user B2. be.

The communication screen 1410 newly includes an evaluation button 1412 along with an add button 813 and a game button 814 . In the example shown in FIG. 14B, the evaluation button 1412 is arranged within the display area of the user output image 812a, but may be displayed outside the display area of the user output image 812a. Also, the evaluation button 1412 may be display-controlled by the display processing unit 274 so as to be displayed for a predetermined time (for example, 10 seconds) after the user B2's character object 811b2 is newly equipped with the wearable object 1411.

The evaluation button 1412 is an object image for inputting an evaluation instruction for transmitting evaluation data indicating the user A's evaluation of the character object 811b2 on which the mountable object 1411 is newly mounted. When the user A operates the input device 24 to select the evaluation button 1412 , the input device 24 inputs an evaluation instruction to the terminal processing device 27 . The input device 24 inputs to the terminal processing device 27 the user ID of the user A stored in the terminal storage device 22 and the user ID of the user B2 related to the character object 811b2 newly equipped with the wearable object, together with the evaluation instruction. . An example of evaluation processing according to an evaluation instruction will be described below.

First, when the display processing unit 274 of the terminal device 2 of the user A acquires the evaluation instruction input from the input device 24, the user ID of the user A, and the user ID of the user B2, the character newly equipped with the wearable object is displayed. Display information indicating a positive evaluation of object 811b2. The display processing unit 274, for example, extracts the name of the user B2 associated with the user ID of the user B2, and displays a character string praising the name of the user B2 as information indicating positive evaluation. Also, for example, the display processing unit 274 may display a predetermined mark image as information indicating a positive evaluation within the display area of the user output image 812b2 of the character object 811b2.

Next, when the transmission unit 272 of the terminal device 2 of the user A acquires the information indicating the evaluation instruction input from the input device 24, the user ID of the user A, and the user ID of the user B2, the transmission unit 272 transmits the acquired information. It is transmitted to the server device 3 via the terminal communication I/F 21 together with destination information (user ID of user B (user B1 to user B3)). Next, the server receiving unit 331 of the server device 3 receives the information indicating the evaluation instruction, the user ID of the user A, the user ID of the user B2, and the destination information via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 transmits the information indicating the evaluation instruction, the user ID of the user A, and the user ID of the user B2 to the user B (user B1 to user B3) via the server communication I/F 31. ) to the terminal device 2. Then, when the receiving unit 273 of the terminal device 2 of the user B receives the information indicating the change evaluation instruction, the user ID of the user A, and the user ID of the user B2 via the terminal communication I/F 21, The display processing unit 274 of the terminal device 2 displays information indicating a positive evaluation of the character object 811b2 to which the new wearable object is attached. The display processing unit 274, for example, extracts the name of the user B2 associated with the user ID of the user B2, and displays a character string praising the name of the user B2 as information indicating positive evaluation. Also, for example, the display processing unit 274 may display a predetermined mark image as information indicating a positive evaluation within the display area of the user output image 812b2 of the character object 811b2. With the above, the evaluation process ends.

As a result, the user A can react with a simple operation when the wearable object is worn on the character object of the user B, and the willingness of the user A to continue communication with the user B is enhanced. becomes possible. In addition, the display screen size of the display device 23 of the terminal device 2 is limited, and when a large amount of information is displayed on the display device 23, it becomes difficult for the user to recognize the displayed information. Ta. In the terminal device 2 of user A according to the present embodiment, an evaluation button 1412 is displayed when the character object or user output image of a user other than user A changes. By operating the evaluation button 1412, the user A can input an evaluation of the character object or the user output image of a user other than the user A after the change. As described above, the terminal device 2 of the user A according to the present embodiment can provide the evaluation button 1412 to the user A at the timing of evaluating the character object or the user output image, thereby improving usability. becomes. The information displayed in response to the user A's evaluation instruction is not limited to positive information, and may be negative information. Also, the displayed evaluation buttons 1412 may be of a plurality of types. For example, two types of evaluation buttons may be displayed: an evaluation button for transmitting positive evaluation data and an evaluation button for transmitting negative evaluation data.

Returning to FIG. 8B, instead of the user output image 812a of the user A, or together with the user output image 812a of the user A, a game display object including a game screen showing the progress of the game played by the user A is displayed. may A game display object may be a so-called filter. For example, when the user A operates the input device 24 to select the game button 814 , the input device 24 inputs a game start instruction to the terminal processing device 27 . In selection screen 1200, an object having the same function as game button 814 may be included as an additional candidate object. In this case, when user A operates the input device 24 to select an additional candidate object for inputting a game start instruction, the input device 24 inputs the game start instruction to the terminal processing device 27 .

First, when the display processing unit 274 acquires a game start instruction input from the input device 24, the display processing unit 274 executes various commands included in a control program (game application program, etc.) stored in the terminal storage device 22. A game is started, and a game display object 1500 including a game screen of the started game is displayed on the display device 23 . FIG. 15A is a diagram showing an example of a game display object 1500 including a game screen displayed on the display device 23 of the user A's terminal device 2 in response to a game start instruction from the user A. FIG.

When the game display object 1500 is displayed, the display processing unit 274 displays the game of the game display object 1500 according to various input information from the user A and various commands included in the control program (game application program, etc.). proceed. Then, the display processing unit 274 ends the game of the game display object 1500 when a predetermined game end condition is satisfied.

Further, when the game display object 1500 is displayed, the generation unit 271 generates game image data showing the game screen of the game played by the user A as needed during the progress of the game. Next, the generation unit 271 generates output information including the generated game image data and the user ID stored in the terminal storage device 22 and outputs the generated output information to the transmission unit 272 .

Next, the transmission unit 272 of the terminal device 2 of the user A transmits the output information of the user A together with the destination information (the user IDs of the users B (users B1 to user B3)) to the server device via the terminal communication I/F 21. Send to 3. Next, the server receiving unit 331 of the server device 3 receives the output information of the user A transmitted from the terminal device 2 of the user A via the server communication I/F 31 . Next, the server transmission unit 332 of the server device 3 transmits the output information of the user A to the terminal devices 2 of the users B (users B1 to B3) via the server communication I/F 31 . When the receiving unit 273 of the terminal device 2 of user B receives the output information via the terminal communication I/F 21, the display processing unit 274 of the terminal device 2 of user B receives the user ID included in the output information. Instead of the user output image 812a of user A indicated by , a game display object 1511 including a game image based on the game image data included in the output information is displayed (FIG. 15(b)).

In the example shown in FIG. 15(a), the game on the game screen included in the game display object 1500 is a fishing game. A game screen included in the game display object 1500 includes a game object 1501 , a target object 1502 , a parameter cursor 1503 and a parameter gauge frame 1504 .

The game object 1501 is an object image imitating a fish, and is display-controlled by the display processing unit 274 so as to automatically move along a predetermined path or a randomly determined path within the game screen. Initial values associated with the game object 1501 are stored in the terminal storage device 22, and the display processing unit 274 sets the parameter values associated with the game object 1501 at the start of the game. Set the initial value.

The target objects 1502 are composed of a first target object 1502a, a second target object 1502b and a third target object 1502c. A first target object 1502a is an object image having a substantially circular shape, a second target object 1502b is an object image having a shape surrounding the first target object 1502a, and a third target object 1502c is an object image having a shape surrounding the first target object 1502a. 2 is an object image having a shape surrounding a target object 1502b.

If the input device 24 is a touch panel, the input device 24 inputs a game input instruction to the display processing unit 274 when the user A taps the inside of the game object 1501 . The display processing unit 274 determines that a predetermined portion (for example, a portion corresponding to a fish mouth) of the game object 1501 at the time when the game input instruction is acquired from the input device 24 is the first target object 152a, the second target object 1502b, and the third target object 1502b. Determine if it is located on any of the target objects 1502c.

When the display processing unit 274 determines that the predetermined part of the game object 1501 is positioned at any one of the first target object 1502a, the second target object 1502b, and the third target object 1502c, the display processing unit 274 changes the parameter value of the game object 1501 ( increase or decrease).

The display processing unit 274 changes the amount of change in the parameter value of the game object 1501 according to the position of the game object 1501 at the predetermined position when the game input instruction is received from the input device 24 . For example, if the predetermined location of the game object 1501 at the time of the game input instruction is located within the first target object 1502a, the display processing unit 274 decreases the parameter value of the game object 1501 by the first decrease value. . Note that if the predetermined location of the game object 1501 at the time of the game input instruction is positioned at the first target object 1502a, the display processing unit 274 increases the parameter value of the game object 1501 by the first increase value. good too. Further, when the predetermined location of the game object 1501 at the time of the game input instruction is outside the first target object 1502a and located inside the second target object 1502b, the display processing unit 274 changes the parameter values of the game object 1501 to , by a second decrement value that is less than the first decrement value. Note that if the predetermined location of the game object 1501 at the time of the game input instruction is located outside the first target object 1502a and inside the second target object 1502b, the display processing unit 274 changes the parameter values of the game object 1501 to , may be increased by a second increase value that is less than the first increase value. In addition, when the predetermined location of the game object 1501 at the time of the game input instruction is outside the second target object 1502b and located inside the third target object 1502c, the display processing unit 274 changes the parameter values of the game object 1501 to , by a third decrement value that is less than the second decrement value. Note that the display processing unit 274, which is located outside the second target object 1502b and inside the third target object 1502c at the time when the game input instruction is given, of the game object 1501, displays the parameter values of the game object 1501 as follows: It may be increased by a third increase value that is smaller than the second increase value. Also, if the predetermined location of the game object 1501 at the time of the game input instruction is located outside the target object 1502 , the display processing unit 274 does not change the parameter values of the game object 1501 .

A parameter cursor 1503 is a cursor object that indicates a parameter value of the game object 1501 . The display processing unit 274 places the parameter cursor 1503 at a position corresponding to the parameter value of the game object 1501 within the parameter gauge frame 1504 . In the example shown in FIG. 13A, the parameter cursor 1503 can move left and right within the parameter gauge frame 1504, and the larger the parameter value of the game object 1501, the further rightward it is placed within the parameter gauge frame 1504. be.

When the parameter value of the game object 1501 becomes equal to or less than a predetermined value (for example, "0" (zero)), the display processing unit 274 determines that the user A has successfully cleared the game, and terminates the game. The display processing unit 274 may determine that the user A has successfully cleared the game when the parameter value of the game object 1501 reaches or exceeds a predetermined value (a value greater than the parameter value at the start of the game). The display processing unit 274 may continue the game until the game is successful, and when a predetermined game period (for example, 30 seconds) has passed since the start of the game, the user A has successfully cleared the game. You may end the game even if you do not

When the game of the game screen included in the game display object 1500 is finished, the game display object may be displayed on any terminal device 2 of the user B. FIG. For example, the transmission unit 272 of the terminal device 2 of the user A transmits information indicating the end of the game to the server device 3, and the server device 3 transmits information indicating the end of the game related to the user A to the terminal device 2 of the user B1. do. When the display processing unit 274 of the terminal device 2 of the user B1 acquires the received information regarding the end of the game, the display processing unit 274 executes various commands included in the control program (game application program, etc.) stored in the terminal storage device 22. A game to be implemented is started, and a game display object 1500 including a game screen of the started game is displayed on the display device 23 . When the game played by the user B1 ends, the terminal device 2 of the user B1 similarly transmits information indicating the end of the game to the terminal device 2 of the user B1 via the server device 3, and the terminal device 2 of the user B1 performs display processing. Part 274 displays game display object 1500 . Furthermore, when the game played by the user B2 ends, information indicating the end of the game is similarly transmitted from the terminal device 2 of the user B2 to the terminal device 2 of the user B3 via the server device 3. The display processing section 274 displays the game display object 1500 . Note that the terminal device 2 of the user who is not playing the game displays the screen of FIG. In this way, each user included in the communication group can play the game in turn, and it is possible to increase the motivation of the users to continue using the communication.

In order to execute the game, the terminal device 2 may call a browser function incorporated in the control program and execute the game program. For example, the terminal device 2 receives a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from the server device 3 using the invoked browser function. The terminal device 2 then executes the game program embedded in the received web page. For example, by using HTML5 as the markup language, the terminal device 2 can easily execute a new game. By adopting the above web application in the information system 1, the creator of the game program can provide the client side (terminal device 2 side) with a new game simply by incorporating the new game program into the web page transmitted by the server device 3. can. As a result, the game program creator can greatly reduce the number of man-hours required to create a new game program. Also, on the client side, a new game can be provided simply by receiving a web page without downloading a new game program. As a result, in the information system 1, it is possible to reduce the load on the communication network, the communication cost and/or the communication time compared to downloading the game program, and it is possible to simplify the user interface.

The game input instruction by the user A through the input device 24 is not limited to input according to a tap operation. For example, the input device 24 may input a game input instruction to the display processing unit 274 in response to a swipe operation by the user A in a predetermined direction. Also, the game input instruction is not limited to being input by the input device 24 . For example, the display processing unit 274 acquires face motion data generated as needed by the generation unit 271 over time based on continuously acquired imaging data, and based on the face motion data, the user A performs a predetermined A game input instruction may be generated when it is determined that the action of The predetermined motion is, for example, user A closing his/her eyes, user A's face facing a predetermined direction, or user's hand performing a predetermined motion. When game input instructions are generated based on face motion data, a game display object 1500 including a game screen showing progress of a game being played by user A may be displayed along with user output image 812a of user A. Simultaneous display of the user output image 812a and the game display object 1500 may be a method in which the user output image 812a and the game display object 1500 are displayed side by side. The displayed method may be used. For example, user output image 812 a may be displayed as a background image for game display object 1500 . Thereby, the user can play the game while confirming the action of the user's character object, and it is possible to improve the operability of the game. Further, when the user A shakes the terminal device 2 having an acceleration sensor (not shown) and the acceleration sensor measures an acceleration equal to or greater than a predetermined value, the display processing unit 274 issues a game input instruction. may be generated.

The game on the game screen included in the game display object 1500 may be a game other than the fishing game. For example, the game may include a character object of user A that moves a predetermined distance each time user A closes his eyes, or a character object of user B that moves a predetermined distance each time user A closes his eyes. It can be a game to play.

Further, when the user B can input a predetermined game input instruction to the game image in the game display object 1511, the game image in the game display object 1511 can be changed by a game input instruction according to one of the user B's operations. The game shown may proceed. Note that when the user B can input a predetermined game input instruction to the game image in the game display object 1511, the input device 24 of the terminal device 2 of the user A is controlled so that the user A cannot input the game input instruction. may Further, when the user B can input a predetermined game input instruction to the game image in the game display object 1511, the game input instruction by the user A and the game input instruction by all the users B cause the game image in the game display object 1511 to The game shown in the game image may progress. In the screen of FIG. 15(b), not only the game display object 1511 played by the user A but also the game display objects 1511 played by each of the users B1, B2 and B3 may be displayed. In this case, for each user, four game display objects 1511 are displayed on the screen of FIG. 15(b), and each user plays the game of the game display object 1511 corresponding to each user.

Thus, user A can provide a game screen while communicating with user B. This makes it possible for user A and user B to increase their willingness to continue communication.

(Operation sequence of information system 1)
FIG. 16 is a diagram showing an example of an operation sequence of the information system 1. As shown in FIG. This operation sequence is executed mainly by the terminal processing device 27 and the server processing device 33 based on the control programs stored in the terminal storage device 22 and the server storage device 32 in advance. performed cooperatively. Hereinafter, it is assumed that the user A operates the terminal device 2a, the user B1 operates the terminal device 2b1, and the user B2 operates the terminal device 2b2. Terminal devices of users other than user A are not limited to the terminal devices 2b1 and 2b2 of users B1 and B2, and terminal devices 2 of three or more users may be connected to the server device 3. FIG.

First, the transmission unit 272 of the terminal device 2a outputs the character moving image data including the motion data of the user A generated by the generation unit 271 based on the imaging data continuously output by the imaging device 25, and the The output information including the obtained voice data of user A and the user ID of user A is transmitted to the server device 3 via the terminal communication I/F 21 (step S101). The output information may include game image data showing a game screen of a game played by the user and user A's user ID. Also, the transmission destination information may be included in the transmission processing of the transmission unit 272 . Since the process of step S101 is continuously executed at predetermined time intervals (for example, every two seconds) until the distribution event ends, the processes of steps S101 to S110 are intermittently executed.

Next, the server transmission unit 332 of the server device 3 transmits the output information received from the terminal device 2a to the terminal device 2b1 via the server communication I/F 31 (step S102) and also to the terminal device 2b2. (Step S103).

In addition, the transmission unit 272 of the terminal device 2b1 transmits output information including the character moving image data of the user B1, the voice data of the user B1, and the user ID of the user B1 to the server device 3 via the terminal communication I/F 21. (Step S104). Next, the server transmission unit 332 of the server device 3 transmits the output information received from the terminal device 2b1 to the terminal device 2a via the server communication I/F 31 (step S105) and also to the terminal device 2b2. .

In addition, the transmission unit 272 of the terminal device 2b2 transmits output information including the character moving image data of the user B2, the voice data of the user B2, and the user ID of the user B2 to the server device 3 via the terminal communication I/F 21. (Step S106). Next, the server transmission unit 332 of the server device 3 transmits the output information received from the terminal device 2b2 to the terminal device 2a via the server communication I/F 31 (step S107) and also to the terminal device 2b1. .

The display processing unit 274 of the terminal device 2a generates a user output image including the character object of the user A, a user output image including the character object of the user B1, and A communication screen including a user output image including the character object of user B2 is displayed on the display device 23 of the terminal device 2a, and voices of users B1 and B2 are output (step S108).

Similar to step S108, the display processing unit 274 of the terminal device 2b1 displays the user output image including the character object of the user A and the character object of the user B1 based on the output information of each of the user A, the user B1, and the user B2. and the user output image including the character object of the user B2 are displayed on the display device 23 of the terminal device 2b1, and the voices of the user A and the user B2 are output (step S109).

Similar to step S108, the display processing unit 274 of the terminal device 2b2 displays the user output image including the character object of the user A and the character object of the user B1 based on the output information of each of the user A, the user B1, and the user B2. and the user output image including the character object of the user B2 are displayed on the display device 23 of the terminal device 2b2, and the voices of the user A and the user B1 are output (step S110).

Next, the transmission unit 272 of the terminal device 2a transmits information indicating any one of the change instruction, the addition instruction, and the evaluation instruction input according to the operation of the input device 24 by the user A, via the terminal communication I/F 21. is sent to the server device 3 (step S111).

Next, the server transmission unit 332 of the server device 3 transmits information indicating any one of the change instruction, the addition instruction, and the evaluation instruction received from the terminal device 2a to the terminal device 2b1 via the server communication I/F 31. (Step S112) and transmits to the terminal device 2b2 (Step S113).

The display processing unit 274 of the terminal device 2a displays the user output image of the user A, the user B1 At least a part of the user output image and the user output image of the user B2 is changed and displayed on the display device 23 of the terminal device 2a (step S114).

The display processing unit 274 of the terminal device 2b1 displays the user output image of the user A, the user At least a part of the output image and the user output image of the user B2 is changed and displayed on the display device 23 of the terminal device 2b1 (step S115).

The display processing unit 274 of the terminal device 2b2 displays the user output image of the user A, the user At least a part of the output image and the user output image of the user B2 is changed and displayed on the display device 23 of the terminal device 2b2 (step S116).

Next, the transmission unit 272 of the terminal device 2b1 transmits information indicating an instruction to add a wearable object, which is input according to the operation of the input device 24 by the user B1, to the server device 3 via the terminal communication I/F 21. (step S117).

Next, the server transmission unit 332 of the server device 3 transmits the information indicating the mounting object addition instruction received from the terminal device 2b1 to the terminal device 2a via the server communication I/F 31 (step S118).

The display processing unit 274 of the terminal device 2a performs a predetermined operation related to acceptance by the user A with respect to the additional object displayed in accordance with the information indicating the instruction to add the mountable object transmitted from the terminal device 2a of the user B1. It is determined whether or not (step S119).

If it is determined that the user A has not performed the predetermined operation regarding acceptance (step S119-No), step S123 is not executed. If it is determined that the user A has performed a predetermined operation related to consent (step S119-Yes), the transmission unit 272 of the terminal device 2a transmits information indicating the consent instruction to the server device 3 via the terminal communication I/F 21. Send (step S120).

Next, the server transmission unit 332 of the server device 3 transmits the information indicating the approval instruction received from the terminal device 2a to the terminal device 2b1 via the server communication I/F 31 (step S121), and the terminal device 2b2 (step S122). Along with the information indicating the approval instruction, an object ID indicating the mounting object related to the information indicating the additional instruction transmitted from the terminal device 2a of the user B1 in step S117 is also transmitted.

The display processing unit 274 of the terminal device 2a displays the user output image including the character object of the user A wearing the wearable object related to the information indicating the additional instruction transmitted from the terminal device 2a of the user B1 in step S117. 2a is displayed on the display device 23 (step S123).

The display processing unit 274 of the terminal device 2b1 displays the user output image including the character object of the user A wearing the wearable object related to the information indicating the additional instruction transmitted from the terminal device 2a of the user B1 in step S117. 2b1 is displayed on the display device 23 (step S124).

The display processing unit 274 of the terminal device 2b2 displays the user output image including the character object of the user A wearing the wearable object related to the information indicating the additional instruction transmitted from the terminal device 2a of the user B1 in step S117. 2b2 is displayed on the display device 23 (step S125).

As described in detail above, in the information system 1 of the present embodiment, in accordance with an instruction from the user A, the user output image of the user A whose at least part has been changed and/or at least A user output image of one user B is displayed. In this way, the information system 1 of the present embodiment allows the output from the terminal device 2 to be changed by the user A or at least one user B during the communication service, thereby enabling the user's continuous use of communication. It is possible to improve the motivation of

(Modification 1)
In addition, the present invention is not limited to this embodiment. For example, when a predetermined condition is satisfied, the display processing unit 274 of the terminal device 2 of user A adds a new addition candidate object to the display area of the selection screen 1200 (FIG. 12(a)). good. Further, when a predetermined condition is satisfied, the display processing unit 274 of the terminal device 2 of user A may delete at least part of the addition candidate objects included in the display area of the selection screen 1200 .

The predetermined condition is that the time during which the user A's character object or the user output image is displayed on the communication screen exceeds a predetermined time. Alternatively, the predetermined condition may be that the time during which the user A's character object or the user output image is displayed on the communication screen does not exceed a predetermined time. Further, the predetermined condition may be a condition that the time during which any character object or user output image of user B is displayed on the communication screen exceeds a predetermined time or a condition that the predetermined time has not been exceeded. Further, the predetermined condition is that the total time during which the character objects or user output images of user A and all users B are displayed on the communication screen exceeds a predetermined time or does not exceed a predetermined time. It's okay. In addition, the predetermined condition is that the total time during which the character objects or user output images of at least two of the user A and all the users B are displayed on the communication screen exceeds a predetermined time, or that the predetermined time is exceeded. It is acceptable if the condition is that the

Further, the predetermined condition may be a condition that the number of times any of the addition candidate objects is selected by user A's operation of the input device 24 exceeds a predetermined number of times. Further, the predetermined condition may be a condition that the number of times any of the addition candidate objects is selected by user A's operation of the input device 24 does not exceed a predetermined number of times. Further, the predetermined condition is that the number of times any of the addition candidate objects has been selected by any user B operating the input device 24 exceeds a predetermined number of times or does not exceed a predetermined number of times. It's okay. Further, the predetermined condition is that the total number of times that any of the additional candidate objects has been selected by the operation of the input device 24 by each of the user A and all the users B exceeds a predetermined number of times, or the predetermined number of times. It is also possible to provide a condition that the In addition, the predetermined condition is that the total number of times that any of the additional candidate objects has been selected by at least two of the users A and all the users B by operating the input device 24 exceeds a predetermined number of times. Alternatively, the condition may be that the predetermined number of times is not exceeded.

New addition candidate objects added to the display area of the selection screen 1200 when a predetermined condition is satisfied are, for example, a specific wallpaper object placed behind the character object in the user output image, and a specific It may correspond to a wearable object or the like.

In this way, when a predetermined condition is satisfied, the range of selection of additional objects is widened (or narrowed), so that it is possible to diversify the output according to the operation of the user A during the communication service. . Therefore, it is possible to increase the willingness of user A to continue using communication with user B.

(Modification 2)
As shown in FIG. 14B, when a new wearable object 1411 is worn on the character object 811b2 of user B2, user A operates an evaluation button 1412 included in the newly displayed communication screen 1410. can. The user A's terminal device 2 may be configured so that the user A can perform an operation other than the operation of the evaluation button 1412 instead of the evaluation button 1412 or together with the evaluation button 1412 .

For example, when a new wearable object 1411 is worn on the character object 811b2 of the user B2, an action button for causing the character object 811a of the user A to automatically perform a predetermined action is displayed on the communication screen 1410 for a predetermined period of time. (for example, 10 seconds) may be included. When an action button is selected by user A's operation of the input device 24, the generation unit 271 suspends the use of the motion data of user A for a predetermined action period (for example, 5 seconds), and stores the motion data in advance in the terminal storage device 22. Information about the predetermined action is extracted, and the character object 611a of the user A is caused to automatically perform the predetermined action for a predetermined action period (for example, 5 seconds). The predetermined action is, for example, a clapping action, a surprise action, or the like. Note that when an action button is selected by user A's operation of the input device 24, the generation unit 271 does not have to stop using part of the motion data of user A. FIG. For example, if the action corresponding to the action button is the action related to the arm of the character object 611a, the face motion data may continue to be used. As a result, the information system 1 can make the character object perform rich expressions in addition to the uniform movement of the action buttons, and can increase the willingness to continue using communication with the user. Become.

Further, each of a plurality of predetermined operations by the user A may be associated with information regarding predetermined actions to be automatically performed by the character object 811a of the user A and stored in the terminal storage device 22 . For example, in association with the upward swipe operation, the terminal storage device 22 stores information related to the motion of making a peace sign, and in association with the downward swipe operation, information related to the jumping motion is stored in the terminal storage device 22. may For example, when the input device 24 detects that the user A has performed an upward swipe operation, the generation unit 271 stops using the motion data of the user A for a predetermined operation period (for example, 5 seconds). Then, the information on the motion of making the peace sign stored in advance in the terminal storage device 22 is extracted, and the character object 811a of the user A is made to automatically perform the motion of making the peace sign for a predetermined motion period (for example, 5 seconds). In this case, the information about the predetermined action associated with the predetermined operation may be changed for a predetermined period (for example, 10 seconds) after the wearable object 1411 is newly mounted on the character object 811b2 of the user B2. For example, for a predetermined period of time (for example, 10 seconds) after the user B2's character object 811b2 is newly equipped with the wearable object 1411, the user B2 associates the upward swipe operation with (changes information about the motion of making a peace sign). ) Information about the action of clapping may be stored in the terminal storage device 22 . As a result, the input device 24 detects that the upward swipe operation has been performed by the user A within a predetermined period (for example, 10 seconds) after the user B2's character object 811b2 is newly equipped with the wearable object 1411. Then, the generating unit 271 stops using the motion data of the user A for a predetermined motion period (for example, 5 seconds), extracts the information about the clapping motion stored in the terminal storage device 22 in advance, and The character object 811a of A is automatically made to perform an action of clapping for a predetermined action period (for example, 5 seconds). In this way, even if the same operation is performed, the character object can be made to perform different actions during normal communication and when another user's character object is equipped with a new wearable object. It is possible to give diversity to the output according to the operation of A. Therefore, it is possible to increase the willingness of user A to continue using communication with user B. Further, the information system 1 causes the character object to perform a predetermined action associated with a predetermined operation, thereby reducing the processing load of the terminal device 2 compared to the case of using motion data. 2 and the server device 3 can be prevented from increasing.

(Modification 3)
The display processing unit 274 may include a selection button on the communication screen for selecting an additional object to be added to the communication screen. For example, when the user A operates the input device 24 to select the select button, the display processing unit 274 selects one object from a group including a plurality of additional objects stored in the terminal storage device 22 based on a predetermined selection method. Or select multiple additional objects. The predetermined selection method includes a method of randomly drawing lots from a group including a plurality of additional objects, a method of selecting from a group including a plurality of additional objects in a predetermined order, and the like. Then, the display processing unit 274 and the transmission unit 272 execute addition processing using the object ID of the selected additional object and the addition instruction. Thereby, the user A's character object can change the user output information based on the selected additional object.

In this case, an equip button may be included in the communication screen of the user B of the character object wearing the equipable object related to the selected additional object. A mounted object related to the additional object is, for example, a mounted object having the same attributes as the additional object or a mounted object having the same color as the additional object. In this case, in the object table T2, information indicating the attribute and/or information indicating the color of each object is stored in association with each object ID. Also, in the user table T1, information indicating the attribute and/or information indicating the color of each user's character object is stored in association with the user ID of each user. An attribute is, for example, the type of object (such as male or female). Also, the attribute may be the type of lottery game from which the object can be obtained. When the wear button is selected by user B's operation of the input device 24, the display processing unit 274 identifies the object ID of the selected additional object, and the display processing unit 274 and the transmission unit 272 transmit the selected additional object. Addition processing is executed using the object ID and the addition instruction. As a result, the wearable object related to the additional object included in the user output image of the user A can be worn on the character object of the user B, and the motivation of the user A to continue using communication with the user B is increased. becomes possible. It should be noted that the installation button may be included in the communication screen of the user B in the user output image to which the wallpaper object related to the selected additional object is applied.

(Modification 4)
If the user A does not perform a predetermined operation on the communication screen 1310 on which the user output image 812a including the additional object 1311 is displayed, the additional object 1311 continues to be displayed on the communication screen 1310 of the user A's terminal device 2. good. In this case, the additional object 1311 may be changed to a mark image. As a result, the user A can change the user character by using the wearable object or the like corresponding to the additional object 1311 at his or her desired timing.

Further, when the number of additional objects 1311 for which a predetermined operation is not performed by user A exceeds a predetermined number, each terminal device 2 of another user B (users B1 to B3), via the server device 3, Information regarding the additional object 1311 for which the user A does not perform a predetermined operation may be transmitted. In this case, the additional object 1311 for which the user A does not perform a predetermined operation is displayed on the terminal device 2 of each user B (users B1 to B3). Then, when the user B performs a predetermined operation on the additional object 1311, the input device 24 of the terminal device 2 of the user B is replaced by the input device 24 of the terminal device 2 of the user A, and the approval instruction is transmitted to the additional object 1311. It is input to the terminal processing device 27 together with the object ID. Thereafter, consent processing is executed. As a result, even if the user A does not wear the wearable object corresponding to the additional object presented by the other user B, another user different from the user A can wear the wearable object corresponding to the additional object. It becomes possible.

(Modification 5)
When a character object is generated based on 3D model data for generating a 3D animation, the user output image of each user is a solid object (3 It may be an image of a character object (a dimensional object) viewed from a predetermined viewpoint in the virtual space. For example, the display processing unit 274 arranges a two-dimensional screen in the line-of-sight direction of a predetermined viewpoint, projects three-dimensional coordinates of various three-dimensional objects arranged in the virtual space onto the two-dimensional screen, and displays various objects on the two-dimensional screen. The image of the projected two-dimensional screen surface is displayed as a user output image. Note that, for example, the line-of-sight direction of the predetermined viewpoint for displaying the user output image of the user A displayed on the terminal device 2 of the user A is the direction in which the character object of the user A faces. Then, the two-dimensional screen for displaying the user output image of the user A displayed on the terminal device 2 of the user A is controlled to move and rotate in accordance with the predetermined movement of the viewpoint and rotation of the line-of-sight direction. be.

For example, when the character object 811a of the user A wears a new wearable object, the communication screen 1110 shown in FIG. 11B is displayed. 274 may automatically move a given viewpoint.

For example, when User A's character object 811a wears a new wearable object, the display processing unit 274 may move a predetermined viewpoint closer to User A's character object 811a. For example, the display processing unit 274 calculates a route connecting a position of a predetermined viewpoint when the character object 811a of the user A wears a new wearable object and the position of the new wearable object, and automatically displays a display along the route. to move. The display processing unit 274 displays, as a user output image, an image showing the virtual space viewed from a predetermined viewpoint that automatically moves in the virtual space.

FIG. 17A is a diagram showing an example of a communication screen 1700 in which a user output image 1012a is an image showing a virtual space viewed from a predetermined viewpoint that automatically moves in the virtual space. In the example shown in FIG. 17A, since the hairstyle of the character object 811a of the user A has been changed to a new wearable object, a user output image 1012a including the enlarged head of the character object 1011a is displayed.

For example, when the character object 811a of the user A wears a new wearable object, the display processing unit 274 sets a path around the character object 1011a after wearing. Then, the display processing unit 274 automatically moves the predetermined viewpoint along the route while aligning the line-of-sight direction of the predetermined viewpoint with the direction facing the character object 1011a. The display processing unit 274 displays, as a user output image, an image showing the virtual space viewed from a predetermined viewpoint that automatically moves in the virtual space.

FIG. 17B is a diagram showing another example of the communication screen 1710 in which the user output image 1012a is an image showing the virtual space viewed from a predetermined viewpoint that automatically moves in the virtual space. In the example shown in FIG. 17A, the user output image 1012a is displayed when the user A is moving around the character object 1011a, and the user output image 1012a including the back of the character object 1011a is displayed.

Further, the display processing unit 274 may control the movement of a predetermined viewpoint and/or the rotation of the line-of-sight direction of the predetermined viewpoint in accordance with a predetermined operation of the input device 24 by the user A on the communication screen. Further, the display processing unit 274 may include an automatic movement button for automatically moving a predetermined viewpoint in the communication screen. In this case, for example, when the user A operates the input device 24 to select the automatic movement button, the display processing unit 274 may automatically move a predetermined viewpoint according to a predetermined movement rule. .

(Modification 6)
When a character object is generated based on 3D model data for generating a 3D animation, the user output image of each user is a solid object (3D model) arranged in a virtual space common to each user. It may be an image of the inside of the virtual space viewed from the user's point of view placed at a predetermined position (for example, the position of the eyes of the character object) of the character object, which is an object). Note that the user's viewpoint may be moved from a predetermined position of the character object according to an operation by the user and/or automatically, and may be positioned behind, above, or in front of the character object.

An example of how to use a communication service in a virtual space common to each user will be described. For example, as shown in FIG. 18, the display processing unit 274 causes a device object S such as a virtual tablet PC to appear within the field of view of the user A's character object C from the user's viewpoint in the virtual space. Various screens described with reference to FIGS. In this case, the user output image of the user A displayed on the display device portion of the device object S is a two-dimensional image of the inside of the virtual space viewed from a virtual camera provided at a predetermined position of the device object S. FIG. User A's voice data is voice data acquired by the microphone 26 of the terminal device 2 .

The transmission unit 272 of the terminal device 2 transmits the user output image of the user A, the voice data of the user A, and the user ID of the user A to all the terminal devices 2 of the user B via the server device 3. . Note that, instead of the user output image of user A, the transmission unit 272 transmits the position and line-of-sight direction of the virtual camera in the virtual space, the position and orientation of the character object of user A in the virtual space, and the direction of the body of user A. Motion data may be transmitted. Further, the terminal device 2 of the user A receives the user output image of each user B, the voice data of each user B, and the user ID of each user B from the terminal device 2 of each user B via the server device 3. to receive Then, the display processing unit 274 of the terminal device 2 of the user A displays the user output image of each user B on the display device portion of the device object S, and outputs the voice data of each user B from the terminal device 2 . As a result, a communication service using the virtual device object S can be realized.

(Modification 7)
At least part of the functions implemented by the terminal processing device 27 of the terminal device 2 may be implemented by a processor other than the terminal processing device 27 of the terminal device 2 . At least part of the functions implemented by the terminal processing device 27 may be implemented by the server processing device 33 of the server device 3, for example. Specifically, part of the functions of the generation unit 271 and the display processing unit 274 may be executed in the server device 3 . For example, the terminal device 2 transmits the continuously acquired imaging data and audio data to the server device 3 together with the user ID of the user who operates the terminal device 2 . Then, the server device 3 uses the functions of the generation unit 271 and the display processing unit 274 to generate character moving image data, and display information for displaying a communication screen including user output images of all users on the display device 23. to generate The server device 3 then transmits the display information together with the audio information (audio data) to the terminal devices 2 of all users. Each terminal device 2 outputs each information received from the server device 3 (so-called "server rendering").

(Modification 8)
The terminal device 2 has a function of displaying a communication screen or the like by executing various commands included in a control program stored in the terminal device 2. Calls a browser function embedded in an application, uses this browser function to receive a web page (a document written in a markup language such as HTML (Hyper Text Markup Language)) from a server device, and embeds it in this web page may execute a control program (so-called "browser rendering"). For example, by using HTML 5 as the markup language, the terminal device 2 can easily execute new information processing. By adopting such web applications to realize communication services in terminal devices, program creators can implement new information processing on the client side (terminals) simply by embedding new programs in web pages transmitted by server devices. device side), and the number of man-hours for creating a new program can be greatly reduced. In addition, on the client side, new services can be provided simply by receiving a web page without downloading a new control program. Time can be reduced and simplification of the user's interface can be realized.

(Modification 9)
The generation unit 271 of the terminal device 2 of the user A may generate output information including face motion data without using imaging data. For example, the terminal storage device 22 of the terminal device 2 stores face motion data corresponding to voice data. to generate output information including the extracted face motion data, voice data, and user A's user ID. Further, the generation unit 271 of the terminal device 2 of the user A generates output information including the voice data of the user A output from the microphone 26 and the user ID stored in the terminal storage device 22 without including the face motion data. may be generated. In this case, the display processing unit 274 of the terminal device 2 of the user B extracts the face motion data corresponding to the voice data of the user A included in the output information of the user A received via the server device 3, and extracts the face motion data of the user A. An animation of the character object may be generated. Note that the process of generating face motion data corresponding to the user's voice data may be a process using a known lip-sync algorithm.

In this way, in the terminal device 2 of user A using the HMD as the display device 23, it is possible to communicate with user B through the character object without acquiring user A's imaging data.

1 information system 2 (2a, 2b1, 2b2) terminal device 21 terminal communication I/F
22 terminal storage device 23 display device 24 input device 25 imaging device 26 microphone 27 terminal processing device 271 generation unit 272 transmission unit 273 reception unit 274 display processing unit 275 audio output unit 3 server device 31 server communication I/F
32 Server storage device 33 Server processing device 331 Server receiving unit 332 Server transmitting unit 4 Base station 5 Mobile communication network 6 Gateway 7 Internet

Claims (15)

A control program for a terminal device of a first user,
to the terminal device of the first user,
displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
transmitting information regarding the display of the first image to the terminal device of the second user;
when acquiring the first voice of the first user, transmitting the first voice to the terminal device of the second user;
displaying the first image including the first object that changes according to the first user's action or the first sound;
when receiving the second voice of the second user, outputting the second voice;
displaying the second image including the second object that changes according to the second user's action or the second sound;
displaying the at least partially changed first image or the at least partially changed second image in response to an instruction by the first user or an instruction by the second user;
and
In the display of the first image at least partially changed, the terminal device of the first user,
including a particular second candidate object in a selection group comprising a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is met; or removing some of the second candidate objects from the selection group;
displaying the first image including a third object corresponding to the second candidate object selected from the selection group in accordance with an instruction by the second user;
A control program for executing In the display of the first image at least partially changed, the terminal device of the first user,
said partially changed based on said first candidate object selected from among one or more first candidate objects associated with said first user according to an instruction by said first user displaying the first image including a first object;
The control program according to claim 1, causing the execution of the first object is a three-dimensional object arranged in a virtual space;
In displaying the first image including the first object, the terminal device of the first user:
displaying, as the first image, an image showing the virtual space seen from a predetermined viewpoint in the virtual space;
and
to the terminal device of the first user,
displaying, as the first image, an image showing the virtual space viewed from the predetermined viewpoint that automatically moves in the virtual space;
3. The control program according to claim 2, causing the execution of 4. The control program according to claim 3, wherein movement of said predetermined viewpoint is controlled so as to approach the changed part of said first object. 4. The control program according to claim 3, wherein movement of said predetermined viewpoint is controlled so as to move around said first object while aligning a line-of-sight direction from said predetermined viewpoint with a direction facing said first object. to the terminal device of the first user,
Based on the second candidate object selected from the selection group according to the instruction by the first user, the third object including the third object corresponding to the selected second candidate object is selected. Displaying two images and transmitting information regarding display of the third object to the terminal device of the second user;
and
6. The control program according to any one of claims 1 to 5 , wherein said predetermined condition includes a condition regarding the number of times said second candidate object is selected by said first user and/or said second user. In the display of the first image and the second image, on the terminal device of the first user,
displaying a plurality of said second images respectively showing a plurality of said second users;
and
to the terminal device of the first user,
The first image and the plurality of the first image and the plurality of the first image, including the third object corresponding to the second candidate object selected based on the second candidate object selected in accordance with an instruction by the second user. displaying a plurality of said second images representing each of a second user;
The control program according to any one of claims 1 to 5 , causing the execution of The first user can instruct the selection of the third object,
to the terminal device of the first user,
displaying the first image including the partially changed first object in response to an instruction by the first user to select the third object;
The control program according to any one of claims 1 to 5 , causing the execution of In the display of the first image at least partially changed, the terminal device of the first user,
displaying an image showing the progress of the game played by the first user and/or the second user in place of at least part of the first image;
The control program according to any one of claims 1 to 5 , causing the execution of to the terminal device of the first user,
When the second candidate object selected according to the instruction by the second user corresponds to a game, an image showing the progress of the game played by the first user and/or the second user is displayed. , displaying in place of at least part of the second image;
The control program according to any one of claims 1 to 5 , causing the execution of A control program for a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user,
Information on display of a first image including a first object representing the first user is received from a terminal device of the first user, and information on display of a second image including a second object representing the second user is received. , received from the terminal device of the second user,
transmitting information for at least displaying the first image to at least the terminal device of the second user, and transmitting information for at least displaying the second image to at least the terminal device of the first user; when receiving the first voice of the first user, transmitting the first voice to the terminal device of the second user;
transmitting information for displaying the first image including the first object that changes according to the first user's action or the first voice to at least the second user's terminal device;
when receiving the second voice of the second user, transmitting the second voice to the terminal device of the first user;
transmitting, to at least the first user's terminal device, information for displaying the second image including the second object that changes according to the second user's motion or the second voice;
transmitting, to at least the second user's terminal device, information for displaying the first image, at least a part of which has been changed, in accordance with an instruction from the first user;
transmitting, to at least the terminal device of the first user, information for displaying the second image, at least a part of which has been changed, in accordance with an instruction from the second user;
and
a specific second candidate object is included in a selection group including a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is satisfied; or removing some of the second candidate objects from the selection group;
In displaying the first image at least partially changed, the server device:
information for displaying the first image including the third object corresponding to the second candidate object selected from the selection group in accordance with the instruction by the second user, at least the terminal of the second user; transmitting to the device;
A control program characterized by executing A terminal device of the first user,
displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
transmitting information regarding the display of the first image to the terminal device of the second user;
when acquiring the first voice of the first user, transmitting the first voice to the terminal device of the second user;
displaying the first image including the first object that changes according to the first user's action or the first sound;
when receiving the second voice of the second user, outputting the second voice;
displaying the second image including the second object that changes according to the second user's action or the second sound;
a processing unit that displays the first image at least partially changed or the second image at least partially changed in response to an instruction by the first user or an instruction by the second user ;
In the display of the first image in which the processing unit is at least partially changed,
including a particular second candidate object in a selection group comprising a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is met; or removing some of the second candidate objects from the selection group;
displaying the first image including a third object corresponding to the second candidate object selected from the selection group in accordance with an instruction by the second user;
A terminal device characterized by: A server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user,
Information on display of a first image including a first object representing the first user is received from a terminal device of the first user, and information on display of a second image including a second object representing the second user is received. , received from the terminal device of the second user,
transmitting information for at least displaying the first image to at least the terminal device of the second user, transmitting information for displaying at least the second image to at least the terminal device of the first user;
when receiving the first voice of the first user, transmitting the first voice to the terminal device of the second user;
transmitting information for displaying the first image including the first object that changes according to the first user's action or the first voice to at least the second user's terminal device;
when receiving the second voice of the second user, transmitting the second voice to the terminal device of the first user;
transmitting, to at least the first user's terminal device, information for displaying the second image including the second object that changes according to the second user's motion or the second voice;
transmitting, to at least the second user's terminal device, information for displaying the first image, at least a part of which has been changed, in accordance with an instruction from the first user;
a processing unit that transmits, to at least the first user's terminal device, information for displaying the second image, at least a part of which has been changed, in accordance with an instruction from the second user ;
a specific second candidate object is included in a selection group including a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is satisfied; or removing some of the second candidate objects from the selection group;
In the display of the first image in which the processing unit is at least partially changed,
information for displaying the first image including the third object corresponding to the second candidate object selected from the selection group in accordance with the instruction by the second user, at least the terminal of the second user; send to the device,
A server device characterized by: A method for controlling a terminal device of a first user,
The terminal device of the first user,
displaying at least a first image including a first object representing the first user and a second image including a second object representing a second user different from the first user;
transmitting information regarding the display of the first image to the terminal device of the second user;
when acquiring the first voice of the first user, transmitting the first voice to the terminal device of the second user;
displaying the first image including the first object that changes according to the first user's action or the first sound;
when receiving the second voice of the second user, outputting the second voice;
displaying the second image including the second object that changes according to the second user's action or the second sound;
displaying the at least partially changed first image or the at least partially changed second image in response to an instruction by the first user or an instruction by the second user;
including
In displaying the first image at least partially changed, the terminal device of the first user,
including a particular second candidate object in a selection group comprising a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is met; or removing some of the second candidate objects from the selection group;
displaying the first image including a third object corresponding to the second candidate object selected from the selection group in accordance with an instruction by the second user;
A control method, further comprising : A control method for a server device capable of communicating with each of a terminal device of a first user and a terminal device of a second user different from the first user,
Information on display of a first image including a first object representing the first user is received from a terminal device of the first user, and information on display of a second image including a second object representing the second user is received. , received from the terminal device of the second user,
transmitting information for at least displaying the first image to at least the terminal device of the second user, transmitting information for displaying at least the second image to at least the terminal device of the first user;
when receiving the first voice of the first user, transmitting the first voice to the terminal device of the second user;
transmitting information for displaying the first image including the first object that changes according to the first user's action or the first voice to at least the second user's terminal device;
when receiving the second voice of the second user, transmitting the second voice to the terminal device of the first user;
transmitting, to at least the first user's terminal device, information for displaying the second image including the second object that changes according to the second user's motion or the second voice;
transmitting, to at least the second user's terminal device, information for displaying the first image, at least a part of which has been changed, in accordance with an instruction from the first user;
transmitting, to at least the terminal device of the first user, information for displaying the second image, at least a part of which has been changed, in accordance with an instruction from the second user;
including
a specific second candidate object is included in a selection group including a plurality of second candidate objects if a predetermined condition including a condition relating to the display time of the first image and/or the display time of the second image is satisfied; or removing some of the second candidate objects from the selection group;
In displaying the first image, at least a portion of which has been changed, the server device
information for displaying the first image including the third object corresponding to the second candidate object selected from the selection group according to the instruction by the second user, at least the terminal of the second user; transmitting to the device;
A control method comprising :

Images