JP2023090814A - Shared information processing device and control program - Google Patents

Abstract

To prevent unexpected workload from increasing in a terminal for displaying elements, concerning a system where the elements are shared among a plurality of users viewing each of common subjects to be viewed.SOLUTION: A shared information processing device (e.g., a server 4) representing one embodiment of the present invention comprises: identifying means (e.g., a replacement processing unit 67) for, among shared elements in which timings to be displayed in a user terminal (e.g., a mobile terminal 3) viewing subjects to be viewed are associated with the subjects to be viewed, identifying the shared elements in a prescribed count as an element group based on the timings; and replacement means (e.g., a replacement processing unit 67) for replacing the element group identified by the identifying means with substitute elements fewer than the prescribed count.SELECTED DRAWING: Figure 9

Description

The present invention relates to a mechanism for sharing elements among multiple users each viewing a common viewing target.

2. Description of the Related Art A communication system is known in which comments input according to reproduced content are shared among a plurality of users. For example, in the communication system described in Patent Literature 1, a user terminal that acquires a comment to be displayed together with content from a comment server before starting playback of the content is displayed in each of a plurality of time segments into which the timeline is divided. Old comments are excluded from display targets so that the number of comments is less than or equal to a predetermined number.

JP 2011-119847 A

In the communication system described in Patent Literature 1, a process of equalizing the number of comments displayed with content over time is executed before starting the reproduction of the content. Therefore, it is not possible to deal with a sudden increase in load caused by comments added during content playback.

The problem to be solved by the present invention is to suppress a sudden increase in load on a terminal that displays an element in a mechanism in which an element is shared among a plurality of users each viewing a common viewing target.

In order to solve the above problems, the present invention selects a predetermined number of the shared elements based on the timing from among the shared elements whose timing to display on the terminal of the user who is viewing the viewing target is associated with the viewing target. A group of elements is specified, and the group of elements is replaced with a number of alternative elements less than the predetermined number.

A shared information processing apparatus, which is one embodiment of the present invention, provides a predetermined number of shared elements whose timing to be displayed on a terminal of a user who is viewing a viewing target is associated with the viewing target based on the timing. It comprises specifying means for specifying a shared element group, and replacing means for replacing the element group specified by the specifying means with alternative elements less than the predetermined number.

ADVANTAGE OF THE INVENTION According to this invention, the sudden load increase in the terminal which displays a shared element can be suppressed.

1 is a configuration diagram of a video distribution system; FIG. (Example) 3 is a block diagram showing the electrical configuration of the server; FIG. (Example) 2 is a block diagram showing the electrical configuration of the mobile terminal; FIG. (Example) It is a figure which shows the display mode of the screen of a portable terminal. (Example) It is a figure which shows the screen which displays the onomatopoeia image. (Example) It is a figure which shows the screen which displays the 2nd image. (Example) 4 is a timing chart showing a case where a combined image is displayed; (Example) 4 is a timing chart showing a case where the second image is replaced; (Example) FIG. 3 is a functional block diagram related to moving image distribution of a server; (Example) It is a functional block diagram about the onomatopoeia display of a portable terminal. (Example) It is a flowchart which shows the flow of onomatopoeia display processing performed with a portable terminal. (Example) It is a flowchart which shows the flow of onomatopoeia display processing performed by a server. (Example) It is a flowchart which shows the flow of onomatopoeia display processing performed by a server. (Example) 4 is a timing chart showing a case where a combined image is displayed; (Modification 2)

[1. embodiment]
[1-1. Main features]
This embodiment relates to a mechanism for sharing an element among a plurality of users each viewing a common viewing target. In this embodiment, in order to suppress a sudden increase in the load on the terminal that displays the element to be shared, the timing of displaying the viewing target on the terminal of the user who is viewing the viewing target is set among the shared elements associated with the viewing target. A predetermined number of the shared elements are specified as an element group based on the timing, and the element group is replaced with a substitute element whose number is less than the predetermined number.

A “audiovisual object” is an audiovisual object that a user can perceive visually and/or aurally. An audiovisual object is a concept that includes images, sounds, and combinations thereof. Audiovisual objects include non-signalized objects and signal-reproduced objects. The signal of the object to be reproduced can be recorded on a recording medium and carried by a carrier wave, for example. The reproduced object is reproduced as at least one of video and audio by a device having a reproduction function, for example.

The “progress” of the viewing target refers to the change over time of the viewing target. Transition is a concept that encompasses constancy and change. Invariance means that visual information or auditory information continues as it is constant. Change means that visual or auditory information is not constant. The changes are, for example, irregular changes, regular changes, periodic repetitions, temporary disappearances, and combinations thereof. If the viewing target is a non-signalized object, the ongoing timing can be specified by the time information. If the viewing target is an object that is reproduced based on a signal, the point in progress can be identified by time information or the elapsed time from the start point of progress (that is, the position on the timeline).

"Viewing" of the viewing target means grasping the progress of the viewing target by at least one of visual and auditory perception. If the viewing target is a non-signalized object, the viewing target is viewed almost simultaneously with the progress. The viewing of non-signalized objects is, for example, watching sports games on stadiums, watching natural phenomena (e.g. solar and lunar eclipses, auroras, sunrises, sunsets, sunsets, rare meteorological phenomena, etc.) as appropriate. It is viewing and listening in places, and viewing and listening events (for example, festivals, street speeches, etc.) in appropriate places. If the viewing target is a signal-based reproduced object, the viewing target is viewed as it is played. Viewing of reproduced objects includes, for example, viewing of television broadcasts, viewing of radio broadcasts, viewing of online-delivered moving images, and viewing of online-delivered music.

A user (viewer) who views a viewing target can associate a predetermined element (shared element) with the viewing target by a predetermined operation (additional operation). A predetermined operation is, for example, a pointing operation with respect to an input element (operation element). The input element is, for example, a button displayed within the screen. A pointing operation is, for example, a combination of a touch operation on a touch panel, a positioning operation of a pointer displayed on the screen, and a click operation. When a viewing target is associated with a predetermined element, the viewing target is associated with the timing of displaying the shared element while the viewing target is in progress. Elements associated with a viewing target may be displayed on the screen at associated display timings while the viewing target is in progress.

The predetermined element is a visual element representing the user's reaction to the viewing target. The user's reaction is, for example, the user's emotion. Visual elements are, for example, still images, animated images, moving images, texts, and combinations thereof. An image that expresses the user's feelings toward the viewing target is sometimes called an emotional image. The emotion image is, for example, an image in which an onomatopoeia (also called an onomatopoeia, an onomatopoeia, a mimetic word, etc.) representing a user's emotion is designed. An image in which onomatopoeia is designed is sometimes called an onomatopoeia image. Also, certain elements may be shared among multiple users viewing a common viewing target. Elements that can be shared are sent and received online and displayed on the screen of the terminal. A terminal having a screen on which a shareable element is displayed may be the same device as the device that reproduces the object to be viewed, or may be a different device.

When identified "based on timing", a predetermined number of shared elements that are in close proximity in display timing may be identified as an element group. That the display timings of two shared elements are close to each other means that the time difference between the timing when one is displayed on the screen and the timing when the other is displayed on the screen is equal to or less than a predetermined threshold. That the display timings of three or more shared elements are close to each other means that the display timings of two shared elements arbitrarily selected from three or more shared elements are close to each other. When judging whether or not there is a proximity relationship with respect to display timing, a point in time between the start and end of display of each shared element is adopted as the display timing. The points of time adopted as display timings are, for example, the start point, the end point, and a predetermined reference point.

An “alternative element” refers to visual information associated with a viewing target instead of an element group (replacement target) including two or more shared elements. The alternative element may be visual information common to the replacement target, or may be visual information different in display mode from the replacement target. The display mode includes, for example, display time, display size, display color, presence/absence of additional visual information, and presence/absence of movement. The change in appearance may indicate to the user the number of overlapping elements. Specifically, it is preferable to change the display mode so that the greater the number of overlaps, the richer the display. Flashiness includes, for example, a long display time, a large display size, a dark display color, a large number of additional visual information, and large movements. Also, the number of overlapping elements may be indicated to the user by changing the display mode. Specifically, visual information indicating the number of overlaps may be added to the alternative element. The visual information indicating the number of overlaps is, for example, an icon displaying the number corresponding to the number of overlaps, and an icon of the number corresponding to the number of overlaps.

[1-2. Note]
The present invention includes at least the following [A] to [C].

[A] Identifying means for identifying, as an element group, a predetermined number of the shared elements based on the timing from among the shared elements associated with the viewing target with respect to the timing of displaying the viewing target on the terminal of the user who is viewing the viewing target and a replacing means for replacing the element group specified by the specifying means with alternative elements less than the predetermined number.

[B] A specification function for specifying, as an element group, a predetermined number of the shared elements based on the timing from among the shared elements associated with the viewing target with respect to the timing of displaying the viewing target on the terminal of the user who is viewing the viewing target. and a replacement function that replaces the element group specified by the specified function with a number of substitute elements less than the predetermined number. Note that the control program may be non-temporarily recorded on a recording medium.

[C] A step of identifying a predetermined number of shared elements as an element group based on the timing from among the shared elements associated with the viewing target with respect to the timing of displaying the viewing target on the terminal of the user who is viewing the viewing target. and a replacing step of replacing the element group identified in the identifying step with a number of alternative elements less than the predetermined number.

The following technical limitations may be added to the "information processing apparatus" in [A] above. Also, the same technical limitation may be added to the above [B] "control program" and the above [C] "information processing apparatus control method".

(1) The identifying means identifies the element group based on an additional attribute added to the shared element, and the replacing means adds to at least one of a plurality of shared elements included in the element group. The alternative element that inherits the additional attribute that is provided is added to the viewing target as the shared element. When identified “based on additional attributes,” a predetermined number of shared elements having additional attributes in common may be identified as an element group. Having commonality means being identical or highly related. Thereby, the alternative element after replacement can be further replaced with an alternative element.

(2) the additional attribute with which the specifying means can specify at least one of a user group to which a user who associates the shared element with the viewing target belongs and an element group to which the shared element associated with the viewing target belongs; Identify the element group based on. As a result, multiple shared elements that tend to overlap temporally are replaced with alternative elements.

A user who can associate a shared element with a viewing target can belong to at least one user group. Each user can voluntarily join any user group. Also, each user may automatically join one of the user groups depending on attributes and circumstances. A shared element associated with a viewing target can belong to at least one element group. Each shared element is pre-assigned to at least one element group.

(3) The identifying means identifies the element group based on the additional attribute capable of identifying both the user group and the element group. As a result, multiple shared elements that tend to overlap temporally are replaced with alternative elements.

(4) When the types of the predetermined number of shared elements included in the element group identified by the identifying means are the same, the replacing means replaces the element group with the substitute element according to the number of overlaps of the types. replace with As a result, a visual presentation is given so that the load does not rise excessively.

(5) The identifying means identifies the set only when a predetermined activation condition is satisfied. As a result, replacement with an alternative element is activated only in situations where a sudden load increase is likely to occur.

The “activation condition” is a condition for determining whether or not to activate the replacement process (the process of replacing a predetermined number of shared elements with fewer than a predetermined number of alternative elements). The activation condition is preferably set as a condition regarding the number of elements in the group to which shared elements in a close relationship belong. The condition regarding the number of elements is, for example, whether or not the number of elements reaches or exceeds a predetermined upper limit. The predetermined upper limit value may be a fixed value or a variable value. The variable value is preferably set according to the processing capability of the terminal. The processing capability of the terminal may be the processing capability for displaying information or the processing capability for receiving information. The processing power for displaying information is, for example, the clock frequency of the control processor of the terminal. The processing capability for information reception is, for example, the speed of the communication line of the terminal.

(6) The predetermined number of the shared elements included in the adjacent element group, except when the number of elements of the adjacent element group including the multiple shared elements that are close in timing does not satisfy the activation condition. are specified as the element group. As a result, replacement with an alternative element is not activated in situations where a sudden load increase is unlikely to occur.

(7) The specifying means specifies the predetermined number of shared elements included in the adjacent element group as the element group, except when the activation condition, which varies according to the processing capability of the terminal, is not satisfied. This makes it possible to flexibly determine whether or not a sudden load increase is unlikely to occur.

[2. Example]
Embodiments of the present invention will be described below with reference to the drawings. The present embodiment relates to a mechanism for displaying an image representing a user's (viewer's) emotion (an image representing a reaction) on a display screen of a moving image (viewing target). In this embodiment, the shared information processing device is assumed to be a server, and the information processing devices (first device and second device) are assumed to be mobile terminals, so-called smart phones. Note that the mobile terminal is not limited to a smart phone, and may be a tablet terminal, a laptop computer, or the like. Further, the information processing device may be a desktop personal computer. Each program according to the embodiment is stored in a shared information processing device or information processing device, but not limited to this, an optical disc such as a CD-ROM (Compact Disk Read Only Memory), or a USB (Universal Serial Bus) flash memory. may be stored in a storage medium such as a portable semiconductor memory.

[2-1. feature]
In this embodiment, a first user (himself) who is watching a moving image (viewing target) displayed on the screen (first screen) of a first mobile terminal (first device) displays a first image (first image) on the moving image. operation information of an addition operation (first operation) performed on the first mobile terminal in order to add one element), and information displayed on the screen (second screen) of the second mobile terminal (second device). operation information of an addition operation (second operation) performed on the second mobile terminal in order to add the first image (second element) to the moving image by the second user (other person) who is watching the moving image and are provided from each portable terminal to the server (first instruction means, shared information processing device). In order to improve entertainment, the server displays on the first screen and the second screen a combined image (third element) based on the combination of the first operation and the second operation corresponding to the provided operation information. .

In addition, in the present embodiment, in order to suppress a sudden increase in the load on the mobile terminal, the timing of displaying the video on the mobile terminal of the user who is viewing the video while the video is being reproduced is associated with the video. Identify two or more (predetermined number) second images as an image group (element group) from among the images (shared elements) based on the display timing proximity relationship (timing) and the commonality (additional attribute) between the subject and the object Then, while invalidating the second images included in the image group, one (smaller than a predetermined number) new second image (alternative element) inheriting the additional attribute is added to the moving image. . A subject means a user who added a second image to a moving image, and a subject having commonality means that the subject belongs to the same user group. Each user belongs to at least one user group. In addition, the object refers to the type of the second image (element) added to the moving image, and the commonality of the objects means that the objects belong to the same element group. Each element belongs to at least one element group.

[2-2. Video distribution system]
A server according to an embodiment is included in a video delivery system and delivers videos to mobile terminals. A mobile terminal according to an embodiment is included in a video delivery system, and videos are delivered from a server.

[2-2-1. Overall configuration of video distribution system]
FIG. 1 is a schematic configuration diagram of a video distribution system 1 according to an embodiment. A video distribution system 1 includes a communication line 2 , a plurality of mobile terminals 3 and a server 4 .

The communication line 2 forms a computer network, and is, for example, a wide area communication line provided by an electric power company.

The mobile terminal 3 is, for example, an information processing terminal such as a smart phone, a tablet terminal, or a notebook computer. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for capturing an image of an object, and the like. As an example, dedicated application software (hereinafter referred to as “moving image viewing application”) is installed on the mobile terminal 3 in order to display the moving images distributed from the server 4 .

The server 4 is a shared information processing device that distributes content to a plurality of mobile terminals 3 via the communication line 2 . A content according to the embodiment is, for example, a moving image. The videos are not particularly limited, and include, for example, animations, dramas, etc., in which a story exists and the story progresses along the story, music videos, live music videos, various sports videos, and the like. In addition, the moving image based on an Example is an animation as an example. Also, the content delivered from the server is not limited to moving images, and may be other services, such as a set of continuous still images, as long as the content can be viewed by the user.

Furthermore, viewing here means that the user recognizes the content at least visually and aurally. It may be sound or the like.

A plurality of mobile terminals 3 to which moving images are distributed are registered in the server 4 as a group 5 (hereinafter referred to as "group registration"), for example. The server 4 simultaneously distributes moving images to the mobile terminals 3 registered in the group. That is, the plurality of mobile terminals 3 registered in the group display the moving images distributed from the server 4 on the touch panel display 3a at the same timing, in other words, in synchronization. As a result, each user of the mobile terminal 3 who is registered as a group can watch the video at the same time.

Note that the moving image may be distributed at a predetermined date and time, or may be distributed at an arbitrary timing for each group 5 . When a moving image is distributed at an arbitrary timing, for example, the mobile terminal 3 serving as a host requests the server 4 to distribute the moving image, and the server 4 that receives the distribution request Deliver videos to.

In FIG. 1, four mobile terminals 3 constitute the group 5, but this is an example, and the number of mobile terminals 3 is not limited as long as the group 5 is composed of a plurality of mobile terminals 3. FIG. Also, the types of the mobile terminals 3 constituting the group 5 are not limited, and smartphones, tablet terminals, notebook computers, and the like may be mixed.

[2-2-2. Server configuration]
FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to the embodiment.

The server 4 according to the embodiment includes a CPU (Central Processing Unit) 20 which is a main control unit that controls the operation of the entire server 4, a ROM (Read Only Memory) 22 in which various programs and various data are stored in advance, and various programs by the CPU 20. A RAM (Random Access Memory) 24 used as a work area or the like when executing the program, and an HDD (Hard Disk Drive) 26 as storage means for storing various programs and various data.

The HDD 26 stores information about the registered group 5, moving images (moving image data) to be distributed to the portable terminal 3, and various other information. Note that the storage means is not limited to the HDD 26, and may be other storage media such as semiconductor memory such as flash memory.

Further, the server 4 includes an operation input unit 28 configured with a keyboard, a mouse, etc., for receiving input of various operations, a monitor 30 such as a liquid crystal display device for displaying various images, and a mobile terminal 3 or the like via the communication line 2. It is provided with an external interface 32 that is connected to other information processing devices or the like and performs transmission/reception of various data with the other information processing devices or the like.

The CPU 20 , ROM 22 , RAM 24 , HDD 26 , operation input section 28 , monitor 30 and external interface 32 are electrically connected to each other via a system bus 34 . Therefore, the CPU 20 accesses the ROM 22, the RAM 24, and the HDD 26, grasps the operation state of the operation input unit 28, displays images on the monitor 30, and exchanges various data with other information processing devices or the like via the external interface 32. Transmission and reception can be performed respectively.

[2-2-3. Electrical Configuration of Mobile Terminal]
FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. As shown in FIG. The mobile terminal 3 includes a main control unit 40, a main storage unit 42, an auxiliary storage unit 44, a communication unit 46, and operation buttons 48 in addition to the configuration shown in FIG.

The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), etc., and controls the overall operation of the mobile terminal 3 .

The main storage unit 42 is composed of, for example, a RAM or a DRAM (Dynamic Random Access Memory), and is used as a work area or the like when the main control unit 40 executes processing based on various programs.

The auxiliary storage unit 44 is, for example, a non-volatile memory such as a flash memory, and stores various data such as images, programs used for processing by the main control unit 40, and the like. The programs stored in the auxiliary storage unit 44 include, for example, an OS (Operating System) for realizing the basic functions of the mobile terminal 3, drivers for controlling various hardware, e-mail and web browsing, and various other programs. It is a program or the like for realizing a function. Further, the auxiliary storage unit 44 pre-stores application software such as a video viewing application for reproducing the video distributed from the server 4 and displaying it on the touch panel display 3a.

The communication unit 46 is, for example, a NIC (Network Interface Controller) and has a function of connecting to the communication line 2 . Note that the communication unit 46 has a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), or a short-range communication such as Bluetooth (registered trademark), instead of or together with the NIC. It may have a function of enabling wireless communication, infrared communication, and the like.

The operation button 48 is provided on the side surface of the mobile terminal 3, and includes a power button for starting or stopping the mobile terminal 3, a volume adjustment button for sound output by the speaker 3b, and the like.

These main control unit 40, main storage unit 42, auxiliary storage unit 44, communication unit 46, touch panel display 3a, speaker 3b, microphone 3c, camera 3d, and operation button 48 are electrically connected to each other via system bus 49. It is connected. Therefore, the main control unit 40 accesses the main storage unit 42 and the auxiliary storage unit 44, displays an image on the touch panel display 3a, grasps the operating state of the touch panel display 3a and the operation buttons 48 by the user, and transmits sound to the microphone 3c. Input, sound output from the speaker 3b, control of the camera 3d, access to various communication networks and other information processing devices via the communication unit 46, and the like can be performed.

[2-3. Onomatopoeia display]
The mobile terminal 3 according to the embodiment can display an image (an onomatopoeia image whose details will be described later) according to the user's operation while displaying the moving image distributed from the server 4 on the touch panel display 3a. In addition, in the following description, the function to display an onomatopoeia image on the touch-panel display 3a is called an onomatopoeia display function, and the touch-panel display 3a is also called screen 3a.

The onomatopoeia display function is, for example, one of the functions that the video viewing application according to the embodiment has. In addition, as an example, onomatopoeia display is not possible for all the videos displayed on the screen 3a using the video viewing application, and the videos for which the onomatopoeia display is possible are determined in advance.

[2-3-1. screen structure]
4 to 6 show display modes of the screen 3a of the mobile terminal 3 on which the video viewing application according to the embodiment is running. A distribution moving image display area 50A, a first image display area 50B, and a second image display area 50C are set on the screen 3a.

50 A of delivery animation display areas are areas which display the animation delivered from the server 4. FIG. The first image display area 50B is an area for displaying the first image 51 (see FIG. 5) according to the user's operation. The second image display area 50C is an area for displaying a second image 52 (see FIG. 6) based on the first image 51 displayed on another mobile terminal 3 by another user's operation. As will be described in detail later, the second image 52 is a reduced image of the first image 51 displayed in the first image display area 50B of the other mobile terminal 3 . In addition, the first image display area 50B and the second image display area 50C according to the embodiment are provided at positions facing each other with the delivered moving image display area 50A interposed therebetween, as an example.

Here, the first image 51 is an image including an emotion image that indicates the emotion of the user who is watching the moving image. The emotion image according to the embodiment is, for example, an image in which an onomatopoeic word (also called an onomatopoeia or a mimetic word) indicating a user's emotion is designed, in other words, an onomatopoeia image. Onomatopoeia images include, for example, "wow", "bad", and "bravo". It should be noted that the designing is, for example, decorative drawing of characters indicating onomatopoeia.

In addition to the onomatopoeia image, the first image 51 includes a character image showing a character appearing in the video, a dialogue image showing the character's dialogue, text input by the user himself, and other various images. good too.

Also, the first image 51 may differ depending on the scene of the moving image. For example, it may be possible to display an onomatopoeia image or a character image that is highly likely to be selected by the user in sympathy with the scene, depending on a happy scene, a sad scene, a confrontation scene between characters, or the like.

Also, the first image 51 that can be displayed may differ depending on the user. For example, the number of types of displayable first images 51 may be increased by acquiring specific items through services such as games related to distributed moving images. Further, for example, the number of types of first images 51 that can be displayed may be increased on condition that a predetermined consideration (for example, monetary value such as currency, virtual currency, or points) is paid.

In the first image display area 50B according to the embodiment, a plurality of display buttons 53A to 53D, which are operation units operated by the user and corresponding to different first images 51, are displayed. Accordingly, the user can display the first image 51 in the first image display area 50B by operating the display buttons 53A to 53D while watching the moving image. The operation of the display buttons 53A to 53D is, for example, touching (pressing) the display buttons 53A to 53D with an indicator. An indicator according to the embodiment is, for example, a user's finger. Further, in the following description, when the display buttons 53A to 53D are not described separately, the alphabetical notation at the end is omitted.

In the examples of FIGS. 4 to 6, four display buttons 53A to 53D are displayed in a row in the horizontal direction below the first image display area 50B. The display button 53A corresponds to the onomatopoeia image stylized "Sugee", the display button 53B corresponds to the onomatopoeia image stylized "Yabai", and the display button 53C corresponds to the stylized onomatopoeia image "Bravo". Corresponds to onomatopoeia images. Furthermore, the display button 53D corresponds to a character image. The display button 53 is designed so that the user can intuitively recognize the corresponding onomatopoeia image, character image, or the like.

By displaying a plurality of display buttons 53 in the first image display area 50B in this manner, the user can easily display the first image 51 that best matches his reaction (emotion) to the moving image on the screen 3a. can be done. In addition, since the first image 51 includes an onomatopoeia image indicating the user's emotion, the user can display his/her own emotion after watching the moving image on the screen 3a.

In the examples of FIGS. 4 to 6, four types of display buttons 53 are displayed in the first image display area 50B, but this is an example, and the number and type of display buttons 53 are not limited to this. . Further, the display button 53 may be switched and displayed as appropriate so that different first images 51 can be displayed according to the scene of the moving image or the like.

Also, the arrangement of the display buttons 53 is also an example. may Furthermore, a display button 53 for displaying a different type of first image 51 may be newly displayed by, for example, the user swiping the lower portion of the screen 3a in the left or right direction.

[2-3-2. Display of first image]
A display mode of the first image 51 will be described in detail with reference to FIG. In the example of FIG. 5, the display button 53C is operated by the user, and an onomatopoeia image in which "Bravo" is designed is displayed on the screen 3a.

The onomatopoeia display function according to the embodiment receives a user's operation for displaying an onomatopoeia image (first image 51) on the screen 3a while a moving image is displayed on the screen 3a. That is, the user operates the display button 53 while watching the moving image to display the onomatopoeia image corresponding to the emotion felt by watching the moving image on the screen 3a.

The onomatopoeia image according to the embodiment is displayed on the screen 3a displaying the moving image while changing according to the operation mode of the display button 53 by the user. As an example, the onomatopoeia image according to the embodiment is displayed so as to extend from the first image display area 50B to the delivery moving image display area 50A according to the operation mode of the display button 53 by the user.

In this way, the user can change the onomatopoeia image displayed on the screen 3a by operating the mobile terminal 3 while watching the moving image. Therefore, the user can easily display on the screen, while watching the moving image, the emotions that the user has felt while watching the moving image.

Moreover, in the example of FIG. 5, the onomatopoeia image is superimposed and displayed on the distribution animation display area 50A beyond the 1st image display area 50B. Thus, the onomatopoeia display function according to the embodiment superimposes the onomatopoeia image on the distribution video display region 50A beyond the first image display region 50B according to the user's operation mode, so that the user can view the video It becomes easier to empathize with the viewer, and the enjoyment of viewing the moving image is enhanced.

In addition, in FIG. 5, the onomatopoeia image is also superimposed on the second image display area 50C beyond the delivery moving image display area 50A. However, this is an example, and the onomatopoeia image may not be superimposed on the second image display area 50C.

Next, the details of the onomatopoeia image that changes according to the operation mode of the display button 53 will be described.

As an example, the display mode of the first image 51 according to the embodiment changes according to the duration of the operation of the display button 53 by the user. The duration of operation of the display button 53 is, for example, the time during which the display button 53 is continuously pressed (hereinafter referred to as “long pressing time”).

When the user presses the display button 53, the onomatopoeia display function according to the embodiment starts displaying the onomatopoeia image starting from the vicinity of the position of the display button 53. FIG. The display of the onomatopoeia image starting from the vicinity of the position of the display button 53 means that the onomatopoeia image is displayed so that the lower end of the onomatopoeia image substantially matches the position of the display button 53, for example.

Then, the onomatopoeia image is expanded (hereinafter referred to as "expansion display") from the first image display area 50B toward the delivery moving image display area 50A according to the long-pressing time of the display button 53. FIG.

In the extended display, for example, the onomatopoeia image is elongated or enlarged. Note that the expansion display may be such that the onomatopoeia image is expanded while changing the form such as the number of characters or the type of characters. For example, if the onomatopoeia image is "Bravo", it may be stretched so that "a" increases between "Bra" and "Bo" depending on the long press time, or It may be extended so that "o" increases between "bravo" and "-".

That is, when the user feels a stronger emotion toward the moving image being viewed, the user keeps pressing the display button 53 for displaying an onomatopoeia image corresponding to this emotion. As a result, the onomatopoeia image is expanded and displayed from the first image display area 50B toward the delivery moving image display area 50A, and displayed on the screen 3a so that the onomatopoeia image is more conspicuous.

In this way, the onomatopoeia image included in the first image 51 changes according to the user's operation mode, so that the user can display his/her own reaction (emotion) to the content on the screen 3a in a more entertaining way.

In addition, since the first image 51 is displayed so as to extend from the first image display area 50B toward the delivery moving image display area 50A according to the operation mode of the user, the user can express his or her reaction (emotion) to the content. ) can be displayed on the screen 3a in a more entertaining manner.

In addition, when the operation (long-press operation) of the display button 53 by the user ends, the display of the onomatopoeia image on the screen 3a is stopped. Stopping the display of the onomatopoeia image, for example, the displayed onomatopoeia image may disappear from the screen 3a momentarily, or the expanded onomatopoeia image may shrink toward the display button 53 and disappear from the screen 3a. good.

In addition, in the example of FIG. 5, the periphery of the display button 53C operated by the user is highlighted in order to display the onomatopoeia image. As a result, the user can easily recognize the display button 53C that the user is operating, and the onomatopoeia display is more entertaining.

In addition, the character image or the like, which is the first image 51 other than the onomatopoeia image according to the embodiment, is displayed without change on the screen 3a. It may be displayed on the screen 3 a displaying the moving image while changing according to the operation mode of the display button 53 .

[2-3-3. Display of second image]
A display mode of the second image 52 will be described in detail with reference to FIG. Note that the second image 52 is a reduced image of the first image 51 displayed on another mobile terminal 3 by another user's operation, as described above.

The onomatopoeia display function according to the embodiment causes the second image 52 to be displayed in the second image display area 50</b>C almost without delay from the timing when the other user causes the other mobile terminal 3 to display the first image 51 . This allows the user to know the reactions of other users viewing the same moving image while viewing the moving image.

In addition, when the second image 52 is superimposed and displayed on the distribution moving image display area 50A like the first image 51, the second image 52 may interfere with the viewing of the moving image and may be annoying to the user. Therefore, the second image 52 is not superimposed and displayed on the distribution moving image display area 50A beyond the second image display area 50C regardless of the operation mode of other users. Thereby, the user can confirm the reactions of other users to the video being viewed without feeling annoyed.

As described above, the first image 51 can also be superimposed on the distribution moving image display area 50A. The superimposition of 51 does not bother the user.

Further, since the screen 3a is set such that the first image display area 50B and the second image display area 50C are different areas, the first image 51 displayed by the user and the image displayed by the other user are displayed. The second image 52 is not mixed and displayed. Accordingly, the user can easily separate and recognize the first image 51 displayed by the user and the second image 52 displayed by another user.

Further, another user according to the embodiment is, for example, another user registered in the same group as the user. That is, when the user is watching the moving image without group registration, the second image 52 is not displayed in the second image display area 50C. In addition, the second image 52 may be displayed in the second image display area 50</b>C according to the operations of all other users who are watching the moving image regardless of whether or not the user is registered in the group.

Also, in the second image display area 50C according to the embodiment, a plurality of second images 52 operated by a plurality of other users are displayed. Further, in the second image display area 50C according to the embodiment, as an example, the position where the second image 52 is displayed for each other user is not determined. That is, in the second image display area 50C, a plurality of second images 52 operated by a plurality of other users are mixed and displayed at random positions. Further, the second image 52 may be displayed while moving vertically and horizontally within the second image display area 50C. In addition, the position which displays the 2nd image 52 for every other user may be determined not only in this but in the 2nd image display area 50C.

In addition, the second image 52 is adjacent to an identification image (hereinafter referred to as “user icon”) 54 indicating another user who displayed the first image 51 as the second image 52 on his mobile terminal 3. displayed. This allows the user to recognize which user is responsible for the second image 52 displayed in the second image display area 50C. In the example of FIG. 6, the user icon 54 is a single alphabetic character displayed in the center of a circle, but this is just an example. and the design of the user icon 54 is not limited.

In addition, the second image 52 shown in FIG. 6 displays the onomatopoeia image horizontally. 52 may display an onomatopoeia image in vertical writing.

Moreover, although the second image 52 is not superimposed on the delivery moving image display area 50A, it may be displayed while being changed (e.g. expanded) within the second image display area 50C in accordance with the operation mode of another user. The second image 52 may be superimposed from the second image display area 50C to the first image display area 50B as long as it is not superimposed on the distribution moving image display area 50A.

[2-3-4. Combined image display]
The mobile terminal 3 according to the embodiment displays a combined image on the screen 3a in accordance with a combination of a user's operation for displaying the first image 51 and another user's operation for displaying the first image 51. It has a combined image display function.

As a result, the portable terminal 3 can display the reactions of the user and other users to the moving image on the screen with a higher level of entertainment. In addition, since an image that cannot be displayed by one user's operation is displayed in combination with another user's operation, the portable terminal 3 can encourage more users to view the content.

Also, the combined image is displayed superimposed on the moving image displayed on the screen 3a. That is, the combination image is displayed not only in the first image display area 50B but also superimposed on the delivery moving image display area 50A and the second image display area 50C. As a result, the user can easily recognize that the combined image has been displayed, and enjoyment of viewing the moving image is enhanced.

When a combined image is displayed on the screen 3a, it is displayed so that the user can recognize that it is a combined image. For example, the user IDs of the other users may be displayed together with the combined image on the screen 3a so that it can be recognized that the other users performed display operations at the same timing.

FIG. 7 is a timing chart showing a case where a combined image is displayed on the screen 3a.

The combined image display function according to the embodiment displays the combined image on the screen 3a when the timing of the user's operation and the other user's operation is within a predetermined time (hereinafter referred to as "combination period"). That is, when a plurality of users perform display operations for the first image 51 within the combination period, it is understood that the user's reaction is due to viewing the same scene. Therefore, the combined image display function displays a combined image based on the reactions of a plurality of users according to the scene of the moving image, so that viewing the moving image is more entertaining for the user.

Note that the start of the combination period is the timing when a certain user starts the display operation of the first image 51 . That is, as shown in FIG. 7A, when the user B performs the display operation within the combination period starting from the timing when the user A starts the display operation, the screen 3a of the mobile terminals 3 of the users A and B is displayed. , the combined image is displayed.

The display period of the combined image is, for example, until the display operation by the user is completed. In the example of FIG. 7A, the combined image is displayed on the screens 3a of the mobile terminals 3 of the users A and B from the start of the display operation by the user B to the end of the display operation.

The example of FIG. 7B shows a case where user C performs a display operation after user A's display operation at the start timing of the combination period is completed. In the example of FIG. 7B, the combined image is displayed on the screens 3a of the portable terminals 3 of the users A, B, and C from when the user B's display operation is started until the user C's display operation is finished.

Note that the display mode of the combined image shown in FIG. 7 is an example, and is not limited to this. For example, there is no concept of combination period, and the combination image may be displayed on the screen 3a at the timing when the user's display operation overlaps with another user's display operation. Alternatively, a display time may be predetermined for each combined image, and the display of the combined image may end when the display time has elapsed.

Also, the combined image according to the embodiment is displayed on the screen 3 a instead of the first image 51 . In the example of FIG. 7A, first, the first image 51 is displayed on the screen 3a of the portable terminal 3 of the user A according to the display operation of the user A. When the display operation of the user B is started, the first image 51 is displayed. The display of the first image 51 ends, and the combined image is displayed on the screen 3a of the mobile terminal 3 of the user A. FIG. On the other hand, on the mobile terminal 3 of the user B, the combined image is displayed without displaying the first image 51 that the user B intended to display on the screen 3a. By displaying the combined image on the screen 3a instead of the first image 51 in this manner, an image different from the first image 51 that the user intended to display is displayed on the screen 3a. The enjoyment of the viewing user is further enhanced.

Also, the combination image according to the embodiment is displayed on the screen 3a when the first image 51 selected by the user and the first image 51 selected by another user are preset images. As a result, there is a possibility that a combined image that the user himself/herself does not anticipate will be displayed according to the image selection by the user himself/herself.

For example, the combined image is displayed on the screen 3a when the first image 51 selected by the user and the first image 51 selected by another user are the same. That is, when users viewing a moving image select the same first image 51, there is a high possibility that the users have similar emotions. Therefore, by displaying the combined image, the user can recognize that he or she empathized with the other user by viewing the moving image. More specifically, for example, when the first image 51 selected by the user and the first image 51 selected by another user are the same first image 51, an enlarged image of the first image 51 is used as the combined image. is displayed on the screen 3a. In addition, not limited to this, a special image that is not displayed by operating the display button 53 may be displayed as a combined image.

Further, even if the first image 51 selected by the user is different from the first image 51 selected by another user, the combined image is displayed when the combination is a preset combination. may In this case, for example, a special image that cannot be displayed by operating the display button 53 is displayed as a combined image.

In addition, in the combination image display function according to the embodiment, a second image based on the combination image displayed on the screen 3a of another user is displayed in the second image display area 50C of the mobile terminal 3 of the user on which the combination image is not displayed. Do not show. However, without being limited to this, the second image based on the combined image displayed on the screen 3a of the other user may be displayed in the second image display area 50C of the mobile terminal 3 of the user on which the combined image is not displayed.

[2-4. Replacement process]
The server 4 according to the embodiment specifies, as an image group, two second images 52 whose display timings are close to each other from among the second images, and performs unit processing of replacing the image group with one new second image 52. is repeated until the image group is no longer identified. As a result, the number of second images 52 displayed on the mobile terminal 3 during playback of the moving image can be reduced, and a sudden increase in the load on the mobile terminal 3 can be suppressed.

FIG. 8 is a timing chart showing the case where the second image is replaced.

As a premise, the timeline of the video is divided into a plurality of sections. For example, the length of each section is assumed to be the same as the length of the combination period. As an example, the replacement processing function according to the embodiment selects, as an image group, two second images 52 having a common subject and object from among the plurality of second images 52 each included in the same section of the display period. A unit process of specifying and invalidating two second images 52 included in the image group and associating one new second image 52 with the timeline of the moving image is repeated until the image group is no longer specified.

The second image 52 contains the subject (the user ID of the user who added the second image to the moving image and the user group to which the user belongs), the object (the type of the second image and the element group to which the type belongs), the display period , the number of times of synthesis, status, etc. are added (second image display information). The type is determined by the object and the number of synthesis times. The pattern of the second image 52 is determined according to the type. It is preferable that the appearance of the pattern reflects the number of times of synthesis, such as displaying a number indicating the number of times of synthesis. As shown in FIG. 8A, both the start of the display period of the second image A and the start of the display period of the second image B are included in the n-th section (n is an appropriate natural number) on the timeline. may be Also, it is assumed that the type of the second image A and the object of the second image B are the same. At this time, the replacement processing function identifies the second image A and the second image B as an image group, invalidates the status of each of the second image A and the second image B, and creates one new second image C to the video timeline.

The second image 52 after replacement (second image C) inherits the additional attributes of the second image 52 before replacement (second image A and second image B). As an example, the second image 52 after replacement inherits the subject and the object as they are. The number of times the second images 52 are combined after replacement is, for example, the total value of the number of times each second image 52 is combined before replacement. The display period of the second image 52 after replacement is, for example, the period from the earlier start to the later end. As shown in FIG. 8A, the display period of the second image C corresponds to the period from the beginning of the second image A to the end of the second image B. As shown in FIG.

The second image 52 before replacement is invalidated and disassociated with the moving image. On the other hand, the second image 52 after replacement forms an image group with other second images 52 and can be replaced with a new second image 52 . As shown in FIG. 8B, both the start of the display period of the second image C and the start of the display period of the second image D are included in the n-th section (n is an appropriate natural number) on the timeline. is The user who added the second image C and the user who added the second image D belong to the same user group, and the type of the second image C and the type of the second image D are the same. At this time, the replacement processing function identifies the second image C and the second image D as an image group, invalidates the status of each of the second image C and the second image D, and creates one new second image E to the video timeline. The display period of the second image E is the period from the beginning of the second image C to the end of the second image D. FIG.

[2-5. Function block]
Next, combined image display and replacement processing according to the embodiment will be described using a functional block diagram. The functions related to the combination image display in the mobile terminal 3 are included in the onomatopoeia display function, and the functions related to the combination image display and replacement processing in the server 4 are included in the moving image distribution function.

[2-5-1. Functional block related to video distribution function of server]
FIG. 9 is a functional block diagram relating to the video distribution function of the server 4 according to the embodiment.

The CPU 20 included in the server 4 includes a group registration unit 60, a moving image distribution unit 61, a first image display information reception control unit 62, a second image display information generation unit 63, a second image display information transmission control unit 64, and a combination image information generation unit. 65 , a combined image information transmission control unit 66 , and a replacement processing unit 67 . Each function provided by the CPU 20 is implemented by a program stored in the HDD 26 .

The group registration unit 60 accepts registration of a group 5 made up of a plurality of mobile terminals 3 . Registration of group 5 is performed by registering a user ID, for example.

The moving image distribution unit 61 distributes moving images to the mobile terminal 3 in response to a request from the mobile terminal 3 .

The first image display information reception control unit 62 receives the first image display information received from the mobile terminal 3 via the external interface 32, and stores the first image display information in the HDD 26 in association with the corresponding moving image.

The first image display information is information related to the first image 51 displayed on the screen 3a of the mobile terminal 3, and is transmitted from the mobile terminal 3 to the server 4 at the timing when the first image 51 is displayed on the mobile terminal 3. . The first image display information includes, for example, the type of the displayed first image 51, the user ID indicating the user who displayed the first image 51, the user's operation mode performed to display the first image 51, It is the elapsed time from the start of the moving image when the first image 51 is displayed, and the like.

The second image display information generation unit 63 generates second image display information based on the received first image display information, and stores the second image display information in the HDD 26 in association with the corresponding moving image.

The second image display information is information related to the second image 52 to be displayed in the second image display area 50C. contains the user ID of the user who displayed the , the display period, the number of times of synthesis, the status, and the like.

The second image display information transmission control section 64 controls the external interface 32 so as to transmit the second image display information to the mobile terminal 3 . The destination of the second image display information is another mobile terminal 3 different from the mobile terminal 3 that sent the first image display information.

The combination image information generation unit 65 performs combination determination based on the received first image display information. Then, the combined image information generation unit 65 generates combined image information indicating a combined image according to the result of combination determination, and stores the combined image information in the HDD 26 .

The combined image information is information relating to a combined image to be displayed on the screen 3a, and includes the type of combined image, user IDs of a plurality of users who performed display operations at the same timing, and the like.

Further, the combination determination is performed for each first image display information, and the display operation of the first image 51 indicated by the first image display information is based on the start timing of the display operation of the first image 51 by another user. It is determined whether or not it is within the set combination period.

The combined image information transmission control section 66 controls the external interface 32 so as to transmit the combined image information to the portable terminal 3 . The mobile terminals 3 to which the combined image information is transmitted are a plurality of mobile terminals 3 corresponding to the user IDs included in the combined image information.

The replacement processing unit 67 executes replacement processing.

[2-5-2. Functional blocks related to onomatopoeia display on mobile terminals]
FIG. 10 is a functional block diagram regarding onomatopoeia display of the mobile terminal 3 according to the embodiment.

The main control unit 40 included in the mobile terminal 3 includes an image display control unit 70, a display area setting unit 71, a first image display operation reception unit 72, an operation state determination unit 73, a first image display information generation unit 74, a first image A display information transmission control section 75 , a second image display information reception control section 76 , and a combined image information reception control section 77 are provided. Each function provided by the main control unit 40 is implemented by a program stored in the auxiliary storage unit 44 .

The image display control unit 70 controls the touch panel display 3a to display an image based on the image data. The image display control unit 70 according to the embodiment displays the moving image distributed from the server 4 in the distributed moving image display area 50A, displays the first image 51 in the first image display area 50B according to the user's operation, and displays other images. The second image 52 based on the first image 51 displayed on the other mobile terminal 3 by the user is displayed in the second image display area 50C. Further, the image display control unit 70 according to the embodiment superimposes the first image 51 beyond the first image display area 50B and displays it on the delivery moving image display area 50A according to the user's operation mode.

The display area setting unit 71 sets the positions and sizes (vertically and horizontally) of the distribution moving image display area 50A, the first image display area 50B, and the second image display area 50C as the display mode of the screen 3a when the moving image is distributed from the server 4. length). Note that the positions and sizes of these are predetermined, but may be arbitrarily set by the user.

The first image display operation accepting unit 72 accepts an operation of the display button 53 by the user while the screen 3a is displaying a moving image. That is, the first image display operation reception unit 72 recognizes the display button 53 operated by the user among the plurality of display buttons 53, and displays the first image 51 corresponding to the display button 53 via the image display control unit 70. to display on the screen 3a. Note that the first image display operation receiving section 72 displays the first image 51 on the screen 3 a so that the display mode of the first image 51 changes according to the determination result of the operation state determining section 73 .

The operation state determination unit 73 determines the operation state of the display button 53 by the user. The operation state referred to here is, for example, an operation mode including a long pressing time of the display button 53 .

The first image display information generating section 74 generates first image display information according to the first image 51 displayed on the screen 3a by the user's operation.

The first image display information transmission control section 75 controls the communication section 46 to transmit the first image display information to the server 4 .

The second image display information reception control section 76 outputs the second image display information received from the server 4 via the communication section 46 to the image display control section 70 .

The combined image information reception control section 77 outputs the combined image information received from the server 4 via the communication section 46 to the image display control section 70 . Accordingly, the image display control unit 70 displays the combined image on the screen 3a displaying the moving image. Note that the image display control unit 70 according to the embodiment displays the combined image on the screen 3a instead of the first image 51 according to the operation of the display button 53 by the user.

[2-6. Processing flow]
Next, the flow of onomatopoeia display processing according to the embodiment in the mobile terminal 3 and the server 4 will be described using a flowchart.

[2-6-1. Flow regarding onomatopoeia display on mobile terminal]
FIG. 11 is a flowchart showing the flow of onomatopoeia display processing executed by the main control unit 40 provided in the mobile terminal 3. As shown in FIG. A program for executing the onomatopoeia display process is stored in advance in a predetermined area of the auxiliary storage unit 44 . As an example, the onomatopoeia display process is started when the portable terminal 3 starts playing a moving image distributed from the server 4, and ends when the playing of the moving image ends.

In step S100, occurrence of a trigger is detected. The trigger in this embodiment is either the first trigger (acceptance of user's operation of the display button 53) or the second trigger (receipt of second image display information). The mobile terminal 3 according to the present embodiment updates the display contents of the screen each time the generation of the trigger is detected. The first trigger and the second trigger can occur in parallel. Therefore, during execution of processing in response to the occurrence of one trigger, processing may be performed in response to the occurrence of the other trigger. Also, multiple secondary triggers may occur in parallel. Therefore, during the execution of the process in response to the generation of one second trigger, the process in response to the generation of another second trigger may be executed.

In step S102, when the detected trigger is the first trigger, the process proceeds to step S106, and when the detected trigger is the second trigger, the process proceeds to step S104.

In step S104, the image display control unit 70 causes the second image 52 indicated by the second image display information to be displayed in the second image display area 50C, and the process ends.

In step S106, the image display control unit 70 causes the first image display area 50B to display the first image 51 corresponding to the display button 53 operated by the user. Note that the first image 51 may be stored in the mobile terminal 3 in advance, or may be downloaded from the server 4 each time the display button 53 is operated by the user.

In step S108, the first image display information generator 74 generates first image display information corresponding to the first image 51 displayed in the first image display area 50B.

In step S<b>110 , the first image display information transmission control section 75 controls the communication section 46 to transmit the first image display information to the server 4 .

In step S112, the operation state determination unit 73 determines whether or not the operation of the display button 53 by the user has ended. If the determination is affirmative, the process proceeds to step S120, and if the determination is negative, the process proceeds to step S114.

In step S114, the combined image information reception control unit 77 determines whether or not the combined image information has been received via the communication unit 46. If the determination is affirmative, the process proceeds to step S118, and if the determination is negative, the process proceeds to step S116. Transition.

In step S116, the image display control unit 70 expands and displays the first image 51 (onomatopoeia image) until the operation of the display button 53 by the user is completed, and the process returns to step S112.

In step S118, the image display control unit 70 displays the combined image indicated by the combined image information on the screen 3a, and the process returns to step S112.

In step S120, the image display control unit 70 ends the display of the first image 51, and ends the process.

In the flowchart shown in FIG. 11, the first image display information is generated together with the display of the first image 51, and the first image display information is transmitted to the server 4; The first image display information may be transmitted to the server 4 after the display is completed. Further, the first image display information transmission control unit 75 transmits the first image display information indicating the start of display of the first image 51 to the server 4 at the timing when the display of the first image 51 is started. The first image display information indicating the end of display of the second image 52 may be transmitted to the server 4 at the timing when the display ends. As a result, the server 4 displays the first image 51 (onomatopoeia image) in a decompressed manner in the reception interval between the first image display information indicating the start of display of the first image 51 and the first image display information indicating the end of display. I can recognize that.

[2-6-2. Flow regarding onomatopoeia display on the server]
FIG. 12 is a flow chart showing the flow of onomatopoeia display processing executed by the CPU 20 of the server 4. As shown in FIG. A program for executing the onomatopoeia display process is pre-stored in a predetermined area of the HDD 26 . As an example, the onomatopoeia display process is started when the server 4 distributes the video to the mobile terminal 3, and ends when the distribution of the video ends.

In step S<b>200 , the first image display information reception control section 62 determines whether or not the first image display information has been received from the mobile terminal 3 . If the determination is affirmative, the process proceeds to step S202, and if the determination is negative, the process enters a waiting state.

In step S<b>202 , the received first image display information is stored in the HDD 26 by the first image display information reception control section 62 .

In step S204, the combined image information generation unit 65 performs combination determination, and if the determination is affirmative, the process proceeds to step S208, and if the determination is negative, the process proceeds to step S206. That is, when the display operation of the first image 51 to be combined is performed by a plurality of users within the combination period, an affirmative determination is made and the process proceeds to step S208. On the other hand, if the operation to display the first image 51 to be combined is not performed by a plurality of users within the combination period, a negative determination is made and the process proceeds to step S206.

In step S<b>206 , the second image display information generator 63 generates second image display information based on the received first image display information, and stores it in the HDD 26 .

In step S208, the combined image information generation unit 65 generates combined image information indicating a combined image corresponding to the display operation of the first image 51 to be combined.

In step S210, the combined image information transmission control unit 66 controls the external interface 32 to transmit combined image information, and the process returns to step S200. The destination of the combined image information is the plurality of mobile terminals 3 that have transmitted the first image display information to be combined, that is, the mobile terminals 3 corresponding to the plurality of user IDs included in the combined image information.

FIG. 13 is a flow chart showing the flow of onomatopoeia display processing executed by the CPU 20 provided in the server 4. As shown in FIG. The flow shown in FIG. 13 targets the second image display information in which the start of the display period is included in the section each time the end of the section in the timeline of the moving image arrives.

In step S300, the replacement processing unit 67 makes a determination, and if the determination is affirmative, the process proceeds to step S302, and if the determination is negative, the process proceeds to step S304. That is, when there is the second image display information having commonality between the subject and the object, an affirmative determination is made and the process proceeds to step S302. On the other hand, when there is no second image display information having commonality between the subject and the object, a negative determination is made and the process proceeds to step S304.

In step S302, the replacement processing unit 67 performs replacement processing. In the unit process of the replacement process, the replacement processing unit 67 identifies two pieces of second image display information having commonality in the subject and object as an image group, and determines the status of the two pieces of second image display information included in the image group. are invalidated, and one new piece of second image display information is generated. In step S302, this unit process is repeated until there are no two pieces of second image display information having commonality between the subject and the object.

In step S304, the second image display information transmission control unit 64 controls the external interface 32 to transmit all second image display information included in the target section to the mobile terminal 3 different from the mobile terminal 3 that has transmitted the first image display information. 2 image display information is transmitted. Therefore, the display period on the portable terminal 3 of the second image 52 based on the second image display information is delayed from the display period associated with the moving image.

As described above, the mobile terminal 3 according to the embodiment allows the first user who is watching the moving image displayed on the screen 3a of the first mobile terminal 3 to add the first image 51 to the moving image. A first image 51 is added to the moving image by a second user who is watching the moving image displayed on the screen 3a of the second mobile terminal 3 and the operation information of the additional operation performed on the first mobile terminal 3. For this purpose, each mobile terminal 3 provides the server 4 with operation information of an additional operation performed on the second mobile terminal 3 . In order to improve entertainment, the server 4 displays a combination image based on the combination of the additional operation by the first user and the additional operation by the second user on the screen 3a of the first mobile terminal 3 and the screen of the second mobile terminal 3. 3a respectively. As a result, an image that cannot be displayed by one user's operation is displayed in combination with another user's operation, so that the mobile terminal 3 can encourage more users to watch moving images.

Further, the server 4 according to the embodiment selects the second image 52 whose display timing is associated with the moving image on the mobile terminal 3 of the user who watches the moving image during reproduction of the moving image, and Two or more second images 52 having common subject and object are replaced with one new second image 52 . As a result, the number of second images 52 to be displayed on the mobile terminal 3 is reduced, so that the server 4 can suppress a sudden increase in load on the mobile terminal 3 .

[3. Modification]
The technical scope of the present invention is not limited to the scope described in the above examples. Various modifications or improvements can be made to the above-described embodiments without departing from the spirit of the invention, and forms with such modifications or improvements are also included in the technical scope of the present invention.

Further, the flow of various processes described in the above embodiment is also an example, and unnecessary steps may be deleted, new steps added, or the processing order changed without departing from the scope of the present invention. good too.

[3-1. Modification 1 (Sharing an image without going through a server)]
In the above embodiment, the server 4 receives the first image display information from the mobile terminal 3, transmits the second image display information generated based on the first image display information to the other mobile terminal 3, and 3 explained the form in which the second image 52 indicated by the second image display information is displayed in the second image display area 50C, but the present invention is not limited to this.

For example, the mobile terminals 3 may display the second image 52 in the second image display area 50</b>C by transmitting and receiving the first image display information between the mobile terminals 3 without going through the server 4 . That is, when the mobile terminal 3 receives the first image display information from the other mobile terminal 3, the mobile terminal 3 reads information about the first image 51 displayed on the other mobile terminal 3 from the first image display information, and based on the information, The second image 52 is displayed in the second image display area 50C.

[3-2. Modification 2 (viewing a video alone)]
The above embodiment has been described as an example of a case in which a plurality of users viewing a moving image at the same time share an image expressing their feelings about the moving image. On the other hand, an image that expresses the emotion of a user who has previously viewed a certain video for the video can be shared with a user who views the video later. Accordingly, it is possible to display a combination image according to a combination of the additional operation by the first user who has viewed the image first and the additional operation by the second user who has viewed the image later.

Below, the timing at which the first user using the first mobile terminal 3 (first device) views a certain video (viewing target) is the timing for the second user using the second mobile terminal 3 (second device). is after the timing of viewing the moving image (FIG. 13(A)). In this case, the additional operation by the first user and the additional operation by the second user may not be closely related in real time. However, both are in a close relationship on the progress axis (timeline) of the moving image (FIG. 13(B)). The first user and the second user may be the person in question and another person, or may be the person in the past and the person in the present.

In Modified Example 2, a second user who is viewing a moving image displayed on the screen 3a (second screen) of the second portable terminal 3 adds a first image (second element) to the moving image. The operation information of the additional operation (second operation) performed on the mobile terminal 3a is transmitted from the second mobile terminal 3a to the server 4
(Second instruction means, shared information processing device). The server 4 provides second image display information (second instruction) for displaying a second image (second element) at a timing corresponding to the additional operation by the second user, when the same moving image is viewed. It is transmitted in advance to the first mobile terminal 3 used by the first user who wants to use it.

In order to improve entertainment, the first mobile terminal 3 allows a first user who is watching a moving image displayed on the screen 3a (first screen) of the first mobile terminal 3 to add a first image (first image) to the moving image. When an addition operation (first operation) performed on the first mobile terminal 3a to add a second image display information is received, the received addition operation (first operation) and the addition related to the second image display information A combined image (third element) based on a combination of operations (second operations) is displayed on the screen 3 a of the first portable terminal 3 . Note that, in Modification 2, since the timing of the second operation performed on the second mobile terminal 3 is known in advance, the received additional operation can be performed without delay after the first operation is received on the first mobile terminal 3 . It is also possible to display a combined image (third element) based on a combination of the (first operation) and the additional operation (second operation) related to the second image display information on the screen 3a of the first portable terminal 3. .

[3-3. Modification 3 (not displaying the first image when a combined image is displayed)]
In the above embodiment, the first image is displayed even when the combined image is displayed. Alternatively, the first image may not be displayed when the combined image is displayed.

Specifically, the image display control unit 70 (second control unit) of the mobile terminal 3 determines that a predetermined period (combination period) has elapsed since the addition operation was received by the first image display operation reception unit 72 (first reception unit). Wait until the time elapses to receive the combined image information. Then, when the combined image information is received within the predetermined time, the image display control unit 70 displays the combined image (the third element) based on the received combined image information, and receives the combined image information within the predetermined time. If not, the first image (first element) is displayed based on the received additional operation.

[3-4. Modification 4 (Displaying the second image even when the combined image is displayed)]
In the above embodiment, the second image is not displayed when the combined image is displayed. Alternatively, the second image may be displayed even when the combined image is displayed. Specifically, when receiving the first image information, the server 4 may generate the second image display information and transmit it to the mobile terminal 3 regardless of the presence or absence of the combined display operation.

[3-5. Modified Example 5 (Limiting Activation of Replacement Processing)]
In the above embodiment, the unit process is repeated every predetermined period (combination period), and a plurality of pieces of second image display information to be replaced are replaced with one piece of second image display information. On the other hand, the replacement process may be omitted in a situation where a sudden increase in load on the mobile terminal 3 is unlikely to occur.

Specifically, the replacement processing unit 67 (specifying means) of the server 4 replaces the number of elements of the adjacent element group including a plurality of pieces of second image display information (shared elements) whose display timings are close to each other when the activation condition is not satisfied. , a unit process (replacement process) for identifying two (predetermined number) of second image display information included in the adjacent element group as an image group (element group) is omitted. The activation condition is, for example, that the number of elements is equal to or greater than a predetermined threshold. The activation condition may be flexibly changed according to the processing capability of the mobile terminal 3 . For example, it is preferable to set a predetermined threshold according to the processing capability of the terminal having the lower capability among the plurality of mobile terminals 3 belonging to group 5 . The processing capacity is, for example, the display processing capacity (DSP clock frequency) and reception processing capacity (communication line speed) of the mobile terminal 3 .

1 video distribution system 2 communication line 3 portable terminal (an example of "first device", "second device", "information processing device")
3a screen (an example of "first screen" and "second screen")
4 Server (an example of "first instruction means", "second instruction means", and "shared information processing device")
5 Group (an example of "information processing system")
50A Delivery video display area 50B First image display area 50C Second image display area 51 First image 52 Second image (an example of "shared element" or "alternative element")
53 Display button 60 Group registration unit 61 Moving image distribution unit 62 First image reception information reception control unit 63 Second image display information generation unit 64 Second image display information transmission control unit 65 Combined image information generation unit 66 Combined image information transmission control unit 67 replacement processing part (an example of "specifying means" and "replacement means")
70 Image display control unit (an example of "first control means", "second control means", "third control means", and "display control means")
71 Display area setting unit 72 First image display operation reception unit (an example of “first reception means” and “second reception means”)
73 Operation state determination unit 74 First image display information generation unit 75 First image display information transmission control unit (an example of “first providing means” and “second providing means”)
76 Second image display information reception control unit (an example of "third reception means")
77 Combined image information reception control unit

Claims (8)

identifying means for identifying, as an element group, a predetermined number of the shared elements based on the timing from among the shared elements associated with the viewing target with respect to the timing of displaying the viewing target on the terminal of the user who is viewing the viewing target;
replacement means for replacing the element group specified by the specifying means with alternative elements in a number less than the predetermined number;
A shared information processing device comprising: The specifying means specifies the element group based on an additional attribute added to the shared element,
The replacement means adds the alternative element inheriting the additional attribute added to at least one of the plurality of shared elements included in the element group to the viewing target as the shared element.
A shared information processing apparatus according to claim 1 . Based on the additional attribute, the identifying means can identify at least one of a user group to which a user who associates the shared element with the viewing target belongs and an element group to which the shared element associated with the viewing target belongs. identifying the group of elements;
3. The shared information processing device according to claim 2. the identifying means identifies the element group based on the additional attribute that can identify both the user group and the element group;
4. The shared information processing device according to claim 3. The replacing means replaces the element group with the alternative element according to the number of overlaps of the types when the types of the predetermined number of the shared elements included in the element group identified by the identifying means are the same. ,
A shared information processing apparatus according to any one of claims 2 to 4. Unless the number of elements of a proximate element group that includes a plurality of the shared elements that are close in timing does not satisfy an activation condition, the specifying means selects the predetermined number of the shared elements included in the proximate element group as the element. identify as a group,
A shared information processing apparatus according to any one of claims 1 to 5. The identifying means identifies, as the element group, the predetermined number of shared elements included in the adjacent element group, except when the activation condition, which varies according to the processing capability of the terminal, is not satisfied;
7. The shared information processing device according to claim 6. a specifying function of specifying, as an element group, a predetermined number of the shared elements based on the timing from among the shared elements associated with the viewing target when the viewing target is to be displayed on the terminal of the user who is viewing the viewing target;
a replacement function that replaces the element group specified by the specified function with alternative elements less than the predetermined number;
A control program that makes the computer of the information processing device realize

Images