JP7373477B2 - Device, program, and method for generating online communication images using original images - Google Patents

Description

The present invention relates to online communication technology such as web conferencing and video conferencing.

In recent years, technology in which users communicate with each other via communication networks using their own terminals has been used in many situations around the world. For example, online communication such as e-mail, SNS (Social Networking Service), and even web conferencing and video conferencing have entered into the daily lives of users widely and have become almost indispensable technologies, whether for private or business purposes. There is.

Users who communicate using such communication technologies are usually located far apart from each other, so even if users who have participated in a web conference want to take a commemorative group photo, they often have to physically Group photography is not possible.

On the other hand, Patent Document 1 discloses a virtual photography service system that allows users in remote locations to enjoy virtual photography services online. Here, in this system, the main terminal activates Service Up by user operation and sends an invitation message to the terminals of multiple shooting subjects, and each terminal that receives the invitation message activates Service Up and The main terminal is connected to the terminal side, and the main terminal specifies the position of each person to be photographed according to the user's instructions, photographs the subject, and sends the photographed image to the server side.

The server then retrieves each subject's image from the received image using a predetermined object segmentation technique and inserts each retrieved image at a specified position on the final image to create an image. The completed image is then transferred to the terminal of each person to be photographed.

Further, Patent Document 2 discloses a group photo forming apparatus that forms a virtual group photo in which a plurality of users with whom a relationship has been established are the subjects when using SNS. Here, in this device, when using a posting log usage page of SNS, etc., an operation (shooting instruction) for the "shoot" button is accepted through the touch panel, and furthermore, the poster image acquisition unit receives the user's face of the user terminal device. The image data of the photo and the image data of the face photos of one or more other users (users with whom a friendship relationship has been established) whose posted information is posted on the usage page of the user are acquired from the SNS. Next, the group photo forming section forms one group photo using the image data of these acquired facial photos.

Furthermore, even without relying on the above-mentioned patent document technology, for example, in a known web conference system, it is possible to display the images of each participant side by side on one screen. It is also possible to activate the shot function and record a screen displaying participant footage. In addition, in a well-known online VR (Virtual Reality) conference, participants' avatars can be placed and displayed in a virtual space, and by using a virtual camera placed in the virtual space, avatars can be viewed from various angles, for example. It is also possible to take virtual photos.

JP 2017-513419 Publication Japanese Patent Application Publication No. 2014-048876

For example, consider a case where users who participated in an online conference want to obtain a commemorative group photo image. In this case, as a matter of course, there is a strong desire to generate a photographic image that has a sense of realism or live performance that looks like a photograph taken during a conference.

Furthermore, while online images displayed on screen during online meetings, for example, are usually of low resolution so that they can be processed online, commemorative group photo images are often of higher resolution and better quality. Something that is highly desired. Therefore, even if a screen shot of the online video is taken, for example, a desired high-resolution group photo image cannot be obtained.

In addition, for example, the technology described in Patent Document 1 is such that each participant individually photographs and generates a photo image while having a voice chat, sends it to a server, and the server synthesizes the photo images. As a result, it is extremely difficult to obtain a group photo image that has a realistic or live feel as described above.

Furthermore, in the technology described in Patent Document 2, a group photo image is generated by collecting face photo data already registered in SNS, so a group photo image that similarly has a realistic or live feeling is generated. It is not possible.

Furthermore, as described above, in known online VR conferences, it is certainly possible to generate a high-resolution group photo image of participants' avatars placed in a virtual space using a virtual camera. However, it is clear that the participants are not satisfied with having a CG (Computer Graphics) image of the avatars gathered together as a commemorative group photo image with a real or live feel.

Therefore, an object of the present invention is to provide a device, a program, and a method that can generate higher-resolution communication images related to online communication using images generated during the online communication.

According to the present invention, by receiving a stream of images obtained by reducing the resolution of original images generated by a plurality of terminals, and displaying an online image generated using the low resolution images generated from the image stream, A communication image generation device capable of generating communication images related to online communication to be carried out, comprising:
image generation management means that accepts or determines the timing of generating the communication image;
Original image acquisition means for acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. A communication image generation device is provided having an image generation means.

In the communication image generation device according to the present invention, the image generation means aligns the number of pixels, aspect ratio, or range of objects in the images between the original image and the low resolution image, and then It is also preferred to determine a scale value .

Further, as an embodiment of the communication image generation device according to the present invention, the image generation means matches the number of pixels, aspect ratio, or range of objects in the images between the original image and the low resolution image. It is also preferable to determine the scale value above .

Furthermore, in the communication image generation device according to the present invention , it is also preferable that the image generation management means notifies the terminal of the user participating in the online communication of the countdown information toward the received or determined timing.

Further, as another embodiment of the communication image generation device according to the present invention, the image generation means includes:
receiving or determining generation plan information that is information related to at least one of the type, structure, image configuration, and settings in the image of the communication image to be generated;
Send the generation plan information to the terminals of users participating in the online communication,
Receiving response information determined by a user to whom the generation plan information was presented and received from a terminal related to this user, the response information relating to consent, selection, or change to the content of the generation plan information;
It is also preferable to generate the communication image based on the generation plan information and the response information.

Furthermore, in the other embodiments described above, the image generating means includes:
Before receiving the response information, the communication image plan as the online image, which uses the low resolution image and is generated based on the generation plan information, is sent to the user participating in the online communication. Send it to the terminal and present it to this user,
After receiving the response information, the communication image proposal generated using the low-resolution image and based on the generation proposal information and the response information received so far is sent to the terminal of the user participating in the online communication. It is also preferable to have the user make the presentation.

Furthermore, in the above-mentioned embodiment of the communication image plan using the low-resolution image, the communication image plan includes a background layer including a background, and a participant image layer related to the participants of the online communication, and the low-resolution image. It is also preferred to have a multi-layered structure including at least one participant image layer with high resolution images.

Further, as an embodiment of the communication image according to the present invention, the communication image includes at least one background layer including a background and a participant image layer related to participants of the online communication using the original image. It is also preferred to have a multi-layer structure including two participant image layers.

Furthermore, as still another embodiment of the communication image generation device according to the present invention, the image generation means receives comment information related to the user's comment received from a terminal related to the user participating in the online communication, and It is also preferable to generate the communication image including comment information.

Furthermore, as yet another embodiment of the communication image generation device according to the present invention, the image generation means is configured to generate audio data and/or audio data generated by the user and/or received from a terminal related to the user participating in the online communication. It is also preferable to receive audio and/or video information related to video data and generate a communication image in which a link to the audio and/or video information is embedded.

Furthermore, as yet another embodiment of the communication image generation device according to the present invention, the communication image generation device includes:
an identifier storage unit that stores an identifier for an electronic transaction service that can provide objects and/or data related to the generated communication image, and that stores an identifier of a user participating in the online communication;
further comprising an electronic transaction access means that uses the identifier to instruct the server associated with the electronic transaction service to provide objects and/or data related to the generated communication image to the user associated with the identifier; It is also preferable to have

According to the present invention, a stream of images obtained by reducing the resolution of original images generated by a plurality of terminals is received, and an online image generated using the low-resolution images generated from the stream of images is displayed. A communication image generation program that causes a computer installed in a device capable of generating communication images related to online communication to function,
image generation management means that accepts or determines the timing of generating the communication image;
Original image acquisition means for acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. A communication image generation program is provided that causes a computer to function as an image generation means.

According to the present invention, a stream of images obtained by reducing the resolution of original images generated by a plurality of terminals is further received, and an online image generated using the low-resolution images generated from the stream of images is displayed. A communication image generation method carried out by a computer installed in a device capable of generating communication images related to online communication carried out by
receiving or determining the timing of generating the communication image;
acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. A communication image generation method is provided, comprising the steps of:

The communication image generation device, communication image generation program, and communication image generation method of the present invention can generate higher resolution communication images related to online communication using images generated during the online communication. .

1 is a schematic diagram showing an online communication service providing system including an embodiment of a communication image generation device according to the present invention. FIG. 2 is a schematic diagram for explaining a communication image proposal and an embodiment of the communication image according to the present invention. 1 is a sequence diagram schematically showing an embodiment of a communication image generation method according to the present invention. FIG. 1 is a sequence diagram schematically showing an embodiment of a communication image generation method according to the present invention. FIG.

Hereinafter, embodiments of the present invention will be described in detail using the drawings.

[Communication image generation device/system]
FIG. 1 is a schematic diagram showing an online communication service providing system including an embodiment of a communication image generation device according to the present invention. Incidentally, FIG. 1 also includes a functional block diagram showing the functional configuration of an embodiment of the communication image generation device.

The online communication (OC) service providing system of this embodiment shown in FIG.
(a) A plurality of terminals 2 related to a plurality of users,
(b) Providing an OC service as a communication image generation device according to the present invention, which provides an OC service, such as a web conference/meeting service or a video conference/meeting service, to (the terminals 2 of) the plurality of users in (a) above. Server 1 and
(c) Provides services related to electronic transactions (including electronic commerce (EC) services) to the plurality of users (terminals 2) in (a) above, and also serves as the OC service providing server 1 in (b) above. It is equipped with an EC server 3 with which data can be exchanged.

Among these, the OC service providing server 1 in (b) above is
(b1) A low-resolution image obtained by reducing (for example, encoding) the "original image" generated by the multiple terminals 2 (cameras built into, attached to, or connected to) in (a) above. Receive a stream, specifically a video stream compressed (lower resolution) using an irreversible compression method such as H.264, MPEG-2, MotionJPEG, or DivX,
(b2) Generate an "online image" (and online video as time-series data) from a "low resolution image" generated from the received image (video) stream (for example, obtained by decoding the stream) Then, the "online image" is displayed on the screen of each terminal 2 in (a) above (for example, each participating user displays the online video projected in each participant window), and the audio of each participating user is also displayed. We provide OC services that output in real time, and
(b3) It is possible to generate a "communication image" related to online communication performed in this OC service providing system, for example, a group photo image of users who participated in an online birthday party (web meeting) via terminal 2. It is a device.

Here, in order to generate the "communication image" (for example, a group photo image) of (b3) above, the OC service providing server 1 specifically:
(A) an original image acquisition/management unit 111 that acquires the “original image” of (b1) above from the terminal 2;
(B) Add the "original image" to the "low-resolution image" portion of the "online image" in (b2) above (for example, the portion containing a predetermined object such as the user's "face" or "upper body"). It has a communication image generation unit 112 that generates a "communication image" by applying the communication image.

In this way, the OC service providing server 1 generates a "communication image" using an "original image" that is the source data for generating an "online image" (online video) displayed during online communication. Here, this "original image" naturally has a higher resolution than the "low resolution image". That is, the OC service providing server 1 can generate a higher resolution "communication image" using the image (original image) generated during the online communication.

In addition, this also makes it possible to generate a high-resolution group photo image (communication image) that has a realistic or live feel that looks like a photo taken during an online birthday party, for example. Of course, it is also possible to go beyond realistic group photo images and create original group photo images more suitable for online birthday parties.

By the way, the "online image" (online video) in (b2) above is generated from a "low resolution image", and for example, an "online image" (online video) of a group of users participating in online communication, For example, it is also possible to record using the screenshot function of the terminal 2. In this case, only a low-resolution (low-quality) group photo image can be obtained, although it does give a sense of realism or live performance.

On the other hand, according to the OC service providing server 1, for example, as mentioned above, it is also possible to generate a high-resolution group photo image (communication image) with a realistic or live feeling. It is also possible to distribute group photo images (communication images) suitable for commemorating and recording online ceremonial occasions and online meetings to participants, attendees, and attendees. Furthermore, for example, a high-resolution two-shot photo image from an online live meet-and-greet can be sent to the relevant person.

In addition, the need for such "communication images" according to the present invention is due to the trend toward more online interaction (such as online meetings, online ceremonial occasions, online drinking parties, online field trips, etc.). It is expected that this will continue to increase as the world continues to change. That is, for example, instead of a conventional commemorative group photo that requires people to physically gather together among people who have interacted online, the "communication image" according to the present invention can be used as a commemoration or record or to create a sense of closeness and solidarity. It is thought that the demand for generating information as a tool to enhance one's senses will continue to increase in the future.

Note that, as one preferable aspect, the OC service providing server 1 also preferably generates a "communication video" from a time-series group of "communication images" generated moment by moment. For example, a ``communication video'' may be generated as a recorded video of a scene from a web birthday party (for example, a scene where everyone congratulates each other by saying ``Congratulations!''). It is also preferable to add an audio message or a video message to the generated "communication image" to make the "communication image" into multimedia. This multimedia conversion will be explained in detail later when the multimedia conversion unit 112b (FIG. 1) is described.

In addition, as another embodiment, the OC service providing server 1 cooperates with the EC server 3 described in (c) above to provide a "communication image" distribution service (for example, to commemorate an online birthday party) to the user of the terminal 2. It is also preferable to provide a group photo printing/distribution service. This distribution service will be explained in detail later when the EC server access unit 123 (FIG. 1) is described.

Furthermore, the service providing device according to the present invention is not limited to the OC service providing server 1 as described above, but, for example, the service providing device according to the present invention can communicate with the OC service providing server that constructs the OC service providing system according to the present invention. It is also possible to use an external device capable of data communication via an API (Application Programming Interface) or the like. In this case, this device (as a service providing device according to the present invention) can be a device including at least the original image acquisition/management section 111 (FIG. 1) and the communication image generation section 112 (FIG. 1).

Further, the service providing device according to the present invention can be a device having both the functions of an EC server and an OC service providing server. In this case, it is also possible to implement a "communication image" distribution service without cooperating with the EC server 3 as described above.

Furthermore, regarding the terminal 2 that generates the "original image", the terminal 2 may generate (shoot) the "original image" with a built-in, attached, or connected camera, and may also In order to capture the "original image" with higher resolution (higher quality), it is equipped with a flash or other shooting light, and when shooting, the brightness of the screen displaying the "original image" or "online image" must be set above a certain level. It is also preferable to have a function to increase the white area in the display area of the screen. Furthermore, the generated (photographed) "original image" may be provided with an original image processing function, such as adding a production effect equivalent to the lighting at the time of shooting, or converting and adjusting brightness and contrast. .

Furthermore, when photographing the "original image" including the participating users, the photograph may be taken against a green background, blue background, or the like. This facilitates the process of cutting out the "face", "upper body", etc. (predetermined objects) of the participating users from the "original image", which will be performed later.

[Device functional configuration]
Similarly, according to the functional block diagram of FIG. 1, the OC service providing server 1 as a communication image generation device according to the present invention includes a communication interface section 101, an OC service information storage section 102, an original image storage section 103, and a communication image generation device. It has a storage unit 104, a corresponding ID storage unit 105, a keyboard (KB) 106, a display (DP) 107, and a processor memory.

Here, this processor memory stores an embodiment of the communication image generation program according to the present invention, further has a computer function, and executes the communication image generation process by executing the communication image generation program. implement. Also, from this, the OC service providing server 1 (communication image generation device) can be a dedicated device for the communication image generation process, but it can also be a general-purpose cloud server equipped with the communication image generation program according to the present invention or a non-standard device. It may be a cloud type server, and furthermore, it may be a personal computer (PC), a notebook type computer, a tablet type computer, or the like.

Furthermore, the above-mentioned processor memory includes an original image acquisition/management unit 111, a communication image proposal generation unit 112a (including a jumble layer generation unit 112aa and a response information reflection unit 112ab), and a multimedia conversion unit 112b. It includes a generation section 112, an image generation management section 113 including a countdown information generation section 113a, a communication control section 121, an OC service implementation section 122, an EC server success section 123, and an input/output control section 124. These functional components can be considered as functions of a communication image generation program stored in the processor memory. Further, the process flow shown by connecting the functional components of the OC service providing server 1 (communication image generation device) with arrows in FIG. 1 can be understood as an embodiment of the communication image generation method according to the present invention. .

Similarly, in the functional block diagram of FIG. 1, the OC service implementation unit 122 provides OC service provision performed by a known OC service provision server, such as a web conference (meeting) service provision server or a video conference (meeting) service provision server. This is a functional component that implements general known processing for Here, this known processing includes a participant window containing images of the "face" and "upper body" of participating users participating in a conference (meeting), for example, from image (video) streams received from a plurality of terminals 2. It also includes the process of generating an "online image" (online video) in which images are arranged, and distributing it to the participating users' terminals 2 and displaying it on their screens.

Furthermore, information such as the service identifier (ID) of the OC service provided, the date and time of the event, the participant ID of the participant in the OC, the address information of the terminal 2, etc. "images" and "online images" (online videos) are stored and managed in the OC service information storage section 102, and the OC service implementation section 122 extracts necessary information from the OC service information storage section 102 as appropriate. It carries out OC service provision processing.

In addition, the input/output control unit 124 receives instructions from the server administrator for operating and managing the OC service providing server 1 via the keyboard 106, and outputs the instructions to necessary locations within the server. It also accepts information from various locations within the server and presents the information to the server administrator via the display 107.

Similarly, in the functional block diagram of FIG. 1, the original image acquisition/management unit 111 receives the OC service provided by the OC service implementation unit 122 (therefore, in order to perform online communication, the original image acquisition/management unit 111 receives the OC service provided by the OC service implementation unit 122). The "original image" from which the image stream is generated is separately obtained from the plurality of terminals 2 (which are transmitting the image stream), and is appropriately stored in the original image storage unit 103 for management. Note that the "original image" obtained here naturally has a higher resolution than the "low resolution image" or "online image" (online video) used in online communication, and has higher image quality. It is an image of.

In addition, regarding the manner and timing of acquiring this "original image", for example, at a preset time, or at the timing when a shooting instruction is received from the terminal 2 of the host participating user who is the host, "communication Only one frame of the "original image" may be received from the user's terminal 2 related to the "image". It is also possible to continue receiving one frame of the "original image" at a given time period periodically at predetermined time intervals (for example, 2 seconds). However, in this embodiment, the original image acquisition/management unit 111 adopts a more suitable acquisition mode and acquisition timing according to instructions from the image generation management unit 113 (FIG. 1), which will be described next.

Similarly, in the functional block diagram of FIG. 1, the image generation management unit 113 accepts or determines the timing to generate the intended “communication image”, and instructs the original image acquisition/management unit 111 to determine the timing based on the timing. An instruction is given to acquire multiple "original images" related to a time interval.

More specifically, in this embodiment, the image generation management unit 113:
(a) Receive a generation instruction from the terminal 2 of the host participating user, or accept generation instruction input by the server administrator as the host via the keyboard 106 and input/output control unit 124;
(b) Based on the received generation instruction (input), determine the ``generation timing'' of the ``communication image'' (for example, the point in time after a predetermined period of time (for example, 5 seconds) has elapsed from the time the generation instruction (input) was received). (Determine "generation timing"),
(c) The countdown information generation unit 113a generates countdown information toward the determined "generation timing" (for example, "5", "4", "3", "2", "1" displayed every second) ", and "0" that is displayed exactly at the "generation timing" (Figure 2).
(d) Send/notify and display the generated countdown information (along with the communication image proposal described later) to each terminal 2 of users participating in online communication,
(e) For the original image acquisition/management unit 111, a predetermined time before and after the "generation timing" (displayed as "0") (for example, from 0.5 seconds before the "generation timing" to 0.5 seconds before the "generation timing") Each terminal 2 acquires an "original image" (for example, several tens of frames) generated in a time interval up to a point in time seconds later.

Here, the reason for acquiring multiple "original images" (for example, several tens of frames) in (e) above is to take into account the delay between the client (terminal 2) and the server (OC service providing server 1). However, this point will be explained later in the communication image generation section 112 (FIG. 1).

Similarly, in the functional block diagram of FIG. 1, the communication image generation unit 112 normally generates a low-resolution image in the “online image” (online video) that is always displayed on the screen of the terminal 2 (while online communication is being performed) The "original image" is applied to an image part that includes a predetermined object, such as the user's "face" or "upper body" in this embodiment, and Generates a ``communication image'' that includes the ``face'' and ``upper body''.

More specifically, in this embodiment, the communication image generation unit 112:
(a) A predetermined object (user's For the "low-resolution image" used to generate the part including the "face" or "upper body", an "original image" that is similar to the "low-resolution image" that satisfies a predetermined condition is selected,
(b) A "communication image" is generated using the selected "original image".

Incidentally, in (a) above, the "online image" related to the determined "generation timing" is read out from the OC service information storage unit 102, while the plurality of "original images" that are candidates are read out from the original image storage unit 103. After that, the two are matched and compared. This eliminates the time lag between the client (terminal 2) and server (OC service providing server 1) due to delays between the two, and for example, even if a participant makes a sudden movement around the "generation timing", the situation will change. It is also possible to reflect this in the ``communication image.'' In addition, it is also possible to obtain a virtual group photo image of a group of participating users gathered at the same virtual location at the exact "generation timing," for example, a realistic photo that looks like a photo taken during online communication. It also becomes possible to generate high-resolution "communication images" that have a realistic or live feel.

Here, the ``original images that are similar enough to satisfy a predetermined condition'' in (a) above means, for example, the ``original images that are similar enough to satisfy a predetermined condition'' are defined as ``the most similar original images according to this metric'' using a publicly known scale for measuring the degree of similarity between various images. It can also be referred to as "the original image that can be said to exist." In addition, it is also possible to select "original images (randomly selected ones, those whose generation time is the earliest, or those whose generation time is closest to the 'generation timing')" that have the scale value equal to or higher than a predetermined threshold. It is.

In this regard, in this embodiment, as a measure of the degree of similarity between images, PSNR (Peak Signal-to-Noise Ratio), which is a well-known measure of image deterioration due to irreversible compression, and SSIM (Structural Similarity). Specifically, the communication image generation unit 112
(a1) The "low-resolution image" used to generate the image part containing the predetermined object (the user's "face" or "upper body") in the online image related to the determined "generation timing" and the "low-resolution image" Calculate the PSNR value or SSIM value between the plurality of "original images" acquired from the terminal 2 that is the transmission source of the "low resolution image",
(a2) The "original image" with the largest PSNR value or SSIM value is selected.

Here, in order to appropriately calculate the PSNR value or SSIM value, the resolution of the "original image" may be lowered to match the number of pixels of the "original image" to the number of pixels of the "low resolution image". Additionally, if necessary, image analysis technology based on machine learning is used to crop, enlarge, or reduce the "original image" so that the captured area is the same between the "original image" and the "low-resolution image." You may. Furthermore, it is also preferable to generate these images so that the aspect ratio and range within the image of the "original image" are the same as those of the "low resolution image". Incidentally, compared to PSNR, SSIM suppresses excessive value fluctuations due to pixel shift, and is said to be able to more accurately capture differences and changes in object shape between comparison images. That is, SSIM is a more suitable measure for understanding the degree of similarity between the "low resolution image" and the "original image" in this embodiment.

Next, the communication image generation unit 112 generates a “communication image” using the selected “original image”. In this embodiment, in generating the “communication image”,
(a) Receive or determine "generation plan information" which is information related to at least one of the type, structure, image configuration, and settings in the image of the "communication image" to be generated;
(b) Send "generation plan information" to the terminals 2 of participating users participating in online communication,
(c) "Response information" determined by the user to whom the "generation plan information" was presented and received from the terminal 2 of the participating user, which is related to consent, selection, or change to the contents of the "generation plan information"; response information” is received,
(d) A "communication image" is generated based on the "generation proposal information" and the "response information." As a result, it is also possible to generate a "communication image" of type, structure, image composition, and settings within the image that are suitable, satisfactory, or can be said to be a consensus among the participating users.

By the way, among the information that constitutes the "generation proposal information" mentioned above,
・Type refers to the category of "communication images" (e.g., group photo images) that corresponds to the type of online communication being conducted (e.g., meetings, ceremonial occasions, drinking parties, field trips, etc.) often,
- It is also preferable that the structure includes information on a multilayer structure (layer structure) as explained later using FIG.
- The image configuration may include the shape, arrangement, transmittance, etc. of the display area in each layer that makes up the "communication image" (group photo image),
・For the image settings of the participant image layer (Figure 2), do you want to cut out only the participant user's "face" or "upper body" (predetermined target) and make other areas transparent, or set the layer display area? Settings such as whether to cut out the shape and make other areas transparent, settings such as the enlargement/reduction ratio (magnification) of the display area in the participant image, the tilt of the participant image, the transparency, etc. is effect settings for participant images (for example, image filter settings such as soft focus, facial skin texture correction settings, and advanced conversion settings such as gender conversion and conversion to superimpose decorations such as headgear on people) etc.) is also preferably included.
・In-image settings for the jumbled layer (Figure 2) include settings such as the enlargement/reduction ratio (magnification) of the jagged display area, the tilt of the jagged image, transparency, etc., and effect settings (text character size, font, etc.) (settings such as type, decoration type, pen type in the case of handwritten characters, etc.) may also be included.

Furthermore, regarding "generation plan information", settings are made so that one can be selected according to the type of online communication from among multiple "generation plan information" (presented) that has the same number of participant image layers as participating users. It is also preferable that the number of participant image layers be automatically determined according to the number of participating users. Furthermore, "generation plan information" generated in the past may be stored, and a setting may be made such that participating users can select one of these and use (present) it again. .

In addition, in such a form in which "generation plan information" and "response information" are exchanged, the communication image plan generation section 112a of the image generation section 112,
(b') Before receiving the "response information" (at the stage (b) above), the "communication image proposal" is an "online image" and uses a "low resolution image", and the above (a) The "communication image proposal" generated based on the "generation proposal information" is transmitted to the terminal 2 of the participating users participating in the online communication and presented to the participating users,
(c') After receiving the "response information" (as in (c) above), the "low-resolution image" is used, and based on the "generation proposal information" and the "response information" received so far, It is also preferable that the "communication image proposal" generated by the response information reflection unit 112ab be transmitted to the terminals 2 of the participating users and presented to the participating users.

In this way, by displaying the "communication image proposal" as an "online image" (an image displayed during online communication) on the screen of the terminal 2, participating users of the terminal 2 can According to the "proposal information", it is possible to visually confirm the content of the communication image to be realized, and as a result, it is also possible to generate more appropriate "response information".

Furthermore, it is also preferable that the above-described exchange of "generation plan information" and "response information" be repeated as appropriate with each participating user. For example, as an example of exchanging intuitive response information, each participating user can tap or drag the "communication image proposal" displayed on the touch panel display of terminal 2 to select the location where they would like to appear. It may also be possible to select and specify the range. Further, such selection/designation among participating users may be adopted on a first-come, first-served basis; for example, if the specified ranges overlap, the image of the participating user specified first is set to be in front (on the upper side). It is also preferable that

Furthermore, participating users may be able to adjust the enlargement/reduction ratio (magnification), tilt, transmittance, etc. of their own images by operating a slide bar displayed on the terminal 2. In addition, by selecting and operating the displayed effect menu, you can set effects for your own image (for example, setting image filters such as soft focus, correction settings for facial skin texture, advanced conversion settings such as gender conversion, etc.) ) is also preferred.

In any case, by displaying the "communication image proposal" that appropriately reflects the "response information" as an "online image" in real time, the advanced settings and adjustments described above can be made among the participating users. It becomes possible to do so. For example, participating users can check the constantly updated "communication image proposal" in real time and create the final "communication image" together (for example, while having fun together). It is.

Furthermore, in the present embodiment, the communication image proposal generation unit 112a (the jumbled layer generation unit 112aa) generates images received from the terminals 2 of participating users participating in online communication, which will be explained in detail later with reference to FIG. , it is also possible to receive "comment information" (for example, messages) related to comments from participating users and to generate a "communication image" including the "comment information".

As a result, it is possible to generate, for example, a "communication image" in which a message (comment) of each participating user is displayed near the "face" or "upper body" of each participating user. Here, this comment may be handwritten data or may be text data including specifications of font, size, etc. Of course, it is also possible to generate a "communication image" that includes "comment information" such as cross-letters, without relying on the "communication image proposal." Note that the specific contents of the process using such "comment" information will be explained later using FIG. 2.

Furthermore, in this embodiment, the communication image generation unit 112 (the multimedia generation unit 112b of the communication image generating unit 112) generates images received from the terminals 2 of the participating users who participate in online communication (for example, using the microphone or camera of the terminal 2). It is also preferable to receive "audio and/or video information" related to the generated audio data and/or video data and generate a "communication image" in which a link to the "audio and/or video information" is embedded.

As a modification, this "audio and/or video information" may include image data in "online images" and "communication image plans" that are displayed in real time (temporarily saved) during online communication, and image data that is saved in parallel. It is also possible to generate the audio data from existing audio data. For example, among such voice data related to a certain participating user, voice data related to utterances that have a predetermined characteristic or more (for example, repeated more than a predetermined number of times) are extracted using a known voice recognition technology, and the voice The data may be acquired as "voice information" of the participating user.

In any case, a "communication image" in which a link to such "audio and/or video information" is embedded is, so to speak, a high-resolution communication image turned into multimedia content.

Therefore, for example, a high-resolution group photo image (communication image) showing the "faces" and "upper bodies" of all participating users who participated in an online birthday party is displayed on the touch panel display of the terminal 2, and participation in this displayed image is displayed. By tapping the image part of the user's "face" or "upper body," it is also possible to output the voice or video message of the participating user (from the speaker of terminal 2 or to the touch panel display). It is. Note that the specific contents of such multimedia processing will be explained later using FIG. 2.

Furthermore, as a modification, multimedia content (communication images) as described above is stored and managed on an external web server, and participating users can access this web server using their own terminals 2. The settings may be such that the multimedia content can be accessed and played back and enjoyed using a web browser installed in the terminal 2. Furthermore, such multimedia content may be set to be automatically played.

The various communication image generation processes that can be performed by the communication image generation unit 112 have been described above, but the “communication image” generated in this way (and also, for example, the “communication image proposal” generated in the middle), It is stored in the communication image storage unit 104, read out from the communication image storage unit 104 as appropriate or in response to a request from (the terminal 2 of) the participating users, and is transmitted via the communication control unit 121 and the communication interface unit 101 (all The information may be distributed to the terminal 2 (of the participating user or the requester).

In another preferred embodiment, the OC service providing server 1 uses the EC server 3 with which it cooperates in providing services, and transfers "communication images" to electronic transactions (in the sense of including electronic commerce (EC) services). It can also be provided as a service. In this case, the EC server 3 provides an EC service that provides objects (e.g., photo prints or printed matter of the image) and/or data (e.g., electronic data of the image) related to the generated "communication image" as the content of the cooperation. It is a server that can implement.

In addition, the corresponding ID storage unit 105 (FIG. 1) of the OC service providing server 1 stores the EC_ID, which is the identifier of the participating user (participating in the online communication) regarding such an EC service, from the EC server 3, for example. Acquired, stored, and managed with user approval.

Furthermore, the EC server access unit 123 (FIG. 1) is an electronic transaction access means, and uses the EC_ID (of the participating user) read from the corresponding ID storage unit 105 to send the generated EC_ID to the EC server 3. Objects (photo prints, printed matter, etc.) and/or data (image electronic data, etc.) related to the "communication image" are provided to participating users associated with the EC_ID. This makes it possible to provide an online print/distribution service, such as delivering a commemorative group photo of participating users virtually gathered online (for example, in a frame).

In addition, as a modification mode, when participating users who participate in the online communication are connected to each other (have a "friend" relationship) on a certain SNS (Social Networking Service), the corresponding ID storage unit 105 stores the EC_ID for each participating user. It is also preferable to store and manage the information and the NS_ID, which is the identifier of the participating user in the SNS, in association with each other.

In this case, the EC server access unit 123 uses the EC_ID searched (from the corresponding ID storage unit 105) with the NS_ID of the target participating user to request the EC server 3 to send information related to the generated “communication image”. (photo prints, printed matter, etc.) and/or data (image electronic data, etc.) may be provided to participating users associated with the NS_ID. In addition, the request to receive such an EC service from the participating user (terminal 2) via the SNS, or the request to receive such an EC service via the SNS may be received. It is also possible to notify the participating users (their terminals 2).

In this case, the participating user can enjoy the EC service without the privacy information such as address and payment information necessary for receiving the EC service being known by the OC service providing server 1 or the planning user of the EC service. can do.

In addition, the planning user (terminal 2) of the EC service may decide the EC service provision plan (for example, the size and image quality of the group photo to be distributed, the type of frame or booklet in which the group photo will be placed, delivery date and time, etc.) A proposal regarding who should be the settler (the person who pays the price, etc.) is notified to each participating user (terminal 2) via the OC service providing server 1, and the proposal is sent to each participating user (terminal 2) according to the notified response from the participating user. It is also preferable to determine the specific content of the EC service. In addition, each participating user can plan an EC service (for example, a print/distribution service for another image) at any time that is different from the EC service (for example, a print/distribution service) planned by the planning user. It's okay.

Furthermore, the EC server access unit 123 may cause the EC server 3 to deliver a recording medium on which multimedia content as a multimedia "communication image" as described above is recorded to the participating users. It is also preferable to have a frame device loaded with such a recording medium and capable of reproducing (eg, displaying) multimedia content, for example, delivered.

[Communication images, image proposals]
FIG. 2 is a schematic diagram for explaining a communication image proposal and an embodiment of the communication image according to the present invention. Here, the "communication image proposal" and the "communication image" of this embodiment each have a multilayer structure (layer structure).

First, the "communication image proposal" is specifically:
(a) 1 background layer,
(b) participant image layers as many as the number of participating users (provided for each participating user);
(c) As many layers as there are participating users (provided for each participating user), and (d) one countdown layer are superimposed, creating a structure in which one image is composed of these layers. . Note that the superimposition order of these layers is not limited to that shown in FIG. 2 and can be set arbitrarily.

Here, the background layer in (a) above is a layer representing a background image agreed upon in "generation plan information" (communication image plan setting information) set by the host participating user, for example, and is prepared in advance. The background image template may be selected from a large number of background image templates.

In addition, the participant image layer in (b) above is a layer generated by layer-converting the "low-resolution image" of the participating user, for example, the "face" or "upper body" of the participant ( It is a target image layer that represents a predetermined object) and makes everything else transparent. By superimposing these participant image layers, it is possible to generate, for example, an image in which the "faces" or "upper bodies" of all participating users are lined up.

Here, the participating user's "low resolution image" is obtained by decoding the image stream (video stream) generated by the participating user's terminal 2 by encoding (irreversible compression) the "original image" after receiving it. This is the image data that is displayed. In addition, the layer conversion performed on this "low-resolution image" is based on the setting content (for example, "face" or "upper body") determined in the "generation plan information" set by the host participating user. Image processing can be performed according to the target to be cropped in the image, the location, size and enlargement rate of the target in the image, the position in the superimposed layers (degree of depth in the image of the participating user), etc. . Incidentally, the extraction of a target (predetermined target) in an image such as a "face" or "upper body" is performed using a known target detector or background subtraction algorithm.

Furthermore, as in this embodiment, by providing a separate participant image layer for each participating user, for example, the setting contents agreed upon in "generation proposal information" and the setting contents changed based on "response information" can be displayed to each participating user. Even if the images are different from each other, the corresponding image processing can be performed appropriately.

Furthermore, the message layer (c) above is generated by the message layer generation unit 112aa (FIG. 1) as described above, and received from the terminal 2 of the participating user, comment information (for example, message information) related to the comment of the participating user. This layer represents text (text). By overlapping these scribble layers with each other and with the participant image layer group, it is possible to generate, for example, an image in which the scribbles by the participating user are displayed near the "face" or "upper body" of the participating user. It is. Here, the display position of this comment may also be set in advance in the above-mentioned "generation proposal information".

Further, the countdown layer (d) above is a layer generated in the countdown information generation unit 113a (FIG. 1) as described above, and is a layer that contains countdown information toward the determined "generation timing", specifically, for example, This layer represents count numbers such as "5", "4", "3", "2", "1", which are displayed every second, and "0", which is displayed at the "generation timing". ing.

Similarly, in Figure 2, next, the "communication image" is specifically:
(e) one high-resolution background layer,
(f) high-resolution participant image layers as many as the number of participating users (provided for each participating user);
(g) As many layers as there are participating users (provided for each participating user), and (h) one link layer are superimposed to form one image. . Note that the superimposition order of these layers is not limited to that shown in FIG. 2 and can be set arbitrarily.

Here, the high-resolution background layer in (e) above is created by taking into account the above-mentioned "response information" received from the terminal 2 of the participating user (who has confirmed the "communication image proposal" including the background layer in (a) above). This is the background layer that was finally determined. This high-resolution background layer differs from the background layer in (a) above, which is delivered in real time during online communication, and uses a higher-resolution background image template that is prepared in association with the background image template that has been decided to be adopted. It can be the layer used.

In addition, the high-resolution participant image layer in (f) above is based on the above-mentioned "response information" received from the terminal 2 of the participating user (who has confirmed the "communication image proposal" including the participant image layer in (b) above). This is the participant image layer that was finally determined by taking into account the following. This high-resolution participant image layer is different from the participant image layer in (b) above, which is distributed in real time during online communication, and is different from the participant image layer in the "communication image proposal" approved by the participating users. The parts to be cut out (predetermined targets) such as "upper body" and "upper body" are replaced with the "original image" which has been subjected to similar layer conversion.

Here, this "original image" is the most similar ( For example, it can be the "original image" (indicating the largest PSNR value or SSIM value). By adopting a high-resolution participant image layer that uses such an "original image", it is possible to create a higher-resolution image layer that reflects the appearance of the actual participating users at the "generation timing" (for example, with a realistic or live feeling). It also becomes possible to generate "communication images."

Incidentally, in this embodiment, the communication image generation unit 112 (FIG. 1) converts the above-described high-resolution participant image layer (f) into one that has been correctly converted from the participant image layer (b) above. You can also check whether it is. In this case, PSNR and SSIM, which are known as image degradation evaluation scales that were also used in the "original image" selection process, are applied, and the PSNR and SSIM values for the high-resolution participant image layer and the participant image layer are If it is greater than or equal to a predetermined threshold, it may be determined that the conversion is correct.

Furthermore, the jumble layer in (g) above may be the same layer as the jumble layer in (c) above, or it may take into account the above-mentioned "response information" received from the terminal 2 of the participating users (for example, the jumble display position may be changed). (after changing), the finally determined jumble layer can be used. In any case, by superimposing these scribble layers on each other and on a group of high-resolution participant image layers, for example, the scribbles by the participant can be displayed near the participant's "face" or "upper body." It also becomes possible to generate high-resolution "communication images."

Further, the link layer (h) is a layer generated by the multimedia conversion unit 112b (FIG. 1) described above, and is a layer that is associated with audio data and/or video data generated by participating users. It is a layer in which link information to "or video information" is embedded in a predetermined position, for example, a position corresponding to the image part of the "face" or "upper body" of the participating user.

Here, specifically, such link embedding processing is performed by, for example, linking using HTML (Hyper Text Markup Language) applying CSS (Cascading Style Sheets) or JavaScript (registered trademark), SMIL (Synchronized Multimedia Integration Language), etc. Describe the layer and link information such as the URL (Uniform Resource Locator) of "audio and/or video information" stored on the OC service providing server 1 or an external storage device using the functions of such a language. This can be achieved by embedding it in the link layer.

[Communication image generation method]
3 and 4 are sequence diagrams schematically showing an embodiment of the communication image generation method according to the present invention.

First, steps S101 to S205 of this embodiment will be sequentially explained using FIG. Initially, an online birthday party is held as online communication provided by the OC service providing server 1 between host participating user A (terminal 2A) and other participating users B to D (terminals 2B to 2D). Suppose that the event is being held. That is, the participating users A to D use the terminals 2A to 2D equipped with the OC program to participate in the online birthday party provided by the OC service providing server 1 with the OC_ID, which is the identifier of this online birthday party. Assume that you are accessing it.

(S101) In order to generate and distribute multimedia content including birthday party commemorative photos as “communication images,” host participating user A (terminal 2A) first receives “generation plan information” from OC service providing server 1. ” (communication image plan setting information), generation plan setting item information (for example, image plan template) that defines setting items, etc. is read out.
(S102) The host participating user A (terminal 2A) generates “generation plan information” based on the read generation plan setting item information and transmits it to the OC service providing server 1.

(S103) The OC service providing server 1 generates a "communication image plan" using the corresponding "low resolution image" based on the received "generation plan information", and uses the "communication image plan" during online communication. The image is displayed on each screen of the terminals 2A to 2D as an "online image" that is displayed in real time.
(S104) Participating users B to D (terminals 2B to 2D) who have confirmed the "communication image plan" on their own terminal screens and who have also confirmed the "generation plan information" sent from the OC service providing server 1, The “response information” is sent to the OC service providing server 1.

(S105) The OC service providing server 1 updates the "communication image plan" that is being generated in real time based on the received "response information", and displays the updated "communication image plan" in real time during online communication. The "online image" is displayed on each screen of the terminals 2A to 2D.

Here, host participating user A (terminal 2A) presents "generation plan information", and the "online image" becomes a "communication image proposal", and participating users B to D (terminals 2B to 2D) A series of steps (steps S102 to S105) in which the host returns "response information" and the displayed "communication image proposal" is updated may be repeated until, for example, the host participating user A approves. As a result, for example, participating users A to D can check the displayed (updated) image proposals in real time and create the final "communication image" together (for example, while having fun together). It is also possible to raise it.

(S201) Host participating user A (terminal 2A) checks the updated "communication image proposal" on his own terminal screen, and also checks the "response information" sent from the OC service providing server 1, Assuming that the current image plan is sufficient, an image generation instruction including "generation timing" information is transmitted to the OC service providing server 1.
(S202) The OC service providing server 1 notifies the participating users A to D (their terminals 2A to 2D) that a countdown will be performed to generate a "communication image", and also ) Based on the "generation timing", generate a countdown layer (Figure 2), display the counting numbers toward the "generation timing" on the screens of terminals 2A to 2D of participating users A to D, and perform a countdown. .

Note that by performing the countdown in this manner, for example, each participating user can react in accordance with the atmosphere of the scene just when the "original image" is being generated (photographed).

(S203) The OC service providing server 1 transmits a request for a group of "original images" including "generation timing" information to the terminals 2A to 2D of the participating users A to D.
(S204) The terminals 2A to 2D of the participating users A to D send a group of "original images" (for example, several tens of frames) generated in a predetermined time range including the received "generation timing" to the OC service providing server. Send to.
(S205) The OC service providing server 1 selects, for each of the participating users A to D, the most similar image to the "low resolution image" related to the "generation timing", for example, the highest An "original image" with a large SSIM value is selected and stored in the original image storage unit 103 as an "original image" used to generate a "communication" image.

By the way, at this stage, (a) the selected "original image", (b) the "communication image" itself to which the selected "original image" has been applied, and (c) the image to which the selected "original image" has been applied At least one of the resolution "communication image proposals" is presented to the terminals 2A to 2D of participating users A to D, and if any of the participating users requests retaking, the "generation timing" information is sent again. It is also preferable that the steps from the step of transmitting a request for a group of "original images" including the "original images" to the step of selecting the "original images" are repeated.

Next, steps S301 to S504 up to the final distribution of multimedia content as a "communication image" will be sequentially explained using FIG.
(S301) Host participating user A (terminal 2A) sends the "comment information" setting items from the OC service providing server 1 in order to generate and distribute multimedia content including a birthday party commemorative photo with a message attached. Read out the message setting item information that specifies etc.
(S302) The host participating user A (terminal 2A) generates “comment setting proposal information” based on the read out comment setting item information, and transmits it to the OC service providing server 1.

(S303) The OC service providing server 1 generates a message blank layer in which the message section is blank based on the received "composition setting plan information", and applies the message to the "communication image proposal" that has been generated and updated so far. A cross-written blank layer is added, and this further updated "communication image proposal" is displayed on each screen of the terminals 2A to 2D as an "online image" displayed in real time during online communication.

(S304) The OC service providing server 1 requests comments (comment information) from the terminals 2A, 2C, and 2D of the participating users A, C, and D, and sends comments from the terminals 2A, 2C, and 2D to the terminals 2A, 2C, and 2D of the participating users A, C, and D. Receive messages (comment information).
Here, the received message (comment information) can be, for example, text information input by a key or handwritten character image information input by a pen via a touch panel. Incidentally, in this step S304, since participating user B is the person whose birthday will be celebrated at this online birthday party, no request for a (congratulatory) message is sent to participating user B's terminal 2B. In addition, this fact is set in the "message setting proposal information".

(S305) The OC service providing server 1 inserts the received message (comment information) into the blank position of the message blank layer, completes the message layer (Fig. 2), and thereby further updates the "communication image proposal". Then, this further updated "communication image proposal" is displayed on each screen of the terminals 2A to 2D as an "online image" displayed in real time during online communication.

(S401) Host participating user A (terminal 2A) sends the OC service providing server 1, in order to distribute the "communication image proposal" generated so far as multimedia content including birthday party commemorative photos. Reads multimedia setting item information that defines setting items of "audio and/or video information" required for multimedia.
(S402) The host participating user A (terminal 2A) generates "multimedia setting plan information" based on the read multimedia setting item information, and transmits it to the OC service providing server 1.

(S403) The OC service providing server 1 starts multimedia processing, such as preparing to embed a link in a setting location, based on the received "multimedia setting plan information".
(S404) The OC service providing server 1 requests an audio message or a video message (audio and/or video information) from the terminals 2A to 2D of the participating users A to 2D, and sends a request from the terminals 2A to 2D to the terminals 2A to 2D of the participating users A to 2D. receive audio and/or video messages (audio and/or video information);
Incidentally, here, the participating user B (terminal 2B), who is the person whose birthday will be celebrated at this online birthday party, is also requested to send an audio message or a video message (for example, a thank you message). In addition, this is set in the "multimedia setting plan information".

In addition, as a specific procedure for collecting audio messages and video messages, the OC service providing server 1 displays a countdown for starting recording or recording on the screens of the terminals 2A to 2D of participating users A to D, and Recording may be performed for a predetermined period of time (for example, 10 seconds) immediately after "0" is displayed. It is also preferable to display a countdown of the remaining time during recording. For example, each participating user, when it is his or her turn to record, can face the microphone or camera of his or her own terminal 2 and input his or her own message into the terminal 2 according to the displayed countdown.

(S501) The OC service providing server 1 stores the received voice message or video message (audio and/or video information) in a predetermined storage location (for example, a dedicated storage within the server 1), and then links the URL of the storage location to Generates multimedia content that is a "communication image" that is embedded in the setting location.
(S502, S503) The OC service providing server 1 stores and manages the generated multimedia content (communication image) in the communication image storage unit 104, and then provides link information for downloading this multimedia content (communication image). is notified to the terminals 2A to 2D of participating users A to D.

(S504) Participating users A to D (terminals 2A to 2D) use the notified link information to create multimedia content including commemorative photo images, messages, and audio/video messages related to the online birthday party that is being held. (communication images) and each person can enjoy the content.

As explained in detail above, according to the present invention, a "communication image" is created using an "original image" that is the source data for generating an "online image" (online video) displayed during online communication. can be generated. That is, a higher resolution "communication image" can be generated using the image (original image) generated during the online communication. Moreover, this also makes it possible to generate a high-resolution group photo image (communication image) that has a realistic or live feel that looks like a photo taken during an online birthday party, for example.

Furthermore, it is expected that the need for such a "communication image" according to the present invention will increase even more now that the mode of interaction between people is changing to be more online. In other words, for example, instead of a commemorative group photo that can only be created by physically gathering between people who have interacted online, the "communication image" according to the present invention can be used as a commemoration/record of that interaction or as a sense of closeness/solidarity. It is thought that the demand for generating information as a tool to enhance one's senses will continue to increase in the future. Furthermore, as can be seen from this, the fields to which the present invention is applied are expected to further expand in the future.

Regarding the various embodiments of the present invention described above, various changes, modifications, and omissions within the scope of the technical idea and viewpoint of the present invention can be easily made by those skilled in the art. The above description is merely an example and is not intended to be limiting in any way. The invention is limited only by the claims and their equivalents.

1 Online communication (OC) service provision server (communication image generation device)
101 Communication interface section 102 OC service information storage section 103 Original image storage section 104 Communication image storage section 105 Corresponding ID storage section 106 Keyboard (KB)
107 Display (DP)
111 Original image acquisition/management unit 112 Communication image generation unit 112a Communication image draft generation unit 112aa Mixed writing layer generation unit 112ab Multimedia conversion unit 112ac Response information reflection unit 113 Image generation management unit 113a Countdown information generation unit 121 Communication control unit 122 OC service Implementation Department 123 EC Server Success Department 124 Input/Output Control Department 2, 2A, 2B, 2C, 2D Terminal 3 EC Server

Claims (13)

For online communication carried out by receiving a stream of images created by lowering the resolution of the original images generated by multiple terminals, and displaying an online image generated using the low-resolution images generated from the stream of images. A communication image generation device capable of generating such a communication image,
image generation management means that accepts or determines the timing of generating the communication image;
Original image acquisition means for acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. 1. A communication image generation device comprising image generation means. 2. The communication image generation device according to claim 1, wherein the image generation means uses PSNR (Peak Signal-to-Noise Ratio) or SSIM (Structural Similarity) as the predetermined measure. The image generation means determines the scale value after aligning the number of pixels, aspect ratio, or range of objects in the images between the original image and the low-resolution image. The communication image generation device according to claim 1 or 2. According to any one of claims 1 to 3, the image generation management means notifies a terminal of a user participating in the online communication of countdown information toward the received or determined timing. The communication image generating device described above. The image generating means includes:
receiving or determining generation plan information that is information related to at least one of the type, structure, image configuration, and settings in the image of the communication image to be generated;
Send the generation plan information to the terminals of users participating in the online communication,
Receiving response information determined by a user to whom the generation plan information was presented and received from a terminal related to the user, the response information relating to consent, selection, or change to the content of the generation plan information;
5. The communication image generation device according to claim 1, wherein the communication image is generated based on the generation plan information and the response information. The image generating means includes:
Before receiving the response information, the communication image plan as the online image, which uses the low resolution image and is generated based on the generation plan information, is sent to the user participating in the online communication. Send it to the terminal and present it to the user,
After receiving the response information, the communication image proposal generated using the low-resolution image and based on the generation proposal information and the response information received so far is sent to the terminal of the user participating in the online communication. 6. The communication image generation device according to claim 5, wherein the communication image generation device displays the communication image to the user. The communication image proposal has a multilayer structure including a background layer including a background, and at least one participant image layer using the low resolution image, which is a participant image layer related to a participant in the online communication. The communication image generation device according to claim 6, characterized in that: The communication image has a multilayer structure including a background layer including a background, and at least one participant image layer using the original image, which is a participant image layer related to a participant in the online communication. The communication image generation device according to any one of claims 1 to 7. 2. The image generating means receives comment information related to a comment of a user who participates in the online communication, received from a terminal related to the user, and generates the communication image including the comment information. 9. The communication image generation device according to any one of 1 to 8. The image generation means receives audio and/or video information related to audio data and/or video data generated by the user, received from a terminal related to the user participating in the online communication, and generates the audio and/or video information. 10. The communication image generation device according to claim 1, wherein the communication image generation device generates a communication image in which a link to video information is embedded. an identifier storage unit that stores an identifier for an electronic transaction service that can provide objects and/or data related to the generated communication image, and that stores an identifier of a user participating in the online communication;
further comprising an electronic transaction access means that uses the identifier to instruct the server associated with the electronic transaction service to provide objects and/or data related to the generated communication image to the user associated with the identifier; The communication image generation device according to any one of claims 1 to 10. For online communication carried out by receiving a stream of images created by lowering the resolution of the original images generated by multiple terminals, and displaying an online image generated using the low-resolution images generated from the stream of images. A communication image generation program that causes a computer installed in a device capable of generating such communication images to function,
image generation management means that accepts or determines the timing of generating the communication image;
Original image acquisition means for acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. A communication image generation program characterized by causing a computer to function as an image generation means. For online communication carried out by receiving a stream of images created by lowering the resolution of the original images generated by multiple terminals, and displaying an online image generated using the low-resolution images generated from the stream of images. A communication image generation method implemented by a computer installed in a device capable of generating such communication images, the method comprising:
receiving or determining the timing of generating the communication image;
acquiring a plurality of original images related to a time interval determined based on the timing from the terminal;
Determine a scale value related to a predetermined scale for measuring the degree of similarity of images for the obtained original image and the low-resolution image used to generate the online image related to the timing, and Select an original image that is similar to the low-resolution image until the determined scale value satisfies a predetermined condition from among the plurality of original images, and generate the communication image using the selected original image. A method for generating a communication image, comprising the steps of:

Images