JP2022152268A - Imaging system, server, terminal, imaging method, and program - Google Patents

Abstract

To provide an imaging system or the like allowing for easily performing imaging by using participant images in an online conference and sharing the photographs after obtaining the consent of the participants.SOLUTION: In an imaging system 1, a server 3 captures an image from a video of each participant in an online conference on the basis of imaging instruction information from a user terminal 5, and saves the image as a captured image. The server 3 sends an approval request as to whether the imaged captured image can be shared to the participant who is the subject, and makes the captured image sharable among the participants (sharers) who has responded as sharable when the participant who is the subject responds with approval. The server manages the captured image as non-shareable when no approval is obtained from the participant who is the subject.SELECTED DRAWING: Figure 1

Description

The present invention relates to a photography system, a server, a terminal, a photography method, and a program, and more particularly to online photography.

Conventionally, at a meeting where many people gather, a group photo was taken with everyone as a commemoration, and it was distributed or sold. However, it is difficult to take a group photo if there is no opportunity to gather. In recent years, an online conference system has been provided that enables a conference to be held remotely from each other by connecting PCs and smartphones of a plurality of users via a network and transmitting and receiving voice and images to each other. This type of conference system is used not only in business situations but also in gatherings of friends, and is used as a tool for people in remote locations to casually gather and have conversations while looking at each other's faces.

In a typical online conference system, a computer terminal or smartphone equipped with a camera, microphone, speaker, display, etc., is used to photograph the participants, acquire the voice, and send it to the terminals of other participants via the server. . The terminals of the other participants receive the image (video) of each participant, display it on the display screen, and output the voice of each participant from the speaker. Windows for the number of participants are provided on the display screen of each terminal, and images transmitted from each participant are displayed in each window. Further, as shown in Patent Document 1, a teleconferencing system has also been proposed that displays an image representing a virtual conference room on a display screen.

JP-A-10-056626

However, it is difficult to take a group photo at an online meeting because it is not a physical meeting. Although it is possible for participants to capture (print screen) the screen using the function of the terminal, it is blunt because the screen is shot as it is, and unnecessary parts of the screen are included. In addition, there was a problem that it was difficult to capture the right timing when people were excited, and there was also a problem that moments that they did not want to be captured were captured.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and its object is to enable the participants to be easily photographed even when gathering online and to share the photographs after obtaining the consent of the participants. It is to provide a possible imaging system and the like.

A first invention for solving the above-described problems is a system capable of executing an online conference between a plurality of terminals connected to a server via a network, wherein the server controls entry and exit from the online conference. and receives video and audio transmitted from the terminals of each participant participating in the online conference and transmits them to the terminals of other participants; capture means for cutting out an image from a video and storing it as a captured image; approval request means for transmitting an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject; and the participant who is the subject a sharing management means for enabling the captured image to be shared with participants who are other subjects when approval is obtained from the participant, and disabling the sharing of the captured image when approval is not obtained from the The imaging system is characterized in that the terminal comprises response means for responding to the approval request from the server as to whether or not the approval is granted.

According to the first invention, an image of each participant is cut out (captured) by using the video of each participant during the online conference, and the captured image is shared when an approval response is obtained from the participant who is the subject. Participants (sharers) who respond as possible can share. This makes it possible to easily take photos during an online meeting and share the photos with the consent of the participants.

In the first invention, the terminal includes a photographing instruction means for designating a participant as a subject and instructing photographing on a screen on which images of participants of the online conference are displayed; transmitting means for transmitting the identification information of the participants and the information on the shooting date and time to the server as shooting instruction information, wherein the capturing means of the server captures the subject based on the shooting instruction information sent from the terminal; Cut out an image from the video of the participant who participates and save it as a captured image. As a result, the participant can operate the terminal to designate the participant to be the subject, and to perform the photographing at the desired timing.

Further, in the first invention, the capture means may monitor the video of each participant, and if a predetermined condition is satisfied, cut out an image from the video of each participant and store it as a captured image. As a result, the server side can determine the timing of image capturing and perform image capturing. For example, even when the conversation is lively, it is possible to take pictures at good timing without forgetting.

The server further comprises image providing means for providing the captured image made sharable by the sharing management means so that it can be used among the participants who have shared it. Further, the image providing means makes the captured image available even after the online conference ends. As a result, the shared captured image can be used (displayed or saved) on each terminal with the consent of the subject. By making the images available even after the conference ends, the conference can be kept as a memorial or record.

The server includes group photo generation means for generating a group photo by combining captured images made sharable by the sharing management means, and the image providing means transmits the group photo to the participants included in the group photo. It is desirable to provide This allows you to create and share a group photo of the participants. Since the group photo is generated on the server side, the participants can easily obtain the group photo without image processing or the like.

Preferably, the server includes transmission means for transmitting the captured images to terminals of the participants during the online conference, and the terminals preferably include image list display means for displaying a list of the captured images received from the server. . Further, it is preferable that the image list display means does not display the captured images that are prohibited from being shared by the sharing management means. As a result, the participants can check the captured images on their own terminals. Also, images that cannot be shared can be prevented from being displayed in the image list. The image list display means displays a message to the effect that the captured images, which are prohibited from being shared by the sharing management means, cannot be shared.

It is preferable that the server includes notification means for transmitting a notification to the terminal of the participant who is the subject when the image is captured by the capture means. This allows the participants to understand that they have been photographed.

It is preferable that the group photo generation means extract keywords from the conversation information during the online conference, acquire images related to the extracted keywords, and set the acquired images as background images of the group photo. As a result, the background representing the state of the meeting can be added to the group photo, thereby improving the quality of the group photo.

A second invention is a server that is connected to the terminals of participants in an online conference for communication via a network, manages entry and exit from the online conference, and manages each participant participating in the online conference. Online conference management means for receiving video and audio transmitted from the terminal of the participant and transmitting them to the terminals of other participants, capturing means for cutting out an image from the video of the participant as a subject and saving it as a captured image; Approval request means for transmitting an approval request as to whether or not the captured image can be shared to a terminal of the participant who is the subject; and sharing management means for disabling sharing of the captured image if approval is not obtained.

The second invention can provide the server in the imaging system of the first invention.

A third aspect of the invention is a terminal of a participant of an online conference, which is connected to a server for communication via a network, and which requests approval whether or not a captured image of the participant himself/herself as a subject can be shared in the online conference. from the server, and response means for responding to the approval request with approval or disapproval of the approval.

The third invention can provide the terminal in the imaging system of the first invention.

A fourth aspect of the invention is a photographing method in a system capable of executing an online conference between a plurality of terminals connected to a server via a network, wherein the server manages entry into and exit from the online conference. receiving the video and audio transmitted from each participant's terminal participating in the online conference and transmitting the video and audio to the other participant's terminal; a step of cutting out an image and saving it as a captured image; a step of the server sending an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject; and, if the server obtains approval from the participant who is the subject, the captured image can be shared with other participants who are subjects. and disabling sharing of the captured image if approval is not obtained.

According to the fourth invention, an image of each participant is cut out (captured) by using the video of each participant during the online conference, and the captured image is shared when an approval response is obtained from the participant who is the subject. Participants (sharers) who respond as possible can share. This makes it possible to easily take photos during an online meeting and share the photos with the consent of the participants.

A fifth aspect of the present invention manages a computer connected to the terminals of the participants of the online conference for communication via a network to enter and leave the online conference, Online conference management means for receiving video and audio transmitted from a terminal and transmitting them to the terminals of other participants, capture means for extracting an image from the video of the participant as a subject and saving it as a captured image, and capturing the captured image. Approval requesting means for transmitting an approval request to the terminal of the participant who is the subject as to whether or not it is possible to share the captured image with the participant who is another subject when approval is obtained from the participant who is the subject It is a program for functioning as sharing management means for enabling sharing between users and disabling sharing of the captured image if approval is not obtained.

The fifth invention enables a computer to function as the server of the imaging system of the first invention or the server of the second invention.

A sixth aspect of the present invention is to determine whether or not a computer used as a terminal of a participant of an online conference, which is connected to a server via a network for communication, can share a captured image of itself as a subject in the online conference. It is a program for functioning as receiving means for receiving an approval request from the server and as responding means for responding to the approval request as to whether approval is granted or not.

The sixth invention enables a computer to function as the terminal of the photography system of the first invention or the terminal of the third invention.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a photography system or the like that can easily take pictures using video of participants in an online conference and share the pictures with the consent of the participants. .

1 is a block diagram showing the system configuration of an imaging system 1 according to the present invention; A diagram showing an example of the hardware configuration of the server 3 FIG. 4 is a diagram showing an example of the hardware configuration of the user terminal 5; Block diagram showing the functional configuration of the imaging system 1 (server 3 and user terminal 5) A diagram showing the configuration of an online conference processing unit 51 and an online conference management unit 31. Flowchart showing the flow of online meeting processing A diagram showing an example of a conference screen 61 displayed on the user terminal 5 Flowchart showing the flow of shooting processing in an online conference A diagram showing an example of an image list 67 A diagram showing the structure of the shared management information 361 Flowchart showing the flow of image provision processing after leaving the room (after the end of the meeting) A diagram showing an example of group photographs 81 and 82 Flowchart showing the flow of imaging processing in the second embodiment

Preferred embodiments of the present invention will be described in detail below with reference to the drawings.

[First embodiment]
FIG. 1 is a diagram showing the system configuration of an imaging system 1 according to the first embodiment of the present invention.

As shown in FIG. 1, the photographing system 1 is configured by connecting a server 3 and user terminals 5 (5a, 5b, 5c, 5d, 5e) for communication via a network 7. As shown in FIG. In the example of FIG. 1, users of five user terminals 5a to 5e are participants in the online conference, but the number of terminals that can participate in the online conference is not limited to five, and any number of terminals can be In the following description, user terminals 5a to 5e are referred to as user terminals 5 when they are not distinguished from each other. Further, in the present invention, the term "meeting" includes not only business meetings but also meetings, lectures, seminars, classes, etc. where users casually talk to each other.

The server 3 is a computer that performs communication processing of video and audio data between the user terminals 5 in an online conference held between a plurality of user terminals 5, and also performs room entry processing and room exit processing for users participating in the conference. .

The user terminal 5 is a terminal used by the participants of the online conference, and is a computer that can be communicatively connected to the server 3 via the network 7 . The user terminal 5 is assumed to have a speaker 508 , a microphone 507 and a camera 506 . For example, a smartphone with built-in speaker 508, microphone 507, and camera 506 may be used, or a PC connected to an external camera, speaker, microphone, or headset may be used.

The network 7 includes a LAN (Local Area Network), a WAN (Wide Area Network) connected for communication over a wider area, a public communication line such as the Internet, a base station, and the like. The network 7 may be wired or wireless.

FIG. 2 is a diagram showing an example of the hardware configuration of the server 3. As shown in FIG. As shown in FIG. 2, the server 3 can be implemented by a computer configured by connecting a control unit 301, a storage unit 302, and a communication unit 303 via a bus 309 or the like. However, without being limited to this, various configurations can be appropriately adopted.

The control unit 301 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls a program stored in the storage unit 302, ROM, recording medium, or the like to a work memory area on the RAM and executes it, drives and controls each device connected via the bus 309, and performs processing performed by the server 3. Realize

The storage unit 302 is a flash memory, a HDD (Hard Disk Drive), or the like, and stores programs executed by the control unit 301, data necessary for executing the programs, an OS (Operating System), and the like. As for programs, a control program corresponding to the OS and an application program for causing the server 3 to execute processing related to the online conference and shooting processing during the online conference are stored. These program codes are read by the control unit 301 as necessary, transferred to the RAM, read by the CPU, and executed as various means. Details of the processing executed by the server 3 will be described later.

The communication unit 303 has a communication control device, a communication port, etc., and is a communication interface that mediates communication between the server 3 and the network 7, and communicates with the user terminal 5 and other computers via the network 7. control is possible. A bus 309 is a path through which control signals, data signals, and the like are transferred between units.

FIG. 3 is a diagram showing the hardware configuration of the user terminal 5. As shown in FIG. As shown in FIG. 3, the user terminal 5 is configured by connecting a control unit 501, a storage unit 502, a communication unit 503, a display unit 504, an input unit 505, a camera 506, a microphone 507, and a speaker 508 via a bus 509. It can be realized by a computer. Note that, as described above, the camera 506, the microphone 507, and the speaker 508 may be externally attached via an interface. The configuration of the user terminal 5 is not limited to the configuration shown in FIG. 3, and various configurations can be used as appropriate.

Functions of the control unit 501, the storage unit 502, and the communication unit 503 are substantially the same as those of the control unit 301, the storage unit 302, and the communication unit 303 described above.

The display unit 504 includes a display such as a liquid crystal panel, and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display. display. Note that the display unit 504 may be a touch panel display in which an input device such as a touch panel is integrally provided on the display screen.
The input unit 505 includes, for example, a touch panel, a keyboard, a pointing device such as a mouse, an input device such as a numeric keypad, and the like, and inputs input data to the control unit 501 .

A camera 506 is a camera that captures an image of the user, and is composed of an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

A microphone 507 acquires voice and inputs it to the control unit 501 .
A speaker 508 outputs the sound input from the control unit 501 .

FIG. 4 is a block diagram showing the functional configuration of the imaging system 1 according to the embodiment of the invention. As shown in FIG. 4, the server 3 includes an online conference management unit 31, a photography instruction reception unit 32, a capture unit 33, a capture notification unit 34, an approval request unit 35, a sharing management unit 36, a group photo generation unit 37, an image provider A portion 38 and the like are provided. The user terminal 5 includes an online conference processing unit 51, a photographing instruction unit 52, a photographing instruction transmission unit 53, an approval approval/disapproval response unit 54, an image list display unit 55, and the like.

FIG. 5 is a diagram illustrating the online conference management unit 31 of the server 3 and the online conference processing unit 51 of the user terminal 5. As shown in FIG.
The online conference management unit 31 includes a communication unit 303, an entry/exit processing unit 31a, an ID setting unit 31b, a decoding unit 31c, a video acquisition unit 31d, an audio acquisition unit 31e, an audio generation unit 31f, a conference screen generation unit 31g, and an audio output. It includes a section 31h, a video output section 31i, an encoding section 31j, and the like.

The room entry/exit processing unit 31a manages entry (participation) into and exit from a virtually set meeting room in an online meeting. The room entrance/exit processing unit 31a sets a virtual conference room in response to a request from the user terminal 5 of the user who is the host of the conference, assigns the conference ID, and receives the setting of the name of the conference. Further, upon receiving a request to participate in the conference from the user terminal 5, the entry/exit processing unit 31a performs login processing of the user terminal 5, and when the login is successful, provides the conference screen 61, and allows participation in the conference (entering the room). To give permission. In the login process, inputs such as names and affiliations of users used in the conference are accepted. Also, upon receiving a request to leave the room from the user terminal 5 , the entering/leaving processing unit 31 a terminates transmission/reception of the conference screen 61 for that user terminal 5 .
The ID setting unit 31b sets different participant IDs (identification information) for participants (user terminals 5) who have entered the conference room. Delete the participant ID when leaving the room.

A communication unit 303 of the server 3 transmits and receives audio data and video data to and from a plurality of user terminals 5 . Audio data and video data are superimposed and transmitted/received as compression-encoded data.

The decoding unit 31c decodes the compression-encoded data received by the communication unit 303, and separates and extracts audio data and video data.

The video acquisition unit 31d acquires the video data extracted by the decoding unit 31c, and inputs it to the conference screen generation unit 31g.
The voice acquisition unit 31e acquires the voice data extracted by the decoding unit 31c, performs signal processing such as noise removal, and inputs the data to the voice generation unit 31f.

The voice generation unit 31f superimposes the voice of each user terminal 5 acquired by the voice acquisition unit 31e, and inputs it to the encoding unit 31j.

The conference screen generator 31g generates, for each user terminal 5, a conference screen 61 in which the video of each participant obtained from each user terminal 5 is laid out. The video output unit 31i inputs the conference screen 61 generated by the conference screen generation unit 31g to the encoding unit 31j.

The encoding unit 31 j superimposes and compresses and encodes the audio data input from the audio output unit 31 h and the conference screen data input from the video output unit 31 i , and transmits the data to the user terminal 5 via the communication unit 303 .

The online conference processing unit 51 of the user terminal 5 includes an encoding unit 51a, a decoding unit 51b, and the like.
The encoding unit 51a superimposes the video input from the camera 506 and the audio input from the microphone 507, and performs compression encoding processing. The decoding unit 51b decodes the superimposed data of audio and video (conference screen 61) received from the server 3, and separates the data into audio data and video data. Audio data is output from the speaker 508 . The video data is input to the display unit 504 and displayed in synchronization with the audio.

Returning to the description of FIG.
The photographing instruction unit 52 of the user terminal 5 designates a participant as a subject and instructs photographing on the screen (meeting screen 61) on which the images of the participants of the online conference are displayed.
The photographing instruction transmission unit 53 transmits the identification information (participant ID) of the participant (subject user) designated by the photographing instruction unit 52 and information on the photographing date and time to the server 3 as photographing instruction information.

The shooting instruction reception unit 32 of the server 3 receives the shooting instruction information transmitted from the user terminal 5 . The photographing instruction information includes the participant ID of the participant who is the subject, and photographing date and time information that is the date and time when the photographing instruction is input.

The capturing unit 33 of the server 3 issues a new shooting ID when receiving the shooting instruction information, cuts out an image from the video of the participant who is the subject based on the received shooting instruction information, and uses it as a captured image. image folder. The capture unit 33 attaches a photographing ID to each captured image and stores the captured image. When a photographing instruction is issued with a plurality of participants as subjects, a captured image is obtained for each participant, and the same photographing ID is assigned to each captured image photographed at the same time.

The capture notification unit 34 of the server 3 transmits a notification indicating that the image has been captured to the user terminal 5 of the subject user when the image is captured by the capture unit 33 . When the user terminal 5 receives the notification, the user terminal 5 notifies the notification by a shutter sound, an effect display of the corresponding window, or the like.

The approval request unit 35 of the server 3 transmits to the user terminal 5 of the subject user an approval request as to whether or not the captured image can be shared.

The approval/non-approval response unit 54 of the user terminal 5 receives an input of approval/non-approval of sharing (approval/rejection) in response to the approval request from the server 3 and transmits (responses) to the server 3 .

The sharing management unit 36 of the server 3 enables sharing of the captured image with other subject users (sharers) when approval is obtained from the user terminal 5 of the subject user. Also, if approval is not obtained, the captured image cannot be shared. The sharing management unit 36 generates, for example, sharing management information 361 shown in FIG. 10 and stores it in the storage unit 302 . The shared management information 361 stores, for each captured image, a photographing ID, photographing date and time, a sharer, approval information, and the like. The shared management information 361 will be described later.

The image list display unit 55 of the user terminal 5 acquires captured images taken during the online conference from the server 3 and displays them as an image list 67 . The image list display unit 55 does not display the captured images that are prohibited from being shared by the sharing management unit 36 and deletes them from the image list 67 . Further, on the user terminal 5 that issued the photographing instruction, a sharing prohibition icon 67e indicating that sharing of the captured image prohibited from being shared is prohibited is displayed.

The group photo generator 37 combines sharable captured images to generate a group photo 81 (82). The group photo generation unit 37 generates, for example, a group photo 81 in which a captured image of a certain participant is placed in the center and captured images of other participants are placed around it. Also, the name of the subject of the captured image included in the group photo 81 is obtained from the login information and displayed near the captured image. Also, the group photo generator 37 acquires the title of the group photo 81 from the name of the meeting and shows it in the group photo 81 . The group photo generation unit 37 also extracts keywords from the conversation information during the online conference, acquires images related to the extracted keywords, and sets the acquired images as background images of the group photo 81 (82). An image is obtained by searching the Web based on the extracted keyword, for example. Alternatively, a method of registering in advance in the storage unit 302 a background image tagged with each keyword can be considered.

The image providing unit 38 provides a sharable captured image so that it can be used (displayed, saved, etc.) among the shared participants. The provision of the image includes, for example, e-mail transmission to the user terminal 5, sharing in the cloud, and the like. The image providing unit 38 also provides the group photo 81 (82) generated by the group photo generating unit 37 to the participants included in the group photo 81 (82).

Next, online conference processing executed between the user terminal 5 and the server 3 will be described with reference to FIG. Note that the storage unit 502 of the user terminal 5 stores a conference application program, which causes the user terminal 5 to execute processing described later. Similarly, a conference management application program is also stored in the storage unit 302 of the server 3, and the server 3 executes processing described later. The flowchart of FIG. 6 shows a case where user terminal 5a (user A), user terminal 5B (user B), and user terminal 5c (user C) participate in the conference.

As shown in FIG. 6, the user terminals 5a, 5b, and 5c access the server 3 and perform room entry processing with the server 3. FIG. In the room entry process, when each user terminal 5a, 5b, 5c transmits login information (user ID, authentication information, etc.) to the server 3, the entry/exit processing unit 31a of the server 3 performs authentication processing. The user is allowed to enter the conference room (steps S101 to S104).

The ID setting unit 31b of the server 3 sets a participant ID for each user (user terminals 5a, 5b, 5c) who entered the conference room (step S105).

When permitted to enter the room, each user terminal 5a, 5b, 5c compresses and encodes the camera image (video) taken by the camera 506 and the voice input from the microphone 507, and transmits them to the server 3 (steps S106, S107, step S108). When the server 3 receives the camera image (video) and audio compression-encoded data transmitted from each of the user terminals 5a, 5b, and 5c of the conference participants, it decodes and separates and extracts the image data and the audio data ( step S109). In this way, the video of each participant (users A, B, and C) is input to the video acquisition unit 31d of the server 3, and when a participant speaks, the voice of this speaker is acquired by the server 3. It is input to the part 31e.

The voice generation unit 31f of the server 3 performs voice signal processing such as noise removal processing on the input speaker's voice, and generates voice for output. Also, the conference screen generation unit 31g of the server 3 generates the conference screen 61 in which the images input from the user terminals 5a, 5b, and 5c are arranged in each window (step S110).

FIG. 7 is a diagram showing an example of the conference screen 61. As shown in FIG. A conference screen 61 is provided with windows 62, 63, 64, and 65 for displaying images of each participant. In the example of FIG. 7, four user terminals 5 are participating in the conference, and four windows 62, 63, 64 and 65 are provided. User A's image on window 62, user B's image on window 63, user C's image on window 64, user D's image on window 65, etc. Images input to 3 are displayed respectively. Each window 62, 63, 64, 65 may display the participant ID or user name, or may be anonymous. The arrangement and size of the windows 62, 63, 64, and 65 are arbitrary, and the configuration may be such that each user terminal 5 can change the layout. Also, instead of the server 3 generating the conference screen 61 , the configuration may be such that the user terminal 5 side generates the conference screen 61 . In that case, the server 3 associates the video input from each user terminal 5 with the participant ID and transmits it to each user terminal 5 . The user terminal 5 displays the video of each participant in the windows 62, 63, 64, 65 of the corresponding participant ID.

An image list 67 and a shooting button 68 are displayed on the conference screen 61, as shown in FIG. The image list 67 and shooting button 68 will be described later.

The audio output unit 31h of the server 3 inputs the output audio generated in step S110 to the encoding unit 31j. Also, the video output unit 31i inputs the conference screen 61 generated in step S110 to the encoding unit 31j. The encoding unit 31j superimposes audio and video, compression-encodes them (step S111), and transmits them to each user terminal 5 via the communication unit 303 (step S112).

Each user terminal 5 receives the video (conference screen 61) and audio transmitted from the server 3 (steps S113, S115, S117), displays them on the display unit 504, and outputs them from the speaker 508 (steps S114, S116, S118). ).

Next, photographing processing in an online conference will be described with reference to the flowchart of FIG. FIG. 8 illustrates the flow of processing when a participant (user B) as a subject is specified and photographed using the user terminal 5a of a certain participant (user A). User C is a user who is a participant in the online conference and is not a subject.

When user A wants to shoot user B as the subject, user A designates user B as the subject on the conference screen 61 displayed on the user terminal 5a (step S201), and presses the shooting button 68 in that state. (Step S202). The designation operation of the subject is, for example, an instruction (click or touch operation) on a window displaying the image of the user to be designated. The user terminal 5a displays the designated windows (windows 62, 63, and 64 in the conference screen 61 of FIG. 7) so that they can be identified by highlighting, inverting, blinking, or the like. Moreover, it is desirable that the shooting button 68 should be capable of switching between shooting a video 68a (moving image) and a photograph 68b (still image). The example of FIG. 7 means taking the picture 68b.

The control unit 501 of the user terminal 5a transmits to the server 3 the participant ID of the designated subject (subject user B) together with the photographing date and time, which is the time at which the photographing button 68 was pressed (step S203).

When receiving the shooting instruction information, the server 3 issues a shooting ID, cuts out (captures) an image of the corresponding shooting date and time from the video of the subject user B based on the shooting instruction information, and stores it in a predetermined image folder in the storage unit 302 . Record (step S204). The server 3 notifies the user terminal 5b of the subject user B that the image has been captured (step S205). Upon receiving the notification from the server 3, the user terminal 5b of the subject user B notifies that the image has been captured (step S206). The notification method is, for example, to sound a shutter sound, or to display an effect in windows 62, 63, and 64 of the corresponding subject user B on the conference screen 61 as indicated by reference numeral 66 in FIG.

The server 3 generates a thumbnail image of the image captured in step S204 (step S207), and transmits the shooting ID, shooting date and time, and thumbnail image of the captured image to the corresponding participant ID (subject user B). An approval request as to whether or not the image may be shared between the photographer and the participant who is the subject is transmitted (step S208). Note that the thumbnail image is transmitted in step S208 in order to reduce the amount of data, and the original captured image may be transmitted.

User terminal 5b of subject user B displays photographing ID 67a, photographing date and time 67b, and captured image 67c (thumbnail image) received from server 3 in image list 67 as shown in FIG. A button 67g and a "Reject" button 67h are displayed, and a response as to approval or rejection is accepted (step S209).

FIG. 9 is a diagram showing an example of the image list 67. As shown in FIG. As shown in FIG. 9, the image list 67 displays a photographing ID 67a, a photographing date and time 67b, and a captured image 67c (thumbnail image) in a list format. Is displayed. If there is no icon display, no response of approval/rejection has been obtained. Also, when the subject is the subject, an "Approve" button 67g and a "Reject" button 67h are displayed in the column of the corresponding photographing ID 67a. When the "Approve (OK)" button 67g is pressed, it is switched to the approved display 67f such as "OK! Done". is erased (hidden). The sharable icon 67d is displayed when approval is obtained from all the participants (sharers) who are subjects with the same photographing ID. If even one of the participants (sharers) who have become subjects with the same shooting ID refuses to share, a sharing prohibition icon 67e is displayed in the line of the shooting ID on the photographer's user terminal 5a, and the user other than the photographer refuses to share. At the user terminals 5b, 5c, .

When the user terminal 5b of the subject user B inputs either an approval/rejection response, the user terminal 5b transmits the photographing ID and approval information (approval or rejection) to the server 3 (step S210). The server 3 records the received approval information in the shared management information 361 (step S211).

FIG. 10 is a diagram showing an example of the sharing management information 361. As shown in FIG. As shown in FIG. 10, in the sharing management information 361, for each "captured image", "shooting ID", "shooting date and time", "sharer", "approval information", and "sharing prohibition flag" are recorded. . For example, in photographing with the photographing ID “#1”, three subjects (users A, B, and C) are designated and photographed simultaneously (19:03 on February 1, 2021). In this case, the subject users "A, B, C" are recorded in the "sharer" column. In the "approval information" column, when no response is obtained from the subject user, a value indicating non-approval ("unapproved") is recorded. Denied” is recorded.

In FIG. 10, three subjects (users A, B, and C) are designated and photographed at the same time (19:10 on February 1, 2021) in the photographing with the photographing ID “#2”. The sharers are subject users A, B, and C, and subject user A and subject user B have responded "approved", but subject user C has responded "rejected". In this case, since permission has not been obtained from all the sharers, the server 3 puts a sharing prohibition flag on the captured image with the corresponding photographing ID "#2".

When the server 3 receives a response of "approval" from all the subject users, the server 3 makes the captured image sharable, and displays the captured image 67c (thumbnail image) and the sharable icon 67d in the image list 67 of all the sharers ( step S212, step S214). If even one of the subject users is not approved ("rejected"), the captured image 67c (thumbnail image) of the corresponding photographing ID is not displayed on the user terminals 5 of the subject user B and the other user C (step S212). , step S215), the user terminal 5 of the photographer displays the captured image 67c (thumbnail image) in the image list 67, but displays the sharing prohibition icon 67e (step S212, step S213).

Next, the processing after the end of the conference will be described with reference to FIG.
As shown in FIG. 11, when the server 3 receives room exit requests from the user terminals 5a, 5b, and 5c (steps S301, S302, and S303), room exit processing is performed (step S304). The conference ends when all participants leave the room. After that, the server 3 extracts from the shared management information 361 the captured images that have been approved by all of the relevant participants, that is, the images without the sharing prohibition flag (step S305), and uses the original images of the extracted captured images to create the group photo 81. is generated (step S306). Alternatively, the original image of the captured image and the generated group photo 81 are provided to the sharer so that they can be shared (step S307). For example, a sharable captured image or a generated group photo 81 can be transmitted to the sharer's user terminal 5 for display and storage, or uploaded to the cloud (image server) and downloaded and displayed on the sharer's user terminal 5. , can be saved.

The server 3 stores the images that cannot be shared for a predetermined period of time, and then deletes them from the image folder (step S308).

FIG. 12 is a diagram showing an example of group photographs 81 and 82. As shown in FIG.
The group photo generation unit 37 of the server 3 combines the sharable captured images to generate a group photo. For example, as shown in FIG. 12(a), the group photo generation unit 37 places a captured image 81a of a certain participant at the center, and capture images 81b, 81c, 81d, and 81e of the other participants. A group photo 81 arranged around it is generated for each participant. Also, the names 81f, 81g, 81h, and 81i of the captured images 81b to 81e included in the group photo 81 are acquired from the login information and laid out nearby, and the title 81k of the group photo 81 is determined from the name of the meeting. , may be laid out in the group photo 81 . In addition, the group photo generation unit 37 extracts keywords from the information of the conversation during the online conference, acquires images related to the extracted keywords from the Web search or the storage unit 302, and displays them as shown in FIG. 12(b). , the background image 82m of the group photo 82 may be set.

The image providing unit 38 provides (email transmission, cloud sharing, etc.) a sharable captured image so that it can be used (displayed, saved, etc.) among the shared participants. The image providing unit 38 provides the group photos 81 and 82 generated by the group photo generating unit 37 to the participants included in the group photos 81 and 82 .

As described above, in the imaging system 1 of the first embodiment, a participant designates a participant as a subject on the screen of the user terminal 5 on which the video of each participant during the online conference is displayed. When the photographing button 68 is operated, the identification information (participant ID) of the designated participant and information on the photographing date and time are transmitted to the server 3 as photographing instruction information, and the server 3 selects the participant as the subject based on the photographing instruction information. An image is cut out from the video and saved as a captured image. The server 3 transmits an approval request to the subject user as to whether or not the photographed captured image can be shared, and when receiving a response of approval from the subject user, each participant who has responded with approval that the captured image can be shared. It is possible to share between users (sharers). If approval is not obtained from the subject user, the captured image is managed as non-shareable. This makes it possible to easily photograph each participant during the online conference. In addition, the captured image can be shared and used by the co-sharers even after the end of the conference with the approval of the participants who took the image. In addition, the server 3 can generate the group photos 81 and 82, and the generated group photos 81 and 82 can be shared and provided to each sharer.

[Second embodiment]
Next, a second embodiment of the invention will be described. FIG. 13 is a flowchart showing imaging processing in the imaging system 1 according to the second embodiment. The system configuration of the imaging system 1 and the hardware configuration of the server 3 and the user terminal 5 of the second embodiment are the same as the system configuration of the imaging system 1, the server 3 and the hardware configuration of the user terminal 5 in the first embodiment. It is the same. Also, the process of entering and leaving an online conference and the process of transmitting and receiving video and audio during a conference (FIGS. 6 and 11) are the same as in the first embodiment.

The difference between the second embodiment and the imaging system 1 of the first embodiment is that the server 3 records the captured image when a predetermined condition is satisfied regardless of the imaging instruction from the user terminal 5 . . The server 3 monitors the video of each participant during the online conference, and when a predetermined condition is satisfied, cuts out an image from the video of each participant and saves it as a captured image.

As shown in the flowchart of FIG. 13, the server 3 monitors images transmitted from the user terminals 5a, 5b, and 5c of the participants in the online conference (step S401). For example, if all the participants perform the same action (step S402; Yes), the server 3 cuts out an image at that moment from the video of each participant (users A, B, and C) and stores it in the storage unit 302 as a captured image. The image is saved in a predetermined image folder (step S403). The server 3 notifies the user terminals 5a, 5b, and 5c of the subject users A, B, and C that the image has been captured (step S404). Upon receiving the notification from the server 3, the user terminals 5a, 5b, and 5c of the subject users A, B, and C make notification indicating that the image has been captured (steps S405, S406, and S407). As in the first embodiment, the method of notification is, for example, to sound a shutter sound, or to effect-display windows 62, 63, 64, and 65 of the subject user on the conference screen 61 in FIG.

The server 3 generates a thumbnail image of the image taken (captured) in step S403 (step S408), and transmits the shooting ID, shooting date and time, and captured image (thumbnail image) to each subject user A, B, and C. At the same time, an approval request is sent as to whether or not the image may be shared among the participants A, B, and C (step S409).

The user terminals 5a, 5b, and 5c of the subject users A, B, and C display the photographing ID 67a, the photographing date and time 67b, and the captured image 67c (thumbnail image) received from the server 3 in the image list 67, )” button 67g and “Reject” button 67h are displayed, and a response as to approval or rejection is accepted (steps S410, S411, S412).

When any response of approval/rejection is input in the user terminals 5a, 5b, and 5c of subject users A, B, and C, the user terminals 5a, 5b, and 5c transmit the photographing ID and approval information (approval or rejection). It is transmitted to the server 3 (steps S413, S414, S415). The server 3 records the received approval information in the shared management information 361 (step S416).

The server 3 makes the captured image 67c sharable when receiving responses of "approval" from all the subject users A, B, and C, and displays the captured image 67c (thumbnail image) and the sharable icon 67d as images of all the sharers. Displayed in list 67. Also, if even one of the subject users A, B, and C is not approved ("rejected"), the captured image 67c (thumbnail image) of the corresponding photographing ID is displayed as non-appropriate on the user terminal 5 of each of the subject users A, B, and C. It is displayed (steps S418 to S421). Processing after the end of the conference is the same as in the first embodiment.

In the process of FIG. 13, the server 3 records the captured image under the condition that all the participants perform the same action, but the condition is not limited to this. For example, the server 3 monitors the video of each participant, analyzes facial expressions and body movements, and detects when all the participants make the same facial expression (for example, laughing) at the same time, or when all the participants make the same gesture at the same time. (for example, toasting), an image of that timing is captured from the video of each participant.

In this manner, the server 3 judges the timing of capture and takes a picture, so that the participants can concentrate on the conference without missing the timing of the picture. In addition, each participant is notified that the image has been captured, and the participant can respond with approval/rejection as to whether or not the image may be shared. Therefore, only images that the user likes can be shared.

Although the preferred embodiments of the imaging system and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications or modifications within the scope of the technical ideas disclosed in the present application, and these also belong to the technical scope of the present invention. Understood.

1 ……………… Shooting system 3 ……………… Server 5, 5a to 5 e ………… User terminal 6 ………… Conference room 7 ……………… Network 10 ……………… Photographing device 31 ………………Online conference management unit 32 ………………Photographing instruction receiving unit 33 ………………Capture unit 33 ………………Image acquisition unit 34 ……………… …Capture notification unit 35 …………Approval request unit 36 ………………Sharing management unit 37 …………Group photo generation unit 38 ………………Image providing unit 39 ………… ……Image output unit 51 ………………Online conference processing unit 52 ………………Photographing instruction unit 53 ………………Photographing instruction transmission unit 54 …………Approval response unit 55 … ………………Image list display unit 61………………Conference screens 62 to 65…………Window 66…………Photographing effect 67………………Image list 67c…………Capture Image (thumbnail image)
67d ………………Shareable icon 67c …………Share prohibited icon 67g …………Approve button 67h …………Reject button 68 …………Photograph button 361 …………Share management information

Claims (16)

In a system that allows online conferences to be held between multiple terminals connected to a server via a network,
The server is
Online conference management means for managing entry into and exit from the online conference, receiving video and audio transmitted from terminals of participants participating in the online conference, and transmitting the video and audio to terminals of other participants. When,
a capturing means for cutting out an image from a video of a participant as a subject and saving it as a captured image;
Approval request means for transmitting an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject;
When approval is obtained from the participant who is the subject, the captured image can be shared with other participants who are subjects, and when approval is not obtained, the captured image cannot be shared. a shared management means;
The terminal is
A photographing system, comprising: response means for responding to approval or rejection of an approval request from the server. The terminal is
a photographing instruction means for designating a participant as a subject and instructing photographing on the screen on which the video of the participant of the online conference is displayed;
transmitting means for transmitting identification information of a participant specified by the photographing instruction means and information on the date and time of photographing to the server as photographing instruction information;
The capture means of the server is
2. The photographing system according to claim 1, wherein an image is clipped from a video of a participant as a subject based on photographing instruction information transmitted from said terminal and saved as a captured image. The capture means is
3. The imaging system according to claim 1, wherein the video of each participant is monitored, and if a predetermined condition is satisfied, an image is clipped from the video of each participant and saved as a captured image. The server is
4. The method according to any one of claims 1 to 3, further comprising image providing means for providing said captured image made sharable by said sharing management means so that it can be used among shared participants. shooting system. 5. The photographing system according to claim 4, wherein the image providing means makes the captured image available even after the online conference ends. The server is
a group photo generation means for generating a group photo by combining the captured images made sharable by the sharing management means;
5. The photographing system according to claim 4, wherein said image providing means provides said group photograph to participants included in said group photograph. The server comprises transmission means for transmitting the captured image to each participant's terminal during the online conference,
7. The photographing system according to claim 1, wherein said terminal comprises image list display means for displaying a list of captured images received from said server. 8. The photographing system according to claim 7, wherein the image list display means does not display the captured images that are prohibited from being shared by the sharing management means. 8. The photographing system according to claim 7, wherein said image list display means displays a message to the effect that the captured images that have been determined to be unshareable by said share management means are unshareable. The server is
10. The photographing system according to any one of claims 1 to 9, further comprising notification means for transmitting a notification to a terminal of a participant who is a subject when the image is captured by said capture means. The group photo generation means is
7. The photographing according to claim 6, wherein a keyword is extracted from the conversation information during the online conference, an image related to the extracted keyword is acquired, and the acquired image is set as a background image of the group photo. system. A server communicatively connected to terminals of participants of an online conference via a network,
Online conference management means for managing entry into and exit from the online conference, receiving video and audio transmitted from terminals of participants participating in the online conference, and transmitting the video and audio to terminals of other participants. When,
a capturing means for cutting out an image from a video of a participant as a subject and saving it as a captured image;
Approval request means for transmitting an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject;
When approval is obtained from the participant who is the subject, the captured image can be shared with other participants who are subjects, and when approval is not obtained, the captured image cannot be shared. a shared management means;
A server characterized by comprising: A terminal of an online conference participant connected to a server via a network for communication,
receiving means for receiving, from the server, an approval request as to whether or not a captured image taken with the user as a subject can be shared in the online conference;
response means for responding to the approval request with approval or disapproval;
A terminal characterized by comprising: A shooting method in a system capable of executing an online conference between a plurality of terminals connected to a server via a network,
The server manages entry into and exit from the online conference, receives video and audio transmitted from terminals of participants participating in the online conference, and transmits them to terminals of other participants. a step;
a step in which the server cuts out an image from the video of the participant as a subject and saves it as a captured image;
a step in which the server transmits an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject;
a step in which the terminal responds to the approval request from the server with approval or disapproval;
When the server obtains approval from the participant who is the subject, the captured image can be shared with the participant who is another subject, and when the approval is not obtained, the captured image is shared. a step of disabling sharing;
A photographing method comprising: Computers that are connected via a network to the terminals of the participants of the online conference,
Online conference management means for managing entry into and exit from the online conference, receiving video and audio transmitted from terminals of participants participating in the online conference, and transmitting the video and audio to terminals of other participants. ,
capture means for cutting out an image from the video of the participant as a subject and saving it as a captured image;
Approval request means for transmitting an approval request as to whether or not the captured image can be shared to the terminal of the participant who is the subject;
When approval is obtained from the participant who is the subject, the captured image can be shared with other participants who are subjects, and when approval is not obtained, the captured image cannot be shared. shared management means,
A program to function as A computer used as a terminal for participants in an online conference that is connected to a server via a network,
Receiving means for receiving from the server an approval request as to whether or not a captured image photographed with itself as a subject can be shared in an online conference;
response means for responding to the approval request with approval or disapproval;
A program to function as

Images