JP6579229B2 - Information processing apparatus, information processing system, control method thereof, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing system, a control method thereof, and a program.

  In recent years, web conferencing systems have become widespread, and the number of users has increased with the spread. An image of a plurality of bases is displayed on one display screen, and the display locations of the plurality of bases are switched or the size is changed.

  By displaying a plurality of images on one screen, there is a problem that the screen becomes difficult to see as the number of participants in the conference increases.

  In Patent Document 1, in order to automatically change the layout of a screen, when a single screen is divided and images from a plurality of locations are displayed, two types of screens, a parent screen and a child screen, are provided. A technique for changing the display position / size of the sub-screen according to the display setting and the combination thereof is described.

JP 2000-350175 A

  In a meeting, there are cases where not only images of participants in the meeting but also materials such as documents are shared and viewed. When viewing a shared material, the conference proceeds based on the material. Therefore, it is necessary for a participant of the conference to view the shared material with priority.

  Although the technique described in Patent Document 1 can automatically change the layout, the priority of image display cannot be shared among a plurality of terminals. Therefore, each client terminal individually desires a desired image. Depending on the user, the conference is continued without displaying the image to be viewed on the screen.

An object of the present invention is to provide a mechanism capable of easily controlling whether images can be exchanged in a display area where different attributes can be displayed .

The present invention provides a display control means for controlling to display an image having a first attribute or an image having a second attribute in a first display area, an image displayed in the first display area, and a second and receiving with means for Ru accepted change instruction of the image display area, when the image displayed in the first display area is the image of the first attribute intends rows incoming Les replacement of an image by the change instruction And a replacement control means for controlling to perform image replacement according to the replacement instruction when the image displayed in the first display area is an image of the second attribute. To do.

According to the present invention, it is possible to provide a mechanism capable of easily controlling whether images can be exchanged in a display area where different attributes can be displayed .

It is a figure which shows an example of a structure of the information processing system in embodiment of this invention. It is a figure which shows an example of the hardware constitutions of various apparatuses in embodiment of this invention. It is a figure which shows an example of a function structure of the various apparatuses in embodiment of this invention. It is a flowchart which shows the flow of the data sharing process in a host terminal in embodiment of this invention. It is a flowchart which shows the flow of a data sharing process in embodiment of this invention. It is a flowchart which shows the flow of the screen display switching process in embodiment of this invention. It is a figure which shows an example of a structure of the various data table in embodiment of this invention. It is a figure which shows an example of the mode of the screen display switching process in embodiment of this invention. It is a figure which shows an example of the full screen display of shared data in embodiment of this invention. It is a flowchart which shows the flow of the display process of the display screen in embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram illustrating an example of the configuration of an information processing system in an embodiment of the present invention.

  As shown in FIG. 1, the information processing system of the present invention includes a server 200 and a client terminal 100 (client terminal 100A, client terminal 100B, client terminal 100C, client terminal 100D / hereinafter, generically referred to as client terminal 100). , And are configured to be communicable via the network 101.

  The server 200 has various functions for executing a web conference using the client terminal 100. For example, creation and storage of a room for a web conference, identification information of the client terminal 100 and a camera image (conference image / video) that is received from each client terminal, for example, image data obtained by imaging a user who is a conference participant, And storage of the association information, control of transmission processing of the image data to each client terminal, and the like.

  The client terminal 100 is a terminal having a camera function and a touch panel function (operation unit). Image data (conference image / video / dynamic data) as a video obtained by logging into the server 200 in response to an operation from the user (for example, in response to an authentication information input / transmission instruction) and captured by the camera. Are sequentially transmitted to the other client terminals 100 entering the same room (participating in the conference) via the server 200.

  In addition to the image data acquired from the camera, data stored in the client terminal 100, for example, static data such as photographic images and document data such as PDF files, can be transferred to other clients via the server 200. A data sharing function for transmitting and sharing to the terminal 100 is provided.

  When the static data is transmitted to another client terminal 100, for example, all the client terminals 100 perform control to display the static data on the display screen. In the data sharing function, the data (shared image) in the same state is displayed on the plurality of client terminals 100 as the data being shared among the plurality of client terminals. That is, when the shared image is enlarged on the client terminal 100A, the shared image being displayed on the other terminals (client terminals 100B to 100D) is also enlarged at the same enlargement position / enlargement rate.

At this time, if operation / editing for the displayed shared image (for example, an image of document data) from all terminals is enabled, operations (processing corresponding to each operation (display control)) compete. Since there is a possibility, here, the client terminal 100 that has started data sharing is used as a host, and control is performed so that only the host terminal can edit the data being shared.
As an example of the operation conflict, there is a case where an enlargement operation and a reduction operation are simultaneously performed from a plurality of terminals.

  Specifically, for example, the client terminal 100A enlarges the document data being shared in response to an operation from a user who wants to enlarge and view the document material, and immediately thereafter, the client terminal 100B reduces the document material. This is a case where the reduction processing of the shared document data is performed according to the operation from the user who wants to browse.

  In this case, immediately after the enlarged image is transmitted from the client terminal 100A to each client terminal 100 via the server 200 and displayed on each client terminal 100, the reduced image is sent from the client terminal 100B to the server 200. Since the data is finally transmitted to each client terminal 100 and displayed on each client terminal 100, the data finally displayed on the display screen of each client terminal 100 is the data after the reduction process, and the operation of the client terminal 100A The user who is a user cannot view the desired document data after the enlargement process.

  The operation conflict is not limited to the enlargement / reduction operation. For example, in an operation such as page feed, there may be an operation conflict such as a process of returning the page immediately after the page is advanced.

  Here, in order to prevent the above-described operation conflict (contention of image display processing corresponding to the operation), each client terminal 100 uses only the client terminal 100 that started data sharing as a host. However, control is performed so that the data being shared can be edited.

  In FIG. 1, for example, the host terminal in the data sharing process = client terminal 100A, and each of the client terminals 100B to 100D is a client terminal for the host. The above is the description of an example of the configuration of the information processing system in the embodiment of the present invention shown in FIG.

  Next, an example of the hardware configuration of various devices in the embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a hardware configuration of various apparatuses according to the embodiment of the present invention.

  In FIG. 2, reference numeral 201 denotes a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, the ROM 202 or the external memory 211 is necessary to realize a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as an OS), which is a control program of the CPU 201, or a function executed by each server or each PC. Various programs to be described later are stored.

  A RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for execution of processing from the ROM 202 or the external memory 211 into the RAM 203 and executing the loaded program.

  An input controller 205 controls input from a keyboard (KB) 209 or a pointing device such as a mouse (not shown). A video controller 206 controls display on a display device such as a CRT display (CRT) 210. In FIG. 2, although described as CRT 210, the display device is not limited to the CRT, but may be another display device such as a liquid crystal display. These are used by the administrator as needed.

  In the embodiment of the present invention, the CRT 210 of the client terminal 100 has a touch panel function. When the CRT 210 has a touch panel function, the CRT 210 also functions as an input device, and the touch operation information detected by the touch panel function (received by the display) is transmitted to the input controller as input information. That is, the input controller 205 performs input control from the CRT 210.

  A memory controller 207 is connected to the hard disk (HD), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, etc. via an adapter. The access to the external memory 211 such as a compact flash (registered trademark) memory is controlled.

  A communication I / F controller 208 connects and communicates with an external device via a network (for example, the network 101 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the CRT 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. In addition, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the CRT 210.

  Various programs to be described later for realizing the present invention are recorded in the external memory 211 and executed by the CPU 201 by being loaded into the RAM 203 as necessary. Furthermore, definition files and various information tables used when executing the program are also stored in the external memory 211, and a detailed description thereof will be described later.

  An acceleration sensor 212 detects, for example, the acceleration of the client terminal 100. When the acceleration sensor is attached to an apparatus such as the client terminal 100 as an external apparatus, for example, the acceleration sensor 212 communicates acceleration information detected by communicating with the input controller 209 to the client terminal 100. The above is an explanation of an example of the hardware configuration of various apparatuses in the embodiment of the present invention shown in FIG.

  Next, with reference to FIG. 3, an example of a functional configuration of various apparatuses in the embodiment of the present invention will be described. FIG. 3 is a diagram illustrating an example of a functional configuration of various apparatuses according to the embodiment of the present invention.

  The room entry request unit 301 is a request unit that makes a room entry request to the room (conference room for web conference) created and stored in the server 200 for the client terminal 100 (a user who is logged in to the server 200 at the client terminal 100). .

  The image data transmission / reception unit 302 transmits video (image data) captured by the camera device installed in the client terminal 100 to another client terminal via the server 200, and from the other client terminal via the server 200. A transmission / reception unit for receiving transmitted image data.

  The image data display unit 311 displays a display screen (for example, the display screen 800 in FIG. 8) that displays the image data transmitted / received by the image data transmission / reception unit 302 and shared data transmitted / received by the data transmission / reception unit described later. It is a display part to do.

  Here, an example of a conference image display screen will be described with reference to FIG. A main screen 801 is a screen (display unit) that displays a camera image from another client terminal on the display screen 800 or an image of data being shared with another terminal by a data sharing function. It is a screen.

  A child screen 802, a child screen 803, a child screen 804, and a child screen 805 are screens for displaying a reduced camera image from another client terminal, and are sub screens in the front screen 8800. In the sub-screens 802 to 805, in addition to camera images from other client terminals, a data sharing function activation icon, a chat function activation icon, and the like are displayed. The “leave room” button 806 is a button for leaving the room being entered.

  Returning to the description of FIG. The screen ID storage unit 303 is a storage unit that stores IDs that are identification information of a plurality of image data display units (screens) displayed on the display screen of the client terminal 100. The storage unit stores the screen ID and the image data transmitted / received by the transmission / reception unit in association with each other, and the client terminal 100 stores the associated image data on the screen indicated by the associated screen ID ( Conference image).

  The shared data specifying unit 304 is a specifying unit that specifies a shared image, which is data shared (displayed in synchronization) with other client terminals by the data sharing function, according to a user operation. The data transmission / reception unit 305 is a transmission / reception unit that transmits the shared image to another client terminal via the server 200 and receives the shared image transmitted from the other client terminal via the server 200.

  The operation target screen specifying unit 306 is a specifying unit that specifies which of the plurality of screens displayed on the display screen is the screen that has received an operation from the user. The operation type specifying unit 307 is a specifying unit that specifies the type of operation received from the user. The type of operation is, for example, an operation such as a tap, a double tap, a swipe, a pinch in, or a pinch out.

  Note that the type of the operation is not limited to the above contents. For example, operations such as click, double click, drag and drop using a mouse device may be used.

  The screen state storage unit 308 is a storage unit that stores the states of a plurality of screens displayed on the display screen. The state of the screen is, for example, the image displayed on the screen (the image data stored in association with the screen ID of the screen), the shared image data, or the result transmission / reception captured by the camera device The image data of the conference image being displayed.

  The image replacement processing unit 309 performs a process of replacing the displayed image among a plurality of screens displayed on the display screen in accordance with an operation from the user. Specifically, when a child screen tap operation (replacement operation) is received (corresponding to a replacement operation reception unit), a display image replacement process with the parent screen is performed.

  The replacement prohibition notification unit 310 is a notification processing unit that notifies the user that the replacement is prohibited when the replacement processing of the image data of the screen that has received an operation from the user is prohibited. Specifically, when the shared image is displayed on the parent screen, the replacement process of the display image of the parent screen and the child screen is prohibited, and when the tap operation on the child screen is accepted, the child that accepted the tap operation is accepted. By vibrating the screen, it is notified that the replacement of the display image of the child screen and the parent screen is prohibited.

  The entrance / exit management unit 321 is a storage unit that stores a user (client terminal 100) entrance / exit state with respect to the web conference room (room) stored in the external memory of the server 200. Specifically, when a room entry request is received from the client terminal 100 and entry is permitted, the login user ID acquired from the requesting client terminal 100 that is logged in to the server 200 is identified with the room identification. When the user leaves the room, the user ID of the logged-in user is deleted from the table stored in association with the room.

  The image data receiving unit 322 is a receiving unit that receives a camera image transmitted from each client terminal 100. The image data storage unit 323 is a storage unit that temporarily stores the received camera image for each client terminal (user ID). The image data transmission unit 324 transmits the camera image from time to time to each client terminal entering the same room as the room where the client terminal 100 that is the transmission source (acquisition source) of the camera image is entering.

  The shared data receiving unit 325 is a receiving unit that receives shared images (photo image data, document data, etc.) to be shared from the client terminal 100 as a host. The shared data storage unit 326 is a storage unit that temporarily stores the received shared image. The shared data transmission unit 327 transmits the shared image to each client terminal that is entering the same room as the room where the client terminal 100 that is the source of the shared image is entering, as needed. The above is an explanation of an example of the functional configuration of various apparatuses in the embodiment of the present invention shown in FIG.

  Next, with reference to FIG. 10, the flow of the display screen display process in the embodiment of the present invention will be described. FIG. 10 is a flowchart showing the flow of display screen display processing according to the embodiment of the present invention.

  The CPU 201 of the client terminal 100 activates the conference application in response to an instruction from the user (step S1001), and transmits user information registered in the conference application to the server 200 in advance to log in to the server 200. That is, a login request is transmitted (step S1002).

  The user information here is information such as the user information 700 of FIG. The user information 700 includes a user ID 701, a user name 702, a password 703, and the like. The user ID 701 is identification information for uniquely identifying the user, and the user name 702 is the name of the user corresponding to the user ID. The password 703 is a password corresponding to the user ID 701 used for logging in to the server 200.

  The CPU 201 of the server 200 receives the login request including the user information 700 (step S1003), performs login processing using the user information (specifically, user ID and password), and displays the result of the login processing. It transmits to the client terminal 100 (step S1004).

  The CPU 201 of the client terminal 100 receives the login result (step S1005). Here, the following explanation will be given on the assumption that the login was successful.

  As a result of successful login, the CPU 201 of the client terminal 100 receives an instruction to select a room to enter from a room selection screen (not shown) displayed on the display screen, and transmits a request to enter the selected room to the server 200. .

  The CPU 201 of the server 200 receives the room entry request and executes room entry processing (step S1007). That is, the user ID (user ID 712) of the logged-in user who entered the room is stored in association with the room identification information (room ID 711 in the room information 710 in FIG. 7) stored in advance in the external memory of the server 200. .

  Thereafter, the result of the room entry process (information on which room the user has entered / room information 710) is transmitted to the client terminal 100. When transmitting the room information, the upper limit value of the communication bandwidth, the upper limit value of the voice quality, the conference value that can be used by each client terminal 100 when using the room, linked to the room ID of the room, in the external memory of the server 200 When the upper limit value of the resolution of the image is stored, the upper limit value is transmitted together.

  When the client terminal 100 receives the upper limit value, the client terminal 100 changes the subsequent conference image transmission and audio transmission to the upper limit value or less and transmits them to the other client terminals 100 via the server 200. It shall be.

  The CPU 201 of the client terminal 100 receives the information on the result of entering the room (step S1009). Then, the camera device is activated to execute an imaging process (step S1010).

  The CPU 201 of the client terminal 100 transmits the camera image (captured image / conference image) acquired by imaging in the imaging process to the server 200 (corresponding to step S1011 / captured image transmission unit and image transmission unit). The CPU 201 receives the camera image (corresponding to step S1011 / image receiving means) and stores the camera image in the external memory in association with the user ID. For example, as shown in the conference image association information 730 in FIG. 7, the user ID 701 of the login user is inserted into the user ID 731, and the camera images that are being received from the client terminal 100 are sequentially stored in the conference image 732. Process.

  The camera image stored in the conference image 732 has only a predetermined data amount (data capacity / number of frames), and is sequentially transmitted to the client terminals 100 of other users who are entering the room. That is, a user ID different from the user ID of the transmission source of the camera image in the room information 710 is specified, and the conference image 732 is transmitted to the client terminal 100 corresponding to the user ID different from the user ID of the transmission source. The camera image stored in is transmitted.

  It is assumed that information in which the user ID is associated with the client terminal is generated by the CPU 201 of the server 200 when the room entry process is completed and stored in the external memory of the server 200. For example, it is information in which the user ID of the user entering the room and the identification information (for example, IP address) of the client terminal 100 received by the server 200 from the client terminal 100 in accordance with the room entry instruction are stored in association with each other.

  The CPU 201 of the server 200 identifies another user who is entering the same room as the room in which the user is entering, with reference to the room information 710, and the user ID 731 of the other user and the meeting corresponding to the user ID 731. The camera image of the image 732 is transmitted to the client terminal 100 (step S1014). That is, the conference image association information 730 is transmitted.

  At this time, the order of user entry into the room (not shown) is transmitted to the client terminal 100 together. It is assumed that the CPU 201 of the server 200 generates information stored in the external memory every time a user enters the room, in the order in which the users enter the room.

  The CPU 201 of the client terminal 100 receives the camera image of the other user (step S1015), and displays the screen (display) of the conference image corresponding to the user for the number of users entering the room including the user of the own terminal. Screen IDs (status 740 and screen ID 741 in FIG. 7) that are IDs of the main screen 801 to the child screen 805 in FIG. 8 and the ID of the user who is entering the same room as the screen ID. Each user ID is stored in association with each other. For example, the user ID is stored in the user ID 742 of the status 740 in FIG. Further, the conference image association information 730 is stored in an external memory (corresponding to an acquisition source storage unit).

  Here, it is assumed that screen IDs are generated and stored in the order of the parent screen 801, the child screen 802, the child screen 803, the child screen 804, and the child screen 805. Further, the screen ID (numerical value) is smaller as the generation order is faster, and is larger as the generation order is later.

  The CPU 201 of the client terminal 100 acquires screen IDs in ascending order, and stores each user ID in association with the screen ID in the user entry order received in step S1015. That is, the user ID of the user who entered the room first is stored in association with the parent screen.

  The status 740 is information indicating the state (status) of each screen in the display screen 800 stored in the external memory of the client terminal 100. The sharing flag 743 is a flag indicating whether or not the screen indicated by the screen ID 741 is being used as a display unit for a shared image shared by the data sharing function. When the data sharing flag 743 = 0, the screen indicated by the screen ID 741 is being used as a conference image display unit. When the sharing flag 743 = 1, the screen indicated by the screen ID 741 is used as a shared image display unit.

  The host flag 744 is a host whose terminal displaying the screen is the sharing source of the shared image when the screen indicated by the screen ID 741 is used as a display unit of the shared image shared by the data sharing function. Indicates whether it is a terminal. When the host flag 744 = 1, the client terminal 100 storing the status 740 = host terminal (shared image sharing source). When the host flag 744 = 0, the client terminal storing the status 740 100 = client terminal (shared image sharing destination) for the host terminal.

  The full screen flag 745 indicates whether or not the screen indicated by the screen ID 741 is in a full screen display state. When the full screen flag 745 = 1, it is a full screen display state (shown in FIG. 9), and when the full screen flag 745 = 0, it is a normal display state (shown in FIG. 8). The screen type 746 indicates whether the screen indicated by the status 740 is a parent screen or a child screen. Note that a value indicating the parent screen is inserted in the screen type 746 corresponding to the smallest screen ID (screen ID generated first), and the child type is assigned to the screen type 746 corresponding to other screen IDs. A value indicating the screen is inserted.

  The CPU 201 of the client terminal 100 stores the image data corresponding to the user ID and the screen ID in association with each other according to the association between the screen ID and the user ID in the status 740 (step S1017). For example, the screen association information 760 in FIG. 7 is generated. The screen ID 761 of the screen association information 760 is identification information of each screen (such as a parent screen 801 to a child screen 805) in the display screen 800.

  The conference image 762 and the shared image 763 are images displayed on the screen indicated by the screen ID 761 stored in association with the screen ID 761. Since either the conference image 762 or the shared image 763 is displayed on the screen indicated by the screen ID 761, if the actual data of the camera image is stored in the conference image 762, the shared image 763 has a null value. Is inserted, and the actual data is not stored. Further, when the shared image real data is being stored in the shared image 763, a null value is inserted into the conference image 762, and the actual data is not stored.

  The CPU 201 of the client terminal 100 generates the display screen 800 of FIG. 8 according to the information of the status 740 and the screen association information 760 (step S1018) and displays it on the display screen (step S1019). The above is the description of the display screen display processing flow in the embodiment of the present invention shown in FIG.

  Next, a flow of data sharing processing in the host terminal in the embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing a flow of data sharing processing in the host terminal according to the embodiment of the present invention.

  In the description after FIG. 4, it is assumed that the client terminal 100A to the client terminal 100D have already logged into the server 200 in response to an operation instruction from the user and are entering the same room.

  In step S401, the CPU 201 of the client terminal 100A receives an instruction from the user, and determines whether the received instruction is a data sharing start instruction (step S402). That is, it is determined whether a tap instruction for the data sharing function activation icon displayed on the display screen 800 has been received.

  If it is determined that the received instruction is a data sharing start instruction (YES in step S402), the server 200 is notified that data sharing is to be started (step S403).

  The CPU 201 of the server 200 receives the notification (step S415), and stores the host ID 721 (identification information for identifying the host terminal / host user) of the host information 720 in FIG. The user ID included in the sharing start notification is updated (step S416). The host information 720 is generated and stored for each conference room (room / room ID). Immediately after the room is generated, a null value is inserted for both the host ID 721 and the shared image 722, and it is assumed to be blank. The shared image 722 stores image data such as a document shared by data sharing.

  The CPU 201 of the server 200 notifies the host ID 721 and the notification of the start of data sharing to the other client terminals (client terminals 100B to 100D) in the room (step S417) and ends the process. The subsequent processing of the client and the terminal that receives the data sharing start notification will be described later with reference to FIG.

  On the other hand, the CPU 201 of the client terminal 100A that has notified the start of data sharing in step S403 refers to the status 740 of FIG. 7 stored in the external memory in association with the screen ID of the parent screen 801, and displays the parent screen. The value of the data sharing flag 743 corresponding to the screen ID 741 is changed from 0 to 1 (step S404). The data sharing flag 743 is a flag indicating that the information displayed on the parent screen is a shared image. Also, the value of the host flag 744 is changed to 1.

  Thereafter, the CPU 201 of the client terminal 100A displays a data selection application selection screen stored in the client terminal 100A on the display screen in order to select data to be shared (step S405). It is assumed that the data selection application is registered and stored in advance in the external memory of the client terminal 100A.

  The data selection application here is, for example, an application for storing and browsing a photographic image that cannot be browsed by the conference application, an application for storing and browsing document data, and the like.

  When the CPU 201 of the client terminal 100A receives, for example, selection of a photo browsing application, the CPU 201A transmits a display request for a photo selection screen of the photo browsing application to the API of the photo browsing application (step S407).

  The API of the photo browsing application transmits the display request to the photo browsing application, and the received photo browsing application (step S418) displays a photo selection screen on the display screen (step S419).

  The CPU 201 of the client terminal 100A accepts an instruction from the user (step S420 / shared image selection accepting unit), and determines whether the accepted instruction is a data (here, photo image) selection instruction (step S421).

  If the received instruction is not a data selection instruction (NO in step S421), the received instruction is determined to be an instruction to end the selection screen for the photo, and the process returns to step S401.

  If the received instruction is a data selection instruction (YES in step S421), the photo browsing application transmits information on the selected data (photo image) to the conference application (step S422), and the process of FIG. Return to step S401. The conference application receives this (step S408), and inserts the shared image (photograph image) into the shared image 763 corresponding to the screen ID 761 of the parent screen in the screen association information 760 of FIG.

  When the shared image is inserted into the parent screen, an additional child screen for displaying the conference image displayed on the parent screen before the insertion processing is additionally generated (that is, one screen ID 741 is generated). Add), a process of inserting the conference image displayed on the parent screen before the insertion process into the added child screen is performed (step S428).

  The CPU 201 of the client terminal 100A transmits the data received in step S408 to the server 200 (corresponding to step S409 / shared image transmitting means), and the CPU 201 of the server 200 receives this (corresponding to step S423 / image receiving means). ) And stored in the shared image 722 in association with the host ID 721 of FIG. 7 (step S424). Thereafter, the process proceeds to step S508 in FIG. The processing after step S508 will be described later with reference to FIG.

  When the CPU 201 of the client terminal 100A determines that the instruction received in step S401 is not a data sharing start instruction (NO in step S402), the received instruction is a display image switching instruction (display image of the parent screen and the child screen). In step S413).

  If the received instruction is a display image switching instruction (YES in step S413), a process of switching the display image and the other parent screen or child screen and display image (replacement process) is performed (step S414). Details of the display image switching process will be described later with reference to FIG.

  If the received instruction is not a display image switching instruction (NO in step S413), it is determined whether the received instruction is an instruction to end the data sharing function (step S410).

  If the data sharing flag 743 = 0 in FIG. 7, the data sharing is not performed, so the process of step S410 is skipped and the process proceeds to step S412.

  If the received instruction is an instruction to end the data sharing function (YES in step S410), a data sharing end notification is transmitted to the server 200 (step S411). The CPU 201 of the server 200 receives the notification (step S425), updates the value of the host ID 721 to null (step S426), and transmits an end notification of the data sharing function to the client terminal 100B to the client terminal 100D ( Step S427). Processing of the client terminal 100B to the client terminal 100D after receiving the notification will be described later with reference to FIG.

  If the received instruction is not an instruction to end the data sharing function (NO in step S410), the CPU 201 of the client terminal 100A determines whether the received instruction is an instruction to end the conference application (step S412). For example, it is determined whether a tap instruction for the “exit” button 806 has been received.

  If the received instruction is a conference application termination instruction (YES in step S412), the process is terminated. If the received instruction is not a conference application termination instruction (NO in step S412), for example, on the main body of the client terminal 100A. When an increase / decrease operation of the installed audio volume is accepted, a process according to the operation instruction is executed, and the process returns to step S401.

  If the received instruction is an instruction to end the conference application, and the data sharing flag 743 = 1 and the host flag 744 = 1 in FIG. 7, that is, data sharing processing is performed. If the terminal is the host terminal, the application is terminated after executing the data sharing termination notification process in step S411. The above is the description of the flow of data sharing processing in the host terminal in the embodiment of the present invention of FIG.

  Next, the flow of data sharing processing in the embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flowchart showing the flow of data sharing processing in the embodiment of the present invention.

  Note that at the time of step S501 in FIG. 5, each of the client terminals 100B to 100D completes the processing in FIG. 10 and is displaying the display screen 800. Here, as an example, the client terminal 100B will be described as a processing subject.

  The CPU 201 of the client terminal 100B receives a notification from the server 200 (step S501), and determines whether the received notification is a data sharing start notification (step S502). If it is determined that it is a data sharing start notification (YES in step S502), the value of the data sharing flag 743 corresponding to the screen ID 741 of the parent screen is referred to with reference to the status 740 of FIG. 7 stored in the external memory. It is changed from 0 to 1 (step S503).

  The CPU 201 of the client terminal 100B inserts the text “Connected” indicating that a shared image is being acquired by connecting to the server on the parent screen 801, and displays the conference image displayed on the parent screen 801. To generate an additional child screen (that is, add one status 740 and one screen ID 741), and store the conference image displayed on the parent screen in association with the added child screen. (Step S504 / Generate the screen association information 760 having the same screen ID as the additionally generated screen ID and insert the conference image into the conference image 762), and display a new display screen 810 (shown in FIG. 8). Generate.

  The CPU 201 of the client terminal 100B transmits a shared image acquisition request to the server 200 (step S507), the CPU 201 of the server 200 receives the request (step S508), and transmits the shared image together with the host ID to the client terminal 100B. (Step S509). That is, the host information 720 is transmitted.

  The CPU 201 of the client terminal 100B receives the shared image (step S510). Then, the received shared image (photo image) is inserted into the shared image 763 corresponding to the screen ID 761 of the parent screen in the screen association information 760 in FIG. 7, and the display screen is displayed on the display screen 820 (shown in FIG. 8). The state is updated and displayed (step S511). Further, the host information 720 is stored in the external memory.

  When determining in step S502 that the notification received in step S501 is not a data sharing start notification (NO in step S502), the CPU 201 of the client terminal 100B determines whether the received notification is a data sharing end notification (step S502). Step S512).

  If it is determined that the received notification is not a data sharing end notification (NO in step S512), the process returns to step S501. If it is determined that the received notification is a data sharing end notification (YES in step S512), the value of the data sharing flag 743 associated with the screen ID of the parent screen is changed to 0 (step S513).

  Also, the child screen newly generated in step S504 is deleted, and the conference image 762 stored in association with the screen ID 761 of the child screen is stored in association with the screen ID 761 of the parent screen. That is, the state of the display screen is returned to the state of the display screen 800 of FIG. 8 and displayed (step S514). The above is the description of the flow of data sharing processing in the embodiment of the present invention of FIG.

  Next, the flow of the screen display switching process in the embodiment of the present invention will be described with reference to FIG. FIG. 6 is a flowchart showing the flow of screen display switching processing in the embodiment of the present invention. The process in FIG. 6 is a detailed process in step S414 in FIG. 4 and is common to all client terminals 100.

  The CPU 201 of the client terminal 100 determines whether the received instruction is a pinch operation (pinch-in or pinch-out operation) (step S601). If the received instruction is a pinch operation (YES in step S601), the CPU 201 accepts the pinch operation. It is determined whether the screen is the parent screen 801 in the display screen 800 or the child screens 802 to 805 (step S602).

  When the screen that has received the pinch operation is a sub-screen in the display screen 800 (NO in step S602), the operation by the pinch operation is prohibited by vibrating the sub-screen that has received the pinch operation. This is notified (step S603). By doing so, the tap operation on the screen is effective (although the client terminal 100 detects the tap operation), but the operation corresponding to the operation is not performed because the operation is prohibited. Can be notified to the user.

  If the screen that accepted the pinch operation is the parent screen 801 in the display screen 800 (YES in step S602), it is determined whether the operation content is a pinch-in operation (step S604). If the operation content is a pinch-in operation (YES in step S604), the image displayed on the parent screen is enlarged (step S606). If the operation content is a pinch-out operation (NO in step S604), the parent The image displayed on the screen is reduced and displayed (step S605).

  If the received operation is not a pinch operation (NO in step S601), the CPU 201 of the client terminal 100 determines whether the received operation is a double tap operation (step S607), and determines that the received operation is not a double tap operation (step S607). NO), it is determined that the accepted operation is a tap operation, and the process proceeds to step S608.

  The CPU 201 of the client terminal 100 determines whether the screen that has received the tap operation is a parent screen or a child screen (step S608), and determines that the screen that has received the tap operation is the parent screen, the screen ID of the parent screen is displayed. Obtained from the external memory, obtains the status 740 including the screen ID of the parent screen, and refers to it (step S609).

  It is determined whether the full screen flag 745 of the status is 1 (that is, it is determined whether the parent screen on which the operation has been received is in the full screen display state / step S610), and in the full screen display state (YES in step S610), Display processing for hiding the toolbar displayed on the display screen is performed (transition processing of the display state from the display screen 900 of FIG. 9 to the display screen 910 is performed). If the display screen is not in full screen display (NO in step S610), Perform display control processing to vibrate the main screen.

  When the CPU 201 of the client terminal 100 determines that the screen that has accepted the tap operation is a child screen (determination in step S608), the CPU 201 acquires the screen ID of the child screen and the screen ID of the parent screen from the external memory. Each status 740 having each screen ID is acquired (step S613).

  Then, it is determined whether the data sharing flag of the parent screen status 740 = 1 (that is, it is determined whether a shared image is being displayed on the parent screen / step S614 / corresponding to image type specifying means), and data sharing of the parent screen status 740 is performed. When flag = 0 (that is, when a conference image is being displayed on the parent screen), it is determined to switch (replace) the images displayed on the child screen that has received the operation and the parent screen, and the determination The image replacement process is performed according to the above.

  Specifically, the value of the user ID 742 of the status 740 having each screen ID is exchanged and updated, and the conference image 762 corresponding to the user ID of the screen association information 760 is changed according to the exchange process. Replace and update. Then, the display screen is updated / generated based on the updated information and displayed (step S618).

  When the data sharing flag of the parent screen status 740 = 1 (that is, when a shared image is being displayed on the parent screen), it is determined not to replace the display image of the child screen that received the tap operation and the parent screen. Then, it is determined whether the host flag 744 = 1 (that is, it is determined whether the client terminal 100 that accepted the operation is a data sharing host terminal / step S615). If the host flag 744 = 1, that is, the host terminal is the host terminal (step S615). YES), a shared image editing process is performed in which a mark indicating a point of interest on the shared image is added to the position on the shared image where the tap operation has been accepted (step S616).

  When the shared image editing process is performed, the CPU 201 of the client terminal 100 transmits the edited shared image to the server 200, and the CPU 201 of the server 200 stores the edited shared image. , To be displayed on another client terminal 100.

  Note that the editing process is not limited to the above-described mark assignment. For example, when a swipe operation is accepted, a process of writing a line along the swipe trajectory may be performed.

  If the host flag 744 = 0, that is, if it is not a host terminal (NO in step S615), display control is performed to vibrate the sub-screen that has accepted the tap operation.

  When the CPU 201 of the client terminal 100 determines that the received operation is a double-tap operation (YES in step S607), the CPU 201 determines whether the screen that has received the operation is a parent screen or a child screen (step S619). If it is determined that the screen on which the operation has been accepted is a child screen, the process proceeds to step S615.

  If it is determined that the screen that has accepted the operation is the parent screen, the screen ID of the parent screen is acquired from the external memory, and the status 740 including the screen ID of the parent screen is acquired and referenced (step S620).

  It is determined whether the full screen flag 745 of the status is 1 (that is, it is determined whether the parent screen on which the operation has been received is in the full screen display state / step S621), and in the full screen display state (YES in step S621), it is full. The screen flag 745 is changed to 0, and the state of the display screen is shifted to a normal display state like the display screen 800 of FIG. 8 (step S622).

  If it is not in the full screen display state (NO in step S621), the full screen flag 745 is changed to 1, and the display screen state is shifted to the full screen display state as shown in the display screen 900 of FIG. 9 (step S623). . The above is the description of the flow of the screen display switching process in the embodiment of the present invention shown in FIG.

  In the above description, when a tap operation on the child screen is received while the conference image is displayed on the parent screen, the child screen and the parent screen display image are switched, and the child screen is displayed while the shared image is displayed on the parent screen. When the tap operation is received, the sub screen is vibrated to notify the user that the process (display image replacement process) by the operation is prohibited. For example, the mode information of FIG. Using information such as 750, it is determined whether the mode at the time of accepting the tap operation on the sub-screen is the shared mode in which the shared image is displayed or the conference mode in which only the conference image is displayed. May prohibit the replacement of the display image, and may allow the replacement of the display image during the conference mode.

  Hereinafter, an embodiment of the present invention using mode information 750 will be described.

  It is assumed that the mode information 750 is generated and stored by the CPU of the server 200 during room generation processing (not shown). The room ID 751 is room identification information, and the mode flag 752 indicates whether the web conference mode is a shared mode in which a shared image is displayed or a conference mode in which only a conference image is displayed. Here, it is assumed that the sharing mode is set when the mode flag 752 = 1, and the conference mode is set when the mode flag 752 = 0. When the mode information 750 is generated, a value of 0 is inserted into the mode flag 752 by default.

  When the display switching process (FIG. 6) is controlled using the mode information 750, the CPU 201 of the server 200 transmits the mode information 750 to the client terminal 100 in step S1008 of FIG.

  Further, in step S404 of FIG. 4, the CPU 201 of the client terminal 100A changes the value of the mode flag 752 of the mode information 750 to 1. When a data sharing end instruction is accepted, the value of the mode flag 752 is changed to zero.

  In step S417, the CPU 201 of the server 200 notifies the other client terminals 100 (100B to 100D) entering the room to change the value of the mode flag 752 of the mode information 750 to 1.

  In step S503 in FIG. 5, the CPU 201 of the client terminal 100B changes the value of the mode flag 752 of the mode information 750 to 1 (corresponding to an image type determination unit). In step S513, the value of the mode flag 752 of the mode information 750 is changed to 0.

  In step S613 in FIG. 6, the mode information 750 is also acquired, and in step S614, it is determined whether the mode flag 752 is 1 or 0. If the mode flag 752 = 1, the process proceeds to step S615. If the mode flag 752 = 0, the process proceeds to step S618. The above is the description of the embodiment of the present invention using the mode information 750.

In the description of the above embodiment, the enlargement / reduction processing of the image displayed on the main screen in steps S406 and S407 in FIG. 4 is performed. For example, the face of the conference partner is displayed like a conference image. Compared to the case where it is only necessary to confirm, when displaying a document or the like like a shared image, there may be a case where details are confirmed by an operation such as enlargement / reduction. However, if the enlargement / reduction processing by pinch operation is always enabled, for example, when the user uses the terminal while holding it in his / her hand, such as a tablet terminal, the hand may suddenly touch the screen. If the screen display is enlarged / reduced each time, the user needs to pay attention not to touch the screen, which is very troublesome.
On the other hand, for example, the validity / invalidity of the enlargement / reduction processing may be switched according to the type of image displayed on the parent screen (image type / conference image or shared image).

  That is, if it is determined in step S602 in FIG. 6 that the received operation is a pinch operation for the parent screen, the status 740 of the parent screen is acquired and referenced from the external memory, and the data sharing flag 743 = status 740 = In the case of 1 (that is, when the shared image is being displayed), the processing ends without performing any enlargement / reduction processing. When the data sharing flag 743 = 0 (that is, when the conference image is being displayed), You may make it transfer to step S405.

  By doing this, it is possible to switch between enabling / disabling of the enlargement / reduction operation according to the type of image displayed on the screen, and when the user needs enlargement / reduction processing according to the operation, The terminal can execute processing corresponding to the operation.

  In the above-described embodiment, the association between the screen ID and the user ID and the association between the screen ID and the conference image according to the association are performed in the order of entering the user's room. Since the conference image includes the conference image of the user who is an operator of the client terminal 100, when the user who is the operator is the first person entering the room, the parent screen (a large image is displayed). On the screen), the operator's own image will be displayed.

  In the case of a web conference, the conference image desired by the user is presumed to be the image of the conference partner. For example, the association between the screen ID and the user ID, and the screen ID and the conference image according to the association The association is performed by removing the user ID of the user of the client terminal 100 that performs the association processing, and the image of the user of the client terminal 100 is displayed on the last child screen (the child screen having the largest numerical value as the screen ID). May be inserted.

  In addition, in order to make it possible for the user of each client terminal 100 to confirm which user has the editing and operation authority for the shared image, which user is the provider of the shared image ( It may be displayed in an identifiable manner, and the user may be notified.

  For example, during display of a shared image, it is conceivable to arrange a conference image (image of a user of the host terminal) transmitted from the host terminal at the top of the child screen. Specifically, the client terminal 100B (others 100C and 100D) refers to the host ID included in the data sharing start notification received in step S501 of FIG. 5 (corresponding to the sharing source specifying unit), and step S504. When the association information between the screen ID and the user ID is updated and the association information between the screen ID and the conference image is updated, the shared image is associated with the parent screen, and the shared image is associated with the parent screen. If it is determined (corresponding to the image type determining means), it is determined to update the user ID indicated by the host ID in association with the top screen ID of the child screen (corresponding to the priority determining means), and the updated This can be realized by generating the display screen 810 based on the information.

  In the above-described embodiment, when the shared image is inserted and displayed on the parent screen, the conference image displayed on the parent screen is added to the end of the child screen (see FIG. 8). For example, the conference image displayed on the main screen is moved down to the first sub-screen (the sub-screen 802 in FIG. 8), and thereafter, each conference image displayed on the sub-screen is screen ID one behind. The screen ID and the user ID and the association information between the screen ID and the conference image may be updated so that the information is displayed on the child screen.

  As described above, according to the present invention, when the display images are exchanged between the screens, it is possible to switch whether the images can be exchanged according to the type of the image displayed on the screen.

  It should be noted that the present invention can be implemented as, for example, a system, apparatus, method, program, or storage medium, and can be applied to a system composed of a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  Note that the present invention includes a software program that implements the functions of the above-described embodiments directly or remotely from a system or apparatus. The present invention also includes a case where the system or the computer of the apparatus is achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let It is also possible to execute the encrypted program by using the downloaded key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

100A client terminal 100B client terminal 100C client terminal 100D client terminal 101 LAN
200 servers

Claims (11)

Display control means for controlling to display an image of the first attribute or an image of the second attribute in the first display area;
A receiving unit for receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
Said when first display image displayed in the area is the image of the first attribute, Ru image and replacement of the image displayed on the first display area and the second display region If the image displayed in the first display area is an image having the second attribute, the image displayed in the first display area is replaced with the image in the second display area. Replacement control means for displaying ,
An information processing apparatus comprising: The instructions of Ru received by the receiving means information processing apparatus according to claim 1, characterized in that the instruction by the user's operation of selecting the display area. The information processing apparatus according to claim 1, further comprising a notification unit that notifies a user that the replacement is not performed when the display of the image is not replaced. Said accepting means, wherein, wherein the receiving an image in which the displayed in the first display area, the entry is replacement El instructed the image displayed on the one of the plurality of the second display region Item 4. The information processing apparatus according to any one of Items 1 to 3.   The first display area displays an image of a first attribute that is an image displayed on another device connected to the information processing device and a user of the other device connected to the information processing device. 5. The information processing apparatus according to claim 1, wherein the information processing apparatus is an area that can display an image having a second attribute that is an image that can be identified. The first display area and the second display area, the information processing apparatus according to any one of claims 1 to 5, characterized in that a region which is shown by a frame of the image. In an information processing device,
A display control step for controlling the display control means to display an image of the first attribute or an image of the second attribute in the first display area;
Reception means, a reception step of receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
When the image displayed in the first display area is an image having the first attribute, the replacement control means displays the image displayed in the first display area in the second display area. without Ru image and replaced, the first display image displayed in the region when an image of the second attribute, displays the image displayed on the first display region the second A replacement control step for switching and displaying the image of the area ;
The control method characterized by including. Information processing device
Display control means for controlling to display an image of the first attribute or an image of the second attribute in the first display area;
A receiving unit for receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
Said when first display image displayed in the area is the image of the first attribute, Ru image and replacement of the image displayed on the first display area and the second display region If the image displayed in the first display area is an image having the second attribute, the image displayed in the first display area is replaced with the image in the second display area. A program for functioning as a replacement control means for displaying . An information processing system including a plurality of information processing devices,
Display control means for controlling to display an image of the first attribute or an image of the second attribute in the first display area;
A receiving unit for receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
Said when first display image displayed in the area is the image of the first attribute, Ru image and replacement of the image displayed on the first display area and the second display region If the image displayed in the first display area is an image having the second attribute, the image displayed in the first display area is replaced with the image in the second display area. Replacement control means for displaying ,
An information processing system comprising: A control method of an information processing system including a plurality of information processing devices,
A display control step for controlling the display control means to display an image of the first attribute or an image of the second attribute in the first display area;
Reception means, a reception step of receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
When the image displayed in the first display area is an image having the first attribute, the replacement control means displays the image displayed in the first display area in the second display area. without Ru image and replaced, the first display image displayed in the region when an image of the second attribute, displays the image displayed on the first display region the second A replacement control step for switching and displaying the image of the area ;
The control method characterized by including. A program for controlling an information processing system including a plurality of information processing devices,
The information processing system;
Display control means for controlling to display an image of the first attribute or an image of the second attribute in the first display area;
A receiving unit for receiving an instruction to display the image displayed on the first display area by replacing the image of the second display area,
Said when first display image displayed in the area is the image of the first attribute, Ru image and replacement of the image displayed on the first display area and the second display region If the image displayed in the first display area is an image having the second attribute, the image displayed in the first display area is replaced with the image in the second display area. A program for functioning as a replacement control means for displaying .

Images