JP6406200B2 - Communication program and communication system - Google Patents

Description

  The present invention relates to a communication program and a communication system for realizing a remote conference by performing communication via a network.

  A remote conference system for performing a remote conference between users of a plurality of communication devices connected to a network is known. Patent Document 1 discloses a system that can provide a video service to a user of each communication device by performing communication between the plurality of communication devices. In this system, setting conditions when a video frame is transmitted from each communication device to the destination communication device are determined based on the relationship between the cost coefficient of each communication device and the CPU resource. The cost coefficient is specified based on the current video frame encoding clock, decoding clock, rendering clock, frame rate, and the like. Specific examples of the setting conditions to be determined include the frame size of the video frame, the frame rate, the type of encoder that generates the video frame, and the like.

Special table 2007-52584 gazette

  The case where a video frame is received in the destination communication apparatus and a video is displayed on the display unit is illustrated. In Patent Document 1, if the destination communication device has a cost coefficient larger than 40% of the usable CPU speed of the destination communication device due to decoding and rendering of the destination communication device, the frame size, the frame rate, and the video frame The type of encoder that generates is changed. For example, the frame size is set small. On the other hand, the image quality of the video displayed on the destination communication device changes according to the resolution (for example, the displayed size) of the video on the display unit of the destination communication device. For example, when the displayed size is increased in a state where an image with a small frame size is displayed, pixels included in the image are complemented according to the displayed size. Thus, for example, if the destination communication device user increases the display size of the video while the consumption of the CPU resource of the destination communication device is reduced, the image quality may be reduced as the video becomes coarse There is sex.

  An object of the present invention is to provide a communication program and a communication system for suppressing deterioration in image quality in accordance with the display size of an image on a display unit.

A communication program according to a first aspect of the present invention is a communication program for causing a computer of a first communication device capable of participating in a remote conference to execute, wherein an image captured by a camera of the first communication device is encoded. a transmission step of transmitting the video data, pre-SL load condition when the second communication device decodes the video data to participate in the teleconference, and image the image data is decoded, the second communication device A receiving step for receiving the resolution when displayed on the display unit, a first determining step for determining whether the load state received by the receiving step is a first load state, and the received by the receiving step A second determination step for determining whether the resolution is equal to or lower than a second predetermined value and the first load state is determined by the first determination step. If the resolution is determined to be less than or equal to the second predetermined value in the second determination step, second video data encoded under a second encoding condition is generated, and the resolution is determined in the second determination step. A first generation step of generating first video data encoded under a first encoding condition having a load greater than that of the second encoding condition in response to determining that is greater than the second predetermined value. The transmission step transmits the first video data as the video data when the first video data is generated by the first generation step, and the second video data by the first generation step. Is generated, the second video data is transmitted as the video data.

  According to the first aspect, the first communication device transmits the second video data when the load state of the second communication device is the first load state and the resolution is equal to or less than the second predetermined value. . The reason is to reduce the load on the second communication device. Note that the load when encoding is performed under the second encoding condition is smaller than that when encoding is performed under the first encoding condition. Therefore, the load when decoding the second video data encoded under the second encoding condition is smaller than the load when decoding the first video data encoded under the first encoding condition. For this reason, the 1st communication apparatus can reduce the load when the 2nd communication apparatus performs a decoding process by transmitting 2nd video data.

  When the first video and the second video are displayed on the display unit with a common resolution, the image quality of the second video may be lower than that of the first video. In contrast, the first communication device transmits the second video data when the resolution of the video displayed on the display unit of the second communication device is equal to or lower than the second predetermined value. The reason is that the resolution of the video displayed on the display unit is equal to or lower than the second predetermined value. For example, even when the second video is displayed on the display unit instead of the first video, the degradation of the video image quality is conspicuous. This is because it is difficult. In this way, the first communication device can reduce the load on the second communication device while suppressing the image quality of the video displayed on the display unit of the second communication device from becoming low.

  On the other hand, the first communication device transmits the first video data when the load state of the second communication device is the first load state and the resolution is greater than the second predetermined value. This is because, for example, when the second video is displayed instead of the first video, the image quality of the video is easily deteriorated, and therefore it is highly necessary to prioritize the image quality of the video. Thus, the 1st communication apparatus can control that the picture quality of the picture displayed on the display part of the 2nd communication apparatus becomes low image quality.

  A communication program according to a second aspect of the present invention is a communication program for causing a computer of a second communication device that can participate in a remote conference to execute the video data from the first communication device that participates in the remote conference. A determination step of determining a load state in the second communication device when decoding, an acquisition step of acquiring a resolution when a video obtained by decoding the video data is displayed on the display unit of the second communication device; Receiving a first video data encoded under a first encoding condition and a second video data encoded under a second encoding condition having a smaller load than the first encoding condition, as the video data; Based on the load state determined by the determination step and the resolution acquired by the acquisition step. And determining whether to decode one of the first video data and the second video data, decoding the video data received in the reception step, and decoding in the first determination step. When the video data is the first video data or when the video data is the second video data, the video data is decoded and the video data received by the receiving step, When the video data determined to be decoded in the determination step is different between the first video data and the second video data, the computer is caused to execute a processing step of not decoding the video data.

  According to the second aspect, the second communication device compares the video data received from the first communication device (first video data or second video data) with the video data specified based on the load state and the resolution. To do. The second communication device decodes the video data when both the video data are the first video data or when both are the second video data. The second communication device does not decode the video data when both video data are different between the first video data and the second video data. Accordingly, the second communication device can suppress the image quality of the video displayed on the display unit from being low, while suppressing the load from becoming relatively high.

A communication system according to a third aspect of the present invention is a communication system including a first communication device and a second communication device that can participate in a remote conference, wherein the first communication device is a camera of the first communication device. transmission means for capturing video transmits the encoded video data, pre-SL load condition when the second communication device decodes the video data, and the video that the video data is decoded, the second First receiving means for receiving a resolution when displayed on a display unit of a communication device; first determining means for determining whether the load state received by the first receiving means is a first load state; A second determination unit that determines whether the resolution received by the first reception unit is equal to or less than a second predetermined value; and when the first determination unit determines that the first load state is present, the second determination unit When the resolution is determined to be less than or equal to the second predetermined value, second video data encoded under a second encoding condition is generated, and the resolution is greater than the second predetermined value by the second determination unit. And a first generation means for generating first video data encoded under a first encoding condition having a load greater than that of the second encoding condition, wherein the transmission means includes the first encoding means. When the first video data is generated by the generation unit, the first video data is transmitted as the video data. When the second video data is generated by the first generation unit, the first video data is transmitted as the video data. 2 video data is transmitted, and the second communication device is configured to determine the load state, an acquisition unit to acquire the resolution, and the video data Second receiving means for receiving the first video data and the second video data transmitted from the first communication device, the load state determined by the determining means, and the resolution acquired by the acquiring means Based on the first video data and the second video data, the third judgment means for judging to decode, the video data received by the second reception means, and the third judgment means for decoding When the video data determined to be the first video data or the second video data, the video data is decoded and received by the second receiving means. The data and the video data determined to be decoded by the third determination means are the first video data and the second video data. If they are different from each other, a processing unit that does not decode the video data is provided. According to the 3rd aspect, there can exist an effect similar to a 1st aspect and a 2nd aspect.

It is a figure which shows the outline | summary of the communication system 1, and the electrical structure of the server 11 and the communication apparatus 12. FIG. It is a figure which shows the 1st table 124A and the 2nd table 124B which were memorize | stored in the memory | storage part 124 of the communication apparatus 12. FIG. It is a flowchart of a 1st process. It is a flowchart of a 2nd process. It is a flowchart of a 3rd process. It is a flowchart of a 4th process. It is a flowchart of a 5th process. It is a figure which shows a communication sequence. It is a figure which shows the table 114A memorize | stored in the memory | storage part 114 of the server apparatus 11. FIG. It is a flowchart of a server process. It is a figure which shows the 1st table 124A and the 2nd table 124B which were memorize | stored in the memory | storage part 124 of 12 A of communication apparatuses.

<Outline of communication system 1>
The communication system 1 will be described with reference to FIG. The communication system 1 includes a server device 11 and communication devices 12A, 12B, and 12C. Hereinafter, the communication devices 12A, 12B, and 12C are collectively referred to as “communication device 12”. The server device 11 and the communication device 12 are connected via the network 14. The communication system 1 is a system for realizing a web conference between users by providing a virtual conference room to each user of the communication device 12.

  The server device 11 is a known multi-point control unit (MCU). The communication device 12 is configured by installing a program of an application (hereinafter referred to as “conference application”) for performing a Web conference on a known general-purpose personal computer (PC). Hereinafter, the IDs of the communication devices 12A, 12B, and 12C are assumed to be “12A”, “12B”, and “12C”. The users of the communication devices 12A, 12B, and 12C are referred to as users 13A, 13B, and 13C. The users 13A, 13B, and 13C are collectively referred to as a user 13. The server device 11 may be configured by installing a server conference application program on a general-purpose server. At least one of the communication devices 12 may be configured by installing a conference application program on a mobile terminal such as a remote conference dedicated terminal, a smartphone, or a tablet terminal.

<Electrical configuration>
The electrical configuration of the server device 11 will be described. The server device 11 includes a CPU 111 that controls the server device 11. The CPU 111 is electrically connected to the ROM 112, the RAM 113, the storage unit 114, the communication I / F 115, and the drive device 116 via an interface circuit (not shown). The ROM 112 stores a boot program, BIOS, and the like. The RAM 113 stores a timer, a counter, flag data, temporary data, and the like. The storage unit 114 includes a computer-readable non-transitory storage medium, such as a hard disk. The storage unit 114 stores a program for processing executed by the CPU 111 and an OS.

  The communication I / F 115 performs wireless communication by connecting to an interface element (for example, a LAN card) for the server apparatus 11 to connect to the network 14 or an access point (not shown) connected to the network 14 by the server apparatus 11. Interface element for performing (for example, Wi-Fi (registered trademark) communication modem). The CPU 111 transmits and receives data to and from other devices connected to the network 14 via the communication I / F 115. The drive device 116 can read information stored in a computer-readable storage medium 116A such as a semiconductor memory or an optical disk. The CPU 111 can read out the program stored in the storage medium 116 </ b> A by the drive device 116 and store it in the storage unit 114.

  The electrical configuration of the communication device 12 will be described. The communication device 12 includes a CPU 121 that controls the communication device 12. The CPU 121 is electrically connected to the ROM 122, the RAM 123, the storage unit 124, the camera 125, the display unit 126, the communication I / F 127, the input unit 128, the microphone 129, the speaker 130, and the drive device 131 via an interface circuit (not shown). Connect to. The ROM 122 stores a boot program, BIOS, and the like. The RAM 123 stores a timer, counter, flag data, temporary data, and the like. The storage unit 124 is configured by a computer-readable non-transitory storage medium, such as a hard disk. The storage unit 124 stores a conference application program and an OS. The camera 125 can output captured video data (hereinafter referred to as “photographing data”) to the CPU 121. The display unit 126 is a display such as a liquid crystal display (LCD). The communication I / F 127 is an interface element for the communication apparatus 12 to connect to the network 14 or an interface element for the communication apparatus 12 to connect to an access point (not shown) connected to the network 14 and perform wireless communication. . The CPU 121 transmits / receives data to / from other devices connected to the network 14 via the communication I / F 127. The input unit 128 includes a keyboard 128A, a mouse 128B, and the like. The drive device 131 can read information stored in a computer-readable storage medium 131A such as a semiconductor memory. The CPU 121 can read out the program stored in the storage medium 131 </ b> A by the drive device 131 and store it in the storage unit 124.

  Note that the storage units 114 and 124 may be configured by other non-temporary storage media such as a flash memory and / or a ROM, for example. The non-temporary storage medium may be any storage medium that can store information regardless of the period in which the information is stored. The non-transitory storage medium may not include a temporary storage medium (for example, a signal to be transmitted). A general-purpose processor may be used as the CPUs 111 and 121. A part of the processing executed by the CPUs 111 and 121 may be executed by another electronic component (for example, ASIC). The processing executed by the CPUs 111 and 121 may be distributed by a plurality of electronic devices (that is, a plurality of CPUs). For example, part of the processing executed by the CPU 111 of the server device 11 may be executed by the CPU 121 of the communication device 12. For example, the program may be downloaded from another server connected to the network 14 (that is, transmitted as a transmission signal) and stored in the storage units 114 and 124. In this case, the program is stored in a non-temporary storage medium such as a hard disk provided in another server.

  When the CPU 121 of the communication device 12 acquires the shooting data output from the camera 125, it can select and encode either the first encoding condition or the second encoding condition. When photographing data is encoded under the first encoding condition and the second encoding condition, the photographing data is converted into data of another format in any case. Hereinafter, data generated when shooting data is encoded under the first encoding condition is referred to as “first video data”. Data generated when shooting data is encoded under the second encoding condition is referred to as “second video data”. The first video data and the second video data are collectively referred to as “video data”. For example, the shooting data is analog data, and the video data is encoded digital data.

  In the present embodiment, it is assumed that the load on the CPU 121 when encoding is performed under the first encoding condition is larger than the load on the CPU 121 when encoding is performed under the second encoding condition. The load on the CPU 121 during encoding is proportional to the bit rate of the encoded data, for example. In the present embodiment, for example, the bit rate is different between the first video data and the second video data. The bit rate of the first video data is larger than the bit rate of the second video data. This difference in bit rate is due to the fact that the frame size and frame rate of the first video data are larger than the frame size and frame rate of the second video data. When the same encoder is used, the first encoding condition may be different from the setting of the encoder, for example, compared to the second encoding condition (for example, the first encoding condition is more than the second encoding condition). Many reference frames). Alternatively, when different encoders are used, the first encoding condition (for example, H.264) and the second encoding condition (for example, MPEG4) may be different.

  Further, the CPU 121 of the communication device 12 can decode the video data generated by the other communication device 12 and generate data that can be displayed on the display unit 126. Hereinafter, the video generated when the first video data is decoded is referred to as “first video”. A video generated when the second video data is decoded is referred to as a “second video”. In the present embodiment, it is assumed that the load on the CPU 121 when decoding the first video data is larger than the load on the CPU 121 when decoding the second video data. The reason is that, while the bit rate of the first video data is larger than the bit rate of the second video data, the load on the CPU 121 at the time of decoding is proportional to the bit rate of the video data, for example.

<Outline of main window 20>
The case where each user 13A-13C of communication apparatus 12A-12C performed the predetermined | prescribed operation for utilizing a virtual conference room (henceforth a "conference room") is mentioned as an example. The predetermined operation is the same as the operation performed by the user when the conference room is used in a known web conference system. For example, the predetermined operation is an operation of selecting a URL notified in advance from the server device 11 to the communication device 12 with the mouse 128B. This URL includes, for example, a conference ID for identifying a conference room to participate as an argument. In this case, the CPU 121 of the communication device 12 communicates with the server device 11 and executes a well-known conference connection process. In the conference connection process, a conference room session is established between the server device 11 and the communication device 12. The server device 11 notifies each communication device 12 of the IDs and address information of all the communication devices 12 participating in the conference room. In the following description, it is assumed that a conference room session is established between the communication devices 12A to 12C and the server device 11, and the communication devices 12A to 12C participate in a common conference room.

  After the session is established, the conference application is executed by the CPU 121 of the communication device 12. When the conference application is executed, the main window 20 of the conference application is displayed on the display unit 126. The main window 20 includes a video window 21. The video window 21 displays a video shot by the camera 125 of another communication device 12 participating in the conference room. The size of the video window 21 can be changed by the user 13 of the communication device 12 operating the mouse 128B.

  In FIG. 1, the main window 20A displayed on the display unit 126 of the communication device 12A includes video windows 22B and 22C. In the video window 22B (resolution: 640 × 480 (dot)), the video of the user 13B taken by the camera 125 of the communication device 12B is displayed. In the video window 22C (resolution: 320 × 240 (dot)), the video of the user 13C taken by the camera 125 of the communication device 12C is displayed. The main window 20B displayed on the display unit 126 of the communication device 12B includes video windows 23A and 23C. In the video window 23A (resolution: 640 × 480 (dot)), the video of the user 13A taken by the camera 125 of the communication device 12A is displayed. In the video window 23C (resolution: 320 × 240 (dot)), the video of the user 13C taken by the camera 125 of the communication device 12C is displayed. The main window 20C displayed on the display unit 126 of the communication device 12C includes video windows 24A and 24B. In the video window 24A (resolution: 320 × 240 (dot)), the video of the user 13A taken by the camera 125 of the communication device 12A is displayed. In the video window 24B (resolution: 640 × 480 (dot)), the video of the user 13B taken by the camera 125 of the communication device 12B is displayed.

  A case in which the first video data is generated by the CPU 121 of the communication device 12A and the first video is displayed on the video window 23A of the communication device 12B and the video window 24A of the communication device 12C based on the first video data. Take an example. As described above, the frame size of the first video data is larger than the frame size of the second video data. Therefore, the image quality when the first video is displayed in the video window 23A is higher than the image quality when the second video is displayed in the video window 23A. Similarly, the image quality when the first video is displayed in the video window 24A is higher than the image quality when the second video is displayed in the video window 24A.

  The resolution (640 × 480 (dot)) of the video window 23A is larger than the resolution 320 × 240 (dot) of the video window 24A. For this reason, the image window 23A is more conspicuous in the image window 23A than in the image window 24A when the second image is displayed instead of the first image. For example, when the pixels of the second video are complemented so that they can be displayed at the respective resolutions of the video windows 23A and 24A, the size of the pixel of the second video displayed in the video window 23A is displayed in the video window 24A. It becomes larger than the pixel size of the displayed second video. For this reason, when the second video corresponding to the second video data whose frame size is originally small is displayed in the video window 23A, each pixel becomes coarser than when the second video is displayed in the video window 24A. The image quality tends to be low. For this reason, it is particularly preferable that the first video is displayed over the second video in the video window 23A of the communication device 12B.

  However, the load on the CPU 121 when decoding the first video data is larger than the load on the CPU 121 when decoding the second video data. For this reason, when the decoding of the first video data is executed, the load on the CPU 121 becomes excessive, which may not be preferable.

  On the other hand, the second video data is generated by the CPU 121 of the communication device 12A, and the second video is displayed on the video window 23A of the communication device 12B and the video window 24A of the communication device 12C based on the second video data. Take the case as an example. In this case, the load on the CPU 121 when decoding the second video data is smaller than the load on the CPU 121 when decoding the first video data. It may be possible to suppress an excessive load. However, as described above, the second video displayed in the video window 23A tends to have lower image quality than the second video displayed in the video window 24A.

  On the other hand, in the present embodiment, the CPU 121 of the communication device 12 determines the first encoding condition and the first encoding condition according to the resolution of the video window 21 of the other communication device 12 and the load state of the CPU 121 of the other communication device 12. It is determined which of the second encoding conditions is used to encode the photographic data. In addition, the CPU 121 of the communication device 12 decodes either the first video data or the second video data and displays the video on the video window 21 according to the load state of the CPU 121 and the resolution of the video window 21. Decide what to do.

<First table 124A>
The first table 124A stored in the storage unit 124 of the communication device 12 will be described with reference to FIG. The first table 124A stores the load state and resolution of the communication device 12. The load state indicates the level of the degree of load on the CPU 121. Details of the load state will be described later. The resolution is the resolution of each video window 21 displayed on the display unit 126 of the communication device 12. The resolution is associated with the ID of the communication device 12 that has transmitted the video data of the video displayed in each video window 21. Hereinafter, the ID of the communication device 12 that has transmitted the video data is referred to as a “display ID”. The “display ID of the communication device 12 that has transmitted the video data of the video displayed in the video window 21” is rephrased as the “display ID of the video window 21”. FIG. 2 shows a first table 124A stored in each storage unit 124 of the communication devices 12A, 12B, and 12C. The description of the second table 124B will be described later.

<First processing>
The first to fifth processes executed by the CPU 121 of the communication device 12 will be described with reference to FIGS. First, the first process will be described with reference to FIG. In the first process, when the conference application is executed after the session with the server device 11 is established and the main window 20 is displayed on the display unit 126, the CPU 121 executes the program stored in the storage unit 124. , And starts at a fixed period (eg, every 5 to 10 minutes) When the CPU 121 executes the first process for the first time, the load level is low (hereinafter referred to as “low state”) as the initial value of the “load state” in the first table 124A (see FIG. 2). “1” is set to indicate that. Hereinafter, setting “1” to the load state is referred to as “setting a low state to the load state”.

  First, the CPU 121 identifies the degree of load on the CPU 121. The degree of load indicates the usage rate of the CPU 121 when video data (first video data or second video data) received in a fifth process (see FIG. 5) described later is decoded. The CPU 121 determines whether the specified degree of load is greater than or equal to a first predetermined value (for example, 40%) (S11). When it is determined that the specified degree of load is equal to or greater than the first predetermined value (S11: YES), the CPU 121 determines whether the load state of the first table 124A is set to a low state (S17). As described above, when the first process is executed first, the load state of the first table 124A is set to the low state. When the CPU 121 determines that the load state of the first table 124A is set to a low state (S17: YES), the CPU 121 is in a state where the load level is medium (hereinafter referred to as “medium state”). “2” indicating this is set to the load state of the first table 124A (S19). Hereinafter, setting “2” to the load state of the first table 124A is referred to as “setting the middle state to the load state”. The CPU 121 transmits the load state of the first table 124A to the server device 11 (S23). The CPU 121 ends the first process. On the other hand, for example, when the load state of the first table 124A is set to the medium state or the high state in the previous first process, the CPU 121 determines that the load state of the first table 124A is not set to the low state. (S17: NO). In this case, the CPU 121 sets “3” indicating that the level of the degree of load is high (hereinafter referred to as “high state”) to the load state of the first table 124A (S21). Hereinafter, setting “3” to the load state of the first table 124A is referred to as “setting a high state to the load state”. The CPU 121 transmits the load state of the first table 124A to the server device 11 (S23). The CPU 121 ends the first process.

  On the other hand, when it is determined that the degree of load is less than the first predetermined value (S11: NO), it is determined whether a high state is set as the load state of the first table 124A (S13). For example, when the load state of the first table 124A is set to a high state in the previous first process, the CPU 121 determines that the high state is set to the load state of the first table 124A (S13: YES). ). In this case, the CPU 121 sets the middle state to the load state of the first table 124A (S19). The CPU 121 transmits the load state of the first table 124A to the server device 11 (S23). The CPU 121 ends the first process. On the other hand, for example, when the load state of the first table 124A is set to the low state or the medium state in the previous first process, the CPU 121 determines that the high state is not set to the load state of the first table 124A. (S13: NO). In this case, the CPU 121 sets a low state as the load state of the first table 124A (S15). The CPU 121 transmits the load state of the first table 124A to the server device 11 (S23). The CPU 121 ends the first process.

<Second processing>
The second process executed by the CPU 121 of the communication device 12 will be described with reference to FIG. The second process is an operation to change the size of any video window 21 included in the main window 20 immediately after the conference application is executed after the session with the server device 11 is established and the main window 20 is displayed. Is started by the CPU 121 executing the program stored in the storage unit 124. When executing the second process for the first time, the CPU 121 sets “0 × 0” (dot) as all the resolutions of the first table 124A (see FIG. 2).

  First, the CPU 121 acquires any resolution of the video window 21 included in the main window 20 (S41). Note that the acquired resolution corresponds to the resolution when the first video or the second video is displayed in the video window 21. The CPU 121 and the resolution acquired in S41 and the resolution associated with the display ID of the video window 21 in which the resolution is acquired in S41 in the first table 124A (hereinafter referred to as “corresponding resolution”). ).

  The CPU 121 determines that the resolution of the video window 21 has been changed when the resolution acquired by the processing of S41 is different from the corresponding resolution of the first table 124A (S43: YES). In this case, the CPU 121 associates the display ID associated with the corresponding resolution in the first table 124A with the resolution acquired by the process of S41, and transmits the display ID to the server device 11 (S45). The CPU 121 changes the corresponding resolution of the first table 124A to the resolution acquired by the process of S41. The CPU 121 advances the process to S47. On the other hand, when the resolution acquired by the process of S41 and the corresponding resolution of the first table 124A match, the CPU 121 determines that the resolution of the video window 21 has not been changed (S43: NO). In this case, the CPU 121 advances the process to S47.

  In the processing of S45 described above, the CPU 121 changes the resolution of the video window 21 when the difference between the resolution acquired by the processing of S41 and the corresponding resolution of the first table 124A is within a predetermined range. You may decide not to. Specifically, for example, when the amount of difference in the number of pixels in the horizontal direction and the vertical direction of each resolution is within 100 dots, the CPU 121 may determine that the resolution of the video window 21 has not been changed.

  The CPU 121 determines whether the resolutions of all the video windows 21 included in the main window 20 have been acquired by the process of S41 (S47). When the video window 21 from which the resolution has not been acquired remains, the CPU 121 determines that the resolutions of all the video windows 21 have not been acquired (S47: NO). The CPU 121 returns the process to S41. The CPU 121 acquires the resolution from any of the video windows 21 for which the resolution has not been acquired (S41), and repeats the processes of S43 and S45. When it is determined that the resolutions of all the video windows 21 have been acquired (S47: YES), the CPU 121 ends the second process.

  Note that when the second process is first executed immediately after the main window 20 is displayed, “0 × 0” is set for all the resolutions in the first table 124A. For this reason, it is determined that the resolution acquired by the process of S41 is different from the corresponding resolution for all the video windows 21 (S43). Accordingly, the display IDs associated with the corresponding resolutions in the first table 124A are associated with the resolutions of all the video windows 21 and transmitted to the server device 11 (S45).

  FIG. 8 shows an outline of a communication sequence between the server device 11 and the communication device 12. When the first process (see FIG. 3) is executed by the CPU 121 of the communication device 12A and the low state is set in the first table 124A, the load state in which the low state is set is changed from the communication device 12A to the server device 11. (Arrow C11). When the second process (see FIG. 4) is executed by the CPU 121 of the communication device 12A, for example, the communication device 12B has the resolution “640 × 480” (dot) of the video window 22B (see FIG. 1). ID “12B” is associated and transmitted from the communication device 12A to the server device 11 (arrow C11). Further, for example, the ID “12C” of the communication device 12C is associated with the resolution “320 × 240” (dot) of the video window 22C (see FIG. 1) and transmitted from the communication device 12A to the server device 11 ( Arrow C11). The same applies when the first process and the second process are executed by the CPU 121 of the communication apparatus 12B, and when the first process and the second process are executed by the CPU 121 of the communication apparatus 12C (arrows C13 and C15). ). In FIG. 8, the load state and the resolution are described as being simultaneously transmitted from the communication device 12, but actually, the load state and the resolution are transmitted when the first process and the second process are executed. So each is sent separately. Of course, the load state and the resolution may be transmitted from the communication device 12 at the same time.

  Although details will be described later, the CPU 111 of the server apparatus 11 receives the load state and resolution transmitted as indicated by arrows C11, C13, and C15. The CPU 111 stores the received load state and resolution in the table 114A (see FIG. 9) stored in the storage unit 114.

<Table 114A>
As illustrated in FIG. 9, in the table 114A, the load state is associated with the IDs “12A”, “12B”, and “12C” of the communication device 12 that transmitted the load state. The resolution is associated with the ID of the communication device 12 that transmitted the resolution and the ID of the communication device 12 associated with the resolution. Hereinafter, the ID of the communication device 12 that has transmitted the load state and the resolution is referred to as a “first ID”. The ID of the communication device 12 associated with the resolution is referred to as “second ID”.

  Each time the CPU 111 receives the load state from the communication device 12, the CPU 111 associates the first ID with the received load state and broadcasts to all the communication devices 12 participating in the conference room (arrows C12, C14, C16). Further, each time the CPU 111 receives the resolution from the communication device 12, the CPU 111 associates the first ID and the second ID with the received resolution, and broadcasts to all the communication devices 12 participating in the conference room (arrows C12 and C14). , C16).

<Third process>
The third process executed by the CPU 121 of the communication device 12 will be described with reference to FIG. The third process is performed when the reception of the load state or the resolution transmitted from the server device 11 is started after the conference application is executed after the session with the server device 11 is established and the main window 20 is displayed. The program is started when the CPU 121 executes the program stored in the unit 124.

  First, the CPU 121 determines whether reception of a load state transmitted from the server device 11 has been started (S81). When it is determined that the load state has been started (S81: YES), the CPU 121 receives all the load states. The CPU 121 associates the received load state with the first ID associated with the received load state and stores them in the second table 124B (see FIG. 2, described later) (S83). The CPU 121 advances the process to S85. If the CPU 121 determines that reception of the load state has not been started (S81: NO), the process proceeds to S85.

The CPU 121 determines whether the reception of the resolution transmitted from the server device 11 has been started (S85). When it is determined that the reception of the resolution is started (S85: YES), the CPU 121 receives all the resolutions. The CPU 121 associates the received resolution with the first ID and the second ID associated with the received resolution and stores them in the second table 124B (see FIG. 2, described later) (S87). The CPU 121 ends the third process. If it is determined that the resolution has not been received (S85: NO), the CPU 121 ends the third process.
To do.

  For example, as shown in FIG. 8, the load state and resolution broadcast transmitted from the server apparatus 11 as indicated by arrows C12, C14, and C16 are executed by the CPU 121 of each of the communication apparatuses 12A, 12B, and 12C. Can be received by. Although not described in the third process of FIG. 5, when the CPU 121 receives a load state and resolution associated with its own ID as the first ID, the CPU 121 displays the load state and resolution in the second table 124B. Do not remember. Specifically, for example, when the CPU 121 of the communication device 12A receives the load state and resolution transmitted from the server device 11 as indicated by the arrow C12, the first ID “12A” that is the same as its own ID is associated with the CPU 121. Therefore, this load state and resolution are not stored in the second table 124B (R11). On the other hand, when the CPU 121 receives the load state transmitted from the server device 11 as indicated by the arrow C14, the CPU 121 associates the first ID with the load state because the first ID “12B” is different from its own ID. It memorize | stores in the memory | storage part 124 (R21). When the CPU 121 receives the resolution transmitted from the server device 11 as indicated by the arrow C14, the CPU 121 stores the first ID and the second ID in association with the resolution in the storage unit 124 (R21). The same applies to the case where the load state and resolution transmitted from the server apparatus 11 as indicated by the arrow C16 are received (R31). When the CPU 121 of the communication device 12B receives the load state and resolution transmitted from the server device 11 as indicated by arrows C12, C14, and C16 (R12, R22, R32), and the CPU 121 of the communication device 12C The same applies to the case (R13, R23, R33) when the load state and the resolution transmitted from the server apparatus 11 as indicated by arrows C12, C14, C16 are received, and the description thereof is omitted.

<Second table 124B>
The second table 124B stored in the storage unit 124 of the communication device 12 will be described with reference to FIG. In the second table 124B, the load state received from the server device 11 is stored in association with the first ID. The second table 124B stores the resolution received from the server device 11 in association with the first ID and the second ID.

<4th process>
With reference to FIG. 6, the 4th process performed by CPU121 of the communication apparatus 12 is demonstrated. The fourth process is started repeatedly in the frame period of the video data after the conference application is executed after the session with the server device 11 is established and the main window 20 is displayed.

  First, the CPU 121 acquires shooting data from the camera 125 (S51). The CPU 121 selects one of the IDs of other communication devices 12 participating in the conference room (S53). The CPU 121 receives the ID and address information of all the communication devices 12 participating in the conference room from the server device 11 after the session with the server device 11 is established. The CPU 121 executes the third process (see FIG. 5) and is associated with the same first ID as the ID selected by the process of S53 among the load states stored in the second table 124B (see FIG. 2). The obtained load state is acquired (S55). Of the resolutions stored in the second table 124B by executing the third process, the CPU 121 is associated with the first ID that is the same as the ID selected by the process of S53 and the second ID that is the same as its own ID. The resolution is acquired (S57).

  The CPU 121 determines whether or not the load state acquired by the process of S55 is set to a low state (S59). When it is determined that the load state is set to the low state (S59: YES), the CPU 121 determines to encode the shooting data acquired by the process of S51 under the first encoding condition (S65). The CPU 121 advances the process to S69.

  When the CPU 121 determines that the low state is not set in the load state acquired by the process of S55 (S59: NO), the CPU 121 determines whether the medium state is set in the load state (S61). When it is determined that the state is set (S61: YES), it is determined whether the resolution acquired by the process of S57 is larger than a second predetermined value (for example, 320 × 240 (dot)). When it is determined that the CPU 121 is larger than the second predetermined value (S63: YES), the CPU 121 determines to encode the shooting data acquired by the process of S51 under the first encoding condition (S65). The CPU 121 advances the process to S69. If the CPU 121 determines that the resolution acquired by the process of S57 is equal to or less than the second predetermined value (S63: NO), the CPU 121 determines to encode the shooting data acquired by the process of S51 under the second encoding condition (S67). ). The CPU 121 advances the process to S69.

  If it is determined that the middle state is not set in the load state acquired by the process of S55 (S61: NO), the CPU 121 advances the process to S67. In this case, the high state is set as the load state acquired by the process of S55. The CPU 121 determines to encode the photographic data acquired by the process of S51 under the second encoding condition (S67). The CPU 121 advances the process to S69.

  The CPU 121 encodes the photographic data acquired by the process of S51 under the condition (first encoding condition or second encoding condition) determined by the process of S65 or S67, and generates video data (S69). . The CPU 121 stores the generated video data in the RAM 123. The CPU 121 determines whether all the IDs of the other communication devices 12 participating in the conference room have been selected by the process of S53 (S71). If the CPU 121 determines that all the IDs of the other communication devices 12 participating in the conference room have not been selected by the process of S53 (S71: NO), the process returns to S53. The CPU 121 selects one of the unselected IDs among the IDs of the other communication devices 12 participating in the conference room (S53), and repeats the process.

  If there are two or more other communication devices 12 participating in the conference room, the first encoding condition or the second encoding condition may be determined twice or more in the process of repeating the process as described above. In such a case, the shooting data acquired by the process of S51 is already encoded under the determined conditions, and the generated video data is already stored in the RAM 123. In this case, although the description is omitted in the fourth process of FIG. 6, the CPU 121 does not execute the encoding (S69) based on the conditions determined after the second time. As a result, the CPU 121 can avoid the same video data being duplicated.

  If the CPU 121 determines that all the IDs of the other communication devices 12 participating in the remote conference have been selected by the process of S53 (S71: YES), the CPU 121 advances the process to S73. The CPU 121 transmits all the video data (first video data or second video data) stored in the RAM 123 to the server device 11 (S73). The CPU 121 ends the fourth process.

  A case where the fourth process is executed by the CPU 121 of the communication device 12A will be specifically described with reference to FIG. The CPU 121 of the communication device 12A acquires shooting data from the camera 125 (S51). The CPU 121 selects the ID “12B” of the communication device 12B among the communication devices 12B and 12C participating in the conference room. (S53). CPU 121 is set to the load state associated with the same first ID as the selected ID “12B” among the load states stored in second table 124B (see FIG. 2A) of storage unit 124. The middle state is acquired (S55). The CPU 121 sets the resolution “640 × 480” associated with the first ID that is the same as the selected ID “12B” and the second ID that is the same as its own ID “12A” among the resolutions stored in the second table 124B. Is acquired (S57).

  The CPU 121 does not set the low state in the acquired load state (S59: NO), sets the medium state (S61: YES), and the resolution “640 × 480” is larger than the second predetermined value. (S63: YES). The CPU 121 determines the first encoding condition (S65), encodes the photographic data acquired by the process of S51, and generates first video data (S69). The CPU 121 stores the first video data in the RAM 123. The CPU 121 returns the process to S53 because the ID “12C” of the communication device 12C participating in the conference room has not been selected by the process of S53 (S71: NO).

Next, the CPU 121 selects the ID “12C” of the communication device 12B among the communication devices 12B and 12C participating in the conference room. (S53). The CPU 121 acquires a high state set to the load state associated with the first ID that is the same as the selected ID “12C” among the load states stored in the second table 124B (S55). The CPU 121 sets the resolution “320 × 240” associated with the first ID that is the same as the selected ID “12C” and the second ID that is the same as its own ID “12A” among the resolutions stored in the second table 124B. Is acquired (S57).

  The CPU 121 determines that the low state is not set in the acquired load state (S59: NO) and that the intermediate state is not set (S61: NO). The CPU 121 determines the second encoding condition (S65), encodes the shooting data acquired by the process of S51, and generates second video data (S69). The CPU 121 stores the second video data in the RAM 123. The CPU 121 selects the first video data and the second video stored in the RAM 123 because the IDs “12B” and “12C” of the communication devices 12B and 12C participating in the conference room are selected in the process of S53 (S71: YES). Data is transmitted to the server device 11 (S73, arrow C17, arrow C19 (see FIG. 8)).

  Although details will be described later, as shown in FIG. 8, the CPU 111 of the server apparatus 11 transmits the first video data transmitted from the communication apparatus 12A as indicated by an arrow C17 and the communication apparatus 12A to C19 as illustrated in FIG. Second video data is received. Although not shown, the CPU 111 similarly receives video data (first video data or second video data) transmitted from the communication devices 12B and 12C. Each time the CPU 111 receives video data from the communication device 12, the CPU 111 associates the ID of the communication device 12 that transmitted the received video data with the received video data, so that all the communication devices 12 that participate in the conference room are associated with each other. Broadcast transmission (arrows C18 and C20). Hereinafter, the communication device 12ID that has transmitted the video data is referred to as a “third ID”.

<Fifth process>
The fifth process executed by the CPU 121 of the communication device 12 will be described with reference to FIG. The fifth process is performed when the reception of video data transmitted from the server device 11 is started after the conference application is executed after the session with the server device 11 is established and the main window 20 is displayed. The program is started when the CPU 121 executes the program stored in.

  The CPU 121 receives all the video data transmitted from the server device 11 and stores it in the RAM 123 (S31). The CPU 121 specifies video data to be decoded by the following method (S33). First, the CPU 121 acquires the load state stored in the first table 124A (see FIG. 2). Next, the CPU 121 obtains a resolution associated with the same display ID as the third ID associated with the received video data from among the resolutions stored in the first table 124A. When the low state is set as the acquired load state, the CPU 121 identifies the first video data as a decoding target regardless of the acquired resolution. The CPU 121 identifies (2) the second video data as a decoding target regardless of the acquired resolution when the high state is set in the acquired load state. The CPU 121 identifies the first video data as a decoding target when (3) the acquired load state is set to the middle state and the acquired resolution is larger than the second predetermined value. The CPU 121 specifies the second video data as a decoding target when (4) the acquired load state is set to the middle state and the acquired resolution is equal to or lower than the second predetermined value.

  The CPU 121 receives the first video data by the process of S31 and the first video data is specified by the process of S33, or the second video data is received by the process of S31. If the second video data is specified by the process of S33, it is determined that the types of the video data match (S35: YES). In this case, when the first video data is received by the process of S31, the CPU 121 decodes the received first video data to generate a first video (S37). When the second video data is received by the process of S31, the CPU 121 decodes the received second video data to generate a second video (S37). The CPU 121 selects the video window 21 having the same display ID as the third ID associated with the video data received by the process of S31, from the video window 21 of the main window 20. The CPU 121 displays the generated first video or second video on the selected video window 21 (S39). The CPU 121 ends the fifth process.

  On the other hand, the CPU 121 receives the first video data by the process of S31 and the second video data is received by the process of S31 when the second video data is specified by the process of S33. When the first video data is specified by the process of S33, it is determined that the types of the video data do not match (S35: NO). In this case, the CPU 121 deletes the first video data or the second video data stored in the RAM 123 by the process of S31 (S40). The CPU 121 ends the fifth process.

  The case where the fifth process is executed will be specifically described with reference to FIG. Although not described in the fifth process of FIG. 7, the CPU 121 deletes the video data from the RAM 123 when it receives video data associated with its own ID as the third ID. Specifically, for example, when the CPU 121 of the communication device 12A receives the first video data transmitted from the server device 11 as indicated by the arrow C18, the same third ID as its own ID “12A” is associated. Therefore, the first video data is deleted from the RAM 123 (R41). When the CPU 121 of the communication device 12A receives the second video data transmitted from the server device 11 as indicated by the arrow C20, the second ID is associated with the same third ID as its own ID “12A”. The video data is deleted from the RAM 123 (R51).

  The CPU 121 of the communication device 12B receives the first video data transmitted from the server device 11 as indicated by the arrow C18 (S31). The CPU 121 acquires the load state “medium state” stored in the first table 124A (see FIG. 2B) stored in the storage unit 124 of the communication device 12B. The CPU 121 acquires the resolution “640 × 480” associated with the same display ID as the third ID “12A” associated with the received first video data in the first table 124A. The CPU 121 determines that the condition (3) is satisfied since the medium state is set and the resolution is greater than the second predetermined value. In this case, the CPU 121 identifies the first video data as a decoding target (S33). The CPU 121 determines that the types of the video data match because the first video data is received by the processing of S31 and the first video data is specified by the processing of S33 (S35: YES). In this case, the CPU 121 decodes the received first video data to generate a first video (S37). The CPU 121 generates the first generated video window 23A (see FIG. 1) having the same display ID as the third ID “12A” associated with the received first video data among the video windows 21 included in the main window 20. One video is displayed (S39, R42 (see FIG. 8)).

  Further, the CPU 121 of the communication device 12B receives the second video data transmitted from the server device 11 as indicated by the arrow C20 (S31). In the same manner as described above, the CPU 121 specifies the first video data as a decoding target (S33). The CPU 121 determines that the types of video data do not match because the first video data has been received by the process of S31 and the second video data has been specified by the process of S33 (S35: NO). In this case, the CPU 121 deletes the received second video data from the RAM 123 (S41, R52 (see FIG. 8)).

  The CPU 121 of the communication device 12C receives the first video data transmitted from the server device 11 as indicated by the arrow C18 (S31). The CPU 121 acquires the load state “high state” stored in the first table 124A (see FIG. 2C) stored in the storage unit 124 of the communication device 12C. The CPU 121 acquires the resolution “320 × 240” associated with the same display ID as the third ID “12A” associated with the received first video data in the first table 124A. Since the high state is set, the CPU 121 determines that the condition (2) is satisfied. In this case, the CPU 121 specifies the second video data as a decoding target regardless of the acquired resolution (S33). The CPU 121 determines that the types of video data do not match because the first video data has been received by the process of S31 and the second video data has been specified by the process of S33 (S35: NO). In this case, the CPU 121 deletes the received first video data from the RAM 123 (S41, R43 (see FIG. 8)).

  Further, the CPU 121 of the communication device 12C receives the second video data transmitted from the server device 11 as indicated by the arrow C20 (S31). In the same manner as described above, the CPU 121 specifies the second video data as a decoding target (S33). The CPU 121 determines that the types of the video data match because the second video data is received by the processing of S31 and the second video data is specified by the processing of S33 (S35: YES). In this case, the CPU 121 decodes the received second video data to generate a second video (S37). The CPU 121 displays the second video in the video window 24A (see FIG. 1) having the same display ID as the third ID “12A” associated with the received second video data among the video windows 21 included in the main window 20. It is displayed (S39, R53 (see FIG. 8)).

<Server processing>
Server processing executed by the CPU 121 of the server device 11 will be described with reference to FIG. The server process is started when the CPU 111 executes the program stored in the storage unit 114 when any one of the load state, the resolution, and the video data is received from the communication device 12. In addition, CPU111 acquires ID and address information of all the communication apparatuses 12A-12C which participate in a meeting room, when a session is established with communication apparatuses 12A-12C. The CPU 111 associates the IDs and address information of the communication devices 12 </ b> A to 12 </ b> C and stores them in the storage unit 114.

  First, the CPU 111 determines whether the load state (C11, 13, 15 (see FIG. 8)) transmitted from the communication device 12 has been received (S101). When it is determined that the load state has been received (S101: YES), the CPU 111 identifies the ID of the communication device 12 that has transmitted the received load state. The CPU 111 associates the identified first ID with the load state as the first ID, and stores the first ID in the table 114A (see FIG. 9) of the storage unit 114 (S103). The CPU 111 associates the first ID with the received load state and broadcasts it to all the communication devices 12 participating in the conference room (S105, C12, 14, 16 (see FIG. 8)). The CPU 111 advances the process to S107. On the other hand, when it is determined that the load state has not been received (S101: NO), the CPU 111 advances the process to S107.

  The CPU 111 determines whether the resolution (C11, 13, 15 (see FIG. 8)) transmitted from the communication device 12 has been received (S107). When it is determined that the resolution has been received (S107: YES), the CPU 111 identifies the ID of the communication device 12 that has transmitted the received resolution as the first ID. The CPU 111 identifies the ID associated with the received resolution as the second ID. The CPU 111 associates the identified first ID and second ID with the received resolution, and stores them in the table 114A of the storage unit 114 (S109). The CPU 111 associates the first ID and the second ID with the received resolution and broadcasts to all the communication devices 12 participating in the conference room (S111, C12, 14, 16 (see FIG. 8)). The CPU 111 advances the process to S113. On the other hand, if it is determined that the resolution has not been received (S107: NO), the CPU 111 advances the process to S113.

  The CPU 111 determines whether the video data (C17, 19 (see FIG. 8)) transmitted from the communication device 12 has been received (S113). When it is determined that the video data has been received (S113: YES), the CPU 111 identifies the ID of the communication device 12 that has transmitted the received video data. The CPU 111 associates the identified ID with the video data as the third ID and broadcasts it to all the communication devices 12 participating in the conference room (S115, C18, 20 (see FIG. 8)). The CPU 111 ends the server process. On the other hand, when it is determined that the video data is not received (S113: NO), the CPU 111 ends the server process.

  Although not described in FIG. 10, when a session is established with the communication device 12, the CPU 111 establishes a session by associating the first ID with each load state stored in the table 114 </ b> A. To the communication device 12. Further, when a session is established with the communication device 12, the CPU 111 associates the first ID and the second ID with each resolution stored in the table 114A, and transmits it to the communication device 12 with the session established. . As a result, the CPU 121 of the communication device 12 can create the first table 124A and the second table 124B and display the video on the video window 21 even immediately after joining the conference room.

<Main functions and effects of this embodiment>
With the first table 124A and the second table 124B shown in FIGS. 2A to 2C stored in the storage unit 124 of the communication devices 12A to 12C, the load state is in the middle state by the CPU 121 of the communication device 12C. (S19) is illustrated as an example. In this case, unlike FIG. 2C, the middle state is stored as the load state in the first table 124A of the communication device 12C. Further, as shown in FIG. 11D, in the second table 124B of the communication device 12A, the middle state is stored as the load state corresponding to the first ID “12C”.

  In the above case, the CPU 121 of the communication device 12A has the load state of the communication device 12C based on the second table 124B in FIG. 11D (S61: YES), and the display unit of the communication device 12C. It is determined that the resolution (320 × 240) of the video window 24A having the display ID “12A” in the video window 21 displayed in 126 is equal to or lower than the second predetermined value (S63: NO). In this case, the CPU 121 determines to encode the shooting data under the second encoding condition (S67), generates the second video data, and transmits it to the server device 11 (S69, S73). The reason is to give priority to reducing the load on the CPU 121 of the communication device 12C. Here, the load on the CPU 121 when decoding the second video data is smaller than the load on the CPU 121 when decoding the first video data. Therefore, the CPU 121 of the communication device 12A can reduce the load when the CPU 121 of the communication device 12C performs the decoding process of the video data by generating and transmitting the second video data.

  When the second video is displayed on the video window 24A, the image quality of the second video may be lower than the image quality when the first video is displayed. On the other hand, since the resolution of the video window 24A is equal to or lower than the second predetermined value, the CPU 121 of the communication device 12A generates and transmits the second video data. The reason is that since the resolution of the video window 24A is low, for example, even when the second video is displayed on the video window 24A instead of the first video, the degradation of the image quality of the video is not noticeable. As described above, the communication device 12A generates and transmits the second video data, thereby suppressing the image quality of the video displayed on the video window 24A of the display unit 126 of the communication device 12C from being lowered. The load on the CPU 121 of the communication device 12C can be reduced.

  In the above case, the CPU 121 of the communication device 12A is based on the second table 124B of FIG. 11D, the load state of the communication device 12B is in the middle state (S61: YES), and the communication device 12B It is determined that the resolution (640 × 480) of the video window 23A with the display ID “12A” in the video window 21 displayed on the display unit 126 is larger than the second predetermined value (S63: YES). In this case, the CPU 121 determines to encode the shooting data under the first encoding condition (S65), generates the first video data, and transmits it to the server device 11 (S69, S73). The reason is that the video window 23A has a high resolution. For example, when the second video is displayed on the video window 23A instead of the first video, the video image quality is likely to deteriorate, so it is necessary to prioritize the video quality. Is high. Note that since the load state of the CPU 121 of the communication device 12B is in the middle state, it is unlikely that the load on the CPU 121 becomes too high due to the decoding of the first video data.

  In a state where the first table 124A and the second table 124B shown in FIGS. 2A to 2C are stored in the storage unit 124 of the communication devices 12A to 12C, the load state is reduced by the CPU 121 of the communication device 12B. The case where it is judged (S15) is illustrated. In this case, unlike FIG. 2B, the low state is stored as the load state in the first table 124A of the communication device 12B. Further, as shown in FIG. 11E, the low state is stored as the load state corresponding to the first ID “12B” in the second table 124B of the communication device 12A.

  In the above case, the CPU 121 of the communication device 12A is based on the second table 124B of FIG. 11E, and the load state of the communication device 12B is low (S59: YES). First, it is determined that the photographic data is encoded under the first encoding condition (S65). In this case, the CPU 121 generates first video data and transmits it to the server device 11 (S69, S73). The reason is that since the load on the CPU 121 of the communication device 12B is already low, the need for further reducing the load on the CPU 121 is low. In this case, the first video is displayed on the video window 23A of the display unit 126 of the communication device 12B. Accordingly, the communication device 12A can display the video with high image quality in the video window 21 of the display unit 126 of the communication device 12B.

  In the above case, the CPU 121 of the communication device 12A is based on the second table 124B of FIG. 11E, and the load state of the communication device 12C is high (S61: NO). Regardless, it is determined to encode the photographic data under the second encoding condition (S69). In this case, the CPU 121 generates the second video data and transmits it to the server device 11 (S69, S73). The reason is that the load on the CPU 121 of the communication device 12C is already high, and therefore there is a high need to reduce the load on the CPU 121. In this case, the CPU 121 of the communication device 12C decodes the second video data and displays the second video on the video window 24A. Therefore, the CPU 121 of the communication device 12A can reduce the load when the CPU 121 of the communication device 12C performs the video data decoding process.

  When the CPU 121 of the communication device 12 receives the video data broadcast from the server device 11 (S31), the CPU 121 determines the video data to be decoded based on the first table 124A and the second table 124B as (1) to It identifies according to the conditions of (4) (S33). When the received video data and the video data to be decoded are both the first video data, or when both are the second video data (S35: YES), the CPU 121 selects the video data. Decode (S37). On the other hand, when both the video data are different between the first video data and the second video data (S35: NO), the CPU 121 deletes the video data from the storage unit 124 without decoding (S41).

  Specifically, as (1), when a low state is set as its own load state, the CPU 121 specifies the first video data as a decoding target regardless of the resolution of the video window 21. This is because the load on the CPU 121 is low, and even if the first video data is decoded, there is a low possibility that the load on the CPU 121 becomes too high. Further, by displaying the first video on the video window 21, the quality of the video can be improved. In addition, as (2), when the high state is set as its own load state, the CPU 121 specifies the second video data as a decoding target regardless of the resolution of the video window 21. This is because the load on the CPU 121 is high, and it is highly necessary to reduce the load on the CPU 121 by decoding the second video data. In addition, as (3), when the medium state is set as the load state of the CPU 121 and the resolution of the video window 21 is larger than the second predetermined value, the CPU 121 decodes the first video data. As specified. This is because the video window 21 has a high resolution, and the deterioration of the image quality of the image is conspicuous, so that it is highly necessary to prioritize the image quality of the image. Moreover, since the load state of the CPU 121 is a medium state, it is unlikely that the load on the CPU 121 becomes too high due to the decoding of the first video data. Further, as (4), the CPU 121 specifies the second video data as a decoding target when the medium state is set as its own load state and the acquired resolution is equal to or lower than the second predetermined value. To do. This is because priority is given to reducing the load on the CPU 121, since the resolution of the video window 21 is low and the deterioration of the image quality of the video is less noticeable.

  As described above, the CPU 121 can maintain the image quality of the video displayed on the video window 21 while suppressing an increase in the load on the CPU 121. Further, the CPU 121 can determine whether or not to decode the video data broadcast from the server device 11 based on the first table 124A and the second table 124B.

  When the load state of the first table 124A is set to the low state, the CPU 121 identifies the first video data as the video data to be decoded (S33). Further, the CPU 121 specifies the usage rate of the CPU 121 when the video data is decoded as the degree of load (see FIG. 3). When the specified load degree is equal to or greater than the first predetermined value (S11: YES), and the load state of the first table 124A is set to a low state (S17: YES), the CPU 121 determines the first table. The medium state is set as the load state of 124A (S19). That is, when the degree of load when the first video data is decoded is equal to or greater than the first predetermined value, the load state is changed from the low state to the middle state. Therefore, when the degree of load when decoding the first video data with a large load on the CPU 121 at the time of decoding is equal to or higher than the first predetermined value, the CPU 121 loads the medium state one level higher than the low state. You can set the state.

  When the load state of the first table 124A is set to the high state, the CPU 121 specifies the second video data as the video data to be decoded (S33). Further, the CPU 121 determines that the specified degree of load is smaller than the first predetermined value (S11: NO) and the load state of the first table 124A is set to a high state (S13: YES). The middle state is set as the load state of the first table 124A (S19). That is, when the degree of load when the second video data is decoded is higher than the first predetermined value, the load state is changed from the high state to the medium state. For this reason, when the degree of load when decoding the second video data with a small load on the CPU 121 at the time of decoding is smaller than the first predetermined value, the CPU 121 takes the intermediate state one step below the high state. The load status can be set. Since the load status is set based on the load status set last time and the current load level, it is more detailed than when the load status is set only based on the current load level. It is possible to set the correct load state.

<Modification>
In addition, this invention is not limited to the said embodiment, A various change is possible. In the above, the communication between the communication devices 12 is executed via the server device 11. On the other hand, communication between the communication devices 12 may be performed directly without using the server device 11. The CPU 121 of the communication device 12 may cause the dedicated device to execute shooting data encoding processing and video data decoding processing. The CPU 121 may acquire video data output from the dedicated device by the encoding process of the dedicated device and video output from the dedicated device by the decoding process of the dedicated device.

  In the above, the CPU 121 specifies the usage rate of the CPU 121 when the video data is decoded as the degree of load (S11). The CPU 121 may specify another parameter as the degree of load. For example, the CPU 121 may specify at least one of the processing speed of the CPU 121, the processing amount, the usage rate of the RAM 123, and the time required to complete the decoding when the video data is decoded as the degree of load. In addition, the CPU 121 calculates a parameter by applying two or more of the usage rate, processing speed, processing amount, usage rate of the RAM 123, and the time required for decoding to a predetermined arithmetic expression. The degree of load may be specified.

  In the above description, the CPU 121 transmits the number of dots in the video window 21 to the server apparatus 11 as the resolution. For this reason, when the size of each dot differs depending on the communication device 12, that is, when the size of the display unit 126 of each communication device 12 is different, the size of the video window 21 is different even if the resolution is the same. There was a case. On the other hand, in addition to the resolution of the video window 21, the CPU 121 determines the vertical and horizontal lengths of the video window 21 (hereinafter referred to as “the size of the video window”) by the process of S45. May be sent to. When receiving the resolution and the size of the video window, the CPU 111 of the server device 11 may broadcast the resolution and the size of the video window to the communication device 12. When the CPU 121 of the communication device 12 receives the resolution and the size of the video window, the processing in S63 causes the resolution to be greater than the second predetermined value and the size of the video window to be greater than the third predetermined value (S63). : YES), it may be determined that the shooting data is encoded under the first encoding condition. On the other hand, if the resolution is equal to or smaller than the second predetermined value or the size of the video window is equal to or smaller than the third predetermined value (S63: NO), the CPU 121 may determine to encode the shooting data under the second encoding condition.

  The CPU 121 may identify one of the low state, the middle state, and the high state as the load state according to the elapsed time of the state where the usage rate is equal to or greater than the first predetermined value. For example, the CPU 121 may specify a larger load state (for example, one level above) when the state where the usage rate is equal to or higher than the first predetermined value continues for a predetermined time or longer. Here, the one-stage load state with respect to the low state is a medium state, and the one-stage load state with respect to the medium state is a high state. Further, the CPU 121 may specify a smaller load state (for example, one level lower) when the state where the usage rate is less than the first predetermined value continues for a predetermined time or longer. Here, the load state one level below the high state is a medium state, and the load state one step below the medium state is a low state. In addition, the CPU 121 identifies the load state in three stages (low state, medium state, and high state). On the other hand, the CPU 121 may specify the load state in two stages (low state and intermediate state, or intermediate state and high state), or may specify in four or more stages.

  In the above, when transmitting video data to the server device 11, the CPU 121 of the communication device 12 may designate and transmit the destination communication device 12 of the video data. Further, when transmitting the video data to the communication device 12, the server device 11 may transmit the video data to a destination designated when the video data is received instead of broadcast transmission. In this case, the CPU 121 may decode all received video data without executing the fifth process.

  In the above, the CPU 121 of the communication device 12 executes the first process (see FIG. 3) at a constant cycle to identify the load state and transmits it to the server device 11. In addition, when the resolution of the video window 21 is changed, the CPU 121 executes the second process (see FIG. 4) to acquire the resolution, and transmits it to the server device 11. On the other hand, the CPU 121 may transmit the request data to the server device 11 at the timing of transmitting the video data. When receiving the request data, the CPU 111 of the server device 11 may broadcast the request data. When receiving the request data, the CPU 121 of the communication device 12 may transmit the load state and the resolution to the server device 11. When the CPU 111 of the server device 11 receives the load state and resolution, the CPU 111 may transmit the load state and resolution to the communication device 12 that transmitted the request data.

<Others>
The process of S73 is an example of a “transmission step” in a communication program (hereinafter referred to as “first communication program”) to be executed by the computer of the first communication apparatus of the present invention. The processes of S83 and S87 are an example of “reception step” in the first communication program of the present invention. The process of S61 is an example of “first determination step” and “fourth determination step” in the first communication program of the present invention. The process of S63 is an example of the “second determination step” in the first communication program of the present invention. The processes of S65, S67, and S69 are examples of “first generation step”, “second generation step”, and “third generation step” in the first communication program of the present invention. The process of S59 is an example of a “third determination step” in the first communication program of the present invention. The processing of S15, S19, and S21 is an example of a “decision step” in a communication program (hereinafter referred to as “second communication program”) that is executed by the computer of the second communication apparatus of the present invention. The process of S41 is an example of an “acquisition step” in the second communication program of the present invention. The process of S31 is an example of a “reception step” in the second communication program of the present invention. The process of S33 is an example of a “first determination step” in the second communication program of the present invention. The processing of S35, S37, and S41 is an example of “processing step” in the second communication program of the present invention. The CPU 121 that performs the process of S73 is an example of the “transmission unit” in the present invention. The CPU 121 that performs the processes of S83 and S87 is an example of the “first receiving means” in the present invention. The CPU 121 that performs the process of S61 is an example of the “first determination unit” in the present invention. The CPU 121 that performs the process of S63 is an example of the “second determination unit” in the present invention. The CPU 121 that performs the processes of S65, S67, and S69 is an example of the “first generation unit” in the present invention. The CPU 121 that performs the processes of S15, S19, and S21 is an example of the “determination unit” in the present invention. The CPU 121 that performs the process of S41 is an example of the “acquiring unit” in the present invention. The CPU 121 that performs the process of S31 is an example of the “reception unit” in the present invention. The CPU 121 that performs the process of S33 is an example of the “third determination unit” in the present invention. The CPU 121 that performs the processes of S35, S37, and S41 is an example of the “processing means” in the present invention. The middle state is an example of the “first load state” in the present invention. The low state is an example of the “second load state” in the present invention. The high state is an example of the “third load state” in the present invention.

1: Communication system 11: Server devices 12, 12A, 12B, 12C: Communication device 111: CPU
114A: Table 121: CPU
124A: First table 124B: Second table 125: Camera

Claims (8)

A communication program for causing a computer of a first communication device capable of participating in a remote conference to execute,
A transmission step of transmitting video data in which video captured by the camera of the first communication device is encoded;
Before SL load condition when the second communication device to participate in a teleconference to decode the video data, and the resolution of the case where the image data is video image which has been decoded, is displayed on the display unit of the second communication device Receiving step for receiving,
A first determination step of determining whether the load state received by the reception step is a first load state;
A second determination step of determining whether the resolution received by the reception step is equal to or less than a second predetermined value;
When it is determined in the first load state by the first determination step,
In response to determining that the resolution is equal to or less than the second predetermined value in the second determination step, generate second video data encoded under a second encoding condition;
When the second determination step determines that the resolution is greater than the second predetermined value, the first video data encoded under the first encoding condition having a load greater than that of the second encoding condition is generated. Causing the computer to execute a first generation step;
The transmitting step includes
When the first video data is generated by the first generation step, the first video data is transmitted as the video data,
When the second video data is generated by the first generation step, the communication program transmits the second video data as the video data. A third determination step for determining whether the load state received by the reception step is a second load state indicating that the load state is smaller than the first load state;
If it is determined in the second load state by the third determination step, the computer further executes a second generation step of generating the first video data encoded under the first encoding condition;
The communication program according to claim 1, wherein the transmission step transmits the first video data generated by the second generation step as the video data. A fourth determination step for determining whether the load state received by the reception step is a third load state indicating that the load state is greater than the first load state;
If it is determined in the third load state by the fourth determination step, the computer further executes a third generation step of generating the second video data encoded under the second encoding condition,
The communication program according to claim 1 or 2, wherein the transmission step transmits the second video data generated by the third generation step as the video data. The first determination step includes
The degree of load when the second communication device decodes the first video data is greater than or equal to a first predetermined value, or the degree of load when the second communication device decodes the second video data. The communication program according to any one of claims 1 to 3, wherein if it is smaller than the first predetermined value, the first load state is determined.   The communication program according to claim 4, wherein the degree of load is a degree of load of a processor of the second communication device. A communication program for causing a computer of a second communication device capable of participating in a remote conference to execute,
A determination step of determining a load state in the second communication device when decoding video data from the first communication device participating in the remote conference;
An acquisition step of acquiring a resolution when a video obtained by decoding the video data is displayed on the display unit of the second communication device;
A receiving step of receiving, as the video data, first video data encoded under a first encoding condition, and second video data encoded under a second encoding condition having a smaller load than the first encoding condition;
A first determination step of determining to decode one of the first video data and the second video data based on the load state determined by the determination step and the resolution acquired by the acquisition step; ,
When the video data received by the receiving step and the video data determined to be decoded by the first determining step are both the first video data or when both are the second video data The video data decoded and received by the receiving step and the video data determined to be decoded by the first determining step are the first video data and the second video data. A communication program for causing the computer to execute a processing step that does not decode the video data if they are different. The determining step includes
Depending on the degree of load on the second communication device when decoding the video data, the load state is changed to a first load state, a second load state having a smaller degree than the first load state, and the It is determined as one of the third load states having a larger degree than the first load state,
The first determination step includes
If the second load state is determined by the determining step, determining that the first video data is to be decoded;
When the third load state is determined by the determining step, it is determined to decode the second video data;
When the first load state is determined by the determining step, and the resolution is greater than a second predetermined value, it is determined to decode the first video data;
7. The method according to claim 6, wherein when the first load state is determined by the determining step and the resolution is equal to or less than the second predetermined value, it is determined that the second video data is to be decoded. Communication program. A communication system including a first communication device and a second communication device capable of participating in a remote conference,
The first communication device is
Transmitting means for transmitting video data encoded with video captured by the camera of the first communication device;
Before SL load condition when the second communication device decodes the video data, and the first video to the video data is decoded, it receives the resolution when displayed on the display unit of the second communication device Receiving means;
First determination means for determining whether the load state received by the first reception means is a first load state;
Second determination means for determining whether the resolution received by the first reception means is equal to or lower than a second predetermined value;
When it is determined by the first determination means that the first load state,
In response to determining that the resolution is equal to or lower than the second predetermined value by the second determination unit, generate second video data encoded under a second encoding condition;
In response to determining that the resolution is greater than the second predetermined value by the second determination unit, first video data encoded under a first encoding condition having a load greater than that of the second encoding condition is generated. First generating means,
The transmission means includes
When the first video data is generated by the first generation means, the first video data is transmitted as the video data,
When the second video data is generated by the first generation means, the second video data is transmitted as the video data,
The second communication device is
Determining means for determining the load state;
Obtaining means for obtaining the resolution;
Second receiving means for receiving the first video data and the second video data transmitted from the first communication device as the video data;
Third determination means for determining to decode one of the first video data and the second video data based on the load state determined by the determination means and the resolution acquired by the acquisition means; ,
The video data received by the second receiving means and the video data determined to be decoded by the third determining means are both the first video data, or both are the second video data. In some cases, the video data is decoded, the video data received by the second receiving means, and the video data determined to be decoded by the third determining means are the first video data and the second video data. A communication system comprising processing means for decoding video data when it differs from video data.

Images