JP6213420B2 - Communication device, communication system, communication method, and program - Google Patents

Communication device, communication system, communication method, and program Download PDF

Info

Publication number
JP6213420B2
JP6213420B2 JP2014162870A JP2014162870A JP6213420B2 JP 6213420 B2 JP6213420 B2 JP 6213420B2 JP 2014162870 A JP2014162870 A JP 2014162870A JP 2014162870 A JP2014162870 A JP 2014162870A JP 6213420 B2 JP6213420 B2 JP 6213420B2
Authority
JP
Japan
Prior art keywords
image data
data
amount
communication
bit rate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2014162870A
Other languages
Japanese (ja)
Other versions
JP2016039553A (en
Inventor
拓也 今井
拓也 今井
健一郎 森田
健一郎 森田
翔 永峯
翔 永峯
Original Assignee
株式会社リコー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社リコー filed Critical 株式会社リコー
Priority to JP2014162870A priority Critical patent/JP6213420B2/en
Publication of JP2016039553A publication Critical patent/JP2016039553A/en
Application granted granted Critical
Publication of JP6213420B2 publication Critical patent/JP6213420B2/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Description

  The present invention relates to a communication device, a communication system, a communication method, and a program.

  As one form of a communication system that transmits and receives data between a plurality of communication terminals, a video conference system that realizes a remote conference using a communication network is known (for example, Patent Document 1).

  In the video conference system, the amount of data such as camera images and shared screens transmitted and received between communication terminals is large. It is common to use a compression technique such as H.264 / SVC. For communication using networks such as video conferencing and recording on media such as DVD / Blu-ray (registered trademark), the amount of data (bit rate) that can be transmitted per unit time is handled more uniformly. Since it is easy, the encoder (encoding device) of the communication terminal has a function of matching the amount of data after encoding with a target value (target bit rate) in a module called RateShaper.

  However, conventionally, for example, when the difference from the target bit rate in the first frame is large, or when the change from the previous frame is too large, a sufficient compression rate cannot be achieved, and the bit rate is reduced. Could not be constant.

To solve the above problems and achieve the object, the present invention is a communication device, when an acquisition unit configured to acquire a plurality of image data in sequence, each time the image data is acquired by the acquisition unit In addition, a determination unit that determines whether or not the amount of change between the image data acquired by the acquisition unit and the image data acquired immediately before exceeds a threshold value, and based on the amount of change, in unit time that is capable of transmitting a control means for performing constant-bit-rate control for constant bit rate representing a data amount of images data after compression encoding, wherein the amount of change by the determining means is the When it is determined that the threshold value is exceeded, the control means continuously performs the constant bit rate control for a predetermined time .

  According to the present invention, the bit rate can be made constant even when the amount of change between frames is large.

FIG. 1 is a diagram showing an example of the configuration of a telecommunications system. FIG. 2 is a conceptual diagram showing a state of transmission / reception of image data and audio data in the telecommunications system. FIG. 3 is a diagram illustrating a hardware configuration example of the communication terminal. FIG. 4 is a diagram illustrating a hardware configuration example of the remote communication management system. FIG. 5 is a diagram illustrating an example of a functional configuration of each terminal, apparatus, and system that configures the telecommunications system. FIG. 6 is a diagram illustrating an example of a functional configuration of the external input device. FIG. 7 is a diagram illustrating an example of a functional configuration of the encoding control unit. FIG. 8 is a flowchart illustrating an operation example when the communication terminal transmits image data after compression encoding to another communication terminal. FIG. 9 is a flowchart showing an example of detailed contents of the constant bit rate processing.

  Hereinafter, embodiments of a communication device, a communication system, a communication method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram showing an example of the configuration of the telecommunications system 1. In the remote communication system 1, for example, the communication terminal 10 is divided and arranged in the area A and the area B, but at least two communication terminals can be used for remote communication.

  The remote communication system 1 includes a plurality of communication terminals (10aa, 10ab,..., 10db), a display (100aa, 100ab,..., 100db) for each communication terminal (10aa, 10ab,. 30a, 30b, 30c, 30d), a remote communication management system 50, a program providing system 90, and an external input device 40.

  In this embodiment, “communication terminal 10” is used to indicate an arbitrary communication terminal among the communication terminals (10aa, 10ab,..., 10db), and any of the displays (100aa, 100ab,..., 100db). "Display 100" is used to indicate the display, and "Relay device 30" is used to indicate any of the relay devices (30a, 30b, 30c, 30d), and the external input device (40aa, 40ab,..., 40db), an “external input device 40” is used to indicate an arbitrary external input device. In the following, a communication terminal 10 as a request source for requesting the start of a video conference, which is an example of remote communication, will be referred to as “request source terminal 10A”, and a communication terminal 10 as a request destination will be referred to as “destination terminal” 10B ".

  The communication terminal 10 transmits / receives image data, audio data, and the like to / from other communication terminals 10. In the present embodiment, the case where the image of the image data is a moving image will be described, but it may be a still image as well as a moving image. The image of the image data may include both a moving image and a still image. The relay device 30 relays image data and audio data between the plurality of communication terminals 10. The remote communication management system 50 centrally manages the communication terminal 10 and the relay device 30.

  As shown in FIG. 2, in the remote communication system 1, management information for transmitting / receiving various types of management information between the request source terminal 10 </ b> A and the destination terminal 10 </ b> B via the remote communication management system 50. Session sei is established. In addition, between the request source terminal 10A and the destination terminal 10B, four pieces of data of high resolution image data, medium resolution image data, low resolution image data, and audio data are transmitted and received via the relay device 30. Four sessions are established. Here, these four sessions are collectively shown as an image / audio data session sed.

  Returning to FIG. 1, the description will be continued. The external input device 40 is connected to the communication terminal 10 and transmits display data for displaying material data to the communication terminal 10. The material data here means data used by using document creation software, spreadsheet software, presentation software, and the like.

  Further, the plurality of routers (70a, 70b,..., 70f) shown in FIG. 1 select an optimum route for image data and audio data. In the present embodiment, “router 70” is used to indicate an arbitrary router among the routers (70a, 70b,..., 70f).

  The program providing system 90 includes an HDD (Hard Disk Drive) (not shown) in which a communication terminal program for causing the communication terminal 10 to realize various functions or various means is stored. Program can be sent. The HDD of the program providing system 90 also stores a relay device program for causing the relay device 30 to realize various functions or various means, and can transmit the relay device program to the relay device 30. . Further, the HDD of the program providing system 90 also stores a remote communication management program for causing the remote communication management system 50 to realize various functions or various means, and the remote communication management system 50 stores the remote communication management program. Can be sent.

  Further, the communication terminal 10aa, the communication terminal 10ab, the relay device 30a, and the router 70a are connected to be communicable with each other via the LAN 2a. The communication terminal 10ba, the communication terminal 10bb, the relay device 30b, and the router 70b are connected to be communicable with each other via the LAN 2b. The LAN 2a and the LAN 2b are communicably connected via a dedicated line 2ab including the router 70c, and are constructed in a predetermined area A. For example, the region A is Japan, the LAN 2a is constructed in a Tokyo office, and the LAN 2b is constructed in an Osaka office.

  On the other hand, the communication terminal 10ca, the communication terminal 10cb, the relay device 30c, and the router 70d are connected to be communicable with each other via the LAN 2c. The communication terminal 10da, the communication terminal 10db, the relay device 30d, and the router 70e are connected to be communicable with each other via the LAN 2d. The LAN 2c and the LAN 2d are communicably connected by a dedicated line 2cd including the router 70f, and are constructed in a predetermined area B. For example, region B is the United States, LAN 2c is built in a New York office, and LAN 2d is Washington D.C. C. Is built in the office.

  Area A and area B are connected to each other via routers (70c, 70f) via the Internet 2i.

  The remote communication management system 50 and the program providing system 90 are communicably connected to the communication terminal 10 and the relay device 30 via the Internet 2i. The remote communication management system 50 and the program providing system 90 may be installed in the region A or the region B, or may be installed in a region other than these.

  In the present embodiment, the communication network 2 of the present embodiment is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d.

  Further, in FIG. 1, four sets of numbers shown under each communication terminal 10, each relay device 30, remote communication management system 50, each router 70, and program providing system 90 are in general IPv4. An IP address is simply shown. For example, the IP address of the communication terminal 10aa is “1.2.1.3”. Further, IPv6 may be used instead of IPv4.

(Hardware configuration)
Next, the hardware configuration of each device will be described. First, the hardware configuration of the communication terminal 10 will be described.

  FIG. 3 is an example of a hardware configuration diagram of the communication terminal 10. As shown in FIG. 3, the communication terminal 10 of this embodiment includes a CPU (Central Processing Unit) 101 that controls the operation of the entire communication terminal 10 and a ROM (Read Only Memory) 102 that stores a communication terminal program. , A RAM (Random Access Memory) 103 used as a work area of the CPU 101, a flash memory 104 for storing various data such as image data and audio data, and control of reading or writing of various data to the flash memory 104 according to the control of the CPU 101 An SSD (Solid State Drive) 105, a media drive 107 that controls reading or writing (storage) of data to a recording medium 106 such as a flash memory, an operation button 108 that is operated when a destination of the communication terminal 10 is selected, Turn on / off the communication terminal 10 A power switch 109 for switching, a network I / F 111 for transmitting data using the communication network 2 described later, a CCD (Charge Coupled Device) 112 for capturing an image of a subject and obtaining image data under the control of the CPU 101, and the CCD 112 Image pickup device I / F 113 for controlling the driving of the sound, microphone 114 for inputting sound, speaker 115 for outputting sound, and sound input / output I for processing input / output of sound signals between the microphone 114 and speaker 115 according to the control of CPU 101 FIG. 3 shows a display I / F 117 that transmits image data to the external display 100 according to the control of the CPU / F 116, an external device I / F 118 that transmits / receives various data to / from an external device, and the above components. Address bus and data for electrical connection A bus line 110 such as a tabas is provided.

  The recording medium 106 is detachable from the communication terminal 10. Further, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used. Further, the CCD 112 is a solid-state imaging device that converts light into electric charges and digitizes a subject image (video). The CCD 112 is not limited to a CCD as long as it captures a subject, and a CMOS (Complementary Metal Oxide Semiconductor) or the like. It may be used. Further, the display 100 is configured by a liquid crystal or an organic EL that displays an image of a subject, an operation icon, or the like, but is not limited thereto, and may be configured by a projector that projects an image on a projection surface such as a screen. Good. In short, the display 100 may be any device that displays an image.

  Further, the communication terminal program may be installed in an installable or executable format, recorded on a computer-readable recording medium such as the recording medium 106, and distributed.

  Next, the hardware configuration of the remote communication management system 50 will be described. FIG. 4 is a diagram illustrating an example of a hardware configuration of the remote communication management system 50 according to the present embodiment. The remote communication management system 50 includes a CPU 201 that controls the overall operation of the remote communication management system 50, a ROM 202 that stores a remote communication management program, a RAM 203 that is used as a work area for the CPU 201, and an HD (Hard Disk) that stores various data. 204, an HDD (Hard Disk Drive) 205 that controls reading or writing of various data to the HD 204 according to the control of the CPU 201, a media drive 207 that controls reading or writing (storage) of data to a recording medium 206 such as a flash memory, and a cursor , A display 208 for displaying various information such as menus, windows, characters, images, etc., a network I / F 209 for transmitting data using the communication network 2 described later, input of characters, numerical values, various instructions, etc. A keyboard 211 having a plurality of keys for selection, a mouse 212 for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like; a CD-ROM (Compact Disk Read Only Memory) as an example of a removable recording medium ) CD-ROM drive 214 for controlling reading / writing of data to / from 213, external device I / F 215 for transmitting / receiving information to / from an external device, and the above components are electrically connected as shown in FIG. For this purpose, a bus line 210 such as an address bus or a data bus is provided.

  The remote communication management program is a file in an installable or executable format, and is recorded on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 for distribution. Good.

  Further, since the external input device 40 has the same hardware configuration as the remote communication management system 50, the description thereof is omitted. However, the ROM 202 stores an external input device program for controlling the external input device 40. Also in this case, the external input device program is a file in an installable or executable format, and is recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Also good.

  Further, since the relay device 30 has the same hardware configuration as the remote communication management system 50, the description thereof is omitted. However, the ROM 202 stores a relay device program for controlling the relay device 30. Also in this case, the relay device program is a file in an installable or executable format, and is recorded on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 for distribution. Good.

  Furthermore, since the program providing system 90 has the same hardware configuration as that of the remote communication management system 50, the description thereof is omitted. However, a program providing program for controlling the program providing system 90 is recorded in the ROM 202. Also in this case, the program providing program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Good.

  In addition, as another example of the removable recording medium, recording is performed on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray (registered trademark) Disc. It may be configured to be provided.

(Functional configuration)
FIG. 5 is a block diagram showing an example of the functional configuration of each terminal, apparatus, and system that constitutes the remote communication system 1 of the present embodiment. In FIG. 5, the communication terminal 10, the relay device 30, and the remote communication management system 50 are connected so that data communication can be performed via the communication network 2. The external input device 40 is connected by, for example, a USB cable so that data can be transmitted to and received from the communication terminal 10. The program providing system 90 shown in FIG. 1 is omitted in FIG. 5 because it is not directly related to the video conference communication.

  First, the functional configuration of the communication terminal 10 will be described. As illustrated in FIG. 5, the communication terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14 a, an image display control unit 14 b, a voice input unit 15 a, a voice output unit 15 b, and a selection unit 16. , An external information transmitting / receiving unit 18, a storage / reading processing unit 19, an arrangement information selecting unit 21, a display data control unit 22, an encoding control unit 23, an encoding unit 24, and a decoding unit 25. Each of these units is a function or means realized by any one of the components shown in FIG. 3 operating according to a command from the CPU 101 according to a program stored in the ROM 102.

  When the external input device 40 is connected to the communication terminal 10, the external input device 40 acquires display data and transmits it to the communication terminal 10. When receiving the display data, the communication terminal 10 transmits the display data to the relay device 30 via the communication network 2.

  The transmission / reception unit 11 of the communication terminal 10 is realized by the network I / F 111 illustrated in FIG. 3, and transmits / receives various data (information) to / from other terminals, devices, or systems via the communication network 2. The operation input receiving unit 12 is realized by the operation button 108 and the power switch 109 shown in FIG. 3 and receives various inputs by the user. For example, when the user turns on the power switch 109 shown in FIG. 3, the operation input receiving unit 12 shown in FIG. 5 receives the power ON and turns on the power. The login request unit 13 is realized by a command from the CPU 101 shown in FIG. 3, and requests the login from the transmission / reception unit 11 to the remote communication management system 50 via the communication network 2 when receiving the power ON. The login request information indicating that the communication is to be performed and the current IP address of the communication terminal 10 are automatically transmitted.

  The imaging unit 14a is realized by the CCD 112 and the imaging element I / F 113 shown in FIG. 3, and images a subject and outputs video data obtained by the imaging. The image display control unit 14 b is realized by the display I / F 117 illustrated in FIG. 3, and performs control for transmitting image data to the external display 100. The image display control unit 14b displays the display of the external input device 40 connected to the communication terminal 10 as its own device, in addition to the video data captured by the communication terminal 10 as its own device and the imaging unit 14a of the other communication terminal 10. Displayed and captured display data displayed on the device 400 is displayed. The image display control unit 14b has a function of controlling a display area for video data and display data.

  The voice input unit 15a is realized by the microphone 114 and the voice input / output I / F 116 shown in FIG. 3. The voice input unit 15a inputs the voice of the user and converts the voice into a voice signal. Output audio data. The audio output unit 15b is realized by the speaker 115 and the audio input / output I / F 116 shown in FIG. 3, and converts an audio signal related to the audio data into audio and outputs the audio. The voice input unit 15a measures the input level of the voice signal and compares it with a threshold value or the like to determine the presence or absence of the voice signal.

  The arrangement information selection unit 21 includes video data captured by the image capturing unit 14a of the communication terminal 10 and the other communication terminal 10 as the own device, and an external device connected to the communication terminal 10 and the other communication terminal 10 as the own device. Arrangement information indicating how to arrange display data (described later) input from the input device 40 on the display 100 is selected, and the selected arrangement information is instructed to the image display control unit 146. The specific content may be the same as the content disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941. In the description of the present embodiment, when image data captured by the imaging unit 14a and display data input from the external input device 40 are not distinguished, they may be simply referred to as “image data”.

  The selection unit 16 selects one relay device 30 from the plurality of relay devices 30. The selection method of the relay device 30 may be the same as that disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  The display data control unit 22 acquires display data of the external input device 40. The display data represents image data obtained by converting an image displayed on the screen of the display device 400 into a format such as JPEG (Joint Photographic Experts Group) or Bitmap, or a rendering command such as GDI (Graphics Device Interface). The display data may include document data or may be a simple desktop screen that does not include document data. The specific content of the display data control unit 22 may be the same as the content disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  The encoding control unit 23 performs control related to compression encoding of image data to be transmitted to another communication terminal 10. Details of the encoding control unit 23 will be described later. The encoding unit 24 performs compression encoding of image data to be transmitted to another communication terminal 10 under the control of the encoding control unit 23. The compression encoding method is arbitrary. H.264 / SVC, H.H. H.264 / AVC, H.H. H.265 (HEVC), Motion JPEG, or the like can be used. For example, as a standard encoding format for encoding image data in a scalable manner, H.264 is used. The H.264 / SVC (H.264 / AVC Annex G) encoding format is known. H. In the H.264 / SVC encoding format, image data is converted into hierarchical data, encoded as a set of a plurality of image data having different qualities, and encoded data corresponding to the image data of each quality is converted into a plurality of channels ( In the example of FIG. 2, transmission / reception can be performed using a channel for high-resolution image data, a channel for medium-resolution image data, and a channel for low-resolution image data. In the present embodiment, this H.264. The encoded data obtained by encoding the image data using the H.264 / SVC encoding format is transmitted / received between the plurality of communication terminals 10, but is not limited thereto.

  The decoding unit 25 has a function of decoding the compression-coded image data received from the other communication terminal 10.

  Next, the functional configuration of the external input device 40 will be described. As shown in FIG. 6, the external input device 40 includes a transmission / reception unit 41, a connection detection unit 42, a display data acquisition unit 451, a display data transmission unit 452, an operation input reception unit 46, and a storage / read processing unit 49. doing. Each of these units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the external input device 40 has a display device 400 constructed by the display 208 shown in FIG. Further, the external input device 40 is equipped with an OS (Operating System) such as Windows (registered trademark), and thereby has a function of executing a program when connected to another device.

  The transmission / reception unit 41 is realized by the network I / F 209 illustrated in FIG. 4, and transmits / receives various data (information) from the communication terminal 10. The connection detection unit 42 detects that data transmission / reception with the communication terminal 10 can be performed by the external device I / F 215. The operation input receiving unit 46 is realized by a keyboard and a mouse and receives an input by a user operation.

  The display data acquisition unit 451 acquires display data of the external input device 40 by capturing the screen of the display device 400, and the display data transmission unit 452 receives the display data acquired by the display data acquisition unit 451. This is transmitted to the communication terminal 10.

  The storage / reading processing unit 49 stores various data using the HDD 205 shown in FIG. 4 as a substance, and reads various data stored in a storage unit (not shown).

  The specific content of the external input device 40 may be the same as the content disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941. Moreover, the specific content of the relay apparatus 30 may be the same as the content disclosed by Unexamined-Japanese-Patent No. 2012-134941, for example.

  Returning to FIG. 5, the functional configuration of the remote communication management system 50 will be described. The remote communication management system 50 includes a transmission / reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state acquisition unit 55, a session management unit 57, and a storage / read processing unit 59. Each of these units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the remote communication management system 50 has a storage unit 5000 constructed by the HD 204 shown in FIG.

  In the storage unit 5000, a relay device management DB 5001 configured by a relay device management table is constructed. In this relay device management table, for each relay device ID of each relay device 30, the operating status of each relay device 30, the reception date and time when the status information indicating the operating status is received by the remote communication management system 50, the relay device 30 The IP address and the maximum data transmission rate (Mbps) in the relay device 30 are associated and managed. Specific contents of the relay device management DB 5001 may be the same as the contents disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  Further, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table is constructed. In this terminal authentication management table, each password is associated with each terminal ID of all communication terminals 10 managed by the remote communication management system 50 and managed. The specific content of the terminal authentication management DB 5002 may be the same as the content disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  In the storage unit 5000, a terminal management DB 5003 configured by a terminal management table is constructed. In this terminal management table, for each terminal ID of each communication terminal 10, the operating state of each communication terminal 10, the reception date and time when the login request information is received by the remote communication management system 50, and the IP address of the communication terminal 10 are associated. Managed. The specific content of the terminal management DB 5003 may be the same as the content disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  Further, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table is constructed. In this destination list management table, all terminal IDs of destination terminals 10B registered as candidates for destination terminal 10B are managed in association with terminal IDs of request source terminal 10A requesting the start of a video conference. Specific contents of the destination list management DB 5004 may be the same as the contents disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  Further, in this storage unit 5000, a session management DB 5005 configured by a session management table is constructed. In this session management table, for each selection session ID used to execute a session for selecting the relay device 30, the relay device ID and request of the relay device 30 used for relaying image data, audio data, and display data. The terminal ID of the original terminal 10A and the terminal ID of the destination terminal 10B are managed in association with each other. The specific content of the session management DB 5000 may be the same as the content disclosed in, for example, JP2012-134941A.

  Next, each functional unit of the remote communication management system 50 will be described. The transmission / reception unit 51 is executed by the network I / F 209 illustrated in FIG. 4, and transmits / receives various data (information) to / from other terminals, devices, or systems via the communication network 2. The terminal authentication unit 52 searches the terminal authentication management DB 5002 in the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and is identical to the terminal authentication management DB 5002. Terminal authentication is performed by determining whether or not the terminal ID and password are managed. In order to manage the operation state of the request source terminal 10A that has requested login, the state management unit 53 stores the terminal ID of the request source terminal 10A, the operation state of the request source terminal 10A, and the remote communication management system 50 in the terminal management table. The reception date and time when the login request information is received and the IP address of the request source terminal 10A are stored and managed in association with each other.

  The terminal extraction unit 54 searches the destination list management table using the terminal ID of the request source terminal 10A that requested the login as a key, and reads the candidate terminal ID of the destination terminal 10B that can remotely communicate with the request source terminal 10A. Then, the terminal ID is extracted. In addition, the terminal extraction unit 54 searches the above-described destination list management table using the terminal ID of the request source terminal 10A that requested the login as a key, and registers the terminal ID of the request source terminal 10A as a candidate for the destination terminal 10B. The terminal ID of the other request source terminal 10A is also extracted.

  The terminal status acquisition unit 55 searches the above-described terminal management table using the candidate terminal ID of the destination terminal 10B extracted by the terminal extraction unit 54 as a search key, and for each terminal ID extracted by the terminal extraction unit 54 Read the operating status. Thereby, the terminal state acquisition unit 55 can acquire the operation state of the candidate of the destination terminal 10B that can remotely communicate with the request source terminal 10A that has requested the login. Further, the terminal status acquisition unit 55 searches the above-described terminal management table using the terminal ID extracted by the terminal extraction unit 54 as a search key, and also acquires the operating status of the request source terminal 10A that has requested login.

  The session management unit 57 stores and manages the selection session ID, the terminal ID of the request source terminal 10A, and the terminal ID of the destination terminal 10B in the above-described session management table. In addition, the session management unit 57 stores and manages the relay device ID of the relay device 30 that is finally selected by the selection unit 16 of the communication terminal 10 for each selection session ID in the session management table. To do.

  The storage / reading processing unit 59 is executed by the HDD 205 shown in FIG. 4 and performs processing for storing various types of data in the storage unit 5000, processing for reading out various types of data stored in the storage unit 5000, and the like.

  The specific contents of the remote communication management system 50 may be the same as the contents disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941.

  Specific contents such as a processing procedure for establishing a session between the communication terminals 10 and a processing procedure for transmitting and receiving image data and audio data between the communication terminals 10 are disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-134941. It may be the same as the contents made.

  Next, specific contents of the encoding control unit 23 of the communication terminal 10 will be described. FIG. 7 is a diagram illustrating an example of a functional configuration of the encoding control unit 23. As illustrated in FIG. 7, the encoding control unit 23 includes an image acquisition unit 301, an image processing unit 302, a comparison determination unit 303, a timer control unit 304, and a constant bit rate control unit 305. These units are realized by the CPU 101 shown in FIG. 3 executing a program stored in the ROM 102.

  The image acquisition unit 301 acquires a plurality of image data in time series. In this example, the image acquisition unit 301 corresponds to “acquisition means” in the claims. In the present embodiment, the image acquisition unit 301 sequentially acquires video data imaged by the imaging unit 14a or display data received by the external information transmission / reception unit 18.

  The image processing unit 302 performs image processing such as format conversion processing of the image data acquired by the image acquisition unit 301, copy processing of the image data, and rescaling processing of the copy image obtained by the copy processing. An example of the format conversion process includes a process of converting a YUV2 / JPG format into RGB. As an example of the rescaling process, for example, a process of reducing the resolution of a copy image to about 80 dots wide and 40 dots high can be cited. If a sufficient memory capacity can be secured, the image processing may be omitted.

  Each time the image acquisition unit 301 acquires image data, the comparison determination unit 303 changes the amount of change (the current frame and the previous frame) between the image data acquired by the image acquisition unit 301 and the image data acquired immediately before the image data. It is determined whether or not the change amount exceeds the threshold value. In this example, the comparison determination unit 303 corresponds to “determination means” in the claims. For example, in the RGB format, the pixel value (density value) of the current frame may be compared with the pixel value of the previous frame for each Red / Green / Blue in units of one dot. For example, for each dot, deltaRGB indicating the sum of the change amount of Red (pixel value of the current frame−pixel value of the previous frame), the change amount of Green and the change amount of Blue is obtained, and the value of deltaRGB is the reference value ( For example, when the number of dots exceeding 16) exceeds a predetermined number (for example, 1000), it can be determined that the amount of change between the current frame and the previous frame exceeds the threshold.

  The timer control unit 304 controls the operation of a timer (not shown) for measuring time. In this embodiment, whenever the comparison determination unit 303 determines that the amount of change between the current frame and the previous frame exceeds the threshold, the timer control unit 304 performs control for resetting the timer (the measured time is set to the initial value). The timer is controlled so as to start counting again.

  The constant bit rate control unit 305 can transmit a unit time (for example, 1 second) based on the amount of change between the image data acquired by the image acquisition unit 301 and the image data acquired immediately before the image data. Then, constant bit rate control is performed to make the bit rate representing the data amount of the image data after compression encoding constant. More specifically, the constant bit rate control unit 305 performs constant bit rate control when the amount of change exceeds a threshold value. In the present embodiment, the constant bit rate control is control for reducing the resolution of the image data (current frame) acquired by the image acquisition unit 301. More specifically, the constant bit rate control is control for reducing the resolution of the image data acquired by the image acquisition unit 301 to ½. In this example, the constant bit rate control unit 305 corresponds to “control means” in the claims.

  For example, in a conventional technique such as RateShaper, when the difference from the target bit rate in the first frame is large or when the change from the previous frame is too large, the data amount after compression encoding is adjusted to the target bit rate. The control was not effective and the bit rate could not be kept constant. In addition, non-natural images such as text data displayed by screen sharing cannot increase the compression ratio because there are many areas with higher spatial frequencies than natural images, and large image changes such as when switching pages are frequent. Therefore, there arises a problem that the bit rate cannot be made constant.

  On the other hand, in the present embodiment, when the amount of change between the current frame and the previous frame exceeds a threshold value, control is performed to reduce the resolution of the current frame. Since the amount of data can be reduced, even when the network bandwidth cannot be sufficiently secured, it is possible to achieve the advantageous effect that the communication delay of the image data can be prevented while the bit rate is kept constant. .

  In this embodiment, as constant bit rate control, control is performed to reduce the resolution of image data (current frame) acquired by the image acquisition unit 301. However, the present invention is not limited to this, and constant bit rate control is Any control may be used as long as the bit rate of the image data after compression encoding is constant. For example, as constant bit rate control, contrary to the above example, control for increasing the resolution of the current frame can be performed. For example, as constant bit rate control, control for discarding the current frame can be performed, and control for increasing / decreasing the compression rate can also be performed. In this example, the constant bit rate control unit 305 can instruct the encoding unit 24 about the compression rate and can also indicate whether the encoded data to be generated is an I picture or a P picture. In this example, the encoding unit 24 is provided independently of the encoding control unit 23, but is not limited thereto, and for example, the encoding unit 24 is provided in the encoding control unit 23. There may be.

  In the present embodiment, when the comparison determination unit 303 determines that the amount of change between the current frame and the previous frame exceeds the threshold, the constant bit rate control unit 305 continues for a predetermined time. Then, constant bit rate control is performed. In this example, the predetermined time is 1 second, but is not limited thereto. If it is determined that the amount of change between the current frame and the previous frame exceeds the threshold, there will be no interval during a predetermined time even if the amount of change between the subsequent frame and the previous frame is less than or equal to the threshold. By performing constant bit rate control under conditions, the bit rate can be kept constant for at least the time.

  FIG. 8 is a flowchart illustrating an operation example when the communication terminal 10 transmits image data after compression encoding to another communication terminal 10. As shown in FIG. 8, first, the communication terminal 10 (image acquisition unit 301) acquires image data (step S1). Next, the communication terminal 10 performs constant bit rate processing on the image data acquired in step S1 (step S2). Details of the constant bit rate processing will be described later. Next, the communication terminal 10 (encoding unit 24) compresses and encodes the image data after the constant bit rate processing (step S3). Next, the communication terminal 10 (transmission / reception unit 11) transmits the image data compressed and encoded in step S3 to another communication terminal 10 (communication terminal 10 serving as a video conference partner) (step S4). Each time image data is acquired by the image acquisition unit 301, the above processing is repeated.

  FIG. 9 is a flowchart showing an example of detailed contents of the constant bit rate processing. As shown in FIG. 9, the image processing unit 302 performs the above-described image processing on the image data acquired in step S1 of FIG. 8 (step S11). Here, an example will be described in which the image processing unit 302 performs a process of reducing the resolution of the copy image of the image data acquired in step S1 to a width of about 80 dots and a height of about 40 dots. Next, the comparison / determination unit 303 performs a process of comparing the 80 × 40 current frame created in step S11 with the previous frame of the same size (step S12), and the amount of change between the current frame and the previous frame is determined. It is determined whether or not the threshold value is exceeded (step S13). This determination method is as described above.

  When the amount of change between the current frame and the previous frame exceeds the threshold value (step S13: Yes), the timer control unit 304 performs control to reset a timer (not shown) (step S14), and starts the timer again. To control. When the time measured by the timer is less than 1 second (step S15: Yes), the constant bit rate control unit 305 performs the above-described constant bit rate control (step S16). In this example, the constant bit rate control unit 305 sets the resolution of the image data acquired in step S1 in FIG. 8 (the original image before the image processing in step S11) to a minimum value of 640 × 360 and the width. And the height is halved. In this example, as a post-process, a process of registering an 80 × 40 current frame (a copy image after the rescale process in step S11) as a previous frame in a memory (not shown) is performed.

  In this example, even when the amount of change between the current frame and the previous frame is equal to or less than the threshold (step S13: No), if the time counted by the timer is less than 1 second (step S15: Yes), The constant bit rate control unit 305 performs the above-described constant bit rate control (step S16). On the other hand, when the time counted by the timer is 1 second or longer (step S15: No), the constant bit rate control is not performed, and the process ends as it is. The above is the detailed content of the constant bit rate processing.

  In the present embodiment, the communication terminal 10 performs the constant bit rate process, and transmits / receives data including image data after compression encoding to / from other communication terminals 10. Therefore, it can be considered that the communication terminal 10 corresponds to the “communication device” in the claims, but is not limited to this. For example, display data (for example, display data of the external input device 40 connected to the communication terminal 10 that is the own device) When transmitting (screen sharing data) to another communication terminal 10, the external input device 40 may perform constant bit rate processing for making the bit rate of display data after compression encoding constant. In this case, the external input device 40 has the function of the encoding control unit 23 described above, and can be considered to correspond to the “communication device” in the claims. In this case, the communication terminal 10 to which the external input device 40 is connected corresponds to the “second communication terminal” in the claims, and the other communication terminal 10 to which the display data after compression encoding is transmitted is claimed. It can be considered that this corresponds to the “first communication terminal”. In this embodiment, the remote communication system 1 corresponds to the “communication system” in the claims, and one communication terminal 10 included in the remote communication system 1 corresponds to the “first communication device” in the claims. The other communication terminal 10 that transmits and receives data to and from the one communication terminal 10 can be considered to correspond to the “second communication device” in the claims.

  As described above, in the present embodiment, when the amount of change between the current frame and the previous frame exceeds the threshold, control is performed to reduce the resolution of the current frame. The amount of image data can be reduced, so that even if the network bandwidth cannot be sufficiently secured, it is possible to prevent a delay in image data communication while maintaining a constant bit rate. Can be achieved.

  Although the embodiments according to the present invention have been described above, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment.

  In the above-described embodiment, the communication terminal 10 and the external input device 40 included in the remote communication system 1 are described as examples of the communication device to which the present invention is applied. However, the communication device to which the present invention is applied is described. Is not limited to this. For example, a communication device to which the present invention is applied includes a conference dedicated terminal, a game device, a tablet-type terminal, a mobile phone, a smartphone, a general-purpose PC terminal, a car navigation terminal mounted on a car, a projection device such as a projector, an electronic blackboard, a wearable It may be a terminal, an industrial device having a communication function, or the like.

  The program executed in the remote communication system 1 of the above-described embodiment is a file in an installable format or an executable format, and is a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk). It may be configured to be recorded on a computer-readable recording medium such as a USB (Universal Serial Bus) and provided or distributed via a network such as the Internet. Further, various programs may be provided by being incorporated in advance in a nonvolatile recording medium such as a ROM.

DESCRIPTION OF SYMBOLS 1 Remote communication system 2 Communication network 10 Communication terminal 23 Encoding control part 24 Encoding part 30 Relay apparatus 40 External input device 50 Remote communication management system 301 Image acquisition part 302 Image processing part 303 Comparison determination part 304 Timer control part 305 Constant bit Rate control unit

JP 2012-134941 A

Claims (7)

  1. A communication device,
    Acquisition means for acquiring a plurality of image data in time series;
    A determination unit that determines whether the amount of change between the image data acquired by the acquisition unit and the image data acquired immediately before the image data exceeds the threshold every time the image data is acquired by the acquisition unit. When,
    Based on the amount of change, which can be transmitted per unit time, and a control means for performing constant-bit-rate control for the bit rate representing a data amount constant of images data after compression coding ,
    When it is determined by the determination means that the amount of change exceeds the threshold, the control means continuously performs the constant bit rate control for a predetermined time.
    Communication device.
  2. The predetermined time is 1 second;
    The communication apparatus according to claim 1 .
  3. The communication device is connected to the second communication terminal to transmit and receive data, including images data after compression coding between the first communication terminal,
    The communication device according to claim 1 or 2 .
  4. The communication device transmits and receives data including images data after compression encoding with other communication terminals,
    The communication apparatus according to any one of claims 1 to 3 .
  5. A communication system including a first communication device and a second communication device that transmits and receives data between the first communication device,
    Each of the first communication device and the second communication device is:
    Acquisition means for acquiring a plurality of image data in time series;
    A determination unit that determines whether the amount of change between the image data acquired by the acquisition unit and the image data acquired immediately before the image data exceeds the threshold every time the image data is acquired by the acquisition unit. When,
    Based on the amount of change, which can be transmitted per unit time, and a control means for performing constant-bit-rate control for the bit rate representing a data amount constant of images data after compression coding ,
    When it is determined by the determination means that the amount of change exceeds the threshold, the control means continuously performs the constant bit rate control for a predetermined time.
    Communications system.
  6. An acquisition step of acquiring a plurality of image data in time series;
    A determination unit that determines whether the amount of change between the image data acquired by the acquisition step and the data acquired immediately before the image data exceeds the threshold every time the image data is acquired by the acquisition step. When,
    Based on the amount of change, which can be transmitted per unit time, and a control step of performing constant-bit-rate control for constant bit rate representing a data amount of images data after compression encoding, the free See
    When the determination step determines that the amount of change exceeds the threshold value, the control step continuously performs the constant bit rate control for a predetermined time.
    Communication method.
  7. On the computer,
    An acquisition step of acquiring a plurality of image data in time series;
    A determination unit that determines whether the amount of change between the image data acquired by the acquisition step and the data acquired immediately before the image data exceeds the threshold every time the image data is acquired by the acquisition step. When,
    Based on the amount of change, which can be transmitted per unit time, running and a control step of performing constant bit rate control for the bit rate representing a data amount constant of images data after compression coding then,
    When the determination step determines that the amount of change exceeds the threshold value, the control step continuously performs the constant bit rate control for a predetermined time.
    program.
JP2014162870A 2014-08-08 2014-08-08 Communication device, communication system, communication method, and program Active JP6213420B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2014162870A JP6213420B2 (en) 2014-08-08 2014-08-08 Communication device, communication system, communication method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2014162870A JP6213420B2 (en) 2014-08-08 2014-08-08 Communication device, communication system, communication method, and program

Publications (2)

Publication Number Publication Date
JP2016039553A JP2016039553A (en) 2016-03-22
JP6213420B2 true JP6213420B2 (en) 2017-10-18

Family

ID=55530296

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2014162870A Active JP6213420B2 (en) 2014-08-08 2014-08-08 Communication device, communication system, communication method, and program

Country Status (1)

Country Link
JP (1) JP6213420B2 (en)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0662392A (en) * 1992-08-05 1994-03-04 Nec Corp High efficient dynamic image encoding system
JPH0795566A (en) * 1993-09-21 1995-04-07 Nippon Telegr & Teleph Corp <Ntt> Method and device for encoding image
JP3985567B2 (en) * 2002-04-03 2007-10-03 三菱電機株式会社 Encoding device and encoding method
WO2010150470A1 (en) * 2009-06-23 2010-12-29 パナソニック株式会社 Moving image encoding method, moving image encoding device, program, and integrated circuit
US8731331B2 (en) * 2010-11-30 2014-05-20 Ricoh Company, Ltd. Apparatus, system, and method of communication, and recording medium storing communication control program
JP5887756B2 (en) * 2010-11-30 2016-03-16 株式会社リコー External input device, communication terminal, display data sharing system, program
JP2014003448A (en) * 2012-06-18 2014-01-09 Ricoh Co Ltd Transmission system, external input device, and program for display resolution conversion

Also Published As

Publication number Publication date
JP2016039553A (en) 2016-03-22

Similar Documents

Publication Publication Date Title
JP6636122B2 (en) Decoding device and decoding method
US9412332B2 (en) Method for wirelessly transmitting content from a source device to a sink device
US10225517B2 (en) Communication management system, communication system, computer-readable recording medium, and maintenance system
CA2790740C (en) Transmission system and transmitting method
KR102052296B1 (en) Establishing a video conference during a phone call
JP2018098795A (en) Transmission terminal, transmission method and program for transmission
CN103222262B (en) For skipping the system and method for Video coding in a network environment
KR101257594B1 (en) Transmission management system, transmission system, computer-readable recording medium, program providing system, and maintenance system
JP5962098B2 (en) Transmission terminal, transmission system, display control method, and program
JP5953689B2 (en) Transmission management system, transmission system, and program for transmission management system
JP5793869B2 (en) Transmission management system, transmission management method, and transmission management program
US8599934B2 (en) System and method for skip coding during video conferencing in a network environment
JP5655499B2 (en) Transmission management system, transmission system, transmission management program, program providing system, and maintenance system
JP5811334B2 (en) Transmission management system, transmission management method, program, and transmission system
CA2820872C (en) Transmission management apparatus
JPWO2013132828A1 (en) Communication system and relay device
JP5887756B2 (en) External input device, communication terminal, display data sharing system, program
RU2679979C1 (en) Representing motion vectors in encoded bitstream
US10045089B2 (en) Selection of encoder and decoder for a video communications session
JP6136217B2 (en) Communication management system, communication system, program, and maintenance system
US20170171513A1 (en) Remote conference system, communication terminal, and program
US10075719B2 (en) Image coding apparatus and method
US9398255B2 (en) Information processing apparatus, information processing system and information processing method
JP2014090387A (en) Information processor, conference system and program
JP5884964B2 (en) Transmission system and transmission method

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20170317

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20170317

A975 Report on accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A971005

Effective date: 20170609

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20170613

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20170614

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20170808

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20170822

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20170904

R151 Written notification of patent or utility model registration

Ref document number: 6213420

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151