JP7400042B2 - Communication terminal and program - Google Patents

Description

The present invention relates to a communication terminal and a program.

Patent Document 1 discloses that when it is determined that congestion occurs in communication between an information terminal and an information distribution server and the loss rate of packets from the information distribution server is increasing, the bit rate of packets constituting distribution data is A technique is described in which an information terminal requests an information distribution server to change the information (for example, see Patent Document 1).
[Prior art documents]
[Patent document]
[Patent Document 1] Japanese Patent Application Publication No. 2015-033095

According to one embodiment of the invention, a communication system is provided. The communication system may include a transmitting terminal. The communication system may include a receiving device. The receiving device may include a data receiving unit that receives data from the transmitting terminal via the network. The receiving device may include a quality determining unit that determines the quality of data. The receiving device may include a process selection unit that selects a process to be executed from among a notification process for the sending terminal, a data correction process, and a communication quality change request process for the network, depending on the quality of the data. The receiving device may include a process execution unit that executes the process selected by the process selection unit.

The processing selection unit selects the notification processing, the correction processing, and the change request processing when the quality of the data is lower than the first threshold, and when the quality of the data is higher than the first threshold, If it is lower than a second threshold that is higher than one threshold, select two of the notification processing, the correction processing, and the change request processing, and the quality of the data is higher than the second threshold and higher than the second threshold. If it is lower than the third threshold, one of the notification process, the correction process, and the change request process may be selected. The processing selection unit selects the notification processing and the correction processing when the quality of the data is higher than the first threshold and lower than the second threshold, and when the quality of the data is higher than the second threshold and the If it is lower than the third threshold, the above notification process may be selected. The quality determining unit may determine that the higher the packet loss rate of communication with the transmitting terminal, the lower the quality of the data.

The processing execution unit may transmit notification information to the sending terminal when the notification processing is selected by the processing selection unit, and the sending terminal may transmit the notification information to the sending terminal in response to receiving the notification information. The bit rate of the data sent to the device may be reduced. When the correction process is selected by the process selection unit, the processing execution unit notifies the transmission terminal of the correction technology to be used, and then uses the correction technology to process the data received by the data reception unit. You may correct it. The processing execution section may transmit request information to the transmission terminal when the correction processing is selected by the processing selection section, and the transmission terminal transmits the data to the transmission terminal in response to receiving the request information. Instead, correction data obtained by correcting the data and having a smaller amount of data than the data may be transmitted to the receiving device. The data may be video data, and the transmitting terminal may transmit the correction data including only feature points extracted from the data to the receiving device in response to receiving the request information. The data may be video data, and the transmitting terminal may transmit the correction data, which is a simplified display of a part of the data, to the receiving device in response to receiving the request information. The communication system may further include the network, and the process execution unit may transmit a communication quality change request to the network when the change request process is selected by the process selection unit, and the process execution unit may transmit a communication quality change request to the network. The network may execute a change process of changing the communication quality of the receiving device in response to receiving the change request. The network may be a mobile network, and in response to receiving the change request, the network performs at least priority control on the receiving device, bandwidth control on the receiving device, and user plane switching of the receiving device. You can do either one.

According to one embodiment of the present invention, a communication device is provided. The communication device may include a data reception unit that receives data from a communication terminal via a network. The communication device may include a quality determination unit that determines the quality of data. The communication device may include a process selection unit that selects a process to be executed from among a notification process for a communication terminal, a data correction process, and a communication quality change request process for a network, depending on the quality of the data. The communication device may include a process execution unit that executes the process selected by the process selection unit.

According to one embodiment of the present invention, a program for causing a computer to function as the communication device is provided.

According to one embodiment of the present invention, a communication terminal is provided. The communication terminal may include a data storage unit that stores data. The communication terminal may include a data transmitter that transmits data to the receiving device via the network. The communication terminal may include a data correction unit that corrects the data in response to receiving the request information from the receiving device that received the data, and generates correction data having a smaller amount of data than the data. The data transmitter may transmit correction data to the receiving device instead of the data. The data may be video data, and the data correction section may generate the correction data including only feature points extracted from the data. The data may be video data, and the data correction section may generate the correction data in which a part of the data is simplified.

According to one embodiment of the present invention, a communication terminal is provided. The communication terminal may include a data storage unit that stores data. The communication terminal may include a data transmitter that transmits data to the receiving device via the network. The communication terminal may include a data receiving unit that receives data received by the receiving device from the receiving device via a network. The communication terminal may include a quality determining unit that determines the quality of data received by the receiving device based on the data transmitted by the data transmitting unit and the data received by the data receiving unit. The communication terminal includes a process selection unit that selects a process to be executed from among a process of changing the bit rate of data transmitted by the data transmitting unit, a process of correcting the data, and a process of requesting a change in communication quality to the network, depending on the quality of the data. may be provided. The communication terminal may include a process execution unit that executes the process selected by the process selection unit.

The process selection unit selects the change process, the correction process, and the change request process when the quality of the data is lower than the first threshold, and selects the change process, the correction process, and the change request process when the quality of the data is higher than the first threshold If it is lower than a second threshold that is higher than one threshold, select two of the change processing, the correction processing, and the change request processing, and the quality of the data is higher than the second threshold and higher than the second threshold. If it is lower than the third threshold, one of the change process, the correction process, and the change request process may be selected. The processing selection unit selects the change processing and the correction processing when the quality of the data is higher than the first threshold and lower than the second threshold, and when the quality of the data is higher than the second threshold and the If it is lower than the third threshold, the above change process may be selected. The quality determination unit uses a mathematical quality estimation model for estimating the quality of the data received by the data reception unit from the data transmitted by the data transmission unit and the data received by the data reception unit. , the quality of the data received by the receiving device may be estimated. The quality determining unit is configured to determine the data transmitted by the data transmitting unit, the received data received by the data receiving unit, and the data received by the receiving device when the data transmitting unit transmits data via the network. Using the quality of the received data as training data, mathematically construct the quality estimation model that estimates the quality of data from the transmitted data transmitted by the data transmitting unit via the network and the received data received by the data receiving unit. You can then memorize it. The quality determining unit determines the quality of the data received by the receiving device using a trained quality estimation model that has undergone machine learning to estimate the quality of the data received by the data receiving unit. good.

According to one embodiment of the present invention, a program for causing a computer to function as the communication terminal is provided.

Note that the above summary of the invention does not list all the necessary features of the invention. Furthermore, subcombinations of these features may also constitute inventions.

1 schematically shows an example of a communication system 10. An example of the functional configuration of a sending terminal 200 and a receiving server 300 is schematically shown. An example of the flow of processing by the receiving server 300 is schematically shown. An example of the functional configuration of a sending terminal 200 and a receiving server 300 is schematically shown. An example of the flow of processing by the sending terminal 200 is schematically shown. FIG. 2 is an explanatory diagram for explaining a communication quality request to the mobile network 100. FIG. An example of the hardware configuration of a computer 1200 functioning as a sending terminal 200 or a receiving server 300 is schematically shown.

In a best effort network that shares resources, such as a mobile network, it is difficult to transmit high quality video with stable quality. For example, packet loss occurs on a transmission path due to the electric field strength of a communication terminal or congestion of a wireless network. Generally, such packet loss is detected on the receiving side and the sender is instructed to change the video quality and bit rate. However, it is difficult to detect when network congestion has converged, and it is difficult to stably transmit data at a desired quality. In the communication system 10 according to the present embodiment, for example, the receiving server or the transmitting terminal detects packet loss or deterioration of video quality, and based on this, the receiving server or the transmitting terminal requests priority control from the mobile network. "reducing the transmission bandwidth on the sending terminal side", and "video correction using a machine learning model that adapts according to congestion on the receiving server side (e.g. packet loss or video quality decoded on the receiving side)" To provide a technology that can contribute to realizing video transmission with stable quality by implementing at least one of the following.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all combinations of features described in the embodiments are essential to the solution of the invention.

FIG. 1 schematically shows an example of a communication system 10. As shown in FIG. The communication system 10 may include a transmitting terminal 200. The communication system 10 may include a reception server 300. Communication system 10 may include a mobile network 100.

Transmitting terminal 200 may be an example of a communication terminal. Transmission terminal 200 may be any communication terminal that has the function of transmitting data. The transmitting terminal 200 may be, for example, a mobile phone such as a smartphone, a tablet terminal, a PC (Personal Computer), or the like. The transmitting terminal 200 may be a so-called IoT (Internet of Things) terminal.

The data transmitted by the transmitting terminal 200 is, for example, video data. The video data may be moving image data. The video data may be, for example, moving image data captured by a camera and transmitted during an online conference. The video data may be broadcast data. These are just examples, and the video data may be any moving image data. The video data may or may not include audio. The data transmitted by the transmitting terminal 200 may be audio data.

The reception server 300 may be an example of a reception device. Reception server 300 may be an example of a communication device. Reception server 300 may be any communication device that has a function of receiving data from transmission terminal 200. The reception server 300 may be, for example, an online conference server, but is not limited thereto, and may be any server. Examples of communication devices include, in addition to the reception server 300, mobile phones such as smartphones, tablet terminals, PCs, and the like.

The sending terminal 200 and the receiving server 300 may communicate via the network 100. Network 100 may include a mobile network. The mobile network may be compliant with any of the following communication methods: LTE (Long Term Evolution) communication method, 5G (5th Generation) communication method, 3G (3rd Generation) communication method, and 6G (6th Generation) communication method. . Network 100 may include the Internet. The transmitting terminal 200 and the receiving server 300 may be able to access the network 100 via a wireless base station, a Wi-Fi (registered trademark) access point, or the like. The reception server 300 may be constructed in MEC (Mobile Edge Computing or Multi-access Edge Computing).

The communication system 10 determines the quality of data received by the receiving server 300 from the transmitting terminal 200, and executes processing according to the quality. This process may be executed mainly by the reception server 300. Further, the processing may be executed mainly by the sending terminal 200. Each case will be explained below.

FIG. 2 schematically shows an example of the functional configuration of the sending terminal 200 and the receiving server 300. Here, a case where the receiving server 300 is the main entity will be described.

In the example shown in FIG. 2, the transmission terminal 200 includes a data storage section 202, a data transmission section 204, a notification information reception section 206, a control section 208, and a data correction section 210. The reception server 300 includes a data reception section 302, a data output section 304, a quality determination section 306, a process selection section 308, and a process execution section 310.

The data storage unit 202 stores various data. For example, the data storage unit 202 stores video data. The data storage unit 202 stores, for example, video data captured by a camera connected to the sending terminal 200. The data storage unit 202 may store video data received from the outside or read from a portable storage medium or the like. The data storage unit 202 may store video data generated by the sending terminal 200. Further, for example, the data storage unit 202 stores audio data. The data storage unit 202 stores, for example, audio data of audio acquired by a microphone connected to the transmitting terminal 200. The data storage unit 202 may store audio data received from the outside or read from a portable storage medium or the like. The data storage unit 202 may store audio data generated at the transmitting terminal 200.

The data transmitting unit 204 transmits the data stored in the data storage unit 202 to the receiving server 300. The data transmitter 204 transmits data to the receiving server 300 via the network 100.

Data receiving section 302 receives data transmitted by data transmitting section 204. Data receiving section 302 receives data via network 100.

The data output section 304 outputs the data received by the data reception section 302. The data output unit 304 displays and outputs the video data on a display included in the reception server 300, for example. The data output unit 304 may transmit and output the video data to, for example, another communication terminal. For example, the data output unit 304 causes a speaker included in the receiving server 300 to output the audio data. The data output unit 304 may transmit and output audio data to, for example, another communication terminal.

The quality determining unit 306 determines the quality of the data received by the data receiving unit 302. The quality determining unit 306 determines the quality of data, for example, based on packet loss in communication with the transmitting terminal 200. The quality determining unit 306 may determine the quality of data based on the packet loss of data received from the transmitting terminal 200. The quality determination unit 306 may determine that the higher the packet loss rate, the lower the quality of the data. The quality determination unit 306 may determine that the lower the packet loss rate, the higher the quality of the data.

The quality determination unit 306 may determine the quality of data by inputting the packet loss rate into a mathematical quality estimation model for estimating the quality of data from the packet loss rate. For example, the quality determination unit 306 uses, as training data, the packet loss rate when data is transmitted via the network 100 and the quality of the received data derived from the data before transmission and the data after reception, and determines the packet loss rate. An estimation model for estimating data quality from the loss rate may be mathematically constructed and stored. The quality determination unit 306 may generate the estimated model by machine learning. The quality determination unit 306 may determine the quality of data by inputting the packet loss rate into a trained quality estimation model that has undergone machine learning to estimate the quality of data from the packet loss rate.

The process selection unit 308 selects a process to be executed depending on the quality of the data determined by the quality determination unit 306. The process selection unit 308 selects a process to be executed, for example, from a notification process to the sending terminal 200, a data correction process, and a communication quality change request process to the network 100. The process execution unit 310 executes the process selected by the process selection unit 308.

The notification process for the sending terminal 200 may be to send notification information to the sending terminal 200. The process execution unit 310 may transmit notification information to the sending terminal 200 when the process selecting unit 308 selects notification processing for the sending terminal 200.

The notification information may be information that triggers the sending terminal 200 to change the bit rate of data sent to the receiving server 300. The notification information is, for example, information that notifies that the quality of data is low. The notification information may be information that notifies that the packet loss rate is high. The notification information may be information that notifies that the network 100 is congested. The notification information may be information requesting a reduction in the data bit rate. The notification information may be information requesting that the transmitting terminal 200 reduce the amount of data transmitted.

The data correction process may be correcting data at the receiving server 300. The processing execution unit 310 may correct the data received by the data reception unit 302 when the processing selection unit 308 selects data correction processing. The processing execution unit 310 may perform correction to improve the quality of data received by the data reception unit 302. When the data is video data, examples of data correction methods include, but are not limited to, super resolution, image noise removal, rendering from feature points, and audio noise removal. When the data is audio data, noise removal, audio super resolution, etc. may be used, but are not limited to these. When data correction processing is selected by the processing selection unit 308, the processing execution unit 310 notifies the sending terminal 200 of the correction technique to be used, and then uses the correction technique to select the data to be received by the data reception unit 302. may be corrected. For example, the processing execution unit 310 specifies a specific format to the sending terminal 200. The sending terminal 200 sends data to the receiving server 300 in the specified format. The processing execution unit 310 executes a correction process suitable for the specified format on data of the specified format.

The data correction process may include requesting the sending terminal 200 to correct the data. The process execution unit 310 may transmit request information for requesting data correction to the sending terminal 200 when the process selection unit 308 selects data correction processing.

The request information may be information that triggers the sending terminal 200 to correct the data. The request information may include content explicitly requesting data correction. The request information does not need to include content requesting data correction. For example, the request information is information that notifies that the quality of data is low. The request information may be information that notifies that the packet loss rate is high. The request information may be information that notifies that the network 100 is congested.

The communication quality change request process for the network 100 may include transmitting a communication quality change request to the network 100. The process execution unit 310 may transmit a communication quality change request to the network 100 when the process selection unit 308 selects the change request process.

In response to receiving the change request, the network 100 executes a change process to change the communication quality of the reception server 300.

For example, the network 100 performs priority control on the receiving server 300. The network 100 may use any method as long as it can prioritize the communication of the receiving server 300. For example, when the network 100 complies with the 5G communication method, the network 100 changes the 5QI (5G QoS (Quality of Service) Indicator) for the receiving server 300 in order to give priority to the communication of the receiving server 300. Further, for example, when the network 100 complies with the LTE communication method, the network 100 changes the QCI (QoS Class Identifier) for the receiving server 300 in order to give priority to the communication of the receiving server 300.

For example, the network 100 performs bandwidth control on the receiving server 300. For example, the network 100 allocates a dedicated BWP (BandWidth Part) to the reception server 300.

For example, the network 100 performs user plane switching of the receiving server 300. For example, the network 100 executes switching to the user plane where the transmission distance is the shortest.

The information receiving unit 206 receives various information from the receiving server 300. The information receiving unit 206 receives notification information from the receiving server 300, for example. The information receiving unit 206 receives request information from the receiving server 300, for example.

In response to the information receiving unit 206 receiving the notification information from the receiving server 300, the control unit 208 controls the bit rate of the data transmitted by the data transmitting unit 204 to the receiving server 300, and transmits the data to the receiving server 300. Reduce the amount of data to be processed. For example, the control unit 208 reduces the data bit rate. When the data is video data, the control unit 208 lowers the resolution of the data, for example.

The data correction unit 210 corrects the data in response to the information receiving unit 206 receiving the request information, and generates correction data having a smaller amount of data than the data. In response to the information receiving unit 206 receiving the request information, the data transmitting unit 204 may transmit correction data generated by the data correcting unit 210 to the receiving server 300 instead of the data.

If the data is video data, for example, the data correction unit 210 generates correction data that includes only feature points extracted from the data. Further, the data correction unit 210 generates correction data using, for example, a video compression technique using known AI. Further, the data correction unit 210 generates correction data by, for example, compressing deep feature data acquired from the data. For example, the data correction unit 210 generates corrected video data in which a part of the video data is simplified. For example, when the video data includes a person, the data correction unit 210 generates corrected video data in which the person is made into an avatar.

If the data is audio data, for example, the data correction unit 210 generates correction data using a known audio compression technique. The data correction unit 210 may use a known audio compression technique using AI.

FIG. 3 schematically shows an example of the flow of processing by the reception server 300. Here, a state in which initial settings are performed when processing according to the quality of video data is executed will be described as a start state.

In step (step may be abbreviated as S) 102, the reception server 300 receives an input of a quality determination model for determining the quality of video data. In S104, the reception server 300 receives input of a determination threshold.

The quality determination model and the determination threshold may be input by the administrator, user, or the like of the reception server 300. Input to the receiving server 300 may be performed using an input device included in the receiving server 300, or may be performed by transmitting input information to the receiving server 300 from the outside. The quality determination unit 306 acquires the input quality determination model. The process selection unit 308 acquires the input determination threshold.

Here, a case is illustrated in which a threshold C, a threshold B higher than the threshold C, and a threshold A higher than the threshold B are input as the determination thresholds. The threshold value C may be an example of a first threshold value. Threshold B may be an example of a second threshold. Threshold A may be an example of a third threshold.

In S106, data receiving section 302 receives video data from transmitting terminal 200. In S108, the quality determining unit 306 determines the quality of the video data received by the data receiving unit 302 in S106, and outputs determination result X. The quality determination unit 306 may determine the quality of the video data using the quality determination model input in S102.

In S110, the process selection unit 308 determines whether X is greater than or equal to threshold A. If it is determined that the value is equal to or greater than the threshold value A, the process proceeds to S122, and if it is determined that the value is not equal to or greater than the threshold value A, the process proceeds to S112. In this example, when the process selection unit 308 determines that the value is equal to or greater than the threshold A, it does not select any of the notification process, correction process, and change request process.

In S112, the process selection unit 308 determines whether X is greater than or equal to threshold B. If it is determined that the value is equal to or greater than the threshold value B, the process proceeds to S114, and if it is determined that the value is not equal to or greater than the threshold value B, the process proceeds to S116. In S114, the process selection unit 308 selects a notification process, and the process execution unit 310 executes the notification process.

In S116, the process selection unit 308 determines whether or not X is equal to or greater than a threshold value C. If it is determined that the value is equal to or greater than the threshold value C, the process proceeds to S118, and if it is determined that the value is not equal to or greater than the threshold value C, the process proceeds to S120. In S118, the process selection unit 308 selects the notification process and the correction process, and the process execution unit 310 executes the notification process and the correction process. When executing the correction process, the process execution unit 310 may execute the correction process on the image portion of the video data. The processing execution unit 310 may further perform correction processing on the audio portion of the video data. In S120, the process selection unit 308 selects the notification process, the correction process, and the change request process, and the process execution unit 310 executes the notification process, the correction process, and the change request process.

In S122, it is determined whether or not the reception of the video data has ended. If it has not, the process returns to S106, and if it is determined that the reception has ended, the process ends.

As described above, the processing selection unit 308 selects notification processing, correction processing, and change request processing when the quality of the video data is lower than the first threshold, and selects the notification processing, correction processing, and change request processing when the quality of the video data is higher than the first threshold. If the quality of the video data is higher than the second threshold and lower than the third threshold, the notification process is selected. As a result, if the quality of the video data is high, no processing is performed, if the quality drops slightly, notification processing is performed, and if the quality drops further, correction processing is performed in addition to the notification processing. However, if the quality further deteriorates, notification processing, correction processing, and change request processing will be executed.

Overall, the correction process involves performing image correction on at least one of the sending terminal 200 and the receiving server 300, and may be more burdensome than the notification process. In addition, the change request process results in adjustment by the network 100, which may affect other communication terminals and communication devices. Therefore, according to the process selection unit 308 according to the present embodiment, it is possible to gradually increase and execute processes that have less influence in response to a decrease in the quality of video data.

Note that the process selected in S114 is not limited to the notification process. Further, the process selected in S118 is not limited to the notification process and the correction process.

The processing selection unit 308 selects notification processing, correction processing, and change request processing when the quality of the video data is lower than the first threshold, and selects the notification processing, correction processing, and change request processing when the quality of the video data is higher than the first threshold and lower than the second threshold. , notification processing, correction processing, and change request processing, and if the quality of the video data is higher than the second threshold and lower than the third threshold, one of the notification processing, correction processing, and change request processing is selected. You may choose one.

In FIG. 3, the case where three determination threshold values are input is described as an example, but the present invention is not limited to this. The number of determination threshold values may be one, two, or four or more. When one determination threshold is input, the process selection unit 308 does not select any process if X is greater than or equal to the threshold, and selects notification processing, correction processing, and change request processing if X is not greater than or equal to the threshold. It's fine. When two determination thresholds are input, the process selection unit 308 selects notification processing, correction processing, and change request processing when X is lower than the first threshold, and selects notification processing, correction processing, and change request processing when X is lower than the second threshold, which is higher than the first threshold. If X is lower than the second threshold, one or two of the notification processing, correction processing, and change request processing is selected, and if X is higher than the second threshold, no processing is selected.

In FIG. 3, the case where the communication system 10 targets video data has been described as an example, but the case where the communication system 10 targets audio data can also be realized by the same flow.

FIG. 4 schematically shows an example of the functional configuration of the sending terminal 200 and the receiving server 300. Here, we will mainly explain the differences from FIG. 2 in the case where the transmitting terminal 200 is the main subject.

In the example shown in FIG. 4, the transmitting terminal 200 includes a data storage section 202, a data transmitting section 204, a data receiving section 212, a quality determining section 214, a processing selecting section 216, and a processing executing section 218. The reception server 300 includes a data reception section 302, a data output section 304, and a processing execution section 312.

The data storage unit 202 stores various data. The data storage unit 202 stores, for example, video data. The data storage unit 202 stores, for example, audio data. The data transmitting unit 204 transmits the data stored in the data storage unit 202 to the receiving server 300.

Data receiving section 302 receives data transmitted by data transmitting section 204. The data output section 304 outputs the data received by the data reception section 302. The data output unit 304 shown in FIG. 4 transmits the data received by the data reception unit 302 to the transmission terminal 200.

The data receiving section 212 receives the data transmitted by the data output section 304. The data receiving unit 212 receives data via the network 100.

The quality determining unit 214 determines the quality of the data received by the data receiving unit 212. The quality determining unit 214 may determine the quality of the data received by the receiving server 300 based on the data transmitted by the data transmitting unit 204 and the data received by the data receiving unit 212.

The quality determination unit 214 uses a mathematical quality estimation model to estimate the quality of the data received by the data reception unit 212 from the data transmitted by the data transmission unit 204 and the data received by the data reception unit 212. , the quality of the data may be estimated. The quality determination unit 214 uses, for example, transmitted data and received data when transmitting data via the network 100, and the quality of the received data as training data, and estimates the quality of data from the transmitted data and received data. A model may be constructed mathematically and memorized. The quality determination unit 214 may generate the estimated model by machine learning. The quality of data may be determined using a learned quality estimation model that has been subjected to machine learning to estimate the quality of data received by the data receiving unit 212.

Furthermore, the quality determining unit 214 calculates a peak signal-to-noise ratio (PSNR) using, for example, the data transmitted by the data transmitting unit 204 and the data received by the data receiving unit 212. do. The quality determining unit 214 may determine that the higher the peak signal-to-noise ratio, the lower the quality of the data.

Furthermore, when the image is video data, the quality determining unit 214 uses, for example, the data transmitted by the data transmitting unit 204 and the data received by the data receiving unit 212 to determine the structural similarity (SSIM). Calculate. The quality determination unit 214 may determine that the lower the structural similarity, the lower the quality of the data.

These are examples, and the quality determination unit 214 uses other known methods to determine the quality of the data received by the data reception unit 212 using the data transmitted by the data transmission unit 204 and the data received by the data reception unit 212. May determine the quality of the data.

The process selection unit 216 selects a process to be executed depending on the quality of the data determined by the quality determination unit 214. The process selection unit 216 selects a process to be executed from, for example, a process of changing the bit rate of data transmitted by the data transmitting unit 204, a process of correcting data, and a process of requesting a change in communication quality to the network 100. The process execution unit 218 executes the process selected by the process selection unit 216.

The process of changing the bit rate of data transmitted by the data transmitter 204 may be a process of reducing the bit rate of data. The process execution unit 218 may reduce the bit rate of data transmitted by the data transmission unit 204 when the process selection unit 216 selects the change process.

The video data correction process may be performed by correcting the data at the transmitting terminal 200. For example, when the process selection unit 216 selects data correction processing, the processing execution unit 218 corrects the data and generates correction data having a smaller amount of data than the data. For example, if the data is video data, the processing execution unit 218 generates correction data using a known video compression technique using AI. The processing execution unit 218 generates correction data, for example, by compressing deep feature data acquired from the data. Further, the processing execution unit 218 generates correction data in which a part of the data is displayed in a simplified manner, for example. For example, when the data includes a person, the processing execution unit 218 generates correction data that represents the person as an avatar. For example, if the data is audio data, the processing execution unit 218 generates correction data using a known audio compression technique. The data transmitting unit 204 may transmit correction data generated by the processing executing unit 218 to the receiving server 300 instead of the data.

The data correction process may include requesting the reception server 300 to correct the data. The process execution unit 218 may transmit request information for requesting data correction to the reception server 300 when the process selection unit 216 selects data correction processing.

The request information may be information that triggers the receiving server 300 to correct data. The request information may include content explicitly requesting data correction. The request information may include content that notifies the correction technique. The request information does not need to include content requesting data correction. For example, the request information is information that notifies that the quality of data is low.

When the data receiving unit 302 receives the request information, the processing execution unit 312 may correct the data received by the data receiving unit 302. The processing execution unit 312 may perform correction to improve the quality of data received by the data reception unit 302. When the data is video data, examples of data correction methods include, but are not limited to, super resolution, image noise removal, rendering from feature points, and audio noise removal. When the data is audio data, noise removal, audio super resolution, etc. may be used, but are not limited to these.

The process of requesting a change in communication quality to the network 100 may be to have the reception server 300 execute a request to change the communication quality to the network 100 . If the change request process is selected by the process selection unit 216, the process execution unit 218 may send an instruction to the receiving server 300 to send a communication quality change request to the network 100. When the data receiving unit 302 receives the instruction, the processing execution unit 312 may transmit a communication quality change request to the network 100.

The communication quality change request process for the network 100 may include transmitting a communication quality change request to the network 100. The process execution unit 218 may transmit a communication quality change request to the network 100 when the process selection unit 216 selects the change request process.

FIG. 5 schematically shows an example of the flow of processing by the sending terminal 200. Here, a state in which initial settings are performed when processing according to the quality of video data is executed will be described as a start state.

In S202, the transmitting terminal 200 receives an input of a quality determination model for determining the quality of video data. In S204, transmitting terminal 200 receives input of a determination threshold.

The quality determination model and the determination threshold may be input by a user of the transmitting terminal 200 or the like. Input to the sending terminal 200 may be performed using an input device included in the sending terminal 200, or may be performed by transmitting input information to the sending terminal 200 from the outside. The quality determination unit 214 acquires the input quality determination model. The processing execution unit 218 acquires the input determination threshold.

Note that, similarly to FIG. 3, here, a case is illustrated in which a threshold C, a threshold B higher than the threshold C, and a threshold A higher than the threshold B are input as the determination thresholds. The threshold value C may be an example of a first threshold value. Threshold B may be an example of a second threshold. Threshold A may be an example of a third threshold.

In S206, the data transmitting unit 204 transmits the video data to the receiving server 300. Data transmitter 204 holds the transmitted video data. In S208, the data output unit 304 transmits the video data received by the data reception unit 302 to the transmission terminal 200, and the data reception unit 212 receives the video data.

In S210, the quality determining unit 214 determines the quality of the video data received by the receiving server 300 based on the video data transmitted by the data transmitting unit 204 in S202 and the video data received by the data receiving unit 212 in S208. Then, the determination result X is output. The quality determination unit 214 may determine the quality of the video data using the quality determination model input in S202.

In S212, the process selection unit 216 determines whether X is greater than or equal to threshold A. If it is determined that the value is equal to or greater than the threshold value A, the process proceeds to S224, and if it is determined that the value is not equal to or greater than the threshold value A, the process proceeds to S214. In this example, if the process selection unit 308 determines that the value is equal to or greater than the threshold value A, it does not select any of the bit rate change process, video data correction process, and change request process.

In S214, the process selection unit 216 determines whether X is greater than or equal to threshold B. If it is determined that the value is equal to or greater than the threshold value B, the process proceeds to S216, and if it is determined that the value is not equal to or greater than the threshold value B, the process proceeds to S218. In S<b>216 , the processing selection unit 216 selects bit rate changing processing, and the processing execution unit 218 reduces the bit rate of the video data that the data transmission unit 204 transmits to the receiving server 300 .

In S218, the process selection unit 216 determines whether or not X is equal to or greater than a threshold value C. If it is determined that the value is equal to or greater than the threshold value C, the process proceeds to S220, and if it is determined that the value is not equal to or greater than the threshold value C, the process proceeds to S222. In S210, the process selection unit 216 selects the bit rate change process and the video data correction process, and the process execution unit 218 executes the bit rate change process and the video data correction process. When executing the correction process, the process execution unit 218 may execute the correction process on the image portion of the video data. The processing execution unit 218 may further perform correction processing on the audio portion of the video data. In S222, the process selection unit 216 selects the bit rate change process, video data correction process, and change request process, and the process execution unit 218 selects the bit rate change process, video data correction process, and change request process. Execute processing.

In S224, it is determined whether or not the transmission of the video data has been completed. If not, the process returns to S206, and if it is determined that the transmission has been completed, the process ends.

As described above, the process selection unit 216 selects the change process, correction process, and change request process when the quality of the video data is lower than the first threshold, and selects the change process, the correction process, and the change request process when the quality of the video data is higher than the first threshold. If the quality of the video data is higher than the second threshold and lower than the third threshold, the modification process and correction process are selected. As a result, if the quality of the video data is high, no processing is performed, if the quality drops a little, the modification processing is performed, and if the quality drops further, the correction processing is performed in addition to the modification processing. However, if the quality deteriorates further, change processing, correction processing, and change request processing will be executed.

Overall, the correction process involves performing video correction on at least one of the sending terminal 200 and the receiving server 300, and may be more burdensome than the changing process. In addition, the change request process results in adjustment by the network 100, which may affect other communication terminals and communication devices. Therefore, according to the process selection unit 216 according to the present embodiment, it is possible to gradually increase and execute processes that have less influence depending on the deterioration of the quality of video data.

Note that the process selected in S216 is not limited to the change process. Further, the process selected in S220 is not limited to the change process and the correction process.

The process selection unit 216 selects the change process, correction process, and change request process when the quality of the video data is lower than the first threshold, and selects the change process, the correction process, and the change request process when the quality of the video data is higher than the first threshold and lower than the second threshold. , change processing, correction processing, and change request processing, and if the quality of the video data is higher than the second threshold and lower than the third threshold, one of the change processing, correction processing, and change request processing is selected. You may choose one.

In FIG. 5, the case where the communication system 10 targets video data has been described as an example, but the case where the communication system 10 targets audio data can also be realized by the same flow.

FIG. 6 is an explanatory diagram for explaining communication quality requests to the network 100. Here, a case will be described in which the network 100 includes a 5GC (5G Core) 110.

The transmitting terminal 200 or the receiving server 300 can control the NF (Network Functions) of the 5GC 110 by transmitting a communication quality change request to the NEF (Network Exposure Function) 114.

When the NEF 114 receives a communication quality change request from the sending terminal 200 or the receiving server 300, the 5GC 110 executes switching of the UPF 112 of the receiving server 300, for example. For example, the 5GC 110 executes switching to the UPF 112 that provides the shortest transmission distance. Switching of the UPF 112 is performed by, for example, an SMF (Session Management Function) of the 5GC 110.

The 5GC 110 may perform priority control on the reception server 300 when receiving a communication quality change request. The 5GC 110 may use any method as long as it can give priority to the communication of the receiving server 300. For example, the 5GC 110 changes the QCI for the receiving server 300 in order to give priority to the communication of the receiving server 300. Such priority control may be performed by the SMF of the 5GC 110, the AMF (Access and Mobility management Function), and the gNB (gNodeB) 120.

The 5GC 110 may perform bandwidth control on the reception server 300 when receiving a communication quality change request. For example, the 5GC 110 allocates a dedicated BWP to the receiving server 300. Such bandwidth control may be performed by the SMF, AMF, and gNB 120 of the 5GC 110.

When the 5GC 110 receives a communication quality change request, the 5GC 110 may give priority to the communication of the receiving server 300 by network slicing. The 5GC 110 may prioritize communication with the receiving server 300 using eMBB (enhanced Mobile Broadband), URLLC (Ultra-Reliable and Low Latency Communications), or the like.

In FIG. 6, a case has been described in which the network 100 includes the 5GC 110 of SA (Stand Alone) system, but the present invention is not limited to this. The mobile network of network 100 may be 4G or 5G NSA (Non-Stand Alone) using EPC (Evolved Packet Core). In the case of 4G, gNB 120 is eNB, in case of EPC, NEF 114 is SCEF (Service Capability Exposure Function), and UPF 112 is SGW (Serving Gateway)/PGW (Packet data network). This may be the U-Plane functional unit of the Gateway.

FIG. 7 schematically shows an example of the hardware configuration of the computer 1200 functioning as the sending terminal 200 or the receiving server 300. The program installed on the computer 1200 causes the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or causes the computer 1200 to perform operations associated with the apparatus according to the present embodiment or the one or more "parts" of the apparatus according to the present embodiment. Multiple units may be executed and/or the computer 1200 may execute a process or a stage of a process according to the present embodiments. Such programs may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. Computer 1200 also includes input/output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to host controller 1210 via input/output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. Storage device 1224 may be a hard disk drive, solid state drive, or the like. Computer 1200 also includes legacy input/output units, such as ROM 1230 and a keyboard, which are connected to input/output controller 1220 via input/output chips 1240.

The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. Graphics controller 1216 obtains image data generated by CPU 1212, such as in a frame buffer provided in RAM 1214 or itself, and causes the image data to be displayed on display device 1218.

Communication interface 1222 communicates with other electronic devices via a network. Storage device 1224 stores programs and data used by CPU 1212 within computer 1200. The DVD drive reads a program or data from a DVD-ROM or the like and provides it to the storage device 1224. The IC card drive reads programs and data from and/or writes programs and data to the IC card.

ROM 1230 stores therein programs that are dependent on the computer 1200 hardware, such as a boot program that is executed by the computer 1200 upon activation. I/O chip 1240 may also connect various I/O units to I/O controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

The program is provided by a computer readable storage medium such as a DVD-ROM or an IC card. The program is read from a computer-readable storage medium, installed in storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by CPU 1212. The information processing described in these programs is read by the computer 1200 and provides coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured to implement the operation or processing of information according to the use of computer 1200.

For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded into the RAM 1214 and sends communication processing to the communication interface 1222 based on the processing written in the communication program. You can give orders. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as a RAM 1214, a storage device 1224, a DVD-ROM, or an IC card under the control of the CPU 1212, and transmits the read transmission data. Data is transmitted to the network, or received data received from the network is written to a reception buffer area provided on the recording medium.

Further, the CPU 1212 causes the RAM 1214 to read all or a necessary part of a file or database stored in an external recording medium such as a storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. Various types of processing may be performed on the data. CPU 1212 may then write the processed data back to an external storage medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on a recording medium and subjected to information processing. CPU 1212 performs various types of operations, information processing, conditional determination, conditional branching, unconditional branching, and information retrieval on data read from RAM 1214 as described elsewhere in this disclosure and specified by the program's instruction sequence. Various types of processing may be performed, including /substitutions, etc., and the results are written back to RAM 1214. Further, the CPU 1212 may search for information in a file in a recording medium, a database, or the like. For example, when a plurality of entries are stored in a recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 selects the first entry from among the plurality of entries. Search for an entry whose attribute value matches the specified condition, read the attribute value of the second attribute stored in the entry, and then set the attribute value to the first attribute that satisfies the predetermined condition. An attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored in a computer-readable storage medium on or near computer 1200. Also, a storage medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby allowing the program to be transferred to the computer 1200 via the network. provide.

Blocks in the flowcharts and block diagrams of the present embodiments may represent stages in a process in which an operation is performed or a "part" of a device responsible for performing the operation. Certain steps and units may be provided with dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and/or provided with computer readable instructions stored on a computer readable storage medium. May be implemented by a processor. Dedicated circuitry may include digital and/or analog hardware circuits, and may include integrated circuits (ICs) and/or discrete circuits. Programmable circuits can perform AND, OR, EXCLUSIVE OR, NAND, NOR, and other logical operations, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. , flip-flops, registers, and memory elements.

A computer-readable storage medium may include any tangible device capable of storing instructions for execution by a suitable device such that a computer-readable storage medium with instructions stored therein may be illustrated in a flowchart or block diagram. A product will be provided that includes instructions that can be executed to create a means for performing specified operations. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable storage media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory). , Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Blu-ray Disc, Memory Stick , integrated circuit cards, and the like.

Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state configuration data, or instructions such as Smalltalk®, JAVA®, C++, etc. any source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as may include.

The computer-readable instructions are for producing means for a processor of a general purpose computer, special purpose computer, or other programmable data processing device, or programmable circuit to perform the operations specified in the flowchart or block diagrams. A general purpose computer, special purpose computer, or other programmable data processor, locally or over a local area network (LAN), wide area network (WAN), such as the Internet, to execute the computer readable instructions. It may be provided in a processor or programmable circuit of the device. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the range described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the embodiments described above. It is clear from the claims that such modifications or improvements may be included within the technical scope of the present invention.

The execution order of each process, such as an operation, a procedure, a step, and a stage in the apparatus, system, program, and method shown in the claims, specification, and drawings, specifically refers to "before" or "before". It should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Even if the claims, specifications, and operational flows in the drawings are explained using "first," "next," etc. for convenience, this does not mean that it is essential to carry out the operations in this order. It's not a thing.

10 communication system, 100 network, 110 5GC, 112 UPF, 114 NEF, 120 gNB, 200 transmission terminal, 202 data storage section, 204 data transmission section, 206 information reception section, 208 control section, 210 data correction section, 212 data reception unit, 214 quality determination unit, 216 process selection unit, 218 process execution unit, 300 reception server, 302 data reception unit, 304 data output unit, 306 quality determination unit, 308 process selection unit, 310 process execution unit, 312 process execution unit , 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input/output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input/output chip

Claims (7)

a data storage section for storing data;
a data transmitter that transmits the data to a receiving device via a network;
a data receiving unit that receives the data received by the receiving device from the receiving device via the network;
a quality determining unit that determines the quality of the data received by the receiving device based on the data transmitted by the data transmitting unit and the data received by the data receiving unit;
Process selection for selecting a process to be executed from among a process for changing a bit rate of the data transmitted by the data transmission unit, a process for correcting the data, and a process for requesting a change in communication quality to the network, depending on the quality of the data. Department and
A communication terminal comprising: a processing execution section that executes the processing selected by the processing selection section. The process selection unit selects the change process, the correction process, and the change request process when the quality of the data is lower than the first threshold, and selects the change process, the correction process, and the change request process when the quality of the data is higher than the first threshold If it is lower than a second threshold that is higher than one threshold, two of the change processing, the correction processing, and the change request processing are selected, and the quality of the data is higher than the second threshold and higher than the second threshold. The communication terminal according to claim 1, which selects one of the change process, the correction process, and the change request process when the value is lower than a third threshold. The processing selection unit selects the change processing and the correction processing when the quality of the data is higher than the first threshold and lower than the second threshold, and when the quality of the data is higher than the second threshold and the The communication terminal according to claim 2, which selects the change process when the communication terminal is lower than a third threshold. The quality determination unit uses a mathematical quality estimation model for estimating the quality of the data received by the data reception unit from the data transmitted by the data transmission unit and the data received by the data reception unit. The communication terminal according to any one of claims 1 to 3, wherein the communication terminal estimates the quality of the data received by the receiving device. The quality determining unit is configured to determine whether the data transmitted by the data transmitting unit, the received data received by the data receiving unit, and the data received by the receiving device when the data transmitting unit transmits data via the network. Using the quality of received data as training data, mathematically construct the quality estimation model that estimates the quality of data from the transmitted data transmitted by the data transmitter via the network and the received data received by the data receiver. The communication terminal according to claim 4, wherein the communication terminal stores: The quality determining unit determines the quality of the data received by the receiving device using a learned quality estimation model that has been subjected to machine learning to estimate the quality of the data received by the data receiving unit. The communication terminal according to any one of claims 1 to 3. A program for causing a computer to function as the communication terminal according to any one of claims 1 to 6 .

Images