JP2022127144A - Data communication device, client device, retransmission instruction method, and retransmission instruction program - Google Patents

Abstract

To provide a technique that guarantees the accuracy of an inference task performed by an inference model while reducing network traffic.SOLUTION: A data communication device according to one aspect of the present invention receives compressed target data from a client device, obtains a result of performing an inference task on the received target data by using an inference model, calculates likelihood of the performance result of the inference task, and when the calculated likelihood is less than a threshold value, transmits, to the client device, an instruction to lower a compression rate and retransmit the target data.SELECTED DRAWING: Figure 1

Description

The present invention relates to a data communication device, a client device, a retransmission instruction method, and a retransmission instruction program.

FIG. 1 schematically shows an example of the configuration of edge computing, which has been actively studied in recent years. As shown in FIG. 1, in edge computing technology, a group of computer servers are arranged near users who use client devices such as user terminals (for example, smartphones) and IoT (Internet of Things) terminals. Compared to cloud network technology, which aggregates a group of computer servers in a data center and provides digital content, edge computing technology provides services with low latency because the user's client device and computer server are close to each other. have the advantage of being able to Edge computing technology is particularly compatible with IoT applications such as autonomous driving, video distribution services, and intelligent video surveillance.

A small-scale communication network tends to be used between the user's client device and the edge server, and reduction of network traffic (that is, improvement of communication band efficiency) is an issue. As one method for reducing network traffic, Non-Patent Documents 1 to 3 propose a method of compressing image data before transmitting the image data to a server device such as a cloud server or an edge server. . In addition, Non-Patent Document 4 proposes a method of reducing network traffic generated during a video call using a deep convolutional adversarial generation network (DCGAN).

J. Chen and X. Ran, "Deep Learning With Edge Computing: A Review", Proc. the IEEE, vol. 107, no. 8, pp. 1655-1674, 2019. J. Ren, et al., "Distributed and Efficient Object Detection in Edge Computing: Challenges and Solutions", IEEE Network, vol. 32, no. 6, pp. 137-143, 2018. H. Li, et al., "AdaCompress: Adaptive Compression for Online Computer Vision Services", Proc. the ACM International Conf. Multimedia, 2019, pp. 2440-2448. S. Watanabe, et al., "Traffic Reduction in Video Call and Chat using DNN-Based Image Reconstruction", Proc. IEEE International Conference on Communications (ICC). 2019.

The inventors of the present invention have found that the above conventional method has the following problems. That is, according to the conventional method, by compressing the image data, it is possible to reduce network traffic generated when the image data is transmitted. However, on the server device side, for example, using an inference model configured by a trained machine learning model etc., it is assumed that an inference task (for example, image recognition) is performed on the obtained image data. . In this case, compressing the image data with a high compression ratio can adversely affect the accuracy of the inference task performed by the inference model. In the conventional method, such a possibility was not assumed, and it was difficult to guarantee the accuracy of the inference task performed on the server device side.

It should be noted that this problem is not peculiar to situations where image data is exchanged. For example, the same problem may occur in situations where data other than image data such as sound data and sensing data are exchanged. Nor is this problem unique to the use of edge computing. A similar problem can arise in any scene where data is exchanged between a data communication device such as a server device and a client device (for example, a scene where cloud network technology is employed).

SUMMARY OF THE INVENTION In one aspect, the present invention has been made in view of such circumstances, and its object is to provide a technique that guarantees the accuracy of inference tasks performed by inference models while reducing network traffic. That is.

The present invention adopts the following configurations in order to solve the above-described problems.

That is, a data communication device according to one aspect of the present invention includes a data receiving unit configured to receive compressed target data from a client device; a result acquisition unit configured to acquire a result of performing an inference task; a result evaluation unit configured to calculate the likelihood of a result of performing the inference task by the inference model; and the calculated likelihood. is less than a threshold, the retransmission instruction unit is configured to transmit to the client device an instruction to retransmit the target data after lowering the compression rate.

In this configuration, network traffic can be reduced by exchanging compressed target data between the client device and the data communication device. Traffic can be reduced as the compression rate is increased. On the other hand, if the compression rate is too high, the inference accuracy of the inference model for the target data may deteriorate. Therefore, in this configuration, the performance result of the inference task by the inference model for the received target data is acquired, and the likelihood of the acquired performance result is calculated. The plausibility indicates whether or not the performance result of the inference task by the inference model is certain (that is, whether or not the inference result of the inference model is reliable). If this likelihood is less than a threshold, the target data with a lower compression rate is retransmitted. This enables the inference model to perform the inference task for target data with a low compression ratio, and improves the likelihood of the inference result. Therefore, according to this configuration, it is possible to guarantee the accuracy of the inference task performed by the inference model while reducing network traffic.

It should be noted that, in one example, the data communication device may operate as a server device (eg, an application server, etc.) that maintains the inference model and is configured to perform inference tasks. In this case, the performance of the inference task by the inference model may be performed in the data communication device. That is, obtaining results of performing an inference task may consist of performing an inference task using an inference model. In another example, the data communication device may operate as a transfer device (eg, relay device, etc.) configured to hold the inference model and transfer the data to another computer to perform the inference task. In this case, the performance of the inference task by the inference model may be performed on another computer. That is, the data communication device may obtain the results of performing the inference task from another computer.

In the data communication device according to the aspect described above, the retransmission instruction unit may be configured to further instruct the extent to which the compression rate is lowered according to the calculated likelihood, together with the retransmission instruction. According to this configuration, it is possible to expect retransmission at an appropriate compression rate, and it is possible to reduce the number of times the target data is retransmitted between the client device and the data communication device. Therefore, according to the configuration, it is possible to suppress an increase in network traffic due to retransmission of the target data.

The data communication device according to the above aspect, in order to prepare for the retransmission instruction when the calculated likelihood is less than a threshold, reduces the compression rate before transmitting the retransmission instruction. The apparatus may further include a preparation instruction unit configured to transmit an instruction to generate data (hereinafter also referred to as a "preparation instruction") to the client device. The processing load for compressing data is relatively large, and the processing takes time. The client device may execute processing for compressing the target data after receiving the resend instruction from the data communication device. be. According to this configuration, the preparation instruction causes the client device to perform data compression processing in advance before the retransmission instruction is given, thereby reducing the time required to retransmit the target data (the compression processing is performed before the retransmission instruction is received). is completed). As a result, it is possible to shorten the time required for the data communication device to re-acquire the compressed target data and to perform the inference task for the target data by the inference model again.

In the data communication device according to one aspect, the inference model may be a trained machine learning model generated by machine learning. According to this configuration, when a trained model generated by machine learning is used to perform an inference task on target data, network traffic is reduced while guaranteeing the accuracy of the inference task performed by the inference model. can do.

In the data communication device according to the aspect described above, the likelihood may be calculated based on uncertainty of a performance result of the inference task. According to this configuration, it is possible to properly evaluate the likelihood of the performance result of the inference task by the trained inference model generated by machine learning. This allows adequate assurance of the accuracy of the inference tasks performed by the inference model.

In the data communication device according to the above aspect, the inference model may be trained by the machine learning to output the performance result of the inference task and the likelihood of the performance result, and calculate the likelihood. Doing may comprise obtaining an output corresponding to the likelihood as a result of performing a computation of the inference model. According to this configuration, it is possible to properly evaluate the likelihood of the performance result of the inference task by the trained inference model generated by machine learning. This allows adequate assurance of the accuracy of the inference tasks performed by the inference model.

In the data communication device according to the above aspect, the target data may be image data, and the inference task may be configured by recognizing an object appearing in the image data. According to this configuration, when image data is exchanged between the client device and the data communication device and image recognition is performed as an inference task, the accuracy of the inference task performed by the inference model can be improved while reducing network traffic. can be guaranteed.

In the data communication device according to the above aspect, the image data may be obtained by a surveillance camera, and recognizing an object appearing in the image data may be configured by identifying a scene in the image data. . According to this configuration, it is possible to guarantee the accuracy of the inference task performed by the inference model while reducing the traffic on the network when identifying the scene shown in the image data obtained by the surveillance camera.

In the data communication device according to the above aspect, the target data may be composed of sound data, and the inference task may be composed of inferring features appearing in the sound data. According to this configuration, when performing an inference task of exchanging sound data between the client device and the data communication device and inferring features appearing in the sound, the inference model is used to reduce network traffic. The accuracy of the inference task performed can be guaranteed.

In the data communication device according to the above one aspect, the sound data may be obtained by recording the voice of a speaker, and inferring features appearing in the sound data may be performed by recognizing the voice of the speaker. It may be configured by According to this configuration, it is possible to guarantee the accuracy of the inference task performed by the inference model while reducing network traffic when speech recognition is performed.

In the data communication device according to the above aspect, the target data may be sensing data, and the inference task may be inferring features appearing in the sensing data. According to this configuration, when performing an inference task of exchanging sensing data between the client device and the data communication device and inferring features appearing in the sensing data, the inference model can be used while reducing network traffic. Accuracy of the inference tasks performed can be guaranteed.

Further, the form of the present invention need not be limited to the data communication device described above. An aspect of the present invention may be a client device configured to transmit compressed target data to the data communication device according to any one of the above aspects.

For example, a client device according to an aspect of the present invention includes a data acquisition unit configured to acquire target data; a data compression unit configured to compress the acquired target data; a data transmission unit configured to transmit the compressed target data to the data communication device according to the aspect of The data compression unit compresses the target data again at a compression rate lower than the previous compression rate in response to the instruction to resend the target data after lowering the compression rate in the data communication device. configured to The data transmission unit is configured to transmit the recompressed target data to the data communication device in response to receiving an instruction to resend the target data. According to this configuration, the accuracy of the inference task performed by the inference model can be guaranteed while reducing network traffic with the data communication device.

Further, in the client device according to the above aspect, before receiving an instruction to resend the target data from the data communication device, the data compression unit performs compression at a lower compression rate than a previous compression rate in preparation for the instruction. It may be configured to start executing a process of compressing the target data again at a rate of According to this configuration, by executing the data compression processing in advance before receiving the retransmission instruction, the time required to retransmit the target data is reduced (if the compression processing is completed by the retransmission instruction, delay). As a result, it is possible to shorten the time required for the data communication device to re-acquire the compressed target data and to perform the inference task for the target data by the inference model again. Note that the process of compressing data in advance before receiving a retransmission instruction in the client device may be executed regardless of the presence or absence of the preparation instruction from the data communication device.

Further, as another aspect of each of the data communication device and the client device according to each of the above aspects, one aspect of the present invention may be an information processing method that realizes all or part of each of the above configurations, It may be a program, or a storage medium that stores such a program and is readable by a computer, other device, machine, or the like. Here, a computer-readable storage medium is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action. Further, one aspect of the present invention may be an estimation system configured by the data communication device and the client device according to any one of the above aspects.

For example, a retransmission instruction method according to one aspect of the present invention includes the steps of a computer receiving compressed target data from a client device, and performing an inference task on the received target data using an inference model. calculating the likelihood of the performance result of the inference task by the inference model; and, if the calculated likelihood is less than a threshold, reduce the compression ratio and then compress the target data. and transmitting to the client device an instruction to resend the .

Also, for example, a retransmission instruction program according to one aspect of the present invention may provide a computer with a step of receiving compressed target data from a client device, and an inference task for the received target data by using an inference model. calculating the likelihood of the performance result of the inference task by the inference model; and if the calculated likelihood is less than a threshold, lowering the compression rate and then and sending an instruction to the client device to resend target data.

According to the present invention, it is possible to guarantee the accuracy of an inference task performed on the server device side while reducing network traffic.

FIG. 1 schematically shows an example of the configuration of edge computing. FIG. 2 schematically shows an example of a scene to which the present invention is applied. FIG. 3 schematically shows an example of a hardware configuration of a server device (data communication device) according to the embodiment. FIG. 4 schematically shows an example of the hardware configuration of the client device according to the embodiment. FIG. 5 schematically shows an example of the software configuration of the server device (data communication device) according to the embodiment. FIG. 6 schematically shows an example of the software configuration of the client device according to the embodiment. 7 is a flowchart illustrating an example of a processing procedure of the server device (data communication device) according to the embodiment; FIG. 8 is a flowchart illustrating an example of a processing procedure of the client device according to the embodiment; FIG. FIG. 9 schematically shows an example of another scene to which the present invention is applied. FIG. 10 schematically shows an example of another scene to which the present invention is applied. FIG. 11 schematically shows an example of another scene to which the present invention is applied. FIG. 12 schematically shows an example of a software configuration of a server device (data communication device) according to a modification. FIG. 13 schematically shows an example of the configuration of a data communication device according to a modification. FIG. 14 schematically shows an example of the configuration of an inference model according to the modification. FIG. 15 shows the configuration of the numerical simulation according to the first experimental example. FIG. 16 shows simulation results of PSNR average values and variance values according to the downsampling rate according to the first experimental example. FIG. 17 shows simulation results of the recognition accuracy of Top-1 and Top-5 in the first experimental example. FIG. 18 shows simulation results of recognition error rates for Top-1 and Top-5 in the first experimental example. FIG. 19A shows simulation results of the recognition error rate of the example (Top-1) and the comparative example in the second experimental example. FIG. 19B shows simulation results of the recognition error rate of the example (Top-5) and the comparative example in the second experimental example. FIG. 20A shows simulation results of the number of image retransmissions in Example (Top-1) of the second experimental example. FIG. 20B shows simulation results of the number of image retransmissions in Example (Top-5) of the second experimental example. FIG. 21 shows simulation results of the dummy traffic throughput of the example (Top-1, Top-5) and the comparative example in the second experimental example. FIG. 22 shows simulation results (cumulative frequency distribution) of dummy traffic delays in the second experimental example (Top-1, Top-5) and the comparative example. FIG. 23 shows simulation results (cumulative frequency distribution) of image traffic delays in the second experimental example (Top-1, Top-5) and the comparative example.

Hereinafter, an embodiment (hereinafter also referred to as "this embodiment") according to one aspect of the present invention will be described based on the drawings. However, this embodiment described below is merely an example of the present invention in every respect. It goes without saying that various modifications and variations can be made without departing from the scope of the invention. That is, in implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Although the data appearing in this embodiment are explained in terms of natural language, more specifically, they are specified in computer-recognizable pseudo-language, commands, parameters, machine language, and the like.

§1 Application Example FIG. 2 schematically shows an example of a scene to which the present invention is applied. As shown in FIG. 2, the communication system according to this embodiment includes a server device 1 and a client device 2 that are connected via a network 4 .

The server device 1 according to this embodiment is a computer configured to receive data from the client device 2 and perform an inference task on the received data. The server device 1 is an example of a data communication device. On the other hand, the client device 2 according to this embodiment is a computer configured to acquire data to be the target of an inference task, compress the acquired data, and transmit the compressed data to the server device 1 .

First, the client device 2 acquires the target data 31 . Next, the client device 2 compresses the acquired target data 31 . Thereby, the client device 2 generates the compressed target data 33 . In addition, in the figure, the compressed target data is described as compressed data. The client device 2 then transmits the compressed target data 33 to the server device 1 .

The server device 1 receives the compressed target data 33 from the client device 2 . The server device 1 uses the inference model 5 to acquire the result of performing the inference task on the received target data 33 . In this embodiment, the server device 1 holds an inference model 5 . The server device 1 uses the inference model 5 to execute processing for performing an inference task on the compressed target data 33, thereby obtaining an execution result (inference result).

The server device 1 calculates the likelihood of the execution result of the inference task by the inference model 5 . The plausibility may be calculated as appropriate to indicate the degree of certainty of the performance result of the inference task by the inference model 5 (that is, whether or not the inference result of the inference model 5 is reliable). If the calculated likelihood is less than the threshold, the server device 1 lowers the compression rate from the previous compression rate and transmits to the client device 2 an instruction to resend the target data.

In response to the server device 1 receiving an instruction to resend the target data after reducing the compression rate, the client device 2 responds to the previous compression rate (that is, the target data previously transmitted to the server device 1 (this In this case, the target data 31 is compressed again at a compression rate lower than the compression rate of the target data 33). As a result, the client device 2 generates the target data 35 compressed at a compression rate lower than the previous compression rate. Then, the client device 2 transmits the re-compressed target data 35 to the server device 1 in response to receiving the instruction to resend the target data.

The server device 1 receives the compressed target data 35 from the client device 2 and obtains the result of performing the inference task on the received target data 35 by using the inference model 5 . The server device 1 may perform the processing subsequent to calculating the likelihood of the execution result for the compressed target data 35 in the same manner as for the compressed target data 33 .

As described above, in this embodiment, the traffic on the network 4 can be reduced by exchanging the compressed target data (33, 35) between the server device 1 and the client device 2. In addition, the server device 1 evaluates the likelihood of the performance result of the inference task by the inference model 5, and if the likelihood is less than the threshold, the target data 35 with a reduced compression rate is resent. This enables the inference model 5 to perform the inference task on the target data 35 with a low compression rate, and improves the plausibility of the inference result. Therefore, according to this embodiment, the accuracy of the inference task performed by the inference model 5 can be guaranteed while reducing the traffic on the network 4 .

Note that the data type of the target data 31 is not particularly limited as long as it can be the target of the inference task, and may be appropriately selected according to the embodiment. The target data 31 may be, for example, image data, sound data, numerical data, text data, sensing data obtained by other sensors, and the like. In one example, the target data 31 may be sensing data obtained by observing an arbitrary target with a sensor. The sensor may be, for example, an image sensor (camera), an infrared sensor, a sound sensor (microphone), an ultrasonic sensor, an optical sensor, a pressure sensor, an air pressure sensor, a temperature sensor, or the like. Also, the sensor may be, for example, an environmental sensor, a vital sensor, a medical examination device, an in-vehicle sensor, a home security sensor, or the like. Environmental sensors may be, for example, barometers, thermometers, hygrometers, sound pressure meters, sound sensors, ultraviolet sensors, illuminometers, rain gauges, gas sensors, and the like. Vital sensors include, for example, sphygmomanometers, pulse meters, heart rate meters, electrocardiographs, electromyographs, thermometers, skin electrometers, microwave sensors, electroencephalographs, magnetoencephalographs, activity meters, blood glucose meters, eyes It may be an electric potential sensor, an eye movement measuring instrument, or the like. The medical examination device may be, for example, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or the like. The in-vehicle sensor may be, for example, an image sensor, a lidar (light detection and ranging) sensor, a millimeter wave radar, an ultrasonic sensor, an acceleration sensor, or the like. Home security sensors include, for example, image sensors, infrared sensors, activity (voice) sensors, gas (CO2, etc.) sensors, current sensors, smart meters (sensors that measure power usage for home appliances, lighting, etc.), etc. good. The target data 31 may be configured to include multiple types of data, such as moving image data including image data and sound data.

The content of the inference task may be determined as appropriate according to the embodiment. In one example, the target data 31 may consist of image data, and the inference task may consist of recognizing objects appearing in the image data. In another example, the target data 31 may consist of sound data, and the inference task may consist of inferring features appearing in the sound data. In yet another example, the target data 31 may consist of sensing data, and the inference task may consist of inferring features appearing in the sensing data.

The type of network 4 is not particularly limited, and may be appropriately selected from, for example, the Internet, wireless communication network, mobile communication network, telephone network, dedicated network, etc., according to the embodiment. The inference model 5 may be appropriately configured to be capable of executing computations that perform inference tasks on target data. In one example of the present embodiment, the inference model 5 is configured by a trained machine learning model (neural network described later) generated by machine learning. At least one of the server device 1 and the client device 2 may be composed of a plurality of computers.

§2 Configuration example [Hardware configuration]
<Server device>
FIG. 3 schematically illustrates an example of the hardware configuration of the server device 1 according to this embodiment. As shown in FIG. 3, the server device 1 according to the present embodiment is electrically connected to a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17. computer. Incidentally, in FIG. 3, the communication interface and the external interface are described as "communication I/F" and "external I/F".

The control unit 11 includes a CPU (Central Processing Unit), which is a hardware processor (processor resource), RAM (Random Access Memory), ROM (Read Only Memory), etc., and executes information processing based on programs and various data. configured as The storage unit 12 is an example of memory (memory resource), and is configured by, for example, a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 12 stores various information such as the inference program 81 and the learning result data 125 .

The inference program 81 is a program for causing the server device 1 to execute information processing (FIG. 7) for performing an inference task using the inference model 5, which will be described later. Inference program 81 includes a series of instructions for the information processing. Inference program 81 is an example of a resend instruction program. The learning result data 125 indicates information about the trained inference model 5 generated by machine learning. Details will be described later.

The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. Server device 1 can use communication interface 13 to perform data communication with other information processing devices via a network. The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting with an external device. The type and number of external interfaces 14 may be arbitrarily selected.

The input device 15 is, for example, a device for performing input such as a mouse and a keyboard. Also, the output device 16 is, for example, a device for outputting such as a display and a speaker. An operator such as a user can operate the server device 1 by using the input device 15 and the output device 16 .

The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as programs stored in the storage medium 91 . The storage medium 91 stores information such as programs by electrical, magnetic, optical, mechanical or chemical action so that computers, other devices, machines, etc. can read various information such as programs. It is a medium that accumulates by At least one of the inference program 81 and the learning result data 125 may be stored in the storage medium 91 . The server device 1 may read at least one of the inference program 81 and the learning result data 125 from the storage medium 91 . In FIG. 3, as an example of the storage medium 91, a disk-type storage medium such as a CD or DVD is illustrated. However, the type of storage medium 91 is not limited to the disk type, and may be other than the disk type. As a storage medium other than the disk type, for example, a semiconductor memory such as a flash memory can be cited. The type of drive 17 may be arbitrarily selected according to the type of storage medium 91 .

Regarding the specific hardware configuration of the server apparatus 1, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, control unit 11 may include multiple hardware processors. The hardware processor may comprise a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 12 may be configured by RAM and ROM included in the control unit 11 . At least one of the external interface 14, the input device 15, the output device 16 and the drive 17 may be omitted. The server device 1 may be composed of a plurality of computers. In this case, the hardware configuration of each computer may or may not match. The server device 1 may be configured by an information processing device designed exclusively for the service provided, a general-purpose server, a PC (Personal Computer), or the like. The server device 1 may be, for example, an edge server, a cloud server, or the like.

<Client device>
FIG. 4 schematically illustrates an example of the hardware configuration of the client device 2 according to this embodiment. As shown in FIG. 4, the client device 2 according to the present embodiment is electrically connected to a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a drive 27. computer.

The control unit 21 to the drive 27 and the storage medium 92 of the client device 2 may be configured similarly to the control unit 11 to the drive 17 and the storage medium 91 of the server device 1, respectively. The control unit 21 includes a hardware processor such as a CPU, a RAM, and a ROM, and is configured to execute various types of information processing based on programs and data. The storage unit 22 is composed of, for example, a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 22 stores various information such as the transmission program 82 and the target data 31 .

The transmission program 82 is a program for causing the client device 2 to execute information processing (FIG. 8), which will be described later, regarding transmission of the compressed target data (33, 35). The transmission program 82 contains a series of instructions for the information processing. At least one of the transmission program 82 and the target data 31 may be stored in the storage medium 92 . The client device 2 may read at least one of the transmission program 82 and the target data 31 from the storage medium 92 .

In one example, the client device 2 may be connected via at least one of the communication interface 23 and the external interface 24 to devices such as cameras, microphones, other sensors, or other computers. The client device 2 may obtain target data 31 from a connected device or other computer.

Regarding the specific hardware configuration of the client device 2, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, the controller 21 may include multiple hardware processors. A hardware processor may comprise a microprocessor, FPGA, DSP, or the like. The storage unit 22 may be configured by RAM and ROM included in the control unit 21 . At least one of the external interface 24, the input device 25, the output device 26, and the drive 27 may be omitted. The client device 2 may be composed of multiple computers. In this case, the hardware configuration of each computer may or may not match. The client device 2 is an information processing device designed exclusively for the provided service, a general-purpose PC, an industrial PC, a tablet PC, a user terminal (for example, a smartphone), a PLC (programmable logic controller), an IoT device, etc. may be

[Software configuration]
<Server device>
FIG. 5 schematically illustrates an example of the software configuration of the server device 1 according to this embodiment. The control unit 11 of the server device 1 expands the inference program 81 stored in the storage unit 12 to RAM. Then, the control unit 11 causes the CPU to interpret and execute instructions included in the inference program 81 developed in the RAM, and controls each component. Accordingly, as shown in FIG. 5, the server device 1 according to the present embodiment is a computer having a data receiving unit 111, a result acquiring unit 112, a result evaluating unit 113, a retransmission instructing unit 114, and an output unit 115 as software modules. works as That is, in this embodiment, each software module of the server device 1 is implemented by the control unit 11 (CPU).

The data receiving unit 111 is configured to receive the compressed target data 33 from the client device 2 . The result acquisition unit 112 is configured to acquire a result of performing an inference task on the received target data 33 by using the inference model 5 . In the present embodiment, the result acquisition unit 112 has the trained inference model 5 generated by machine learning by holding the learning result data 125, and is configured to operate as an inference unit. . That is, the result acquisition unit 112 uses the trained inference model 5 to perform processing for performing an inference task on the compressed target data 33, thereby acquiring an execution result (inference result). configured as

The result evaluation unit 113 is configured to calculate the likelihood of the performance result of the inference task by the inference model 5 . The retransmission instruction unit 114 is configured to transmit an instruction to the client device 2 to retransmit the target data after lowering the compression rate than the previous compression rate when the calculated likelihood is less than the threshold. be done. In this embodiment, the retransmission instructing unit 114 may be configured to further instruct the extent to which the compression rate is lowered according to the calculated likelihood along with the retransmission instruction. In response to the retransmission instruction, the data receiving unit 111 is configured to receive from the client device 2 the target data 35 compressed at a compression rate lower than the previous compression rate. The result acquisition unit 112 is configured to acquire a result of performing an inference task on the target data 35 by using the inference model 5 . The result evaluation unit 113 may be configured to calculate the likelihood of performance results of the inference task on the compressed target data 35 . If the calculated likelihood is less than the threshold, the retransmission instruction unit 114 retransmits the instruction to the client device 2 to retransmit the target data with a lower compression rate than the previous compression rate. may be configured. Thereby, the server device 1 may be configured to repeatedly obtain compressed target data and perform the inference task by the inference model 5 until the likelihood of the performance result of the inference task exceeds the threshold. The output unit 115 is configured to output information about the performance result of the inference task.

(An example of an inference model)
The inference model 5 may be appropriately configured, for example, to be able to execute arithmetic processing for performing an inference task corresponding to target data, using a predetermined algorithm, predetermined rules, functional expressions, or the like. The output of the inference model 5 may be appropriately configured to identify the results of performing the inference task. In one example of this embodiment, the inference model 5 is configured by a trained machine learning model generated by machine learning. The machine learning model has parameters that are adjustable by machine learning. The configuration and type of the machine learning model may be appropriately selected according to the embodiment. In one example, a neural network may be used for the machine learning model that makes up the inference model 5, as shown in FIG.

In one example of FIG. 5 , the inference model 5 comprises an input layer 51 , one or more intermediate (hidden) layers 52 and an output layer 53 . The number of intermediate layers 52 may be appropriately determined according to the embodiment. Intermediate layer 52 may be omitted. The number of neural network layers forming the inference model 5 may be appropriately determined according to the embodiment. The input layer 51 may be suitably configured to receive target data. Output layer 53 may be suitably configured to output a value corresponding to the inference result. The input layer 51 may be configured to receive information other than the target data, and the output layer 53 may be configured to output information other than the information corresponding to the inference result.

Each layer 51-53 comprises one or more nodes (neurons). The number of nodes included in each layer 51-53 may not be particularly limited, and may be determined as appropriate according to the embodiment. Each node contained in each layer 51-53 may be connected to all nodes in adjacent layers. As a result, the inference model 5 may be composed of a fully-connected neural network. However, the connection relationship of each node may not be limited to such an example, and may be determined as appropriate according to the embodiment. For example, each node may be connected to a specific node in an adjacent layer, or connected to a node in a layer other than the adjacent layer.

A weight (connection weight) is set for each connection of each layer 51-53. A threshold is set for each node, and basically the output of each node is determined depending on whether or not the sum of the products of each input and each weight exceeds the threshold. The threshold may be expressed by an activation function. In this case, the output of each node is determined by inputting the sum of the product of each input and each weight into the activation function and executing the operation of the activation function. The type of activation function may be chosen arbitrarily. The weight of the connection between nodes included in each layer 51 to 53 and the threshold value of each node are examples of parameters used in the calculation process of the inference model 5 .

In machine learning, the parameter values of the inference model 5 determine its ability to perform a desired inference task using learning data (e.g., multiple learning data sets each composed of a combination of training data and correct labels). adjusted accordingly to obtain In one example, machine learning is performed by inputting training data (data of the same type as target data) into the inference model 5 for each learning data set so that the inference task performance results obtained from the inference model 5 match the corresponding correct labels. It is constructed by training the inference model 5 (adjusting the values of the parameters) so that As the machine learning method, for example, a known method such as error backpropagation may be adopted according to the machine learning model.

Learning result data 125 may be configured as appropriate to include information for reproducing trained inference model 5 . In one example, the learning result data 125 may include information indicating the value of each parameter of the inference model 5 obtained by adjusting the machine learning. Depending on the case, the learning result data 125 may further include information indicating the structure of the inference model 5 . The structure may be specified, for example, by the number of layers from the input layer to the output layer in the neural network, the type of each layer, the number of nodes included in each layer, the connection relationship between nodes in adjacent layers, and the like. If the structure of the machine learning model is shared between devices, the information indicating the structure of the inference model 5 may be omitted.

<Client device>
FIG. 6 schematically illustrates an example of the software configuration of the client device 2 according to this embodiment. The control unit 21 of the client device 2 expands the transmission program 82 stored in the storage unit 22 to RAM. Then, the control unit 21 causes the CPU to interpret and execute commands included in the transmission program 82 developed in the RAM, thereby controlling each component. As a result, as shown in FIG. 6, the client device 2 according to this embodiment operates as a computer having a data acquisition unit 211, a data compression unit 212, and a data transmission unit 213 as software modules. That is, in the present embodiment, each software module of the client device 2 is realized by the control unit 21 (CPU) as well as the server device 1 .

The data acquisition unit 211 is configured to acquire the target data 31 . The data compression unit 212 is configured to generate compressed target data 33 by compressing the acquired target data 31 . The data transmission unit 213 is configured to transmit the compressed target data 33 to the server device 1 . In addition, in response to the occurrence of the instruction to resend the target data after reducing the compression rate in the server device 1, the data compression unit 212 re-transmits the target data 31 at a compression rate lower than the previous compression rate. The compression is configured to generate compressed target data 35 . The data transmission unit 213 is configured to transmit the recompressed target data 35 to the server device 1 in response to receiving the instruction to resend the target data.

<Others>
Each software module of the server device 1 and the client device 2 will be described in detail in operation examples described later. In this embodiment, an example is described in which each software module of the server device 1 and the client device 2 is implemented by a general-purpose CPU. However, some or all of the software modules may be implemented by one or more dedicated processors. Each module described above may be implemented as a hardware module. Further, with regard to the software configurations of the server device 1 and the client device 2, software modules may be omitted, replaced, or added as appropriate according to the embodiment.

§3 Operation example [Server device]
FIG. 7 is a flow chart showing an example of a processing procedure regarding an inference method by the server device 1 according to this embodiment. The processing procedure of the server device 1 described below is an example of a retransmission instruction method. However, the processing procedure of the server apparatus 1 described below is merely an example, and each step may be changed as much as possible. Further, in the following processing procedures, steps may be omitted, replaced, or added as appropriate according to the embodiment.

(Step S101 and Step S102)
In step S<b>101 , the control unit 11 operates as the data receiving unit 111 and receives the compressed target data 33 from the client device 2 via the network 4 . Upon receiving the compressed target data 33, the control unit 11 advances the process to the next step S102.

In step S<b>102 , the control unit 11 operates as the result acquisition unit 112 and acquires the result of executing the inference task on the received target data 33 by using the inference model 5 . In this embodiment, the control unit 11 refers to the learning result data 125 and sets the trained inference model 5 . Then, the control unit 11 inputs the compressed target data 33 to the trained inference model 5 and executes the arithmetic processing of the trained inference model 5 . In the above example, the control unit 11 inputs the compressed target data 33 to the input layer 51 of the trained inference model 5 and executes forward propagation arithmetic processing. As a result of this arithmetic processing, the control unit 11 acquires the result of performing the inference task on the compressed target data 33 from the trained inference model 5 (the output layer 53 in the example above). After acquiring the performance result of the inference task, the control unit 11 advances the process to the next step S103.

(Step S103)
In step S<b>103 , the control unit 11 operates as the result evaluation unit 113 and calculates the likelihood of the performance result of the inference task by the inference model 5 . A method for evaluating plausibility may be appropriately determined according to, for example, the configuration of the inference model 5, the process in which the inference model 5 derives an inference result, and the like.

In this embodiment, the inference model 5 is composed of a trained machine learning model. In this case, the higher the plausibility of the inference result, the more likely the inference model 5 outputs a definite value, and the lower the plausibility of the inference result, the more the inference model 5 tends to output a less definite value. As a specific example, assume the case where the inference task is class identification and the inference model 5 is configured to output the posterior probability of belonging to each class. In this case, the closer the posterior probability that the output of the inference model 5 belongs to one class is to 1 and the posterior probability that the other classes belong to the rest is closer to 0, the more certain the inference result by the inference model 5 is. It can be evaluated that the plausibility is determined and the likelihood is high. On the contrary, the more similar the output of the inference model 5 with the posterior probabilities belonging to each class is, the lower the plausibility of the inference result by the inference model 5 becomes. It is possible.

Therefore, in one example, the control unit 11 may calculate the likelihood based on the uncertainty of the execution result of the inference task by the inference model 5 . Uncertainty is an index that indicates the extent to which the inference result is uncertain. Uncertainty may be translated into terms such as ambiguity and entropy. The control unit 11 may calculate the plausibility value by evaluating the uncertainty of the inference result, for example, using a calculation method such as variance, kurtosis, and average information amount (information entropy). In one example, the control unit 11 calculates at least one of the variance, the kurtosis, and the average amount of information of the posterior probability belonging to each class, and the larger the calculated value, the higher the likelihood. The smaller the value, the lower the likelihood may be calculated. Note that even when a model other than the machine learning model and having the same output tendency as the inference model 5 is adopted as the inference model 5, the control unit 11 uses the same calculation method to determine the likelihood of the inference result. can be calculated. According to these calculation methods, it is possible to appropriately evaluate the uncertainty and appropriately calculate the likelihood.

As a result, in the above cases, the former case can be calculated so that the uncertainty is low and the plausibility is high, and the latter case can be calculated so that the uncertainty is high and the plausibility is low. . In the above case, it may be assumed that the identification target is included in the target data. In this case, the class to be set may be given according to the identification target. On the other hand, if there is no such premise, the class to be set may include a class corresponding to the non-existence of the identification object. As a result, it is possible to calculate the likelihood of the inference result by distinguishing it from the fact that the identification target does not exist in the target data.

The method of calculating likelihood based on uncertainty need not be limited to the above method. As another method, in the above case, assuming a posterior probability distribution of deterministic or indeterminate inference results, Based on the difference, the plausibility of the inference result may be calculated. That is, the difference between the assumed posterior probability distribution and the posterior probability distribution of the inference result may be used as an index for evaluating uncertainty. As an example of the assumed posterior probability distribution, the posterior probability distribution of uncertain inference results may be assumed to be a uniform distribution. That is, it may be assumed that the inference result in which the posterior probability of each class is the reciprocal of the number of classes C, 1/C, is the most uncertain. As another example, if there is a bias in the frequency of each class in the learning data used for machine learning (for example, the number of training data associated with the correct label indicating the target class), the class with the higher frequency is identified. easier to be Therefore, the posterior probability distribution of uncertain inference results may be assumed as the prior distribution of classes in the learning data. When adopting this method, in order to evaluate the difference between the distributions, the control unit 11 uses the inference result (the distribution of the posterior probability of each class) obtained by the inference model 5 in step S102 and the assumed posterior probability distribution. You may calculate the distance between The distance may be calculated by an index such as KL divergence, JS divergence, or the like. When an uncertain posterior probability distribution is assumed, the smaller the calculated distance is, the higher the uncertainty is, and the control unit 11 may calculate the likelihood to be a lower value. The certainty is low, and the likelihood may be calculated as a high value. Conversely, when a deterministic posterior probability distribution is assumed, the smaller the calculated distance is, the lower the uncertainty is, and the higher the likelihood may be calculated, the larger the calculated distance is. The higher the uncertainty, the lower the likelihood may be calculated.

A plurality of inference models 5 may be prepared as another method of evaluating uncertainty. In this case, each inference model 5 is, for example, a different machine learning model to be adopted, a different structure of at least a part of the machine learning model, a different parameter configuration of at least a part of the machine learning model, at least a part may be configured to have different attributes from the other inference model 5 because machine learning is performed using different learning data, the order of learning data adopted in machine learning is different, or the like. The control unit 11 may acquire an inference result for the target data 33 by each inference model 5 in step S102. In step S<b>103 , the control unit 11 may calculate the degree of disagreement between the obtained inference results of the inference models 5 as the uncertainty of the inference results of the inference model 5 . The control unit 11 may calculate a lower value of likelihood as the degree of mismatch increases, and a higher value of likelihood as the degree of mismatch decreases. This method may be adopted regardless of the type of inference task. Similarly, when using a model other than the machine learning model, a plurality of inference models 5 having different attributes may be prepared, and the likelihood may be calculated based on the degree of disagreement between the inference results of each inference model 5.

Furthermore, as another method of evaluating uncertainty, the parameters of the inference model 5 may be assumed with probability distributions (see, for example, Yarin Gal, "Uncertainty in Deep Learning", [online], [Order Retrieved on February 19, 1998], Internet <URL: https://www.cs.ox.ac.uk/people/yarin.gal/website/thesis/thesis.pdf>”). In this case, in the learning stage, each parameter of the inference model 5 may be modeled with an arbitrary probability distribution (eg, Gaussian distribution). In machine learning, the learning data may be used to train the inference model 5 accordingly. During inference processing, a plurality of inference models 5 composed of fixed-value parameters may be prepared by sampling the value of each parameter from the probability distribution of each parameter obtained by training. Thereafter, similarly to the method of preparing the plurality of inference models 5, the control unit 11 may obtain the inference result for the target data 33 by each inference model 5 in step S102. In step S<b>103 , the control unit 11 may calculate the degree of disagreement between the obtained inference results of the inference models 5 as the uncertainty of the inference results of the inference model 5 . The control unit 11 may calculate a lower value of likelihood as the degree of mismatch increases, and a higher value of likelihood as the degree of mismatch decreases. This method may be adopted regardless of the type of inference task. When adopting this method, the inference model 5 may use a Bayesian neural network.

After calculating the likelihood of the inference result, the control unit 11 proceeds to the next step S104.

(Step S104)
In step S104, the control unit 11 determines a branch destination of the process according to the likelihood value calculated in the process of step S103. When the calculated likelihood is less than the threshold, the control unit 11 advances the process to step S105. On the other hand, when the likelihood exceeds the threshold, the control unit 11 advances the process to step S106. If the likelihood is equal to the threshold, the processing may branch to any destination. The threshold value may be given by any method such as operator designation, setting value within a program, or the like.

In the above case where the inference task is class identification, the deterministic posterior probability value (determination threshold) for determining that an object belongs to the target class may differ from class to class. For example, for the first class, if the posterior probability is 0.9 or more, it is not determined to belong deterministically, whereas for the second class, if the posterior probability is 0.6 or more, it definitely belongs This is a case where it is determined that To accommodate this, the threshold may be variably set for each class. In this case, in one example, the likelihood may be calculated for each class. The control unit 11 may compare the likelihood and the threshold for each class. In another example, the likelihood may be calculated for the entire inference result. The control unit 11 determines the calculated plausibility and the class that is likely to belong (for example, the class that simply has the highest posterior probability, and if each posterior probability is lower than the determination threshold, the posterior probability that is closest to the determination threshold. class, etc.). Similarly, when the inference task is regression, multiple thresholds (for example, thresholds for each range) may be set, or a single threshold may be set.

Also, the relationship between the likelihood and its value may be appropriately determined according to the embodiment. When a state with high plausibility is represented by a large value, evaluating the plausibility to a high value corresponds to calculating the plausibility to a large value, and evaluating the plausibility to a low value corresponds to calculating a small value. corresponds to calculating In this case, the fact that the calculated likelihood is less than the threshold corresponds to the fact that the value indicating likelihood is less than the threshold. On the other hand, when a state with high plausibility is represented by a small value, evaluating the plausibility to a high value corresponds to calculating the plausibility to a small value, and evaluating the plausibility to a low value corresponds to This corresponds to calculating the likelihood as a large value. In this case, the fact that the calculated likelihood is less than the threshold corresponds to the fact that the value indicating likelihood exceeds the threshold.

(Step S105)
In step S105, the control unit 11 operates as the retransmission instruction unit 114, and transmits an instruction to the client device 2 to retransmit the target data with a lower compression rate than the previous compression rate.

At this time, the control unit 11 may further instruct the extent to which the compression rate is to be lowered according to the calculated likelihood along with the instruction to retransmit. Instructing the degree to which the compression ratio is to be reduced may be constituted by instructing the amount of reduction in the compression ratio, or may be constituted by directly instructing the compression ratio after the reduction. . Basically, the control unit 11 instructs the client device 2 to increase the degree of reduction in the compression rate as the plausibility is low, and to decrease the degree of reduction in the compression rate as the plausibility is high. good. The extent to which the compression ratio is lowered may be defined stepwise by discrete values or may be defined by continuous values.

In one example, a rule may define a correspondence relationship between the likelihood and the extent to which the compression ratio is lowered. In this case, the control unit 11 may determine the extent to which the compression rate is lowered from the likelihood calculated in step S103 according to the rule. Then, the control unit 11 may notify the client device 2 of the extent to which the determined compression rate is lowered. Alternatively, the control unit 11 may notify the client device 2 of the likelihood value, and cause the client device 2 to execute the process of determining the extent to which the compression rate is lowered according to a rule. Note that, using the inference model 5 and test target data, the correspondence relationship between the likelihood of the inference task execution by the inference model 5 and the compression rate of the target data may be confirmed in advance. Based on the results of that verification, the rules may be created.

However, the method of designating the extent to which the compression ratio is lowered need not be limited to such an example. In another example, the extent to which the compression ratio is reduced may be determined without depending on likelihood (eg, a constant value). In this case, the extent to which the compression rate is to be lowered may be specified in advance by, for example, an operator's instruction, a set value in the program, etc., and the instruction of the extent to which the compression rate is to be lowered from the server device 1 to the client device 2 is omitted. you can

After transmitting the retransmission instruction to the client device 2, the control unit 11 returns the process to step S101 and repeats the series of processes. In response to the client device 2 responding to the retransmission instruction, in step S101, the control unit 11 operates as the data receiving unit 111, and transmits the target data 35 compressed at a compression rate lower than the previous compression rate to the client device. Receive from 2. In step S<b>102 , the control unit 11 operates as the result acquisition unit 112 and acquires the result of performing the inference task on the target data 35 by using the inference model 5 . In step S<b>103 , the control unit 11 calculates the likelihood of the execution result of the inference task for the compressed target data 35 . In step S104, the control unit 11 determines a branch destination of the process according to the calculated likelihood value. Thereby, the control unit 11 may be configured to repeatedly execute the processing of steps S101 to S105 until the likelihood of the performance result of the inference task exceeds the threshold (or the compression ratio cannot be lowered).

However, the processing procedure of the server device 1 need not be limited to such an example. As another example, the control unit 11 may omit the processes of steps S103 and S104 in the repeated process after executing the process of step S105. In this case, the control unit 11 may proceed to step S106 after acquiring the performance result of the inference task for the compressed target data 35 by executing the process of step S102. Thereby, the control unit 11 may be configured to transmit the retransmission instruction to the client device 2 only once.

(Step S106)
In step S106, the control unit 11 operates as the output unit 115 and outputs information about the performance result (inference result) of the inference task obtained in step S102.

The content of the information to be output may be appropriately determined according to the embodiment. In one example, the control unit 11 may directly output the inference result obtained in step S102. In another example, the control unit 11 may execute arbitrary information processing based on the obtained inference result. Then, the control unit 11 may output the result of executing the information processing as information about the inference result. The output of the result of executing this information processing may include controlling the operation of the controlled device according to the inference result.

Similarly, the information output destination may be appropriately determined according to the embodiment. In one example, the output destination of information may be the output device 16, an output device of another computer, a device to be controlled, or the like. As another example, the control unit 11 may transmit information about the inference result to the client device 2 . The client device 2 may display the received information on the output device 26 as it is. Further, the client device 2 may execute arbitrary information processing based on the received information and output the result of executing the information processing.

When the inference task is performed multiple times on the compressed target data (33, 35) by repeating the branching process in step S104, in one example, the control unit 11 outputs all the inference results. good. As another example, the control unit 11 may output only a part of the inference results (for example, the inference result with the highest likelihood).

When the output of the information about the performance result of the inference task is completed, the control unit 11 terminates the processing procedure of the server device 1 according to this operation example. Note that the control unit 11 may execute a series of information processing from step S101 to step S106 each time the target data 33 is received from the client device 2 .

[Client device]
FIG. 8 is a flow chart showing an example of the processing procedure of the client device 2 according to this embodiment. The processing procedure of the client device 2 described below is an example of the data transmission method. However, the processing procedure of the client device 2 described below is merely an example, and each step may be changed as much as possible. Further, in the following processing procedures, steps may be omitted, replaced, or added as appropriate according to the embodiment.

(Step S201)
In step S<b>201 , the control unit 21 operates as the data acquisition unit 211 and acquires the target data 31 .

Target data 31 is a sample of a given type of data on which an inference task is performed. A method for acquiring the target data 31 may be appropriately determined according to the type of data. As an example, the target data 31 may be obtained by observing the inference target with a sensor (eg, camera, microphone, or other sensor). Also, the route for acquiring the target data 31 may not be particularly limited, and may be appropriately selected according to the embodiment. As an example, when the client device 2 is directly connected to a device such as a sensor, the control unit 21 may directly acquire the target data 31 from the device. As another example, the control unit 21 may acquire the target data 31 via another computer, the storage medium 92, or the like. After obtaining the target data 31, the control unit 21 advances the process to the next step S202.

(Step S202)
In step S<b>202 , the control unit 21 operates as the data compression unit 212 and compresses the acquired target data 31 to generate the compressed target data 33 .

A method for determining the compression rate may not be particularly limited, and may be appropriately selected according to the embodiment. The compression rate may be preset to a predetermined value. Alternatively, the compression ratio may be determined according to any index. As an example of the index, the compression rate may be determined based on PSNR (Peak signal-to-noise ratio). In this case, a PSNR reference value may be set in advance. The control unit 21 generates compressed target data 33 by compressing the target data 31 . The control unit 21 calculates the PSNR of the compressed target data 33 and evaluates whether the calculated PSNR value satisfies the reference value. The control unit 21 may generate the compressed target data 33 that satisfies the reference value, for example, by repeating generation of the compressed target data 33 and PSNR evaluation.

As another example of the index, the compression rate may be determined according to communication conditions. In this case, the control unit 21 measures the traffic congestion degree of the network 4 with the server device 1 . Any method may be adopted as a method of measuring the degree of congestion. For example, the control unit 21 may measure the response speed with the server device 1 using a ping command as an index for evaluating the degree of congestion. The controller 21 can evaluate that the faster the response speed, the lower the congestion degree, and the slower the response speed, the higher the congestion degree. Alternatively, for example, a method using SNMP (Simple Network Management Protocol) for monitoring the amount of discarded packets or the like may be adopted. In this case, the control unit 21 may evaluate the degree of congestion based on the results of SNMP monitoring. Alternatively, for example, a flow monitoring method for monitoring the congestion state for each flow such as NetFlow, SFlow, AppFlow, etc. may be employed. In this case, the control unit 21 may evaluate the degree of congestion based on flow state information obtained by the flow monitoring method. The control unit 21 may set a higher compression rate as the degree of congestion is higher, and may set a lower compression rate as the degree of congestion is lower. The compression rate according to the degree of congestion may be defined stepwise by discrete values, or may be defined by continuous values.

Any compression method may be employed. In one example, the compression method may employ a method such as downsampling, a method of transferring each frequency component (eg, progressive JPEG), or the like. Any compression method that results in the amount of data being transmitted being smaller than the original data may be employed. After generating the compressed target data 33, the control unit 21 proceeds to the next step S203.

(Steps S203 and S204)
In step S<b>203 , the control unit 21 transmits the compressed target data 33 to the server device 1 via the network 4 . As a result of executing the process of step S203, the server apparatus 1 receives the compressed target data 33 in the process of step S101. After transmitting the compressed target data 33, the control unit 21 advances the process to the next step S204.

In step S204, the control unit 21 determines the branch destination of the process according to whether or not the retransmission instruction in step S105 has been received from the server device 1 or not. When receiving the retransmission instruction, the control unit 21 returns the process to step S202, and repeats the processes of steps S202 and S203.

In step S202, the control unit 21 generates compressed target data 35 by compressing the target data 31 again at a compression rate lower than the previous compression rate. As described above, the degree to which the compression ratio is lowered may be determined according to the likelihood of the inference result, or may be determined regardless of the likelihood. Also, the degree to which the compression ratio is to be lowered may be instructed by the server apparatus 1 together with the retransmission instruction. Alternatively, the control unit 21 may appropriately determine the extent to which the compression rate is to be lowered, without depending on the instruction from the server device 1 .

In step S<b>203 , the control unit 21 transmits the recompressed target data 35 to the server device 1 via the network 4 . As a result of executing the process of step S203, the server apparatus 1 receives the compressed target data 35 in the process of step S101 after executing the process of step S105.

On the other hand, when the retransmission instruction is not received, the control unit 21 terminates the processing procedure of the client device 2 according to this operation example. As an example of a method for judging the end, if the server device 1 does not decide to branch to step S105 (that is, decides to branch to step S106) in step S104, the server device 1 A notification may be sent to the client device 2 to inform them. In the output process of step S106, when information about the inference result is transmitted to the client device 2, the transmission information may serve as the notification. By receiving this notification, the control unit 21 may determine that no retransmission instruction has occurred, and terminate the processing procedure of the client device 2 according to this operation example.

Note that the control unit 21 may execute a series of information processing from step S201 to step S204 at any timing. In one example, the control unit 21 may execute a series of processes from step S201 to step S204 according to the operator's operation. As another example, the control unit 21 may continuously and repeatedly execute a series of information processing from step S201 to step S204. The timing of repetition may be appropriately determined according to the embodiment.

[feature]
As described above, in the present embodiment, in the processes of steps S101 and S203, the compressed target data (33, 35) are exchanged between the server device 1 and the client device 2, so that network 4 traffic can be reduced. In addition, in the server device 1, the likelihood of the performance result of the inference task by the inference model 5 is evaluated by the processing of steps S103 to S105. The object data 35 with a low compression ratio is retransmitted. This enables the inference model 5 to perform the inference task on the target data 35 with a low compression rate (that is, less information loss), and improves the plausibility of the inference result. Therefore, according to this embodiment, the accuracy of the inference task performed by the inference model 5 can be guaranteed while reducing the traffic on the network 4 . Further, by properly calculating the likelihood in the process of step S103, the accuracy of the inference task performed by the inference model 5 can be appropriately guaranteed.

Further, in the present embodiment, in the process of step S105, the server apparatus 1 may further instruct the extent to which the compression rate is lowered according to the calculated likelihood along with the retransmission instruction. As a result, in the processes of steps S202 and S203 that are executed again, retransmission at an appropriate compression rate can be expected, and retransmission of the compressed target data 35 between the server device 1 and the client device 2 is executed. (that is, the number of times the processes of steps S101 to S105/steps S202 and S204 are repeated) can be suppressed. Therefore, according to this embodiment, an increase in traffic on the network 4 due to retransmission of the compressed target data 35 can be suppressed.

§4 Modifications Although the embodiments of the present invention have been described in detail, the above description is merely an example of the present invention in every respect. It goes without saying that various modifications or variations can be made without departing from the scope of the invention. For example, the following changes are possible. In addition, below, the same code|symbol is used about the component similar to the said embodiment, and description is abbreviate|omitted suitably about the point similar to the said embodiment. The following modified examples can be combined as appropriate.

<4.1>
The communication system according to the above embodiments may be applied to any situation in which any inference task is performed on a given type of data. The inference task may be, for example, recognizing an object appearing in image data, inferring features appearing in sound data, inferring features appearing in sensing data, and the like. The type of target data 31 may be appropriately selected according to the inference task. Modified examples with limited application scenes are shown below.

(A) Image Recognition Scene FIG. 9 schematically illustrates an example of an application scene of the communication system according to the first specific example. This modified example is an example in which the above-described embodiment is applied to a scene of recognizing an object appearing in image data. The communication system according to this modification is an example of the communication system according to the above embodiment, and includes the server device 1 and the image acquisition device 2A. The server device 1 and the image acquisition device 2A are connected to each other via the network 4 in the same manner as in the above embodiment.

In this modification, the target data is composed of image data 31A in which the target object RA can be captured. The image data 31A may be configured to indicate images such as still images, moving images, and 3D images, for example. The image data 31A may be obtained by the camera SA, may be generated by appropriately processing raw data obtained by the camera SA, or may be generated by arbitrary image processing independent of the camera SA. may be Camera SA may be, for example, a general RGB camera, a depth camera, an infrared camera, or the like. The image acquisition device 2A is a computer configured to acquire image data 31A as target data. The image acquisition device 2A is an example of the client device 2 according to the above embodiment. The inference task is constructed by recognizing the object RA appearing in the image data 31A that constitutes the object data. Recognizing the target object RA may include, for example, identifying the type of the target object RA, extracting a region in which the target object RA is captured (segmentation), and the like.

The object RA may be a person or any object. The range captured in the image data 31A may be a specific portion (for example, face) of the object RA, or may be the entire object RA. When the target object RA is a person, the recognition target may be, for example, a part of the body such as the face. Identifying the type of a person may be, for example, estimating an individual or estimating body parts (face, arms, legs, joints, etc.). The same is true for arbitrary objects.

As an example, as shown in FIG. 9, the image data 31A forming the target data may be obtained by the surveillance camera SA1. In this case, recognizing the object RA appearing in the image data 31A may be identifying the scene RA1 in the image data 31A. The monitoring camera SA1 is an example of the camera SA, and the scene RA1 is an example of the object RA. Identifying the scene RA1 means, for example, determining whether or not a specific person exists, or determining whether or not there is a sign of danger in a predetermined place (eg, street, station, airport, hall, etc.). It may include estimating and the like.

As another example, the image data 31A forming the target data may be obtained by a camera that monitors the state of products produced on the manufacturing line. In this case, recognizing the object RA shown in the image data 31A may be determining whether or not the product shown in the image data 31A has a defect (that is, performing a visual inspection of the product). Determining whether or not a product has defects includes, for example, identifying the presence or absence of defects, estimating the probability that the product contains defects, and determining the types of defects contained in the product ("defect free"). ), extracting the range of defects contained in the product, or a combination thereof.

The product may be, for example, a product that is transported in a manufacturing line for electronic equipment, electronic parts, automobile parts, medicines, food, and the like. The electronic components may be, for example, substrates, chip capacitors, liquid crystals, windings of relays, and the like. Automotive parts may be, for example, connecting rods, shafts, engine blocks, power window switches, panels and the like. The drug may be, for example, packaged tablets, unpackaged tablets, and the like. A product may be a final product produced after the manufacturing process is completed, an intermediate product produced during the manufacturing process, or an initial product prepared before the manufacturing process. may Defects may be, for example, scratches, stains, cracks, dents, burrs, color unevenness, foreign matter contamination, and the like.

Except for these points, the communication system according to this modification may be configured in the same manner as the above embodiment.

(Server device)
In this modification, the server device 1 may be configured to perform the inference task and transmit a retransmission instruction by the same processing procedure as in the above embodiment. That is, in step S101, the control unit 11 receives the compressed image data 33A from the image acquisition device 2A. In step S102, the control unit 11 acquires the result of performing the inference task on the received image data 33A by using the inference model 5A. The inference model 5A may be appropriately configured to be capable of executing arithmetic processing for performing the task of recognizing an object appearing in given image data. As in the above embodiment, the inference model 5A may be a trained machine learning model generated by machine learning.

In step S103, the control unit 11 calculates the likelihood of the performance result of the inference task by the inference model 5A. In step S104, the control unit 11 determines a branch destination of the process according to the calculated likelihood value. When the calculated likelihood is less than the threshold, the control unit 11 advances the process to step S105. On the other hand, when the likelihood exceeds the threshold, the control unit 11 advances the process to step S106. If the likelihood is equal to the threshold, the processing may branch to any destination.

In step S105, the control unit 11 transmits an instruction to the image acquisition device 2A to resend the image data after setting the compression rate lower than the previous compression rate. After that, the control unit 11 may return the process to step S101 and repeat the series of processes in the same manner as in the above-described embodiment. In step S101, the control unit 11 acquires the image data 35A compressed at a compression rate lower than the previous compression rate. In step S102, the control unit 11 acquires the result of performing the inference task on the image data 35A by using the inference model 5A. The control unit 11 may also perform the processing after step S103 for the acquired image data 35A.

At step S106, the control unit 11 outputs information about the performance result of the inference task. The output destination may be, for example, the output device 16, an output device of another computer, a device to be controlled, the image acquisition device 2A, or the like. In one example, the control unit 11 may output the result of recognizing the target object RA as it is. As another example, the control unit 11 may execute arbitrary information processing according to the result of identifying the target object RA. As a specific example, in the case of identifying the scene RA1, when it is recognized that there is a sign of danger, for example, because the probability of occurrence of danger exceeds a threshold value, the control unit 11 sends a notification to notify that fact. may be output. As another specific example, in the case of the visual inspection of the product, the production line may include a conveyor device that conveys the product. The control unit 11 may control the conveyor device so that defective products and non-defective products are conveyed on separate lines based on the result of determining whether or not the products are defective. In this case, the control unit 11 may cause the image acquisition device 2A to control the operation of the conveyor device as described above.

(Image acquisition device)
The hardware configuration and software configuration of the image acquisition device 2A may be the same as those of the client device 2 according to the above embodiment. In this modification, the image acquisition device 2A may be connected to the camera SA via a communication interface or an external interface. Alternatively, the camera SA may be connected to another computer, and the image acquisition device 2A may be configured to be able to acquire the image data 31A from the camera SA by connecting to the other computer. The image acquisition device 2A may incorporate a camera SA.

The image acquisition device 2A may perform processing for transmitting the compressed image data (33A, 35A) according to the same processing procedure as the client device 2 described above. That is, in step S201, the control section of the image acquisition device 2A acquires the image data 31A. In step S202, the control unit generates compressed image data 33A by compressing the acquired image data 31A. In step S203, the control unit transmits the compressed image data 33A to the server device 1 via the network 4. FIG.

In step S<b>204 , the control unit determines a branch destination of the process depending on whether or not a retransmission instruction has been received from the server device 1 . When the retransmission instruction is received, the control unit returns the process to step S202 and repeats the processes of steps S202 and S203. In step S202, the control unit generates compressed image data 35A by compressing the image data 31A again at a compression rate lower than the previous compression rate. In step S203, the control unit transmits the recompressed image data 35A to the server apparatus 1 via the network 4. FIG. On the other hand, if the resend instruction is not received, the control unit terminates the procedure for data transmission.

(feature)
According to the first specific example, when image data is transmitted from the image acquisition device 2A to the server device 1 and an inference task is performed on the image data, the traffic on the network 4 is reduced and the inference model 5A is used to perform the task. The accuracy of the inference task performed can be guaranteed. Image data has a relatively large amount of data, and can be a cause of heavy traffic. Therefore, the effect can be exhibited particularly effectively.

(B) Sound Recognition Scene FIG. 10 schematically illustrates an example of an application scene of the communication system according to the second specific example. This modified example is an example in which the above-described embodiment is applied to a situation in which features appearing in sound data are inferred. The communication system according to this modification is an example of the communication system according to the above embodiment, and includes the server device 1 and the sound acquisition device 2B. The server device 1 and the sound acquisition device 2B are connected to each other via the network 4 in the same manner as in the above embodiment.

In this modified example, the target data is composed of sound data 31B in which the characteristics of the target RB can appear. The sound data 31B may be obtained by observing the target RB with the microphone SB, may be generated by appropriately processing raw data obtained by the microphone SB, or may be generated by arbitrary sound generation processing. may be generated without relying on the microphone SB. The type of microphone SB may be appropriately selected according to the embodiment. The sound acquisition device 2B is a computer configured to acquire sound data 31B as target data. The sound acquisition device 2B is an example of the client device 2 according to the above embodiment. The inference task is configured by inferring the features of the target RB appearing in the sound data 31B forming the target data.

As an example, as shown in FIG. 10, the sound data 31B forming the target data may be obtained by recording the speaker's voice RB1 with the microphone SB. A speaker's voice RB1 is an example of a target RB. In this case, inferring features appearing in the sound data 31B may consist of recognizing the speaker's speech RB1. Recognizing the speech RB1 of the speaker may be, for example, identifying the speaker, analyzing the utterance content (language understanding), or the like.

As another example, the sound data 31B forming the target data may be obtained by observing environmental sounds or machine operating sounds. In this case, inferring the features appearing in the sound data 31B may be inferring the state of the environment or inferring the state of the machine. Inferring the state of the environment may be, for example, determining whether an accident has occurred, determining whether there is a sign of the occurrence of an accident, determining the weather, and the like. Inferring the state of the machine may be, for example, determining whether the machine is operating normally, whether the machine has signs of failure, and the like.

Except for these points, the communication system according to this modification may be configured in the same manner as the above embodiment.

(Server device)
In this modification, the server device 1 may be configured to perform the inference task and transmit a retransmission instruction by the same processing procedure as in the above embodiment. That is, in step S101, the control unit 11 receives the compressed sound data 33B from the sound acquisition device 2B. In step S102, the control unit 11 acquires the result of performing the inference task on the received sound data 33B by using the inference model 5B. The inference model 5B may be appropriately configured to be able to execute arithmetic processing that accomplishes the task of inferring features appearing in given sound data. As in the above embodiment, the inference model 5B may be a trained machine learning model generated by machine learning.

In step S103, the control unit 11 calculates the likelihood of the execution result of the inference task by the inference model 5B. In step S104, the control unit 11 determines a branch destination of the process according to the calculated likelihood value. When the calculated likelihood is less than the threshold, the control unit 11 advances the process to step S105. On the other hand, when the likelihood exceeds the threshold, the control unit 11 advances the process to step S106. If the likelihood is equal to the threshold, the processing may branch to any destination.

In step S105, the control unit 11 transmits an instruction to the sound acquisition device 2B to retransmit the sound data with a lower compression rate than the previous compression rate. After that, the control unit 11 may return the process to step S101 and repeat the series of processes in the same manner as in the above-described embodiment. In step S101, the control section 11 acquires the sound data 35B compressed at a compression rate lower than the previous compression rate. In step S102, the control unit 11 acquires the result of performing the inference task on the sound data 35B by using the inference model 5B. The control unit 11 may also execute the processes after step S103 for the acquired sound data 35B.

At step S106, the control unit 11 outputs information about the performance result of the inference task. The output destination may be, for example, the output device 16, an output device of another computer, a controlled device, the sound acquisition device 2B, or the like. In one example, the control unit 11 may directly output the result of inferring the characteristics of the target RB. As another example, the control unit 11 may execute arbitrary information processing according to the inference result. As a specific example, in the case of recognizing the speaker's voice RB1, the control unit 11 may determine the response content according to the speaker's utterance content, and output the determined response content. Alternatively, the control unit 11 may execute a language search (for example, a term search, a popular song search, etc.) based on the utterance content of the speaker and output the search result. These specific examples may be exclusively applied when the sound acquisition device 2B is a user terminal such as a smart phone, and information regarding the inference result is output to the sound acquisition device 2B. As another specific example, in the case of inferring the state of the machine from the mechanical sound, the control unit 11 determines that the target machine is out of order or has a sign of failure based on the inference result. For example, a process for coping with the failure or its sign may be executed, such as stopping the operation of the machine or outputting a notification to notify the fact.

(sound acquisition device)
The hardware configuration and software configuration of the sound acquisition device 2B may be the same as those of the client device 2 according to the above embodiment. In this modification, the sound acquisition device 2B may be connected to the microphone SB via a communication interface or an external interface. Alternatively, the microphone SB may be connected to another computer, and the sound acquisition device 2B may be configured to be able to acquire the sound data 31B from the microphone SB by connecting to the other computer. The sound acquisition device 2B may incorporate a microphone SB.

The sound acquisition device 2B may perform processing for transmitting compressed sound data (33B, 35B) by the same processing procedure as the client device 2 described above. That is, in step S201, the control unit of the sound acquisition device 2B acquires sound data 31B. In step S202, the control unit generates compressed sound data 33B by compressing the acquired sound data 31B. In step S203, the control unit transmits the compressed sound data 33B to the server device 1 via the network 4. FIG.

In step S<b>204 , the control unit determines a branch destination of the process depending on whether or not a retransmission instruction has been received from the server device 1 . When the retransmission instruction is received, the control unit returns the process to step S202 and repeats the processes of steps S202 and S203. In step S202, the control unit generates compressed sound data 35B by compressing the sound data 31B again at a compression rate lower than the previous compression rate. In step S<b>203 , the control unit transmits the recompressed sound data 35</b>B to the server device 1 via the network 4 . On the other hand, if the resend instruction is not received, the control unit terminates the procedure for data transmission.

(feature)
According to the second specific example, when sound data is transmitted from the sound acquisition device 2B to the server device 1 and an inference task is performed on the sound data, the inference model 5B is used to reduce traffic on the network 4. The accuracy of the inference task performed can be guaranteed. As with image data, sound data also has a relatively large amount of data, and can be a cause of heavy traffic. Therefore, the effect can be exhibited particularly effectively.

(C) Scene of Recognizing State of Sensing Target FIG. 11 schematically illustrates an example of an application scene of the communication system according to the third specific example. This modified example is an example in which the above-described embodiment is applied to a situation in which features appearing in sensing data are inferred. The communication system according to this modification is an example of the communication system according to the above embodiment, and includes a server device 1 and an observation device 2C. The server device 1 and the observation device 2C are connected to each other via the network 4 as in the above embodiment.

In this modification, the target data is composed of sensing data 31C that can represent the features of the target object RC. The sensing data 31C may be obtained by observing the object RC with the sensor SC, or may be generated by appropriately processing raw data obtained by the sensor SC (for example, extracting feature amounts). Alternatively, it may be generated by simulating the operation of the sensor SC. The sensing data 31C may be composed of a single type of data, or may be composed of a plurality of types of data. The sensors SC may be, for example, cameras, microphones, encoders, environmental sensors, vital sensors, medical examination equipment, vehicle sensors, home security sensors, and the like. The observation device 2C is a computer configured to acquire sensing data 31C as target data. The observation device 2C is an example of the client device 2 according to the above embodiment. The inference task is configured by inferring the features of the object RC appearing in the sensing data 31C forming the object data.

The type of sensor SC may be appropriately selected according to the inference task. As an example, the object RC may be the object person RC1, and inferring features about the object RC may be inferring the state of the object person RC1. In this case, the sensor SC1 that observes the state of the target person RC1 may be composed of at least one of a camera, a microphone, a vital sensor, and a medical examination device, for example. The target person RC1 is an example of the target object RC, and the sensor SC1 is an example of the sensor SC. As a specific example of the content of inference, inferring the state of the target person RC1 may be inferring health conditions such as the probability of developing a predetermined disease or the probability of a change in physical condition. As another specific example of the inference content, the target person RC1 may be the driver of the vehicle, and inferring the state of the target person RC1 may be based on the driver's state (for example, drowsiness, fatigue, leisure time, etc.). degrees, etc.).

As another example, the object RC may be an industrial machine, and inferring the characteristics of the object RC may be inferring (detecting or predicting) whether or not the industrial machine has an abnormality. good. In this case, the sensor SC may be composed of, for example, at least one of a microphone, an encoder, and an environmental sensor. A sample of sensing data may be composed of a motor encoder value, temperature, operation sound, and the like.

As another example, the object RC may be an object existing outside the vehicle, and inferring features about the object RC may be inferring a situation outside the vehicle. In this case, the sensor SC may be composed of at least one of a camera and an in-vehicle sensor, for example. Inferring the situation outside the vehicle may be, for example, inferring the attributes of objects existing outside the vehicle, inferring the congestion situation, inferring the risk of an accident, or the like. Objects existing outside the vehicle may be, for example, roads, traffic lights, obstacles (persons, objects), and the like. Inferring attributes of objects present outside the vehicle may include, for example, inferring the occurrence of an event such as a person or vehicle jumping out, starting abruptly, stopping abruptly, changing lanes, or the like.

As another example, the target object RC may be an object that exists in a specific location such as outdoors or a predetermined indoor location (for example, inside a vinyl house). It may be to infer the situation of the place of In this case, the sensor SC may be configured by at least one of a camera, a microphone, and an environment sensor, for example. As a specific example, the object RC may be a plant, and inferring the situation of a specific place may be inferring the cultivation situation of the plant.

As another example, the object RC may be, for example, an object present in a house, and inferring features about the object RC may be inferring a situation inside the house. In this case, the sensors SC may for example consist of cameras, microphones, environmental sensors and/or home security sensors.

Except for these points, the communication system according to this modification may be configured in the same manner as the above embodiment.

(Server device)
In this modification, the server device 1 may be configured to perform the inference task and transmit a retransmission instruction by the same processing procedure as in the above embodiment. That is, in step S101, the control unit 11 receives the compressed sensing data 33C from the observation device 2C. In step S102, the control unit 11 acquires the result of performing the inference task on the received sensing data 33C by using the inference model 5C. The inference model 5C may be appropriately configured to be capable of executing arithmetic processing for performing the task of inferring features appearing in given sensing data. As in the above embodiment, the inference model 5C may be configured by a trained machine learning model generated by machine learning.

In step S103, the control unit 11 calculates the likelihood of the execution result of the inference task by the inference model 5C. In step S104, the control unit 11 determines a branch destination of the process according to the calculated likelihood value. When the calculated likelihood is less than the threshold, the control unit 11 advances the process to step S105. On the other hand, when the likelihood exceeds the threshold, the control unit 11 advances the process to step S106. If the likelihood is equal to the threshold, the processing may branch to any destination.

In step S105, the control unit 11 transmits an instruction to the observation device 2C to retransmit the sensing data with a lower compression rate than the previous compression rate. After that, the control unit 11 may return the process to step S101 and repeat the series of processes in the same manner as in the above-described embodiment. In step S101, the control unit 11 acquires the sensing data 35C compressed at a compression rate lower than the previous compression rate. In step S102, the control unit 11 acquires the result of performing the inference task on the sensing data 35C by using the inference model 5C. The control unit 11 may also perform the processes after step S103 for the acquired sensing data 35C.

At step S106, the control unit 11 outputs information about the performance result of the inference task. The output destination may be, for example, the output device 16, an output device of another computer, a controlled device, an observation device 2C, or the like. In one example, the control unit 11 may directly output the result of inferring the features of the object RC. In another example, the control unit 11 may execute arbitrary information processing according to the inference result. As a specific example, in the case of inferring the health condition of the target person RC1, the control unit 11 determines that the health condition of the target person RC1 is abnormal (for example, it is determined that the probability of developing a predetermined disease is high). ), a warning may be output to inform you of this. As another specific example, in the case of inferring the state of the driver, the control unit 11 automatically sends a message prompting a driver to take a break when it is determined that the degree of drowsiness or fatigue of the driver is high. Information such as prohibiting switching from driving to manual driving may be output. As another specific example, in the case of inferring the situation outside the vehicle, the control unit 11 determines an operation command for the vehicle according to the inferred situation outside the vehicle, When the vehicle is detected to run out, the vehicle may be temporarily stopped) may be output. These specific examples may be exclusively applied when information about the inference result is output to the observation device 2C. As another specific example, in the case of inferring whether or not there is an abnormality in the industrial machine, if it is determined that the industrial machine has an abnormality or there is a sign of it, the control unit 11 notifies it. may output a warning for

(observation device)
The hardware configuration and software configuration of the observation device 2C may be the same as those of the client device 2 according to the above embodiment. In this modification, the observation device 2C may be connected to the sensor SC via a communication interface or an external interface. Alternatively, the sensor SC may be connected to another computer, and the observation device 2C may be configured to be able to acquire the sensing data 31C from the sensor SC by connecting to the other computer. The observation device 2C may incorporate the sensor SC.

The observation device 2C may perform processing for transmitting the compressed sensing data (33C, 35C) according to the same processing procedure as the client device 2 described above. That is, in step S201, the control unit of the observation device 2C acquires the sensing data 31C. In step S202, the control unit generates compressed sensing data 33C by compressing the acquired sensing data 31C. In step S<b>203 , the controller transmits the compressed sensing data 33</b>C to the server device 1 via the network 4 .

In step S<b>204 , the control unit determines a branch destination of the process depending on whether or not a retransmission instruction has been received from the server device 1 . When the retransmission instruction is received, the control unit returns the process to step S202 and repeats the processes of steps S202 and S203. In step S202, the control unit generates compressed sensing data 35C by compressing sensing data 31C again at a compression rate lower than the previous compression rate. In step S<b>203 , the control unit transmits the recompressed sensing data 35</b>C to the server device 1 via the network 4 . On the other hand, if the resend instruction is not received, the control unit terminates the procedure for data transmission.

(feature)
According to the third specific example, when sensing data is transmitted from the observation device 2C to the server device 1 and an inference task is performed on the sensing data, traffic on the network 4 is reduced while the inference model 5C performs the task. It can guarantee the accuracy of the inference task. Sensing data, like the image data and the like, has a relatively large amount of data and can be a cause of heavy traffic. Therefore, the effect can be exhibited particularly effectively.

<4.2>
In the above embodiment, the client device 2 generates the target data 35 compressed at a compression rate lower than the previous compression rate in response to receiving the retransmission instruction from the server device 1 through the process of step S204. there is However, recompressing the target data at a lower compression rate than the previous compression rate in response to the generation of the retransmission instruction in the data communication device need not be limited to such a form. As another example, the client device 2 may start generating the target data 35 compressed at a compression rate lower than the previous compression rate before receiving the retransmission instruction from the server device 1 . In this case, generation of the compressed target data 35 does not have to be completed by the time the retransmission instruction is received. Further, the process of generating the compressed target data 35 in advance may be executed in response to a preparation instruction from the server device 1 .

FIG. 12 schematically shows an example of the software configuration of the server device 1 according to this modification. In this modification, the server device 1 operates as a computer further including a preparation instructing unit 116 as a software module by executing the inference program 81 . Before transmitting the retransmission instruction, preparation instruction section 116 issues a preparation instruction to generate target data 35 with a reduced compression rate in order to prepare for the retransmission instruction when the calculated likelihood is less than the threshold. It is configured to transmit to the client device 2 .

In the processing procedure described above, the control unit 11 may transmit the preparation instruction to the client device 2 at any timing before executing the processing of step S105. As an example, assume that the client device 2 transmits a plurality of different pieces of target data to the server device 1 . In this case, the control unit 11 may transmit a preparation instruction to the client device 2 at the timing when the first compressed target data 33 is acquired from the client device 2 by the process of step S101. The client device 2 may apply the received preparation instruction to other target data to be transmitted at a later timing in addition to the target data 33 transmitted at the first time. Alternatively, the control unit 11 may transmit a preparation instruction to the client device 2 in the initial stage of operation, before the process of step S101.

In response to this, before receiving the instruction to resend the target data from the server apparatus 1, the data compression unit 212 of the client device 2 compresses data at a lower compression rate than the previous compression rate in preparation for the resend instruction. It may be configured to start execution of the process of compressing the target data 31 again. In the above processing procedure, the control unit 21 may start repeatedly executing the process of step S202 before executing the process of step S204. Then, in response to receiving the retransmission instruction from the server device 1, the control unit 21 selects repetition through the process of step S204, and transmits the generated compressed target data 35 to the server through the repetition process of step S203. It may be sent to device 1 .

When the extent to which the compression rate is lowered is determined according to the likelihood, the control unit 21 may compress the target data 31 at a plurality of candidate compression rates. When the correspondence relationship between the likelihood and the extent to which the compression rate is lowered is given by a rule, the control unit 21 may determine a plurality of candidate compression rates according to the rule. Then, after receiving the retransmission instruction from the server device 1, the control unit 21 selects the target data 35 with the corresponding compression rate from among the plurality of candidates, and transmits the selected target data 35 to the server device 1. good too.

The preparation of the compressed target data 35 in the client device 2 may be executed without depending on the preparation instruction from the server device 1 . That is, even if the server device 1 does not have the preparation instruction unit 116, the client device 2 performs processing to re-compress the target data 31 at a compression rate lower than the previous compression rate before receiving the retransmission instruction. may start executing. Each of the devices 2A-2C in each of the above embodiments may likewise generate compressed data 35A-35C with a lower compression rate than the previous compression rate before receiving the retransmission indication.

According to this modification, the compressed target data 35 is transmitted (that is, the target data is retransmitted) by causing the client device 2 to perform compression processing of the target data 31 in advance before the retransmission instruction is given. can reduce the time it takes. If the compression process is completed by the time the retransmission instruction is received, the delay caused by the compression process can be eliminated. As a result, it is possible to shorten the time required for the server device 1 to re-acquire the compressed target data and for the inference model 5 to perform the inference task on the target data again.

<4.3>
In the above embodiment, the server device 1 is provided with the inference model 5 and is configured to execute processing for performing an inference task on target data. However, the configuration of the data communication device need not be limited to such an example. The process of performing the inference task on the target data by the inference model 5 may be executed by another computer, and the data communication device may acquire the performance result of the inference task from the other computer.

FIG. 13 schematically shows an example of the configuration of a data communication device according to this modification. In this modification, a relay device 101 and an inference device 102 are provided on the server device side. The hardware configurations of the relay device 101 and the inference device 102 may be configured in the same manner as the server device 1 described above. The relay device 101 is connected to the client device 2 via the network 4 in the same manner as the server device 1 described above. Also, the relay device 101 is connected to the inference device 102 via an arbitrary network.

The relay device 101 is configured to transfer the target data (33, 35) received from the client device 2 to the inference device 102, and to execute the processing of the server device 1 other than the arithmetic processing of the inference model 5. you can The inference device 102 may include the inference model 5 and may be configured to execute processing for performing an inference task on the target data (33, 35) transferred from the relay device 101. FIG. In the processing of step S102 of the above procedure, the relay device 101 (result acquisition unit 112) transfers the compressed target data (33, 35) to the inference device 102, and the inference task by the inference model 5 from the inference device 102. You may also obtain performance results. The relay device 101 according to this modification is an example of a data communication device.

The relay device 101 may hold a retransmission instruction program, and may be configured to execute each of the above processes according to the retransmission instruction program. The retransmission instruction program may be configured by replacing the instruction for executing the arithmetic processing of the inference model 5 included in the inference program 81 with an instruction for acquiring the inference result from another computer (the inference device 102). According to this modification, the accuracy of the inference task performed by the inference model 5 can be guaranteed while reducing the traffic on the network 4 between the client device 2 and the relay device 101 .

<4.4>
In the above embodiment, the server device 1 calculates the likelihood based on the uncertainty of the performance result of the inference task obtained by the inference model 5 . However, the method of calculating the likelihood of performance results need not be limited to such an example. In another example, the inference model may be trained by machine learning to output the performance results of the inference task as well as the likelihood of the performance results. Correspondingly, computing the plausibility may comprise obtaining an output corresponding to the plausibility as a result of performing the computation of the inference model.

It should be noted that in the above-described embodiment, the arithmetic processing of the inference model may be executed in the server device 1 . In a modification of the relay device 101 and the reasoning device 102 described above, the reasoning model arithmetic processing may be executed in the reasoning device 102 . In other words, calculating the likelihood may include obtaining the result of executing the arithmetic processing of the inference model, and may not include executing the arithmetic processing of the inference model.

FIG. 14 schematically shows an example of the configuration of an inference model 500 according to this modified example. The inference model 500 according to this modification may be configured in the same manner as the inference model 5 described above, except that it includes an output layer 530 having a configuration different from that of the output layer 53 of the inference model 5 described above. The output layer 530 of the inference model 500 according to this modification is divided into two parts (531, 533). A first portion 531 includes nodes configured to output inference results. The first part 531 may be configured to operate similarly to the output layer 53 of the inference model 5 above.

The second part 533, on the other hand, contains nodes configured to output the likelihood of the inference result. In one example, the activation function for the nodes in the second portion 533 may use a softmax function, such that the nodes in the second portion 533 output likelihoods in the range [0, 1]. may be configured. The number of nodes included in the second portion 533 may be determined as appropriate according to the embodiment. In one example, the second portion 533 may be configured to include one node, which may be configured to output the likelihood of the overall inference result. As another example, in the context of the above case where the reasoning task is class identification, the second part 533 may be configured to contain a number of nodes corresponding to the number of classes to identify, each node corresponding to a corresponding It may be configured to output the likelihood of class identification results. In the example of FIG. 14, the layers of the first portion 531 and the layers of the second portion 533 are the same, but the layers of the layers of the first portion 531 and the layers of the second portion 533 may differ.

In machine learning, the correct label for the first part 531 may be configured in the same way as the inference model 5 described above. On the other hand, the correct answer label for the second portion 533 may be configured to indicate the maximum likelihood value (eg, "1" in the case of the softmax function above). Other parts may be the same as inference model 5 above. By this machine learning, the inference model 5 can be trained to output the performance results of the inference task from the first part 531 and the likelihood of the performance results from the second part 533 . The control unit 11 can acquire the plausibility of the inference result from the second part 533 in the process of step S103. This method also allows the plausibility to be properly calculated and, as a result, the accuracy of the inference task performed by the inference model 500 to be properly guaranteed.

In another example, the server device 1 may hold training data used for machine learning of the inference model 5 . A trained inference model 5 generated by machine learning acquires the ability to perform inference tasks well within the range of training data used for machine learning. Therefore, when the given target data is out of the training data distribution, the inference accuracy of the inference model 5 for the target data tends to be low, and the likelihood of the inference result tends to be low. Therefore, in the process of step S103, the control unit 11 may calculate the distance between the training data distribution and the target data, and derive the likelihood of the inference result according to the calculated distance. The control unit 11 may evaluate the plausibility lower as the calculated distance increases, and evaluate the plausibility higher as the calculated distance decreases. This method can also properly calculate the likelihood, and as a result, can properly guarantee the accuracy of the inference task performed by the inference model 5 . It should be noted that the above other method may also be employed in the case of calculating the likelihood of the performance result of the inference task by the inference models 5A-5C according to the above modifications.

<4.5>
In the above embodiments and modifications, the inference model 5 (5A-5C, 500) is composed of a fully connected neural network. However, the type of neural network that constitutes inference model 5 (5A-5C, 500) need not be limited to such an example. In another example, the inference model 5 (5A-5C, 500) may consist of a convolutional neural network, a recurrent neural network, a neural network with an attention mechanism, or the like. The neural network that makes up the inference model 5 (5A-5C, 500) may include other types of layers, such as convolution layers, pooling layers, normalization layers, dropout layers, and the like.

In addition, in the above embodiments and modifications, the inference model 5 (5A-5C, 500) uses a neural network as a machine learning model. However, the machine learning model used for inference model 5 (5A-5C, 500) is not limited to the neural network, and may be appropriately selected according to the embodiment. As another example, the inference model 5 (5A-5C, 500) can be derived from other machine learning models such as support vector machines, decision trees, multivariate analysis (e.g., Fisher discriminant analysis, canonical correlation analysis, etc.). It may be configured by a model. Models other than machine learning models may be employed as the inference model 5 (5A-5C) as long as they can perform the inference task and evaluate the plausibility of the inference results.

§5 Experimental Example (I) First Experimental Example In the first experimental example, numerical simulation was used to verify the effect of target data compression on the accuracy of the inference task. Image data was adopted as the target data, and image recognition was adopted as the reasoning task. An IoT device is assumed for the client device, and an edge server is assumed for the server device. Downsampling was adopted as the image compression method. PSNR was adopted as an index of compression rate.

FIG. 15 shows the configuration of the numerical simulation according to the first experimental example. Note that in order to calculate the PSNR, the server device was configured to use an upsampler to adjust the downsampled image to have the same resolution as the image before compression. The image data used was the ILSVRC2012 validation dataset (ImageNet). Down-sampling by the down-sampler was configured to be performed by a bicubic method using 16 pixels (4×4). The calculation formula (formula 1) of the bicubic method is shown below.

ここで、ｘは画素間の距離を示し、ａは任意の係数である。

where x indicates the distance between pixels and a is an arbitrary coefficient.

The OpenCV library was used for bicubic calculation and PSNR calculation. In addition, the inference model (image recognition model) is an image classification model pre-trained with OpenVINO (Intel Corporation, "Openvino toolkit," 2019. https://docs.openvinotoolkit.org/latest/index.html.) Adopted AlexNet.

Next, since the resolution of each image included in the verification dataset is different, the resolution of the compressed image was adjusted to match the size of the input layer of AlexNet (227×227). Specifically, the resolution of each image was adjusted by enlarging or reducing the short side of the image to 227 pixels using a resizing filter and trimming the long side to match the size of the input layer. In the numerical simulation, the image recognition task was performed by inputting the image after adjusting the resolution to the input layer of AlexNet and executing the arithmetic processing of forward propagation of AlexNet. Then, the performance results of the image recognition task and the correct label assigned to each image were compared to calculate the recognition accuracy. In the first experimental example, the recognition accuracy R _k of Top-k corresponding to the compression rate was used as an evaluation index. Let the output value of AlexNet be C ₀ and the set C _k be {C _k ⊂C ₀ |Top k in C ₀ }. In this case, the recognition accuracy R _k of Top-k can be calculated by Equation 2 below. In the first experimental example, the recognition accuracy of Top-1 and Top-5 was calculated.

FIG. 16 shows simulation results of PSNR average values and variance values according to the downsampling rate according to the first experimental example. 17 and 18 show simulation results of recognition accuracy and recognition error rate for Top-1 and Top-5 in the first experimental example. As shown in each figure, it was found that the PSNR and recognition accuracy decreased and the recognition error rate increased as the downsampling rate increased. It was found that when the compression ratio exceeded 50%, the recognition accuracy decreased significantly.

(II) Second Experimental Example In the second experimental example, a wireless network simulator was used to simulate traffic reduction by the method of the above embodiment, and throughput and delay characteristics were verified.

In the second experimental example, it is assumed that 10 IoT devices (client devices) are connected to the network, and 5 IoT devices among them are configured to transmit image data to an access point (AP). configured to The image traffic arrival interval was randomly determined based on an exponential distribution, and the average arrival interval was set to 0.2 seconds. Once the image traffic was generated, the IoT device was configured to select images from the ILSVRC2012 validation dataset and transmit the selected images. Note that the image size of the ILSVRC2012 validation dataset is smaller than the practical image size that can be obtained by shooting with a user terminal such as a smartphone. Therefore, in the second experimental example, in order to approximate the actual environment, the data size of each image is simply multiplied by 5, and the traffic is designed to continuously transmit three identical images. The IoT device was configured to compress image data using a downsampler. Also, an image classification model (inference model) was constructed using AlexNet in the edge server (server device) on the AP. The remaining IoT devices were configured to continuously transmit dummy traffic to reproduce background traffic. Each IoT device was configured to generate traffic every 0.05 seconds. The data size of traffic was set to 250 Kbytes. The time for one simulation was set to 60 seconds and the number of trials was set to 10. That is, the radio network was simulated for a total of 10 minutes.

A wireless LAN system based on IEEE802.11n was adopted as the wireless network system to be simulated. That is, CSMA/CA (Carrier Sense Multiple Access with Collision Avoidance) was adopted as a multi-access control protocol. Table 1 below shows the parameters of the wireless LAN system.

Greenfield mode was adopted for the frame format. The PHY rate was set to 300 Mbps, assuming the option of 40 MHz transmission and a short guard interval. A static communication environment was assumed and the PHY rate was fixed. The main causes of packet errors in wireless communication are packet collision of signals from IoT devices, packet loss during transmission, and transmission waiting time due to carrier sense. Therefore, in this simulator, the packet transmission timeout is assumed to be 10 ms, and the number of packet retransmissions is set to 10 times. That is, in this simulator, in addition to the retransmission control by the method of the above embodiment, packet retransmission control that is generally employed in wireless networks is prepared.

The image compression ratio in the IoT device was determined by the PSNR threshold. From the above-described first experimental example, it was found that when the compression rate exceeds 50%, the reduction in recognition accuracy becomes significant. Therefore, a value with a compression rate of around 50% is set as the PSNR threshold. Specifically, the PSNR threshold was set to 25 dB. In addition, since the correct label is assigned to the validation data set of ILSVRC2012, the recognition accuracy for each image can be calculated. Therefore, in the second experimental example, the image recognition accuracy is adopted as an index of plausibility. For the same reason that the PSNR threshold was determined, the threshold for Top-1 recognition accuracy (likelihood) was set to 29.9%, and the threshold for Top-5 recognition accuracy (likelihood) was set to 56.4. set to %. The upper limit of the number of retransmissions of an image by the method of the above embodiment is set to two. In the first retransmission of the image, the standard compression ratio is lowered by 30% from the first transmission, and in the second retransmission of the image, the original image is transmitted as it is without compression processing. It is assumed that the processing time of the edge server is 10 ms in total for recognition processing, image retransmission instruction, reception of the retransmitted image, and propagation. Thus, a communication system according to the embodiment was constructed.

On the other hand, the communication system according to the comparative example is configured to perform only packet retransmission control and not to perform retransmission control according to the method of the above embodiment. Also, in the communication system according to the comparative example, the IoT device is configured to transmit the image as it is to the edge server without compressing the image. Other conditions of the communication system according to the comparative example were the same as those of the communication system according to the embodiment.

As evaluation items, five items were adopted: recognition error rate, number of image retransmissions, throughput of dummy traffic, waiting time of dummy traffic, and waiting time of image traffic. For each item, simulations were performed for Examples (Top-1, Top-5) and Comparative Example (Conv.). Note that the throughput Th was calculated by Equation 3 below.

ここで、Ｔ₀は、基準時間である。Ｔ₀には、１００ｍｓを代入した。Ｎ_pは、ＩｏＴ装置がＡＰに正常に送信できたパケット数である。

where T ₀ is the reference time. 100 ms was substituted for T ₀ . N _p is the number of packets successfully transmitted by the IoT device to the AP.

FIGS. 19A and 19B show simulation results of recognition error rates of examples (Top-1 and Top-5) and comparative examples in the second experimental example. 20A and 20B show simulation results of the number of image retransmissions in examples (Top-1 and Top-5) in the second experimental example. FIG. 21 shows simulation results of the dummy traffic throughput of the example (Top-1, Top-5) and the comparative example in the second experimental example. 22 and 23 show simulation results (cumulative frequency distributions) of delays of dummy traffic and image traffic in the example (Top-1, Top-5) and the comparative example in the second experimental example.

As shown in FIGS. 19A and 19B, there was almost no difference in recognition error rate between the example that adopted image compression and the comparative example that did not adopt image compression. In other words, in the example, recognition accuracy equivalent to that in the case of transmitting target data without compression (comparative example) could be ensured. Also, as shown in FIGS. 20A and 20B, in the example, retransmission occurred a certain number of times. When image traffic occurs, throughput decreases due to network congestion. After the transmission of the image traffic is completed, the over-the-air traffic is transmitted at the same time, increasing the throughput of the whole network. This increases the variation in throughput. In this respect, as shown in FIG. 21, the examples (Top-1, Top-5) had more stable variations in throughput than the comparative example. Also, as shown in FIGS. 22 and 23, in the examples (Top-1 and Top-5), delays in dummy traffic and image traffic could be reduced compared to the comparative example. Note that downsampling is one of the simplest methods of image compression, and it was presumed that similar trends would be exhibited even if other compression methods were employed. From the above results, it was verified that according to the present invention, the accuracy of the inference task performed by the inference model can be guaranteed while reducing network traffic.

1 ... server device (data communication device),
11... control unit, 12... storage unit, 13... communication interface,
14 ... external interface,
15... input device, 16... output device, 17... drive,
81... inference program (retransmission instruction program), 91... storage medium,
111... data receiving unit, 112... result obtaining unit,
113... Result evaluation unit, 114... Retransmission instruction unit,
115... output section, 116... preparation instruction section,
125... Learning result data,
2... client device,
21... control unit, 22... storage unit, 23... communication interface,
24 ... external interface,
25... input device, 26... output device, 27... drive,
82... transmission program, 92... storage medium,
211 ... data acquisition unit, 212 ... data compression unit,
213 ... data transmission unit,
31 ... target data, 33 and 35 ... compressed target data,
5 ... inference model

Claims (15)

a data receiver configured to receive compressed target data from a client device;
a result acquisition unit configured to acquire a result of performing an inference task on the received target data using an inference model;
a result evaluation unit configured to calculate the likelihood of a result of performing the inference task by the inference model;
a retransmission instruction unit configured to transmit an instruction to the client device to retransmit the target data with a lower compression rate when the calculated likelihood is less than a threshold;
comprising
Data communication equipment. The retransmission instruction unit is configured to further instruct the extent to which the compression rate is to be reduced according to the calculated likelihood along with the retransmission instruction.
The data communication device according to claim 1. In order to prepare for the retransmission instruction when the calculated likelihood is less than a threshold, the client device is instructed to generate the target data with a lower compression rate before transmitting the retransmission instruction. further comprising a readiness indicator configured to transmit to
3. A data communication device according to claim 1 or 2. The inference model is composed of a trained machine learning model generated by machine learning,
4. The data communication device according to claim 1. The likelihood is calculated based on the uncertainty of the performance result of the reasoning task.
5. The data communication device according to claim 4. The inference model is trained by the machine learning to output the performance result of the inference task together with the likelihood of the performance result,
Calculating the plausibility comprises obtaining an output corresponding to the plausibility as a result of performing arithmetic processing of the inference model.
5. The data communication device according to claim 4. The target data is composed of image data,
wherein the inference task is configured by recognizing an object appearing in the image data;
A data communication device according to any one of claims 1 to 6. The image data is obtained by a surveillance camera,
recognizing an object in the image data comprises identifying a scene in the image data;
8. A data communication device according to claim 7. The target data is composed of sound data,
wherein the inference task consists of inferring features appearing in the sound data;
A data communication device according to any one of claims 1 to 6. The sound data is obtained by recording a speaker's voice,
inferring features appearing in the sound data comprises recognizing the speech of the speaker;
10. A data communication device according to claim 9. The target data is composed of sensing data,
The inference task is configured by inferring features appearing in the sensing data.
A data communication device according to any one of claims 1 to 6. a data acquisition unit configured to acquire target data;
a data compression unit configured to compress the acquired target data;
a data transmission unit configured to transmit the compressed target data to the data communication device according to any one of claims 1 to 11;
A client device comprising:
In response to the instruction to resend the target data after lowering the compression rate, the data compression unit compresses the target data again at a compression rate lower than the previous compression rate. is configured to
The data transmission unit is configured to transmit the recompressed target data to the data communication device in response to receiving an instruction to resend the target data.
client device. Before receiving an instruction to resend the target data from the data communication device, the data compression unit performs a process of compressing the target data again at a compression rate lower than the previous compression rate in preparation for the instruction. configured to start running,
13. A client device according to claim 12. the computer
receiving compressed target data from a client device;
obtaining results of performing an inference task on the received data of interest using an inference model;
calculating the likelihood of performance of the inference task by the inference model;
a step of transmitting an instruction to the client device to retransmit the target data after lowering the compression rate when the calculated likelihood is less than a threshold;
run the
Retransmission instruction method. to the computer,
receiving compressed target data from a client device;
obtaining results of performing an inference task on the received data of interest using an inference model;
calculating the likelihood of performance of the inference task by the inference model;
a step of transmitting an instruction to the client device to retransmit the target data after lowering the compression rate when the calculated likelihood is less than a threshold;
to run the
Resend instruction program.

Images