JP2021089582A - Information processing device, relay device, system, information processing device control method, relay device control method, and program - Google Patents

Abstract

To suppress the decrease in inference accuracy due to a learnt model even if the resource state of a terminal changes dynamically.SOLUTION: An information processing device includes reception means for receiving a resource state of a terminal together with a request of a learnt model from the terminal, control means for controlling compression of the learnt model that is a target of the request with a compression degree according to the resource state, and transmission means for transmitting the compressed learnt model to the terminal.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, a relay device, a system, a control method of the information processing device, a control method of the relay device, and a program.

Patent Document 1 proposes a technique of compressing image data on a screen and transmitting it to a client terminal. In the technique of Patent Document 1, the server terminal compresses image data in at least a part of a display image including a plurality of types of graphical user interface (GUI) components on the screen and transmits the image data to the client terminal.

Japanese Unexamined Patent Publication No. 2010-55280

For example, assume a case where an edge terminal (terminal) performs inference processing using a learned model acquired from a server. In general, edge terminals do not have abundant hardware resources, so it is difficult to hold many trained models. Therefore, it is conceivable to compress the trained model and hold the compressed trained model in the edge terminal. However, when the trained model is compressed, the inference accuracy using the trained model decreases, so it is desirable that the degree of compression of the trained model is low. On the other hand, the resource state of the hardware of the edge terminal changes dynamically. Therefore, it is difficult to control the degree of compression of the trained model according to the change in the resource state of the edge terminal, it is difficult to use the trained model compressed to an appropriate degree, and the inference accuracy is lowered. There is.

An object of the present invention is to suppress a decrease in inference accuracy due to a trained model even if the resource state of the terminal changes dynamically.

In order to achieve the above object, the information processing apparatus of the present invention comprises a receiving means for receiving the resource state of the terminal together with the request of the learned model from the terminal, and the target of the request with a compression degree according to the resource state. It is characterized by including a control means for controlling compression of the trained model, and a transmission means for transmitting the compressed trained model to the terminal.

According to the present invention, even if the resource state of the terminal changes dynamically, it is possible to suppress a decrease in inference accuracy due to the trained model.

It is a figure which shows the configuration example of the system of this embodiment. It is a block diagram which shows an example of the configuration example of an information processing apparatus. It is a block diagram which shows an example of the configuration example of an image pickup apparatus. It is a table showing the correspondence between the resource state and the compression degree of the trained model. It is a table which shows an example of management of a trained model. It is a flowchart which shows the process flow of the image pickup apparatus. It is a flowchart which shows an example of the processing flow of an information processing apparatus. It is a flowchart which shows an example of the generation process of a trained model. It is a figure which shows the configuration example of the system of the modification example.

Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in the respective embodiments.

Hereinafter, the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of the system 100 of the present embodiment. In the system 100, the image pickup apparatus 101 and the information processing apparatus 102 are connected to each other via networks 106 and 107. Networks 106 and 107 are networks capable of wired communication or wireless communication. In the present embodiment, the image pickup device 101 is applied as an edge terminal (terminal), but as the edge terminal, for example, any terminal such as a smartphone or a tablet terminal may be applied. The information processing device 102 holds or caches (saves) a plurality of trained models 103. The trained model 103 is a compressed trained model or an uncompressed trained model (original trained model). The information processing device 102 selects an arbitrary trained model from the plurality of trained models 103, and compresses the selected trained model. Then, the information processing apparatus 102 compresses the selected trained model to generate the compressed trained model 105. The compressed trained model 105 is referred to as the trained model A'in FIG. The information processing device 102 is, for example, a cloud server or the like.

The trained model is a model obtained by arbitrary machine learning. The trained model may be, for example, a trained neural network whose parameters have been adjusted by an error back propagation method or the like. The trained model may be a model other than the neural network. For example, the trained model may be a model trained by a support vector machine.

The image pickup apparatus 101 transmits a request for the trained model and its own resource state to the information processing apparatus 102 via the network 106. The resource state is information indicating the load of the image pickup device 101 that dynamically changes according to the processing state of the image pickup device 101, such as the CPU usage rate, the memory usage rate, the housing temperature, and the battery capacity of the image pickup device 101. .. The resource state may be an index that combines the CPU usage rate, the memory usage rate, the housing temperature, and the battery capacity. Further, the resource state may include performances and functions such as the original memory capacity and processing specifications of the image pickup apparatus 101.

The information processing device 102 determines the degree of compression of the trained model 104 that is the target of the request from the request and the resource state received from the image pickup device 101. The trained model 104 that is the target of the request is referred to as the trained model A'in FIG. When the information processing device 102 caches the learned model of the compression degree according to the request from the image pickup device 101 and the resource state, the information processing device 102 transmits the cached learned model to the image pickup device 101 via the network 107. To do. On the other hand, when the information processing apparatus 102 does not cache the trained model of the compression degree, the information processing apparatus 102 compresses the trained model of the request target with the determined compression degree. Then, the information processing device 102 generates and caches the compressed learned model 105. Further, the information processing device 102 transmits the generated compressed learned model to the image pickup device 101 via the network 107. The networks 106 and 107 may be an integrated network or a separate network. Further, the networks 106 and 107 may be wireless networks.

As described above, the information processing device 102 compresses the trained model with the degree of compression determined from the resource state of the image pickup device 101. Although the resource state of the image pickup apparatus 101 changes dynamically, the information processing apparatus 102 compresses the trained model with the lowest possible compression degree in response to the request from the image pickup apparatus 101 and the resource state. Then, the image pickup apparatus 101 can perform inference using the compressed trained model. As a result, it is possible to suppress a decrease in accuracy when the image pickup device 101 makes an inference without impairing the original function of the image pickup device 101.

FIG. 2 is a block diagram showing an example of a configuration example of the information processing device 102. The information processing device 102 includes a CRT display 201, a VRAM 202, a BMU 203, a keyboard 204, a PD 205, a CPU 206, a ROM 207, a RAM 208, and an HDD 209. Further, the information processing device 102 has a flexible disk 210, a network I / F 211, and a bus 212. The configuration of the information processing device 102 is not limited to the example of FIG. The CRT (Cathode Ray Tube) display (CRT display 201) displays icons, messages, menus, and other user interface information. These interface information is managed by the information processing device 102.

An image to be displayed on the CRT display 201 is drawn on the VRAM 202. The image data generated in the VRAM 202 is transferred to the CRT display 201 according to a predetermined specification. As a result, the image is displayed on the CRT display 201. The BMU (bit move unit) 203 is used, for example, for data transfer between memories (for example, between VRAM 202 and another memory) and data between the memory and each I / O device (for example, network I / F211). Control the transfer. The keyboard 204 has various keys for inputting characters and the like. The PD (pointing device) 205 is used, for example, for an operation such as instructing an icon, a menu or other contents displayed on the CRT display 201, or dragging and dropping an object.

The CPU 206 controls each device based on a control program such as an OS or a program to be performed later, which is stored in the ROM 207, the HDD 209, or the flexible disk 210. The CPU 206 corresponds to the control means. The ROM 207 stores various control programs and data. The RAM 208 has a work area of the CPU 206, a data save area at the time of error processing, a control program load area, and the like. The processing of the present embodiment may be realized by the CPU 206 executing the control program loaded in the RAM 208. The HDD 209 stores data such as each control program executed in the information processing apparatus 102 and temporarily stored data. Each of the above-described trained models may be cached in, for example, the cache memory of the RAM 207, the HDD 209, or the CPU 206.

The network I / F 111 is connected to the networks 106 and 107 and communicates with the image pickup apparatus 101. The network I / F 111 corresponds to a receiving means and a transmitting means. The network I / F 111 can also communicate with other information processing devices, printers, and the like via the network. Bus 212 includes an address bus, a data bus and a control bus. The control program for the CPU 206 may be provided from the ROM 207, the HDD 209, or the flexible disk 210, or may be provided from another server or the like via the network via the network I / F211.

Next, the image pickup apparatus 101 will be described. FIG. 3 is a block diagram showing an example of a configuration example of the image pickup apparatus 300. The image pickup device 300 of FIG. 3 is the image pickup device 101 of FIG. The image pickup device 300 may be a mobile terminal having an image pickup function. The imaging device 300 includes a photographing lens 301 including a zoom lens and a focus lens, a shutter 302 having an aperture function, and an imaging unit 303 composed of a CCD, a CMOS element, or the like that converts an optical image into an electric signal. The A / D converter 304 converts the analog signal output from the imaging unit 303 into a digital signal.

By covering the photographing lens 301 and the like, the barrier 305 prevents the imaging system including the photographing lens 301, the shutter 302, the imaging unit 303 and the like from being soiled or damaged. The image processing unit 306 performs predetermined image processing (resizing processing such as pixel interpolation and reduction, color conversion processing, etc.) on the image data output by the A / D converter 304 or the image data acquired from the memory control unit 307. Give. In addition, the image processing unit 306 performs predetermined arithmetic processing using the captured image data. Then, the system control unit 308 performs exposure control, distance measurement control, and the like based on the calculation result. The image data output by the A / D converter 304 is written to the memory 309 via the image processing unit 306 and the memory control unit 307, or only via the memory control unit 307.

The system control unit 308 controls the entire image pickup apparatus 101. The system control unit 308 realizes the processing of the image pickup apparatus 101 by executing the program stored in the non-volatile memory 312. The memory 309 stores the image data obtained by the image pickup unit 303 and converted into digital data by the A / D converter 304, and the image data to be displayed on the display unit 310. The memory 309 has a storage capacity sufficient to store a predetermined number of still image data, moving image data for a predetermined time, audio data, and the like. Further, the memory 309 also serves as a memory (video memory) for displaying an image.

The D / A converter 311 converts the image data for image display stored in the memory 309 into an analog signal and supplies it to the display unit 310. As a result, the image data for display written in the memory 309 is displayed on the display unit 310 via the D / A converter 311. The display unit 310 displays on a display such as an LCD according to the analog signal from the D / A converter 311. The non-volatile memory 312 is a memory that can be electrically erased and recorded, and for example, EEPROM or the like is used. The non-volatile memory 312 stores constants for operation of the system control unit 308, control programs such as a program for cooperation described later, and the like. The system memory 313 is, for example, a RAM. In the system memory 313, constants and variables for the operation of the system control unit 308, a program read from the non-volatile memory 312, and the like are expanded.

The image pickup apparatus 300 includes a mode changeover switch 314, a first shutter switch 315, a second shutter switch 316, and an operation unit 317. The mode changeover switch 314, the first shutter switch 315, the second shutter switch 316, and the operation unit 317 are operation units for inputting various operation instructions to the system control unit 308. The mode changeover switch 314 switches the operation mode of the system control unit 308 to any one of various modes such as a still image recording mode, a moving image recording mode, and a playback mode. The first shutter switch 315 is turned on by a so-called half-press (shooting preparation instruction) during the operation of the shutter button provided on the image pickup apparatus 101, and the first shutter switch signal SW1 is generated. The second shutter switch 316 is turned on when the operation of the shutter button is completed, so-called full pressing (shooting instruction), and the second shutter switch signal SW2 is generated. When the second shutter switch signal SW2 is generated, the system control unit 308 starts a series of shooting processes from reading the signal from the imaging unit 303 to writing the image data to the recording medium 325.

Each operation member of the operation unit 317 is assigned a function as appropriate for each scene by selecting and operating various function icons displayed on the display unit 310, and acts as various function buttons. The function buttons include, for example, a confirmation button, an end button, a back button, an image feed button, a jump button, a narrowing down button, an attribute change button, and the like. For example, when the menu button is pressed, various settable menu screens are displayed on the display unit 310. The user can intuitively make various settings by using the menu screen displayed on the display unit 310, the four-way button, the SET button, and the like.

The controller wheel 318 is a rotation-operable operation member included in the operation unit 317, and is used when instructing a selection item together with a direction button. The system control unit 308 controls each unit of the image pickup apparatus 101 based on the pulse signal. In addition, the system control unit 308 can determine the angle at which the controller wheel 318 is rotated, the number of rotations, and the like based on the pulse signal. Any member may be used for the controller wheel 318 as long as it is an operating member capable of detecting the rotation operation. The controller ring 319 is a rotation operation member included in the operation unit 317. The controller ring 319 can be operated to rotate around the lens barrel around the optical axis. For example, by operating the controller ring 319, an electric pulse signal corresponding to the amount of rotation (operation amount) is generated. The system control unit 308 controls each unit of the image pickup apparatus 101 based on the pulse signal. Further, when the function switching button of the controller ring 319 included in the operation unit 317 is pressed, a menu screen on which the function assigned to the controller ring 319 can be changed is displayed on the display unit 310. The controller ring 319 and controller wheel 318 are used to select normal mode items and change their values.

The power switch 320 is a switch for switching between power on and power off of the image pickup apparatus 300. The power supply control unit 321 includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like. The power supply control unit 321 detects whether or not a battery is installed, the type of battery, the remaining battery level, and the like. Further, the power supply control unit 321 controls the DC-DC converter based on the detection result and the instruction of the system control unit 308, and supplies a necessary voltage to each unit including the recording medium 325 for a necessary period. The power supply unit 322 may be a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter or the like. The communication unit 323 transmits the image stored in the recording medium 325 to the external device. The interface 324 is an interface for controlling the recording medium 325. The recording medium 325 is composed of a semiconductor memory, a magnetic disk, or the like. The network I / F 326 is controlled by the system control unit 308 to communicate via the networks 106 and 107. The network I / F326 corresponds to the request means. The image pickup apparatus 300 can communicate with the information processing apparatus 102 via the networks 106 and 107.

Next, the determination of the degree of compression of the trained model will be described. FIG. 4 is a table showing the correspondence between the CPU usage rate of the image pickup apparatus 300 and the degree of compression of the trained model. The table shown in FIG. 4 is held by the information processing device 102. As the compression method of the trained model, for example, any model weight reduction technique such as quantization, pruning, weight sharing, and distillation can be applied. Quantization is a technique for reducing the number of bits in a parameter. Pruning is a technique for reducing less important parameters. Weight sharing is a method of sharing parameter weighting factors.

The table of FIG. 4 contains items 401 and 402. Item 401 shows the CPU usage rate as the resource state of the image pickup apparatus 300. Item 401 may be, for example, a memory usage rate or the like. Item 402 indicates the degree of compression of the trained model, which is determined according to the CPU usage rate. The trained model with a compression degree of 32 bits is the original trained model. As the degree of compression of the trained model increases, the number of bits decreases and the compression rate increases. As the degree of compression of the trained model increases, the capacity of the trained model decreases and the inference accuracy decreases. Each value of CPU utilization indicates a threshold value that determines the degree of compression of the trained model. For example, when the CPU usage rate exceeds 60%, the trained model that can be used by the image pickup apparatus 300 is quantized to a parameter of 8 bits or less.

FIG. 5 is a table showing an example of management of the trained model. The table shown in FIG. 5 is held by the information processing device 102. The information processing device 102 holds the original trained model and also caches the compressed trained model. That is, the information processing device 102 manages the trained models of the same type and different degrees of compression. In the table of FIG. 5, item 501 represents what kind of subject the trained model managed by the information processing apparatus 102 corresponds to.

For example, item 501 "flower" corresponds to a trained model for flower composition assistance. The trained model for assisting the composition of a flower is a trained model that makes inferences for finding the optimum composition when photographing a flower as a subject. Further, in the example of FIG. 5, a plurality of trained models having different compression degrees for "persons" are managed as subjects of the same type. For example, the information processing device 102 may generate a trained model for each subject by performing machine learning with the subject recognized from the image as an input and the optimum composition when shooting the subject as correct answer data. Good.

Item 502 represents the degree of compression of the trained model for each subject. Item 503 represents the version of the trained model and is used to manage the update status of the trained model. Here, when the original trained model itself is updated, trained models of the same type but different compression degrees may be updated or discarded at once.

Next, the operation of the image pickup apparatus 300 will be described. FIG. 6 is a flowchart showing a processing flow of the image pickup apparatus 300. In the present embodiment, the image pickup apparatus 300 applies a learned model for composition assist according to the type of the subject recognized at the time of shooting. The composition assist function of the present embodiment is merely an example, and is not limited to the embodiment of the present embodiment. In step S601, the image pickup apparatus 300 receives an instruction from the user to start shooting and starts shooting. The system control unit 308 determines whether the instruction to start shooting has been received in response to the activation of the imaging device 30 by pressing the power switch 320 or the generation of the first shutter switch signal SW1 by pressing the first shutter switch 315. You may.

In step S602, the system control unit 308 recognizes the subject from the live view image being captured by the user. The subject recognition may be performed by inference using a trained model for subject recognition, or another subject recognition method may be used. In step S603, the system control unit 308 determines whether the recognition subject has changed or the resource state of the image pickup apparatus 300 has changed. The resource state is, for example, the usage rate of the CPU, memory, etc. of the image pickup apparatus 300. The system control unit 308 holds, for example, information indicating the resource status. Here, regarding the change in the resource state, the system control unit 308 may perform the determination process in step S603 based on the information held in step S605 described later. The system control unit 308 may determine whether or not the resource state has changed based on whether or not processing that is expected to change the resource state, such as switching the shooting mode by pressing the mode changeover switch 314, has been performed.

When the system control unit 308 determines that the recognition subject has changed or the resource state has changed, the system control unit 308 proceeds to the process in step S604. In step S604, the system control unit 308 controls to transmit the request for acquiring the trained model and the current resource state and the resource state at the time of the previous request transmission to the information processing device 102. The request here is a trained model for composition assist for shooting a recognized subject. The network I / F 326 transmits the current resource status and the resource information at the time of the previous request transmission to the information processing apparatus 102 via the network 106 together with the request. In step S605, the system control unit 308 holds information on the resource state of the current image pickup apparatus 300. The retained resource state information can be used to determine a change in the resource state in step S603. Further, the retained resource state information can also be used as the resource state when the previous request is transmitted when the next request is transmitted in step S604.

The network I / F 326 receives the trained model compressed according to the resource state of the image pickup apparatus 300 from the information processing apparatus 102 via the network 107. In step S606, the system control unit 308 applies the trained model received from the information processing device 102. In step S603, when the system control unit 308 determines that the recognition subject has not changed and the resource state has not changed, the system control unit 308 proceeds to the process in step S607. Further, after the process of step S606, the system control unit 308 advances the process to step S607. In step S607, the system control unit 308 executes inference for composition assist using the trained model applied to the image pickup apparatus 300.

When the system control unit 308 inputs the subject recognized in step S602 into the previously applied trained model or the newly applied trained model, the optimum composition for the recognized subject is inferred as a result. can get. In step S608, the system control unit 308 executes composition assist according to the inference result using the trained model. Inference using the trained model may be performed by, for example, a GPU (graphics processing unit) or the like.

In step S609, the system control unit 308 determines whether to continue shooting. If the system control unit 308 determines in step S609 to continue shooting, the process returns to step S602. In step S609, when the system control unit 308 determines that the shooting is not continued, the system control unit 308 ends the process. The system control unit 308 may determine whether to continue shooting based on whether a certain period of time has elapsed after the execution of composition assist is completed. Further, the system control unit 308 determines whether to continue shooting, whether the imaging device 300 is stopped by pressing the power switch 320, or whether the second shutter switch signal SW2 is generated by pressing the second shutter switch 316, and the like. , May be judged as a reference.

As described above, when the recognition subject changes or the resource state changes, the imaging device 300 transmits a request for the trained model to the information processing device 102 together with the resource state information. The information processing device 102 transmits the learned model compressed to the degree of compression according to the resource state of the image pickup device 300 to the image pickup device 300. Then, the image pickup apparatus 300 switches the trained model to be applied. As a result, the imaging device 300 does not need to hold many types of trained models, and can make inferences according to the resource state.

Next, the operation of the information processing device 102 will be described. FIG. 7 is a flowchart showing an example of the processing flow of the information processing apparatus 102. In step S701, the CPU 206 of the information processing device 102 receives the current resource state of the image pickup device 300 and the resource state at the time of the previous request together with the request of the learned model transmitted by the image pickup device 300. In step S702, the CPU 206 determines the limit compression degree applicable to the image pickup apparatus 300 from the current resource state of the image pickup apparatus 300. In the present embodiment, the CPU 206 determines the degree of compression (item 402) corresponding to the current resource state (item 401) of the image pickup apparatus 300 with reference to the table shown in FIG. The CPU 206 may determine the degree of compression according to the resource state by any method.

In step S703, the CPU 206 determines whether the learned model of the degree of compression determined in step S702 is cached in the information processing apparatus 102. When the CPU 206 determines that the trained model of the degree of compression determined in step S702 is not cached, the CPU 206 advances the process to step S704. In step S704, the CPU 206 generates a trained model of the degree of compression determined in step S702. The detailed processing related to the generation of the trained model in step S704 will be described later. Here, it is assumed that the information processing apparatus 102 caches a trained model of the same type as the trained model to be generated and having a compression degree higher than a predetermined degree. In this case, the CPU 206 may control the transmission of the trained model to the image pickup apparatus 300 and apply it to the image pickup apparatus 300 before generating the trained model of the degree of compression determined in step S703. As a result, it is possible to reduce the waiting time until the compressed trained model (trained model for composition assist) is generated.

If the CPU 206 determines in step S703 that the learned model of the degree of compression determined in step S702 is cached in the information processing apparatus 102, the CPU 206 advances the process to step S705. In step S705, the CPU 206 controls to transmit the trained model of the target to the image pickup apparatus 300. The target trained model is the trained model generated in step S704 or the trained model cached in the information processing device 102. The target trained model is a trained model compressed according to the resource state of the image pickup apparatus 300. The trained model of the target is transmitted from the information processing device 102 to the image pickup device 300 via the network 107.

In step S706, the CPU 206 compares the current resource state of the image pickup apparatus 300 with the resource state at the time of the previous request (the resource state of the image pickup apparatus 300 when the request was transmitted last time). In step S707, the CPU 206 predicts a future change in the resource state of the image pickup apparatus 300 from the comparison result in step S706. For example, from the history of past changes in the resource state of the imaging device 300, when the CPU usage rate as the resource state is continuously increasing, it can be expected that the CPU usage rate of the imaging device 300 will continue to increase in the future. Further, if the CPU usage rate of the image pickup device 300 at the time of the previous request has increased significantly (when the increase exceeds a predetermined amount), it can be expected that the CPU usage rate of the image pickup device 300 will continue to increase in the future. .. In step S707, the CPU 206 determines whether it has predicted that the trained model currently applied to the imaging device 300 needs to be recompressed. When the CPU 206 predicts that the trained model does not need to be recompressed, the process ends.

When the CPU 206 predicts that the trained model needs to be recompressed, the process proceeds to step S708. In step S708, the CPU 206 determines whether the learned model of the degree of compression determined in step S702 is cached in the information processing apparatus 102. When the CPU 206 determines that the trained model of the determined compression degree is cached, the process ends. On the other hand, when the CPU 206 determines that the trained model of the determined compression degree is not cached, the CPU 206 advances the process to step S709. In step S709, the CPU 206 generates a trained model in the same manner as in step S704. The generated trained model is cached. Then, the process ends.

As described above, the CPU 206 predicts the need for compression of the trained model based on the current resource state of the imaging device 300 and the resource state of the imaging device 300 at the time of the previous request. The CPU 206 holds the resource state of the image pickup apparatus 300 before the previous request, and may predict the necessity of compression of the trained model based on each held resource state. For example, as described above, if the CPU usage is continuously increasing, it is predicted that compression of the trained model is necessary. Therefore, the information processing apparatus 102 can pre-compress and hold the composition assist trained model in accordance with the prediction of the change in the resource state of the imaging apparatus later. Further, the information processing apparatus 102 determines the degree of compression of the trained model in step S702, and then compresses and caches the trained models other than the requested trained model with the degree of compression determined in step S702. May be good. As a result, the information processing device 102 can pre-compress and hold the learned model for composition assist other than the requested subject.

Next, the generation of the trained model in step S704 and step S708 will be described. FIG. 8 is a flowchart showing an example of the trained model generation process. In step S801, the CPU 206 determines whether the trained model of the degree of compression to be requested requires higher compression than the cached trained model. If the CPU 206 determines in step S801 that the trained model needs to be compressed to a higher degree than the currently cached trained model, the process proceeds to step S802. In step S802, the CPU 206 recompresses the compressed trained model currently cached in the information processing apparatus 102 to a degree of compression determined in step S702. This will generate a recompressed trained model.

If the CPU 206 determines in step S801 that it is not necessary to compress the trained model to a higher degree than the currently cached trained model, the process proceeds to step S803. In step S803, the CPU 206 compresses the uncompressed original trained model to the degree of compression determined in step S702. This will generate a compressed trained model. In step S804, the CPU 206 caches the trained model generated in step S802 or step S803. As a result, the process of step S704 or step S709 ends.

As described above, the information processing device 102 receives the resource state together with the request for the trained model from the image pickup device 300. Then, the information processing device 102 compresses the trained model to be requested by the degree of compression according to the resource state of the image pickup device 300, and transmits the compressed learned model to the image pickup device 300. As a result, even if the resource state of the image pickup apparatus 300 changes dynamically, the image pickup apparatus 300 can make inferences using the trained model corresponding to the dynamic change. As a result, it is possible to suppress a decrease in the inference accuracy of the trained model. In the above-described embodiment, an example in which the trained model for composition assist for each subject is applied as the trained model has been described, but any trained model may be applied. For example, the trained model of the present embodiment may be a trained model that recognizes a subject.

Further, the information processing device 102 may transmit information indicating the degree of compression of the trained model to be transmitted to the image pickup device 300 together with the compressed trained model. At this time, the system control unit 308 of the image pickup apparatus 300 may display information indicating the degree of compression together with the current resource state on the display unit 310. For example, the display unit 310 may display information indicating that the CPU usage rate is 65% as the current resource state and information indicating that the information indicating the degree of compression is 8 bits. Thereby, it is possible to present to the user of the image pickup apparatus 300 how much the inference accuracy of the trained model is.

Next, a modified example will be described. FIG. 9 is a diagram showing a configuration example of the system 900 in the modified example. In the system 900, the image pickup device 101 and the relay device 901 are connected via networks 902 and 905. The relay device 901 and the information processing device 102 are connected to each other via networks 903 and 904. The information processing device 102 holds a plurality of original trained models 103. The relay device 901 carries out the processes shown in FIGS. 7 and 8. As the configuration of the relay device 901, the configuration of FIG. 2 can be adopted. The relay device 901 acquires the original trained model 104 from the information processing device 102 via the networks 903 and 904 in step S803 of FIG. Then, the relay device 901 compresses the acquired original trained model with the degree of compression determined in step S702 of FIG. 7. That is, the relay device 901 is responsible for most of the processing of the information processing device 102, and the communication performed by the information processing device 102 is only the communication with the relay device 901 (communication related to the original learned model). As a result, the number of communications of the information processing device 102 can be reduced. For example, when the information processing device 102 is a cloud server as described above, the processing load of the cloud server can be reduced. The above-mentioned information indicating the current resource state and the degree of compression may be displayed on the screen of the relay device 901.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and modifications can be made within the scope of the gist thereof. The present invention supplies a program that realizes one or more functions of each of the above-described embodiments to a system or device via a network or storage medium, and one or more processors of the computer of the system or device implements the program. It can also be realized by the process of reading and executing. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 System 101 Imaging device 102 Information processing device 206 CPU
211 Network I / F
308 System Control Unit 326 Network I / F
901 relay device

Claims (16)

A receiving means for receiving the resource status of the terminal together with the request of the trained model from the terminal, and
A control means that controls compression of the trained model that is the target of the request with a compression degree according to the resource state, and
A transmission means for transmitting the compressed trained model to the terminal, and
An information processing device characterized by being equipped with. The information processing apparatus according to claim 1, wherein the control means caches the compressed trained model. When the trained model compressed with the degree of compression according to the resource state is cached, the control means controls to transmit the cached trained model without performing the compression control. The information processing apparatus according to claim 2. The information processing apparatus according to any one of claims 1 to 3, wherein the control means caches a plurality of trained models of the same type and different degrees of compression. The terminal is an imaging device and
The receiving means according to any one of claims 1 to 4, wherein the receiving means receives the resource state together with the request when the subject recognized by the image pickup apparatus changes or the resource state of the image pickup apparatus changes. The information processing device according to any one of the items. When a trained model of the same type as the trained model to be the target of the request and having a compression degree higher than a predetermined degree is cached, the control means sets the trained model higher than the predetermined degree as the target of the request. The information processing apparatus according to any one of claims 1 to 5, wherein the terminal is controlled to transmit the trained model before the transmission of the trained model. The control means predicts the resource state of the terminal based on the resource state of the terminal received together with the request and the resource state of the terminal when the request was last transmitted, and sets the predicted resource state. The information processing apparatus according to any one of claims 1 to 6, wherein it determines whether to recompress the compressed trained model based on the method. The information processing apparatus according to claim 7, wherein the control means caches the recompressed learned model when it is determined to perform the recompression. When the control means controls to compress the trained model that is the target of the request, it controls to compress the trained models other than the trained model that is the target of the request with the same degree of compression. The information processing apparatus according to any one of claims 1 to 8. The information processing apparatus according to any one of claims 1 to 9, wherein compression of the trained model is performed by quantization of parameters. A receiving means that receives the resource state of the terminal together with the request of the trained model from the terminal and receives the original trained model from the information processing device.
A control means that controls compression of the original learned model that is the target of the request received from the terminal and received from the information processing device with a compression degree according to the resource state.
A transmission means for transmitting the compressed trained model to the terminal, and
A relay device characterized by being provided with. A system equipped with a terminal and an information processing device.
The terminal
A request means for transmitting a resource state to the information processing apparatus together with a request for a trained model is provided.
The information processing device
A receiving means for receiving the request and the resource state, and
A control means that controls compression of the trained model that is the target of the request with a compression degree according to the resource state, and
A transmission means for transmitting the compressed trained model to the terminal, and
A system characterized by being equipped with. The process of receiving the resource status of the terminal together with the request of the trained model from the terminal,
A process of controlling compression of the trained model to be the target of the request with a compression degree according to the resource state, and a step of performing compression.
The process of transmitting the compressed trained model to the terminal, and
A control method for an information processing device, which comprises. The process of receiving the resource status of the terminal together with the request of the trained model from the terminal and receiving the original trained model from the information processing device.
A step of controlling compression of the original learned model that is the target of the request received from the terminal and received from the information processing device at a compression degree according to the resource state.
The process of transmitting the compressed trained model to the terminal, and
A method for controlling a relay device, which comprises. A program for causing a computer to execute each means of the information processing apparatus according to any one of claims 1 to 10. A program for causing a computer to execute each means of the relay device according to claim 11.

Images