CN114781640A

CN114781640A - Model deployment method, system, storage medium and electronic device

Info

Publication number: CN114781640A
Application number: CN202210677363.3A
Authority: CN
Inventors: 商磊; 李瑞晨; 王立威; 孙佰贵; 李�昊
Original assignee: Alibaba Damo Institute Hangzhou Technology Co Ltd
Current assignee: Alibaba Damo Institute Hangzhou Technology Co Ltd
Priority date: 2022-06-16
Filing date: 2022-06-16
Publication date: 2022-07-22
Anticipated expiration: 2042-06-16
Also published as: CN114781640B

Abstract

The application discloses a model deployment method, a model deployment system, a storage medium and electronic equipment. Wherein, the method comprises the following steps: the method comprises the steps of obtaining an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area of the plurality of areas; based on the target remote sensing image, determining a pruning proportion corresponding to the original model; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; and deploying the target model to the target device. The method and the device solve the technical problem that the efficiency is low when the model runs due to the fact that the parameter quantity of the model is large in the related technology.

Description

Model deployment method, system, storage medium and electronic device

Technical Field

The present application relates to the field of model deployment, and in particular, to a model deployment method, system, storage medium, and electronic device.

Background

At present, neural networks are widely applied to the aspects of computer vision, natural language processing, reinforcement learning and the like. The neural network is used for carrying out applications such as ground feature classification, change detection and the like in the field of remote sensing. However, the better the model expression ability is, the larger the number of neural network parameters corresponding to the model expression ability is. The huge parameters cause that the network models are usually operated very slowly when actually deployed, and the algorithm processing speed is influenced.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the application provides a model deployment method, a model deployment system, a storage medium and electronic equipment, so as to at least solve the technical problem that the efficiency of a model in operation is low due to large parameter quantity of the model in the related technology.

According to an aspect of an embodiment of the present application, there is provided a model deployment method, including: acquiring an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas, the target remote sensing image is the remote sensing image acquired in a target area corresponding to the target equipment, and the target area is a partial area in the plurality of areas; determining a pruning proportion corresponding to the original model based on the target remote sensing image; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; and deploying the target model to the target device.

According to an aspect of an embodiment of the present application, there is provided a model deployment method, including: responding to an input instruction acting on an operation interface, and displaying a target remote sensing image on the operation interface, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area in a plurality of areas; responding to a deployment instruction acting on an operation interface, and displaying a deployment result on the operation interface, wherein the deployment result is used for representing a result of deploying a target model to target equipment, the target model is obtained by carrying out pruning training on an original model on a target remote sensing image according to a pruning proportion, the original model is obtained by training remote sensing images collected in a plurality of areas, and the pruning proportion is determined based on the target remote sensing image.

According to an aspect of an embodiment of the present application, there is provided a model deployment method, including: the cloud server receives a target remote sensing image sent by the client, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area of a plurality of areas; the method comprises the steps that a cloud server obtains an original model to be deployed to target equipment, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas; the cloud server determines a pruning proportion corresponding to the original model based on the target remote sensing image; the cloud server carries out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; the cloud server deploys the target model to the target device.

According to an aspect of an embodiment of the present application, there is provided a model deployment method, including: displaying a target remote sensing image on a display picture of Virtual Reality (VR) equipment or Augmented Reality (AR) equipment, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in a plurality of areas; acquiring an original model to be deployed to target equipment, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas; based on the target remote sensing image, determining a pruning proportion corresponding to the original model; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; deploying the target model to target equipment to obtain a deployment result; and driving the VR device or the AR device to display the deployment result on the presentation screen.

According to an aspect of an embodiment of the present application, there is provided a model deployment system including: the target equipment is used for deploying an original model, wherein the original model is obtained by training remote sensing images collected in a plurality of areas; the client is used for sending a target remote sensing image, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in the plurality of areas; and the cloud server is connected with the client and the target equipment and used for determining a pruning proportion corresponding to the original model based on the target remote sensing image, performing pruning training on the original model on the target remote sensing image based on the pruning proportion to obtain a target model, and deploying the target model to the target equipment.

According to an aspect of the embodiments of the present application, there is also provided a computer-readable storage medium including a stored program, where the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform any one of the methods described above.

According to an aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory for executing a program stored in the memory, and a processor for executing the method of any one of the above when the program is executed.

In the embodiment of the application, an original model to be deployed to target equipment and a target remote sensing image are obtained firstly, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, the target area is a partial area in the plurality of areas, and a pruning proportion corresponding to the original model is determined based on the target remote sensing image; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; the target model is deployed to the target equipment, and the purpose that the target model with higher running speed is obtained by compressing the initial model through pruning training is achieved.

It is easy to note that the initial model is obtained by training according to the remote sensing images acquired in the plurality of areas, however, when the model is actually deployed in the corresponding area, a large number of redundant parameters exist in the model, in the application, the pruning proportion corresponding to the original model can be determined according to the target remote sensing image corresponding to the target area which is actually deployed, and as the pruning proportion is specific to the target area, the target model obtained by pruning and training the original model according to the pruning proportion can be specific to the remote sensing image in the target area, that is, when the pruned target model is deployed when the target device is running, the running speed can be increased on the premise of ensuring the running accuracy, and the technical problem that the running efficiency of the model is low due to the large parameter quantity of the model in the related technology is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a block diagram of a hardware structure of a computer terminal (or mobile device) for implementing a model deployment method according to an embodiment of the present application;

FIG. 2 is a flow chart of a model deployment method according to embodiment 1 of the present application;

FIG. 3 is a parameter value diagram of an original model without pruning according to embodiment 1 of the present application;

fig. 4 is a parameter quantity diagram of an original model in a pruning process according to embodiment 1 of the present application;

FIG. 5 is a parameter diagram of a trained model at the end of pruning training according to embodiment 1 of the present application;

FIG. 6 is a flowchart of another model deployment method according to embodiment 1 of the present application;

FIG. 7 is a flow chart of a model deployment method according to embodiment 2 of the present application;

FIG. 8 is a flow chart of a model deployment method according to embodiment 3 of the present application;

FIG. 9 is a flowchart of a model deployment method according to embodiment 4 of the present application;

FIG. 10 is a schematic view of a model deployment apparatus according to example 5 of the present application;

FIG. 11 is a schematic view of a model deployment apparatus according to example 6 of the present application;

FIG. 12 is a schematic view of a model deployment apparatus according to example 7 of the present application;

FIG. 13 is a schematic view of a model deployment apparatus according to example 8 of the present application;

FIG. 14 is a schematic diagram of a hardware environment for a model deployment method according to an embodiment of the application;

FIG. 15 is a diagram of a hardware environment for another method of delivering media files, according to an embodiment of the invention;

fig. 16 is a block diagram of a computer terminal according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions of the present application better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

network model pruning: the aim of compressing the network model is achieved by cutting out unimportant parameters in the network model. The cut-off weight does not participate in the operation of the network model, and the storage space is not occupied, so that the acceleration of the network can be obtained, and the memory occupation can be reduced.

Currently, remote sensing models are often trained using data from a large area, such as global data, and are usually deployed only for tasks in a specific area. This results in a large amount of parameter redundancy in the model of the actual deployment. Therefore, research on model compression of remote sensing models is required. Pruning is a common means for compressing the neural network model, the memory occupation and the operation time consumption are reduced by the pruning model, and the method is more friendly to deployment in an application scene. The current commonly used pruning method comprises iterative pruning, direct pruning on a sparse network and the like, and is designed based on a conventional picture classification task, so that a certain problem exists in the processing of a remote sensing task.

Common pruning schemes include iterative amplitude pruning and direct pruning.

The iterative amplitude pruning method takes the modular length of the weight as a criterion, and removes the weight with smaller amplitude in proportion. After the weights are removed, parameters of the neural network change, and the retraining of the network can play a role in improving the accuracy of the network. The iterative method is to repeat the above processes, keep the cut parameters at 0 all the time, and gradually increase the pruning rate to obtain better compression effect. In order to ensure the effect, the network is retrained once when the iterative pruning rate is increased once. This results in inefficiency in the scheme. The design of the scheme aims at a conventional picture classification task, and the parameter redundancy rate in the conventional task is far lower than that of a remote sensing model, so that multi-step iteration is needed for deep and fine pruning.

The direct pruning method described above tends to suffer from loss of precision with the high pruning rate of the model normally trained. Therefore, a special training method is needed to train the model without pruning to obtain a network with sparse property, and the method is difficult to be applied to the model obtained by conventional training. Meanwhile, the remote sensing model trained by global data does not necessarily have sparsity, so that the scheme may not be capable of obtaining a proper non-pruning model. The scheme can be directly applied to a conventional model while the pruning efficiency is improved, and has a greater application prospect.

In addition, the common practice is to prune against the training set, and does not optimize against different scenes. The method and the device mainly aim at pruning optimization of different remote sensing downstream scenes, and are more suitable for actual scenes. The method and the device are developed aiming at the remote sensing task, so that a reasonably compressed model can be obtained without multi-step iteration, and the pruning efficiency is greatly improved.

The application provides a method for compressing a remote sensing model when the remote sensing model is deployed. The conventional remote sensing neural network model contains a large number of parameters, so that the model is huge in size and is not favorable for being deployed on equipment with limited memory resources, such as terminals. Meanwhile, when the remote sensing model is deployed, the remote sensing model is usually only specific to a specific area. Models trained with global data may therefore have a large number of redundant parameters at a particular deployment. Aiming at the problem, the method for pruning the remote sensing model to achieve the purpose of model compression is provided, unimportant weight parameters are removed, the parameter amount of the model during deployment is reduced under the condition that the recognition effect is basically kept unchanged, the time consumption of the model is reduced, and the memory resource occupation is reduced.

Example 1

There is also provided, in accordance with an embodiment of the present application, a model deployment method embodiment, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.

The method provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar operation device. Fig. 1 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing the model deployment method. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module or incorporated, in whole or in part, into any of the other elements in the computer terminal 10 (or mobile device). The data processing circuit acts as a processor control (e.g., selection of a variable resistance termination path connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the model deployment method in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the model deployment method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with the user interface of the computer terminal 10 (or mobile device).

It should be noted here that in some alternative embodiments, the computer device (or mobile device) shown in fig. 1 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that fig. 1 is only one example of a particular specific example and is intended to illustrate the types of components that may be present in the computer device (or mobile device) described above.

Under the operating environment, the application provides a model deployment method as shown in fig. 2. Fig. 2 is a flowchart of a model deployment method according to embodiment 1 of the present application. As shown in fig. 2, the method may include the steps of:

step S202, an original model to be deployed to target equipment and a target remote sensing image are obtained.

The original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in the target area corresponding to the target device, and the target area is a partial area in the plurality of areas.

The target device may be a device that processes the remotely sensed image in the target area.

The plurality of regions may be a plurality of regions of the whole world, the whole province, or the whole city, wherein the target region may be a partial region of the plurality of regions, for example, a region of a country of the plurality of regions of the whole world, a region of a province of the plurality of regions of the province, or a region corresponding to a city of the plurality of regions corresponding to the city.

The scenes corresponding to the multiple regions can be cloud layer scenes, agriculture and forestry scenes, water conservancy scenes, disaster scenes, city scenes and the like. The agriculture and forestry scenes can comprise a greenhouse scene, a field scene, a farmland scene, a plot scene and the like; the water conservancy scene can comprise a water body scene, a barrage scene and the like; the disaster scene can comprise a mud-rock flow scene, a typhoon scene and the like; the city scene may include a building scene, a road network scene, and the like. The target device can be applied to the remote sensing image detection task corresponding to the scene.

The target area may be one or more of a plurality of areas, which is not limited herein.

The original model may be a model obtained by training remote sensing images acquired in a plurality of regions, optionally, the remote sensing images acquired in the plurality of regions may be global data, and the original model may be a remote sensing model suitable for the world.

In an optional embodiment, an original model to be deployed to a target device may be obtained first, and for the original model, since the original model is obtained by training remote sensing images acquired in a plurality of regions, when the original model is used to process remote sensing images in a part of the regions of the plurality of regions, most parameters in the original model have no practical significance, so that a large amount of parameter redundancy exists in the model, therefore, the original model needs to be pruned, redundant parameters in the original model are removed, so as to improve the running speed of the model, training data for pruning may be determined according to a target region to which the deployed target device is directed, so that the target model obtained by pruning may be more suitable for a processing scene of the target region, in the present application, by obtaining a target remote sensing image in the target region corresponding to the target device as a basis for pruning the original model, the target model obtained by pruning can be more concerned in the remote sensing processing task of the target area, so that the accuracy of the target model obtained by pruning is ensured, and the operation efficiency of the target model is improved.

Illustratively, if the original model is obtained by training remote sensing images of a cloud layer area, a farmland area and a city area, if the model deployed to the target device only needs to process the remote sensing images of the cloud layer area, pruning can be performed on the original model based on the target remote sensing images of the cloud layer area, and the parameter quantity obtained by training according to the farmland area and the city area is reduced, so that the model deployed to the target device is more focused on the remote sensing image processing of the cloud layer area, and the operation efficiency of the target model is improved while the accuracy of the target model obtained by pruning is ensured.

And S204, determining a pruning proportion corresponding to the original model based on the target remote sensing image.

The pruning proportion refers to the proportion between the volume sum of the original model which is not activated through the target remote sensing image and all the volume sums.

In an optional embodiment, the original model obtained by training the remote sensing images acquired in the multiple regions has corresponding feature expression capability for features in the remote sensing images of the multiple regions, such as features in the form of landforms, vegetation and buildings, so that the obtained original model has wider adaptability, but the model is generally formed only for specific geographic conditions and buildings in the deployment process, and only features generated by a limited number of convolution kernels actually contribute to a final model generation result in actual use, so that a typical remote sensing sample of a target region, namely the target remote sensing original model, can be combined to perform activation analysis on the target remote sensing image to obtain a better pruning proportion.

And S206, carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model.

The pruning training is to determine the constraint condition corresponding to each parameter in the original model based on the pruning proportion, and sort the parameters from large to small according to the parameter magnitude, namely, sort in a descending order. Constraint conditions can be added to the parameter amplitudes in the later ranking so that the amplitude of the parameter approaches to 0 in the subsequent training process, and the constraint conditions can not be added to the parameters in the parameter amplitudes in the earlier ranking so that the parameters in the earlier ranking can be updated through training of the target remote sensing image in the training process so as to keep the feature expression capability of the remote sensing image corresponding to the target area.

In an optional embodiment, the convolution sum parameters of the original model may be sorted according to parameter amplitudes to obtain a distribution histogram of the parameter amplitudes of the original model, and the number M of the pruned parameters may be determined according to a pruning proportion M and a total parameter number N, where M = M × N, the remote sensing model parameters may be sorted from small to large according to the amplitudes, and the amplitude of the parameter at M is used as a threshold

And increasing a constraint condition on the parameters lower than the threshold value to ensure that the amplitude of the part of parameters is close to 0, and the parameters higher than the threshold value have no constraint, and updating the parameters higher than the threshold value through training of the target remote sensing image to keep the feature expression capability of the remote sensing image corresponding to the target area.

Step S208, deploying the target model to the target device.

The target device may be a computer terminal, a server, etc.

In an alternative embodiment, after the target model is trained, the target model may be deployed on a target device so as to process a remote sensing image corresponding to the target area through the target device. The pruned target model reserves the characteristic expression capability of the remote sensing image corresponding to the target area, and can remove the convolution kernel which has little relation with the remote sensing image corresponding to the target area, thereby reducing the size of the model, reducing the occupation of the memory after the pruned model is deployed, and improving the reasoning speed of the target model.

In another alternative embodiment, the target device may correspond to different application scenarios, for example, the target device may be a device for detecting a remote sensing image, the target device may be a device for classifying the remote sensing image, and the target device may also be a device for detecting a target object in the remote sensing image.

In another optional embodiment, the model deployment method may be provided externally in a cloud calling manner, and the original model and the target remote sensing image may be obtained first, and the original model and the target remote sensing image may be transmitted to a corresponding device for processing, for example, directly transmitted to a computer terminal (e.g., a laptop, a personal computer, etc.) of a user for processing, or transmitted to a cloud server for processing through the computer terminal of the user. It should be noted that, because processing of the original model and the target remote sensing image requires a large amount of computing resources, in the embodiment of the present application, a processing device is taken as an example of a cloud server for description.

For example, in order to facilitate a user to upload an original model and a target remote sensing image, an interactive interface may be provided for the user, where the user interface may include icons such as "select model", "select image", "upload" and the like for the user to operate, the user may select a target remote sensing image of a target area from stored remote sensing images of multiple areas through a "select image" button, select an original model to be used from a trained model through a "select model" button, and upload the original model and the target remote sensing image selected by the user to a cloud server for processing by clicking an "upload" button. In addition, in order to facilitate the user to confirm whether the selected original model and the target remote sensing image are the original model and the target remote sensing image required by the user, the target remote sensing image selected by the user can be displayed in the image display area, the name of the model can be displayed in the model display area, and after the user confirms that the model is correct, the data is uploaded by clicking an upload button.

Further, the cloud server can prune the original model according to the target remote sensing image to obtain the target model.

In the embodiment of the application, an original model to be deployed to target equipment and a target remote sensing image are obtained firstly, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in a target area corresponding to the target equipment, the target area is a partial area in the plurality of areas, and a pruning proportion corresponding to the original model is determined based on the target remote sensing image; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; the target model is deployed to the target equipment, and the purpose that the target model with higher running speed is obtained by compressing the initial model through pruning training is achieved.

In the above embodiments of the present application, determining, based on the target remote sensing image, the pruning proportion corresponding to the original model includes: processing the target remote sensing image by using the original model to obtain a feature map output by each layer in the original model; generating a feature matrix based on the feature map; and determining the pruning proportion based on the singular values in the feature matrix.

In an optional embodiment, a target remote sensing image can be transmitted in a forward direction on an original model to obtain a feature map output by each layer, the feature map can be spread into one-dimensional vectors, a feature matrix is formed according to channels, n singular values of the feature matrix are calculated, the singular values can be sorted from large to small, k values are selected, the sum of the first k singular values accounts for 99% of the sum of all singular values, k and n of each layer are recorded, after the values of k and n are calculated by each layer, the sum of all k and the sum of all n can be divided to calculate the ratio a of the convolution sum activated by the model to the deployment scene, and then the ratio of the pruned branches is determined to be m =1-a, namely the pruning ratio.

In the foregoing embodiment of the present application, based on the feature map, generating the feature matrix includes: unfolding the feature map into a one-dimensional vector; and generating a feature matrix according to the one-dimensional vector.

In an alternative embodiment, the feature map may be expanded into a one-dimensional vector, and the feature matrix may be generated using the one-dimensional vector generated by the expansion of the feature map.

In the above embodiment of the present application, determining the pruning ratio based on the singular value in the feature matrix includes: sequencing a plurality of singular values in the feature matrix to obtain sequenced singular values; determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; and determining the pruning proportion based on the target quantity threshold and the quantity of the plurality of singular values.

The predetermined ratio may be 99%.

In an optional embodiment, the plurality of singular values in the feature matrix may be sorted to obtain sorted singular values, and the target number threshold may be determined according to a product of a preset ratio and a sum of the plurality of singular values, wherein a ratio of an activated convolution of the remote sensing image corresponding to the target region by the original model may be calculated according to the target number threshold, and then a ratio of an inactivated convolution may be determined according to the ratio. I.e. the pruning ratio as described above, the inactive convolutions and corresponding parameters can be reduced in accordance with the pruning ratio as described above.

In another optional embodiment, an original model may be obtained by training remote sensing images of multiple regions, and parameter redundancy analysis is performed by using target remote sensing images corresponding to target regions, because the redundancy degree of the original model is high, multiple convolution sums of each layer in the original model respectively correspond to scene features of different regions, so that analysis of the number of activated convolution kernels in the original model may be performed by using real data of the target regions, that is, the target remote sensing images, so as to obtain a reasonable pruning proportion.

In the above embodiment of the present application, determining the pruning proportion based on the target number threshold and the number of the plurality of singular values includes: obtaining the sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; acquiring the sum of the number of a plurality of singular values corresponding to all layers in the original model to obtain a second sum; acquiring the ratio of the first sum value to the second sum value to obtain a target ratio; and obtaining a difference value between the preset value and the target ratio to obtain a pruning ratio.

The preset value may be 1. It should be noted that the preset value 1 is 100%, and the pruning proportion can be obtained by subtracting the target proportion from 100%.

In an optional embodiment, a sum of target quantity thresholds corresponding to all layers in the original model may be obtained to obtain a first sum, a sum of quantities of a plurality of singular values corresponding to all layers in the original model may be obtained to obtain a second sum, a ratio of activated convolution sums, that is, the target ratio, may be determined according to a ratio of the first sum to the second sum, and a ratio of inactivated convolution kernels may be determined according to a difference between 1 and the target ratio.

In another alternative embodiment, since the scene features of the target area are generally a fixed limited number of features belonging to various types such as terrain, landform, vegetation and buildings, a large pruning ratio can be obtained according to the above steps, so that a large-amplitude pruning can be performed.

In the above embodiment of the present application, performing pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain the target model includes: determining a first parameter in the original model based on the pruning proportion; adding constraint conditions to the first parameters in the original model to obtain a training target; training a training target by using a target remote sensing image to obtain a trained model; and removing the second parameter in the trained model according to the pruning proportion to obtain the target model.

The first parameter may be a parameter with a smaller amplitude in the convolution kernel, and the second parameter may be a parameter with a smaller amplitude in the convolution kernel, where the second parameter may be a parameter close to 0.

The training target may be a target for determining quantization training in the original model.

The constraint may be 12-norm

Wherein, in the step (A),

in order to be a constraint condition, the method comprises the following steps of,

for the constraint term coefficients, w is a parameter,

for distinguishing a first parameter of the plurality of parameters.

In an alternative embodiment, a threshold may be determined according to the pruning proportion, a plurality of parameters in the original model are distinguished according to the threshold, a parameter lower than the threshold is used as a first parameter, the training target may be trained by using the target remote sensing image so as to increase the feature expression capability of the training target for the remote sensing image corresponding to the target region, so that the amplitude of the first parameter is continuously reduced, because a constraint condition is added to the first parameter during the training process, the amplitude of the first parameter approaches to 0, at the initial stage of the training, a small part of the parameters with smaller amplitudes are attracted to the vicinity of 0, and the rest of the larger parameters maintain the original expression capability, so that with further training, the penalty coefficient is gradually increased, the number of the parameters attracted to the vicinity of 0 is gradually increased, and the expression capability of the parameters which are not attracted to 0 is not additionally limited, the processing capability of the remote sensing image corresponding to the target area can be fitted.

In the above embodiments of the present application, determining the first parameter in the original model based on the pruning ratio includes: obtaining target parameters based on the pruning proportion and the number of a plurality of original parameters in the original model; sequencing the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain sequenced original parameters; obtaining the amplitude of the original parameters at the position corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold; and determining the original parameter with the amplitude smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

In an optional embodiment, the number of the target parameters may be determined according to a product of a pruning proportion and the number of the plurality of original parameters, then the plurality of original parameters are sequenced from small to large according to the amplitudes of the plurality of original parameters to obtain sequenced original parameters, the amplitudes of the original parameters at positions corresponding to the number of the target parameters in the sequenced original parameters may be obtained to obtain an amplitude threshold, and the original parameter with the amplitude smaller than the amplitude threshold in the plurality of original parameters is used as the first parameter.

In the above embodiment of the present application, adding a constraint condition to a first parameter in an original model to obtain a training target includes: determining a current constraint item coefficient based on the current training iteration number of pruning training; generating a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; and adding constraint conditions to the first parameters to obtain a training target.

In an optional embodiment, a constraint term coefficient is initialized first, the initialized constraint term may be updated according to the current training iteration number of pruning training to obtain a current constraint term coefficient, and the current constraint term coefficient may be updated according to the current training iteration number during subsequent training. Constraint conditions can be generated according to the current constraint term coefficient and the amplitude of the first parameter, so that the first parameter approaches to 0, and a training target after pruning is obtained.

In another alternative embodiment, a more stable pruning process can be obtained by gradually raising the coefficients of the constraint terms. In the initial training stage, a small part of parameters with smaller amplitudes are attracted to be close to 0, and the rest of larger parameters keep the original expression capacity, so that the penalty coefficient is gradually increased along with further training, the number of the parameters attracted to be close to 0 is gradually increased, the expression capacity of the parameters which are not attracted to 0 is not limited additionally, and the processing capacity of the remote sensing image corresponding to the target area can be fitted. It should be noted that the penalty coefficient is also a constraint coefficient. The stability of the pruning process is maintained by gradually increasing the constraint term coefficients.

In the above embodiment of the present application, removing the second parameter from the trained model according to the pruning proportion to obtain the target model includes: obtaining target parameters based on the pruning proportion and the number of a plurality of target parameters in the trained model; sequencing the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters; determining the target parameters arranged behind the positions corresponding to the target parameters in the sorted target parameters as second parameters; and deleting the second parameters, and combining the rest parameters in the trained model to obtain the target model.

The target parameter number may be obtained according to a product of the pruning ratio and a number of the plurality of target parameters in the trained model.

In an optional embodiment, according to the pruning proportion and the number of the plurality of target parameters in the trained model, the plurality of target parameters may be sorted in a descending order according to the amplitudes of the plurality of target parameters to obtain sorted target parameters, the target parameter which is ranked at a position corresponding to the number of the target parameters in the determined target parameters is determined as a second parameter with a smaller amplitude, the second parameter with the smaller amplitude is deleted, and the remaining parameters in the trained model are combined to obtain the target model.

In another alternative embodiment, after completing a pruning training process by using a target remote sensing image, M parameters with the minimum amplitude sequence are directly removed according to the previously determined proportion, and the remaining parameters are reorganized to generate a new convolution structure to form a new network. The process of the step involves two removing processes, firstly, the convolution kernel with small amplitude of the layer of parameter is directly removed, and simultaneously, because the convolution kernel with small amplitude of the layer is also removed, the number of input channels of the layer of convolution kernel is correspondingly reduced. The pruned model is used as the target model.

In another alternative embodiment, the M parameters may be removed in order of magnitude according to all parameters. For layer I, the parameter of the convolution kernel with the minimum amplitude of the first layer is

The second parameter for reducing the number of input channels of the layer is

，

。

In the above embodiments of the present application, the second parameter includes at least one of: in the parameters of the current layer of the trained model, the parameters are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer; and inputting parameters of a second parameter with reduced channel number into the parameters of the current layer of the trained model.

The first parameter quantity and the second parameter quantity described above may be used to indicate the quantity of the parameters.

In an optional embodiment, in the parameters of the current layer of the trained model, the parameters arranged after the position corresponding to the first parameter quantity corresponding to the current layer may be the parameters with smaller parameter amplitude in the layer, that is, the second parameters; in the parameters of the current layer of the trained model, the parameter of the second parameter with the reduced number of input channels may be the number of channels corresponding to the parameter with the smaller amplitude, where the parameter of the reduced second parameter may be the second parameter.

Fig. 3 is a parameter quantity schematic diagram of an original model without pruning according to embodiment 1 of the present application, fig. 4 is a parameter quantity schematic diagram of an original model during pruning according to embodiment 1 of the present application, and fig. 5 is a parameter quantity schematic diagram of a trained model at the end of pruning training according to embodiment 1 of the present application.

The horizontal axis in fig. 3, 4, and 5 represents the modulus length of the parameters, and the vertical axis represents the number of parameters. Fig. 3, 4 and 5 represent the distribution of model parameters that change with the pruning process. Fig. 3 is a parameter distribution of the original model. As pruning proceeds, the parameters in fig. 4 appear to split between 0 and non-0. At the tail stage of pruning, the distribution of the non-0 parameters with larger amplitude is closer to the distribution of the parameters after direct pruning and parameter adjustment are finished, and the rest parameters are almost completely 0. At this stage of fig. 5, the distribution of the model parameters after the pruning training and before the pruning is mainly shown, and the parameters of the part of the parameters whose amplitudes are close to 0 are pruned once without causing extra loss to the model, so that it is ensured that no extra parameter adjustment is needed, and the final target model can be obtained after the pruning training is finished.

In the above embodiment of the present application, after determining the pruning proportion corresponding to the original model based on the target remote sensing image, the method further includes: outputting the pruning proportion; receiving a first feedback result, wherein the first feedback result comprises: a pruning proportion or a new pruning proportion, wherein the new pruning proportion is obtained by modifying the pruning proportion; and carrying out pruning training on the original model on the target remote sensing image for one time based on the first feedback result to obtain the target model.

In an optional embodiment, the pruning proportion may be output to a client device of the user, the user may modify the pruning proportion in the client device or directly determine the pruning proportion, a first feedback result may be generated according to an operation condition of the user, the first feedback result is fed back, and a pruning training may be performed on the original model on the target remote sensing image according to the pruning proportion determined by the user or a new pruning proportion formed after the modification, so as to obtain a final target model. The pruning proportion is determined by the user, so that the obtained pruning proportion is more accurate, and the lower precision of model training caused by the overlarge pruning proportion is avoided.

In the above embodiment of the present application, before deploying the target model to the target device, the method further includes: receiving a test remote sensing image; processing the test remote sensing image by using the target model to obtain a processing result of the test remote sensing image; outputting a processing result; receiving a second feedback result, wherein the second feedback result is used for representing whether the processing precision of the processing result meets the preset precision or not; and deploying the target model to the target equipment under the condition that the processing precision meets the preset precision as a second feedback result.

The preset precision can be set by a user.

The test remote sensing image can be a remote sensing image used for testing a target model, wherein the test remote sensing image can be a remote sensing image collected in a target area.

In an optional embodiment, the test remote sensing image may be received, the target model may be used to process the test remote sensing image to obtain a processing result, the processing result may be output to the client device, a user may check the processing result at the client device to determine whether the processing precision of the processing result meets the preset precision, and generate a second feedback result, and the target model may be deployed to the target device when the received second feedback result is that the processing precision meets the preset precision, so that the model precision deployed to the target device meets the requirement.

Fig. 6 is a flowchart of another model deployment method according to an embodiment of the present application, the method including:

step S601, acquiring an original model;

step S602, determining a pruning proportion according to target remote sensing data of a target area;

step S603, sequencing a plurality of original parameters of the feature map corresponding to the original model;

step S604, determining thresholds corresponding to the sequenced original parameters according to the pruning proportion;

optionally, a pruning training operation may be performed on the target remote sensing image corresponding to the target area once according to the pruning ratio to complete the pruning acceleration of the model.

Step S605, determining a first parameter of the plurality of original parameters, wherein the amplitude value of the first parameter is smaller than the amplitude value threshold value, based on the threshold value;

step S606, adding constraint conditions to the first parameters to obtain training targets;

step S607, training a training target based on the target remote sensing image to obtain a trained model, and judging whether a round of iteration is completed, if so, executing step S608, otherwise, executing step S603;

step S608, parameters of the trained model with parameter amplitudes in the pruning range are removed from small to large, and the rest parameters in the trained model are combined to obtain the target model.

Through the steps, punishment constraint can be applied to the parameters with the amplitude ordered from small to large within the preset pruning amplitude in the pruning process, the expression capability of the parameters with the larger amplitude is still kept, and the original target of the remote sensing model is minimized. The pruning process can be completed by one-time pruning process, and the pruning operation is carried out on the original remote sensing model according to a specific proportion on the premise of keeping the model effect. Compared with iterative amplitude pruning, the iterative adjustment process for multiple times after the pruning amplitude is gradually increased is not needed, the time is saved, compared with a method for directly pruning by a sparse network, the sparsity of an original model is not needed, and the pruning can be carried out according to the determined amplitude.

In addition, the remote sensing model is pruned by fully utilizing the target remote sensing image corresponding to the target area. Specifically, the pruning amplitude is automatically determined by the target remote sensing image. After the pruning amplitude is given, the algorithm utilizes the downstream target remote sensing image to carry out targeted pruning instead of pruning on the original training set, thereby achieving the ideal pruning effect.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art will recognize that the embodiments described in this specification are preferred embodiments and that acts or modules referred to are not necessarily required for this application.

Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method according to the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (such as a ROM/RAM, a magnetic disk, and an optical disk), and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device) to execute the method of the embodiments of the present application.

Example 2

Fig. 7 is a flowchart of a model deployment method according to embodiment 2 of the present application, and as shown in fig. 7, the method may include the following steps:

and step S702, responding to an input instruction acted on the operation interface, and displaying the target remote sensing image on the operation interface.

The target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in the multiple areas.

Step S704, responding to the deployment instruction acting on the operation interface, and displaying a deployment result on the operation interface.

The deployment result is used for representing a result of deploying the target model to the target device, the target model is obtained by carrying out pruning training on the original model on the target remote sensing image once according to the pruning proportion, the original model is obtained by training the remote sensing images collected in the plurality of areas, and the pruning proportion is determined based on the target remote sensing image.

In the above embodiment of the present application, the method further includes: processing the target remote sensing image by using the original model to obtain a characteristic diagram output by each layer in the original model; generating a feature matrix based on the feature map; and determining the pruning proportion based on the singular values in the feature matrix.

In the above embodiment of the present application, based on the feature map, generating the feature matrix includes: unfolding the feature map into a one-dimensional vector; and generating a feature matrix according to the one-dimensional vector.

In the above embodiment of the present application, determining the pruning proportion based on the target number threshold and the number of the plurality of singular values includes: acquiring the sum of target quantity thresholds corresponding to all layers in an original model to obtain a first sum; acquiring the sum of the number of a plurality of singular values corresponding to all layers in the original model to obtain a second sum; obtaining the ratio of the first sum value to the second sum value to obtain a target ratio; and obtaining a difference value between the preset value and the target ratio to obtain a pruning ratio.

In the above embodiment of the present application, performing pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain the target model includes: determining a first parameter in the original model based on the pruning proportion; adding constraint conditions to the first parameters in the original model to obtain a training target; training a training target by using the target remote sensing image to obtain a trained model; and removing the second parameter in the trained model according to the pruning proportion to obtain the target model.

In the above embodiment of the present application, determining the first parameter in the original model based on the pruning ratio includes: obtaining target parameters based on the pruning proportion and the number of a plurality of original parameters in the original model; sequencing the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain sequenced original parameters; obtaining the amplitude of the original parameters at the positions corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold; and determining the original parameter with the amplitude smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

In the above embodiments of the present application, the second parameter includes at least one of: in the parameters of the current layer of the trained model, the parameters are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer; the parameters of the second parameter with reduced number of channels are input into the parameters of the current layer of the trained model.

It should be noted that the preferred embodiments described in the above examples of the present application are the same as the schemes, application scenarios, and implementation procedures provided in example 1, but are not limited to the schemes provided in example 1.

Example 3

There is also provided, in accordance with an embodiment of the present application, a model deployment method embodiment, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.

Fig. 8 is a flowchart of a model deployment method according to embodiment 3 of the present application, and as shown in fig. 8, the method may include the following steps:

step S802, the cloud server receives the target remote sensing image sent by the client.

The target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area of the multiple areas.

Step S804, the cloud server obtains an original model to be deployed to the target device.

The original model is obtained through training of remote sensing images collected in a plurality of areas.

Step 806, the cloud server determines a pruning proportion corresponding to the original model based on the target remote sensing image.

And step S808, carrying out pruning training on the original model on the target remote sensing image by the cloud server based on the pruning proportion to obtain a target model.

Step S810, the cloud server deploys the target model to the target device.

In the above embodiment of the present application, the determining, by the cloud server, the pruning proportion corresponding to the original model based on the target remote sensing image includes: the cloud server processes the target remote sensing image by using the original model to obtain a feature map output by each layer in the original model; the cloud server generates a feature matrix based on the feature map; and the cloud server determines the pruning proportion based on the singular values in the characteristic matrix.

In the above embodiment of the present application, the generating, by the cloud server, the feature matrix based on the feature map includes: the cloud server expands the feature map into a one-dimensional vector; and the cloud server generates a feature matrix according to the one-dimensional vector.

In the above embodiment of the present application, the determining, by the cloud server, the pruning ratio based on the singular value in the feature matrix includes: the cloud server sequences a plurality of singular values in the feature matrix to obtain sequenced singular values; the cloud server determines a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; and the cloud server determines a pruning proportion based on the target quantity threshold and the quantity of the plurality of singular values.

In the above embodiment of the present application, the determining, by the cloud server, the pruning ratio based on the target number threshold and the number of the plurality of singular values includes: the cloud server obtains the sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; the cloud server obtains the sum of the number of the plurality of singular values corresponding to all layers in the original model to obtain a second sum; the cloud server acquires the ratio of the first sum value to the second sum value to obtain a target ratio; and the cloud server acquires a difference value between the preset value and the target ratio to obtain a pruning ratio.

In the above embodiment of the application, the performing, by the cloud server, pruning training on the original model on the target remote sensing image once based on the pruning proportion to obtain the target model includes: the cloud server determines a first parameter in the original model based on the pruning proportion; the cloud server adds constraint conditions to the first parameters in the original model to obtain a training target; the cloud server trains a training target by using the target remote sensing image to obtain a trained model; and the cloud server removes the second parameters in the trained model according to the pruning proportion to obtain the target model.

In the above embodiment of the present application, the determining, by the cloud server, the first parameter in the original model based on the pruning proportion includes: the cloud server obtains a target parameter number based on the pruning proportion and the number of the plurality of original parameters in the original model; the cloud server sequences the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain the sequenced original parameters; the cloud server acquires the amplitude of the original parameter at the position corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold; the cloud server determines the original parameter of which the amplitude is smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

In the above embodiment of the present application, adding, by the cloud server, a constraint condition to the first parameter in the original model, and obtaining the training target includes: the cloud server determines a current constraint item coefficient based on the current training iteration number of pruning training; the cloud server generates a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; and the cloud server adds constraint conditions to the first parameters to obtain a training target.

In the above embodiment of the application, the removing, by the cloud server, the second parameter in the trained model according to the pruning proportion to obtain the target model includes: the cloud server obtains target parameters based on the pruning proportion and the number of the target parameters in the trained model; the cloud server sequences the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters; the cloud server determines that the target parameters arranged at the positions corresponding to the target parameters in the sequenced target parameters are second parameters; and deleting the second parameters by the cloud server, and combining the rest parameters in the trained model to obtain the target model.

Example 4

Fig. 9 is a flowchart of a model deployment method according to embodiment 4 of the present application, and as shown in fig. 9, the method may include the following steps:

and S902, displaying the target remote sensing image on a display picture of the virtual reality VR device or the augmented reality AR device.

Step S904, an original model to be deployed to the target device is obtained.

And step S906, based on the target remote sensing image, determining the pruning proportion corresponding to the original model.

And step S908, carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model.

Step S910, deploying the target model to the target device to obtain a deployment result.

And step S912, driving the VR device or the AR device to display the deployment result on the display screen.

In the above embodiment of the present application, determining the pruning proportion corresponding to the original model based on the target remote sensing image includes: processing the target remote sensing image by using the original model to obtain a characteristic diagram output by each layer in the original model; generating a feature matrix based on the feature map; and determining the pruning proportion based on the singular values in the feature matrix.

In the above embodiment of the present application, determining the pruning ratio based on the singular value in the feature matrix includes: sequencing a plurality of singular values in the feature matrix to obtain sequenced singular values; determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; and determining the pruning proportion based on the target quantity threshold value and the quantity of the plurality of singular values.

In the above embodiment of the present application, determining the pruning proportion based on the target number threshold and the number of the plurality of singular values includes: obtaining the sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; acquiring the sum of the number of a plurality of singular values corresponding to all layers in the original model to obtain a second sum; obtaining the ratio of the first sum value to the second sum value to obtain a target ratio; and obtaining a difference value between the preset value and the target ratio to obtain a pruning ratio.

In the above embodiment of the present application, performing pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain the target model includes: determining a first parameter in the original model based on the pruning proportion; adding a constraint condition to a first parameter in the original model to obtain a training target; training a training target by using the target remote sensing image to obtain a trained model; and removing the second parameters in the trained model according to the pruning proportion to obtain the target model.

In the above embodiment of the present application, adding a constraint condition to the first parameter in the original model to obtain the training target includes: determining a current constraint term coefficient based on the current training iteration times of the original model; generating a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; and adding constraint conditions to the first parameters to obtain a training target.

In the above embodiment of the present application, removing the second parameter from the trained model according to the pruning proportion to obtain the target model includes: obtaining target parameters based on the pruning proportion and the number of the plurality of target parameters in the trained model; sequencing the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters; determining the target parameters arranged behind the positions corresponding to the target parameters in the sorted target parameters as second parameters; and deleting the second parameters, and combining the rest parameters in the trained model to obtain the target model.

In the above embodiments of the present application, the second parameter includes at least one of: the parameters of the trained model are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer in the parameters of the current layer; and inputting parameters of a second parameter with reduced channel number into the parameters of the current layer of the trained model.

Example 5

According to an embodiment of the present application, there is further provided a model deployment apparatus for implementing the model deployment method, where fig. 10 is a schematic diagram of a model deployment apparatus according to embodiment 5 of the present application, and as shown in fig. 10, the apparatus includes: an acquisition module 1002, a determination module 1004, a training module 1006, and a deployment module 1008.

The acquisition module is used for acquiring an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas, the target remote sensing image is a remote sensing image acquired in a target area corresponding to the target equipment, and the target area is a partial area in the plurality of areas; the determining module is used for determining the pruning proportion corresponding to the original model based on the target remote sensing image; the training module is used for carrying out pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; the deployment module is used for deploying the target model to the target device.

It should be noted here that the obtaining module 1002, the determining module 1004, the training module 1006, and the deploying module 1008 correspond to steps S202 to S208 in embodiment 1, and the four modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in embodiment 1. It should be noted that the above modules as a part of the apparatus may operate in the computer terminal 10 provided in embodiment 1.

In an embodiment of the present application, the determining module includes: the device comprises a first processing unit, a first generating unit and a first determining unit.

The first processing unit is used for processing the target remote sensing image by using the original model to obtain a feature map output by each layer in the original model; the first generation unit is used for generating a characteristic matrix based on the characteristic diagram; the first determining unit is used for determining the pruning proportion based on the singular values in the feature matrix.

In an embodiment of the present application, the first generation unit includes: expanding the sub-unit and generating the sub-unit.

The unfolding subunit unfolds the feature map into a one-dimensional vector; the generating subunit is used for generating a feature matrix according to the one-dimensional vector.

In an embodiment of the present application, the first determining unit includes: a first ordering subunit and a second determining subunit.

The first sorting subunit is used for sorting a plurality of singular values in the feature matrix to obtain sorted singular values; the second determining subunit is used for determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; the second determining subunit is configured to determine a pruning proportion based on the target number threshold and the number of the plurality of singular values.

In the embodiment of the application, the second determining subunit is further configured to obtain a sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; the second determining subunit is further configured to obtain a sum of numbers of the plurality of singular values corresponding to all layers in the original model, so as to obtain a second sum; the second determining subunit is further configured to obtain a ratio of the first sum to the second sum, so as to obtain a target ratio; the second determining subunit is further configured to obtain a difference between the preset value and the target ratio, so as to obtain a pruning ratio.

In an embodiment of the present application, the training module includes: the device comprises a second determining unit, an adding unit, a training unit and a removing unit.

The second determining unit is used for determining a first parameter in the original model based on the pruning proportion; the adding unit is used for adding a constraint condition to the first parameter in the original model to obtain a training target; the training unit is used for training a training target by using the target remote sensing image to obtain a trained model; and the removing unit is used for removing the second parameters in the trained model according to the pruning proportion to obtain the target model.

In an embodiment of the present application, the second determining unit includes: a third determining subunit, a second sorting subunit and an acquiring subunit.

The third determining subunit is used for obtaining a target parameter quantity based on the pruning proportion and the quantity of the plurality of original parameters in the original model; the second sequencing subunit is used for sequencing the plurality of original parameters according to the sequence from small to large of the amplitudes of the plurality of original parameters to obtain sequenced original parameters; the obtaining subunit is configured to obtain amplitudes of the original parameters at positions corresponding to the target parameters in the sorted original parameters, and obtain an amplitude threshold; the third determining subunit is further configured to determine, as the first parameter, an original parameter of the plurality of original parameters whose amplitude is smaller than the amplitude threshold.

In an embodiment of the present application, the adding unit includes: a fourth determining subunit and a generating subunit.

The fourth determining subunit is configured to determine a current constraint term coefficient based on the current training iteration number of pruning training; the generating subunit is used for generating a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; the fourth determining subunit is further configured to add a constraint condition to the first parameter to obtain a training target.

In an embodiment of the present application, the removing unit includes: a fifth determining subunit, a third sorting subunit and a combining subunit.

The fifth determining subunit is used for obtaining target parameters based on the pruning proportion and the number of the plurality of target parameters in the trained model; the third sorting subunit is used for sorting the plurality of target parameters according to the sequence of the amplitudes of the plurality of target parameters from large to small to obtain sorted target parameters; the fifth determining subunit is further configured to determine, as the second parameter, a target parameter that is ranked behind a position corresponding to the target parameter number in the ranked target parameters; and the combination subunit is used for deleting the second parameter and combining the rest parameters in the trained model to obtain the target model.

In an embodiment of the present application, the second parameter includes at least one of: the parameters of the trained model are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer in the parameters of the current layer; and inputting parameters of a second parameter with reduced channel number into the parameters of the current layer of the trained model.

In the embodiment of the present application, the apparatus further includes: the device comprises an output module and a receiving module.

The output module is used for outputting the pruning proportion; the receiving module is used for receiving a first feedback result, wherein the first feedback result comprises: a pruning proportion or a new pruning proportion, wherein the new pruning proportion is obtained by modifying the pruning proportion; the training module is further used for carrying out pruning training on the original model on the target remote sensing image based on the first feedback result to obtain the target model.

In the embodiment of the application, the receiving module is further used for receiving the test remote sensing image; the processing module is also used for processing the test remote sensing image by using the target model to obtain a processing result of the test remote sensing image; the output module is also used for outputting the processing result; the receiving module is further used for receiving a second feedback result, wherein the second feedback result is used for representing whether the processing precision of the processing result conforms to the preset precision; the deployment module is further used for deploying the target model to the target device under the condition that the second feedback result is that the processing precision meets the preset precision.

It should be noted that the preferred embodiments described in the foregoing examples of the present application are the same as the schemes, application scenarios, and implementation procedures provided in example 1, but are not limited to the schemes provided in example 1.

Example 6

According to an embodiment of the present application, there is also provided a model deployment apparatus for implementing the model deployment method, fig. 11 is a schematic diagram of a model deployment apparatus according to embodiment 6 of the present application, and as shown in fig. 11, the apparatus 1100 includes: a first display module 1102, a second display module 1104.

The first display module is used for responding to an input instruction acting on the operation interface and displaying a target remote sensing image on the operation interface, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area of a plurality of areas; the second display module is used for responding to a deployment instruction acting on the operation interface and displaying a deployment result on the operation interface, wherein the deployment result is used for representing a result of deploying the target model to the target device, the target model is obtained by carrying out pruning training on the original model on the target remote sensing image according to a pruning proportion, the original model is obtained by training the remote sensing images collected in a plurality of areas, and the pruning proportion is determined based on the target remote sensing image.

It should be noted that the first display module 1102 and the second display module 1104 correspond to steps S702 to S704 of embodiment 2, and the two modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above as part of the apparatus may be run in the computing terminal 10 provided in the first embodiment.

Example 7

According to an embodiment of the present application, there is further provided a model deployment apparatus for implementing the model deployment method, where fig. 12 is a schematic diagram of a model deployment apparatus according to embodiment 7 of the present application, and as shown in fig. 12, the apparatus 1200 includes: a receiving module 1202, a first deployment module 1204, a determining module 1206, a training module 1208, a second deployment module 1210.

The receiving module is used for receiving a target remote sensing image sent by a client through a cloud server, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area in a plurality of areas; the first deployment module is used for acquiring an original model to be deployed to target equipment through a cloud server, wherein the original model is obtained through training of remote sensing images acquired in a plurality of areas; the determining module is used for determining a pruning proportion corresponding to the original model based on the target remote sensing image through the cloud server; the training module is used for carrying out primary pruning training on the original model on the target remote sensing image through the cloud server based on the pruning proportion to obtain a target model; the second deployment module is used for deploying the target model to the target device through the cloud server.

It should be noted here that the receiving module 1202, the first deploying module 1204, the determining module 1206, the training module 1208, and the second deploying module 1210 described above correspond to steps S802 to S810 of embodiment 3, and the five modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the above modules as part of the apparatus may be run in the computing terminal 10 provided in the first embodiment.

Example 8

According to an embodiment of the present application, there is further provided a model deployment apparatus for implementing the model deployment method, where fig. 13 is a schematic diagram of a model deployment apparatus according to embodiment 8 of the present application, and as shown in fig. 13, the apparatus 1300 includes: a receiving module 1302, an obtaining module 1304, a determining module 1306, a training module 1308, a deployment module 1310, and a driving module 1312.

The receiving module is used for displaying a target remote sensing image on a display picture of the virtual reality VR device or the augmented reality AR device, wherein the target remote sensing image is a remote sensing image acquired in a target area corresponding to the target device, and the target area is a partial area of the multiple areas; the acquisition module is used for acquiring an original model to be deployed to target equipment, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas; the determining module is used for determining the pruning proportion corresponding to the original model based on the target remote sensing image; the training module is used for carrying out pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; the deployment module is used for deploying the target model to the target equipment to obtain a deployment result; the driving module is used for driving the VR equipment or the AR equipment to display the deployment result on the display picture.

It should be noted here that the receiving module 1302, the obtaining module 1304, the determining module 1306, the training module 1308, the deploying module 1310, and the driving module 1312 described above correspond to steps S902 to S912 of embodiment 4, and the six modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the modules described above as part of the apparatus may be run in the computing terminal 10 provided in the first embodiment.

In addition, the model deployment method used by the model deployment apparatus can be applied to the hardware environment shown in fig. 14 and composed of the server 102 and the AR-VR device 104, and fig. 14 is a schematic diagram of the hardware environment of a model deployment method according to an embodiment of the present application. As shown in fig. 14, the server 102 is connected to the AR-VR device 104 via a network, which may be a server corresponding to a media file operator, including but not limited to: the AR-VR device 104 may be a virtual reality VR device or an augmented reality AR device, where the virtual reality VR device is not limited to: virtual reality helmets, virtual reality glasses, virtual reality all-in-one machines and the like.

Optionally, the AR-VR device 104 includes: memory, processor, and transmission means. The memory is for storing an application program operable to perform: acquiring a target image, wherein the target image comprises a target hand-drawn pattern; detecting the target image to obtain a target detection result of a target element contained in the target hand-drawn pattern, wherein the target detection result is used for representing a target position and a target category of the target element; and grading the target hand-drawn pattern based on the target detection result to obtain a target grading result of the target hand-drawn pattern.

The processor of this embodiment may invoke the application stored in the memory via the transmission device to perform the steps described above. The transmission device may receive the target image sent by the server through the network, and may also be used for data transmission between the processor and the memory.

Alternatively, in the AR-VR device 104, a Head Mounted Display (HMD) with eye tracking is provided, a screen in the HMD is provided for displaying real-time images, an eye tracking module in the HMD is provided for acquiring a real-time movement track of the user's eyes, a tracking system is provided for tracking the position information and movement information of the user in a real three-dimensional space, and a calculation processing unit is provided for acquiring the real-time position and movement information of the user from the tracking system and calculating three-dimensional coordinates of the user's Head in a virtual three-dimensional space, and a visual field orientation of the user in the virtual three-dimensional space.

Fig. 15 is a schematic diagram of a hardware environment of another media file delivery method according to an embodiment of the present invention. As shown in fig. 15, the AR-VR device 104 is connected to the terminal 106, and the terminal 106 is connected to the server 102 via a network, and the AR-VR device 104 is not limited to: the terminal 104 is not limited to a PC, a mobile phone, a tablet computer, etc., and the server 102 may be a server corresponding to a media file operator, where the network includes but is not limited to: a wide area network, a metropolitan area network, or a local area network.

Optionally, the AR-VR device 104 of this embodiment functions as in the above-described embodiment, and the terminal of this embodiment may be configured to perform: acquiring a target image, wherein the target image comprises a target hand-drawn pattern; detecting the target image to obtain a target detection result of a target element contained in the target hand-drawn pattern, wherein the target detection result is used for representing a target position and a target category of the target element; and grading the target hand-drawn pattern based on the target detection result to obtain a target grading result of the target hand-drawn pattern.

Optionally, the AR-VR device 104 of this embodiment has an eye tracking HMD display and an eye tracking module that function the same as those in the above-described embodiments, that is, a screen in the HMD display is used for displaying real-time images, and the eye tracking module in the HMD is used for obtaining a real-time movement track of the user's eyes. The terminal of the embodiment acquires the position information and the motion information of the user in the real three-dimensional space through the tracking system, and calculates the three-dimensional coordinates of the head of the user in the virtual three-dimensional space and the visual field orientation of the user in the virtual three-dimensional space.

Example 9

According to an embodiment of the present application, there is also provided a model deployment system, including: the target equipment is used for deploying an original model, wherein the original model is obtained by training remote sensing images collected in a plurality of areas; the client is used for sending a target remote sensing image, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area of the plurality of areas; and the cloud server is connected with the client and the target equipment and used for determining a pruning proportion corresponding to the original model based on the target remote sensing image, performing pruning training on the original model on the target remote sensing image based on the pruning proportion to obtain a target model, and deploying the target model to the target equipment.

Example 10

The embodiment of the application can provide a computer terminal, and the computer terminal can be any one computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.

In this embodiment, the computer terminal may execute the program code of the following steps in the model deployment method: the method comprises the steps of obtaining an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area of the plurality of areas; based on the target remote sensing image, determining a pruning proportion corresponding to the original model; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; and deploying the target model to the target device.

Alternatively, fig. 16 is a block diagram of a computer terminal according to an embodiment of the present application. As shown in fig. 16, the computer terminal a may include: one or more processors (only one shown), memory.

The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the model deployment method and apparatus in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the model deployment method described above. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory remotely located from the processor, and these remote memories may be connected to terminal a through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas, the target remote sensing image is the remote sensing image acquired in a target area corresponding to the target equipment, and the target area is a partial area in the plurality of areas; determining a pruning proportion corresponding to the original model based on the target remote sensing image; carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; and deploying the target model to the target device.

Optionally, the processor may further execute the program code of the following steps: in the above embodiment of the present application, determining the pruning proportion corresponding to the original model based on the target remote sensing image includes: processing the target remote sensing image by using the original model to obtain a characteristic diagram output by each layer in the original model; generating a feature matrix based on the feature map; and determining pruning proportion based on singular values in the characteristic matrix.

Optionally, the processor may further execute the program code of the following steps: unfolding the characteristic diagram into a one-dimensional vector; and generating a feature matrix according to the one-dimensional vector.

Optionally, the processor may further execute the program code of the following steps: sequencing a plurality of singular values in the feature matrix to obtain sequenced singular values; determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; and determining the pruning proportion based on the target quantity threshold value and the quantity of the plurality of singular values.

Optionally, the processor may further execute the program code of the following steps: obtaining the sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; acquiring the sum of the number of a plurality of singular values corresponding to all layers in the original model to obtain a second sum; obtaining the ratio of the first sum value to the second sum value to obtain a target ratio; and obtaining a difference value between the preset value and the target ratio to obtain a pruning ratio.

Optionally, the processor may further execute the program code of the following steps: determining a first parameter in the original model based on the pruning proportion; adding constraint conditions to the first parameters in the original model to obtain a training target; training a training target by using a target remote sensing image to obtain a trained model; and removing the second parameters in the trained model according to the pruning proportion to obtain the target model.

Optionally, the processor may further execute the program code of the following steps: obtaining target parameters based on the pruning proportion and the number of a plurality of original parameters in the original model; sequencing the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain sequenced original parameters; obtaining the amplitude of the original parameters at the positions corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold; and determining the original parameter with the amplitude smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

Optionally, the processor may further execute the program code of the following steps: determining a current constraint term coefficient based on the current training iteration times of the original model; generating a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; and adding constraint conditions to the first parameters to obtain a training target.

Optionally, the processor may further execute the program code of the following steps: obtaining target parameters based on the pruning proportion and the number of the plurality of target parameters in the trained model; sequencing the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters; determining the target parameters arranged behind the positions corresponding to the target parameters in the sorted target parameters as second parameters; and deleting the second parameters, and combining the rest parameters in the trained model to obtain the target model.

Optionally, the processor may further execute the program code of the following steps: the second parameter includes at least one of: the parameters of the trained model are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer in the parameters of the current layer; the parameters of the second parameter with reduced number of channels are input into the parameters of the current layer of the trained model.

In this embodiment, the computer terminal may execute the program code of the following steps in the model deployment method: responding to an input instruction acting on an operation interface, and displaying a target remote sensing image on the operation interface, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area in a plurality of areas; responding a deployment instruction acting on an operation interface, and displaying a deployment result on the operation interface, wherein the deployment result is used for representing a result of deploying a target model to target equipment, the target model is obtained by carrying out pruning training on an original model on a target remote sensing image according to a pruning proportion, the original model is obtained by training remote sensing images collected in a plurality of areas, and the pruning proportion is determined based on the target remote sensing image.

In this embodiment, the computer terminal may execute the program code of the following steps in the model deployment method: the cloud server receives a target remote sensing image sent by the client, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area of a plurality of areas; the method comprises the steps that a cloud server obtains an original model to be deployed to target equipment, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas; the cloud server determines a pruning proportion corresponding to the original model based on the target remote sensing image; the cloud server carries out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; the cloud server deploys the target model to the target device.

In this embodiment, the computer terminal may execute the program code of the following steps in the model deployment method: displaying a target remote sensing image on a display picture of Virtual Reality (VR) equipment or Augmented Reality (AR) equipment, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in a plurality of areas; acquiring an original model to be deployed to target equipment, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas; determining a pruning proportion corresponding to the original model based on the target remote sensing image; carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; deploying the target model to target equipment to obtain a deployment result; and driving the VR equipment or the AR equipment to display the deployment result on a presentation screen.

In the embodiment of the application, an original model to be deployed to target equipment and a target remote sensing image are obtained firstly, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, the target area is a partial area in the plurality of areas, and a pruning proportion corresponding to the original model is determined based on the target remote sensing image; carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; the target model is deployed to the target equipment, and the purpose that the target model with higher running speed is obtained by compressing the initial model through pruning training is achieved.

It can be understood by those skilled in the art that the structure shown in fig. 16 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, etc. Fig. 16 does not limit the structure of the electronic device. For example, the computer terminal a may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in fig. 16, or have a different configuration than shown in fig. 16.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Example 11

Embodiments of the present application also provide a storage medium. Optionally, in this embodiment, the storage medium may be configured to store the program code executed by the model deployment method provided in the embodiment.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the processor can call the information and application program stored in the memory through the transmission device to execute the following steps: the method comprises the steps of obtaining an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area of the plurality of areas; based on the target remote sensing image, determining a pruning proportion corresponding to the original model; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; and deploying the target model to the target device.

Optionally, the storage medium is further configured to store program code for performing the following steps: in the above embodiments of the present application, determining, based on the target remote sensing image, the pruning proportion corresponding to the original model includes: processing the target remote sensing image by using the original model to obtain a feature map output by each layer in the original model; generating a feature matrix based on the feature map; and determining pruning proportion based on singular values in the characteristic matrix.

Optionally, the storage medium is further configured to store program code for performing the following steps: unfolding the characteristic diagram into a one-dimensional vector; and generating a feature matrix according to the one-dimensional vector.

Optionally, the storage medium is further configured to store program code for performing the following steps: sequencing a plurality of singular values in the feature matrix to obtain sequenced singular values; determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion; and determining the pruning proportion based on the target quantity threshold and the quantity of the plurality of singular values.

Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining the sum of target quantity thresholds corresponding to all layers in the original model to obtain a first sum; acquiring the sum of the number of a plurality of singular values corresponding to all layers in the original model to obtain a second sum; acquiring the ratio of the first sum value to the second sum value to obtain a target ratio; and obtaining a difference value between the preset value and the target ratio to obtain a pruning ratio.

Optionally, the storage medium is further configured to store program code for performing the following steps: determining a first parameter in the original model based on the pruning proportion; adding constraint conditions to the first parameters in the original model to obtain a training target; training a training target by using the target remote sensing image to obtain a trained model; and removing the second parameters in the trained model according to the pruning proportion to obtain the target model.

Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining target parameters based on the pruning proportion and the number of a plurality of original parameters in the original model; sequencing the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain sequenced original parameters; obtaining the amplitude of the original parameters at the positions corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold; and determining the original parameter with the amplitude smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

Optionally, the storage medium is further configured to store program code for performing the following steps: determining a current constraint term coefficient based on the current training iteration times of the original model; generating a constraint condition based on the current constraint term coefficient and the amplitude of the first parameter; and adding constraint conditions to the first parameters to obtain a training target.

Optionally, the storage medium is further configured to store program code for performing the following steps: obtaining target parameters based on the pruning proportion and the number of the plurality of target parameters in the trained model; sequencing the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters; determining the target parameters arranged behind the positions corresponding to the target parameters in the sorted target parameters as second parameters; and deleting the second parameters, and combining the rest parameters in the trained model to obtain the target model.

Optionally, the storage medium is further configured to store program code for performing the following steps: the second parameter includes at least one of: the parameters of the trained model are arranged behind the position corresponding to the first parameter quantity corresponding to the current layer in the parameters of the current layer; and inputting parameters of a second parameter with reduced channel number into the parameters of the current layer of the trained model.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: responding to an input instruction acting on an operation interface, and displaying a target remote sensing image on the operation interface, wherein the target remote sensing image is a remote sensing image acquired in a target area corresponding to target equipment, and the target area is a partial area in a plurality of areas; responding a deployment instruction acting on an operation interface, and displaying a deployment result on the operation interface, wherein the deployment result is used for representing a result of deploying a target model to target equipment, the target model is obtained by carrying out pruning training on an original model on a target remote sensing image according to a pruning proportion, the original model is obtained by training remote sensing images collected in a plurality of areas, and the pruning proportion is determined based on the target remote sensing image.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the cloud server receives a target remote sensing image sent by the client, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area in a plurality of areas; the method comprises the steps that a cloud server obtains an original model to be deployed to target equipment, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas; the cloud server determines a pruning proportion corresponding to the original model based on the target remote sensing image; the cloud server carries out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; the cloud server deploys the target model to the target device.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: displaying a target remote sensing image on a display picture of Virtual Reality (VR) equipment or Augmented Reality (AR) equipment, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in a plurality of areas; acquiring an original model to be deployed to target equipment, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas; based on the target remote sensing image, determining a pruning proportion corresponding to the original model; performing pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model; deploying the target model to target equipment to obtain a deployment result; and driving the VR equipment or the AR equipment to display the deployment result on a presentation screen.

In the embodiment of the application, an original model to be deployed to target equipment and a target remote sensing image are obtained firstly, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is the remote sensing image collected in a target area corresponding to the target equipment, the target area is a partial area in the plurality of areas, and a pruning proportion corresponding to the original model is determined based on the target remote sensing image; carrying out pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model; the target model is deployed to the target equipment, and the purpose that the target model with higher running speed is obtained by compressing the initial model through pruning training is achieved.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be implemented in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, or portions or all or portions of the technical solutions that contribute to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present application and it should be noted that, as will be apparent to those skilled in the art, numerous modifications and adaptations can be made without departing from the principles of the present application and such modifications and adaptations are intended to be considered within the scope of the present application.

Claims

1. A method of model deployment, comprising:

the method comprises the steps of obtaining an original model to be deployed to target equipment and a target remote sensing image, wherein the original model is obtained through training of remote sensing images collected in a plurality of areas, the target remote sensing image is a remote sensing image collected in a target area corresponding to the target equipment, and the target area is a partial area in the plurality of areas;

determining a pruning proportion corresponding to the original model based on the target remote sensing image;

performing pruning training on the original model on the target remote sensing image for one time based on the pruning proportion to obtain a target model;

deploying the target model to the target device.

2. The method of claim 1, wherein determining a pruning proportion corresponding to the original model based on the target remote sensing image comprises:

processing the target remote sensing image by using the original model to obtain a feature map output by each layer in the original model;

generating a feature matrix based on the feature map;

and determining the pruning proportion based on the singular values in the characteristic matrix.

3. The method of claim 2, wherein generating a feature matrix based on the feature map comprises:

unfolding the feature map into a one-dimensional vector;

and generating the feature matrix according to the one-dimensional vector.

4. The method of claim 2, wherein determining the pruning proportion based on singular values in the feature matrix comprises:

sequencing the plurality of singular values in the feature matrix to obtain sequenced singular values;

determining a target quantity threshold value based on the sorted singular values, wherein the sum of the singular values corresponding to the target quantity threshold value in the sorted singular values is equal to the product of the sum of the singular values and a preset proportion;

determining the pruning proportion based on the target number threshold and the number of the plurality of singular values.

5. The method of claim 1, wherein performing pruning training on the original model on the target remote sensing image once based on the pruning proportion to obtain a target model comprises:

determining a first parameter in the original model based on the pruning proportion;

adding a constraint condition to the first parameter in the original model to obtain a training target;

training the training target by using the target remote sensing image to obtain a trained model;

and removing the second parameter in the trained model according to the pruning proportion to obtain the target model.

6. The method of claim 5, wherein determining the first parameter in the original model based on the pruning ratio comprises:

obtaining target parameters based on the pruning proportion and the number of the plurality of original parameters in the original model;

sequencing the plurality of original parameters according to the sequence of the amplitudes of the plurality of original parameters from small to large to obtain sequenced original parameters;

obtaining the amplitude of the original parameters at the position corresponding to the target parameter number in the sorted original parameters to obtain an amplitude threshold;

and determining the original parameter with the amplitude smaller than the amplitude threshold value in the plurality of original parameters as the first parameter.

7. The method of claim 5, wherein adding a constraint condition to the first parameter in the original model to obtain a training target comprises:

determining a current constraint item coefficient based on the current training iteration number of the pruning training;

generating the constraint condition based on the current constraint term coefficient and the amplitude of the first parameter;

and adding the constraint condition to the first parameter to obtain the training target.

8. The method of claim 5, wherein removing the second parameter from the trained model according to the pruning proportion to obtain the target model comprises:

obtaining target parameters based on the pruning proportion and the number of the plurality of target parameters in the trained model;

sequencing the target parameters according to the sequence of the amplitudes of the target parameters from large to small to obtain sequenced target parameters;

determining the target parameters arranged behind the position corresponding to the target parameter number in the sorted target parameters as the second parameters;

and deleting the second parameters, and combining the rest parameters in the trained model to obtain the target model.

9. The method according to claim 1, wherein after determining a pruning proportion corresponding to the original model based on the target remote sensing image, the method further comprises:

outputting the pruning proportion;

receiving a first feedback result, wherein the first feedback result comprises: the pruning proportion or a new pruning proportion is obtained by modifying the pruning proportion;

and performing pruning training on the original model on the target remote sensing image based on the first feedback result to obtain the target model.

10. The method of claim 1, wherein prior to deploying the target model to the target device, the method further comprises:

receiving a test remote sensing image;

processing the test remote sensing image by using the target model to obtain a processing result of the test remote sensing image;

outputting the processing result;

receiving a second feedback result, wherein the second feedback result is used for representing whether the processing precision of the processing result meets the preset precision or not;

and deploying the target model to the target equipment under the condition that the second feedback result is that the processing precision accords with the preset precision.

11. A method of model deployment, comprising:

the method comprises the steps that a cloud server receives a target remote sensing image sent by a client, wherein the target remote sensing image is a remote sensing image collected in a target area corresponding to target equipment, and the target area is a partial area of a plurality of areas;

the cloud server acquires an original model to be deployed to target equipment, wherein the original model is obtained by training remote sensing images acquired in the multiple regions;

the cloud server determines a pruning proportion corresponding to the original model based on the target remote sensing image;

the cloud server carries out pruning training on the original model on the target remote sensing image on the basis of the pruning proportion to obtain a target model;

the cloud server deploys the target model to the target device.

12. A model deployment system, comprising:

the target equipment is used for deploying an original model, wherein the original model is obtained by training remote sensing images acquired in a plurality of areas;

the client is used for sending a target remote sensing image, wherein the target remote sensing image is acquired in a target area corresponding to the target equipment, and the target area is a part of the areas;

and the cloud server is connected with the client and the target equipment and used for determining a pruning proportion corresponding to the original model based on the target remote sensing image, performing pruning training on the original model on the target remote sensing image based on the pruning proportion to obtain a target model, and deploying the target model to the target equipment.

13. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the method of any one of claims 1-11.

14. An electronic device, comprising: a memory and a processor for executing a program stored in the memory, wherein the program when executed performs the method of any one of claims 1 to 11.