JP2021532498A - Video memory processing methods, devices and recording media based on convolutional neural networks - Google Patents

Abstract

The present application provides a video memory processing method, an apparatus and a recording medium based on a convolutional neural network in the field of a neural network, in which the video memory processing method temporarily stores input data, output data, input error and output error. Depending on the step of creating the temporary storage space, which is the storage space, and the type and direction of the data to be processed, the temporary storage space corresponding to the data to be processed is called, and the temporary storage space to which the data to be processed is called is used. A step to read, a step to perform predetermined processing on the data to be processed in the called temporary storage space, and data in the called temporary storage space according to the type and direction of the processed data. Includes a step to write to the specified external storage space. The present application can save a lot of video memory and improve the parallelism of GPU calculation. [Selection diagram] FIG. 5

Description

This application claims the priority of the patent application of Chinese Patent Application No. 201910497396.8 (Filing date June 10, 2019, title of invention "Video memory processing method, apparatus and recording medium based on convolutional neural network"). It is something to do.

The present application relates to the technical field of a convolutional neural network (CNN), and more particularly to a video memory processing method, an apparatus and a recording medium based on a convolutional neural network.

The video memory is a temporary memory of the GPU display core, and temporarily stores the core data to be processed. Its role is the same as that of the CPU and memory. The capacity of the video memory determines the ability of the video memory to temporarily store the data, and if the core of the video card is strong enough, the large capacity video memory can reduce the number of times the data is read and reduce the delay. can. In the current convolutional neural network model training process, the applicant repeatedly stores the model's input / output data in different video memory spaces, causing unnecessary video memory overhead and reducing the number of batch processes for model training. Therefore, we focused on affecting the accuracy of model training.

For example, the Concat layer and the Addition layer are commonly used layers in current deep learning classification networks and target detection networks. The Concat layer performs merge processing on a plurality of input data in the feature dimension, and the Addition layer performs cumulative processing on a plurality of input data. Traditional deep learning network training frameworks such as Caffe and TensorFlow do not perform video memory optimization for the Concat and Addition layers, so I / O data is repeatedly stored in different video memory spaces and unnecessary video. It affects the training accuracy of the model by introducing memory overhead and reducing the number of batch processes for training the model. At the same time, the video memory space also limits the search space of the optimization method of the automated machine learning technology autoML.

The present application is compared with conventional frameworks with the main purpose of creating a shared temporary storage space and reading or writing the data into the corresponding temporary storage space according to the type and instructions of the data to be processed. , Video memory processing based on convolutional neural network, which allows the user to arbitrarily combine with various modules to form a new CNN structure, which can save a lot of video memory and improve the parallelism of GPU computation. Methods, devices and computer-readable recording media are provided.

In order to achieve the above object, the video memory processing method based on the convolutional neural network applied to the electronic device according to the present application is a storage space for temporarily storing input data, output data, input error and output error. A step of creating a temporary storage space, a step of calling a temporary storage space corresponding to the processing target data according to the type and direction of the processing target data, and a step of reading the processing target data into the called temporary storage space, and the above-mentioned The data in the called temporary storage space is specified as an external storage according to the step of performing a predetermined process on the data to be processed in the called temporary storage space and the type and direction of the processed data. Includes steps to write to space.

The video memory processing system based on the convolutional neural network according to the present application has a space creation unit for creating a temporary storage space, which is a storage space for temporarily storing input data, output data, input error, and output error, and a space creation unit for processing target data. A data calling unit that calls the temporary storage space corresponding to the processing target data and reads the processing target data into the called temporary storage space according to the type and direction, and the processing target data in the called temporary storage space. A pre-processing unit that performs predetermined processing on the data, and a data writing unit that writes data in the called temporary storage space to a designated external storage space according to the type and direction of the data after processing. include.

Further, in order to achieve the above object, the electronic device according to the present application includes a memory and a processor, and when the memory is executed by the processor, the step of the video memory processing method based on the convolutional neural network described above is realized. Includes a video memory processor based on a convolutional neural network.

Further, in order to achieve the above object, the computer-readable recording medium according to the present application, when executed by a processor, realizes a step of a video memory processing method based on the convolutional neural network, a video based on a convolutional neural network. Includes memory processing program.

The video memory processing method, system, electronic device and computer readable recording medium based on the convolutional neural network according to the present application set a shared temporary storage space and correspond to the corresponding temporary storage according to the type and instruction of the data to be processed. It can be applied to CNN algorithms by calling space and reading or writing data into the corresponding temporary storage space for arithmetic processing, and compared to other frameworks, the Dense, Memory, and Injection modules. A new CNN structure can be formed by any combination with and can save about half of the video memory and improve the parallelism of GPU calculation.

Achievements, functional features and advantages of the present application will be further described in combination with examples with reference to the drawings.
It is a schematic diagram of the application environment of the video memory processing method based on the convolutional neural network which concerns on embodiment of this application. It is a schematic diagram of the module of the specific embodiment of the video memory processing program based on the convolutional neural network in FIG. 1. It is a schematic diagram of the partial structure of the conventional CNN structure. It is a schematic diagram of the partial structure of FIG. 3 after the video memory optimization process. It is a flowchart of the video memory processing method based on the convolutional neural network which concerns on embodiment of this application. It is a schematic diagram of the logical structure of the video memory processing system based on the convolutional neural network which concerns on embodiment of this application.

It should be noted that the specific examples described here are merely for interpreting the present application and do not limit the present application.

(Example 1)
The present application provides a video memory processing method based on a convolutional neural network applied to the electronic device 1. As shown in FIG. 1, it is a schematic diagram of the application environment of a preferable embodiment of the video memory processing method based on the convolutional neural network of the present application.

In this embodiment, the electronic device 1 may be a terminal device having a calculation function, such as a server, a smartphone, a tablet computer, a portable computer, or a desktop computer.

The electronic device 1 includes a processor 12, a memory 11, a network interface 14, and a communication bus 15.

The memory 11 includes at least one type of readable recording medium, and may be a non-volatile recording medium such as a flash memory, a hard disk, a multimedia card, or a card-type memory 11. In some embodiments, the readable recording medium may be the internal storage of the electronic device 1, for example, the hard disk of the electronic device 1. In another embodiment, the readable recording medium may be the external memory 11 of the electronic device 1, for example, a plug-in hard disk mounted on the electronic device 1, a SmartMedia Card (SMC), and the like. It is a secure digital (SD) card and a flash card (FlashCard).

In this embodiment, the readable recording medium of the memory 11 generally stores a video memory processing program 10 or the like based on a convolutional neural network mounted on the electronic device 1. Further, the memory 11 can temporarily store the output data or the data to be output.

In some embodiments, the processor 12 may be a central processing unit (CPU), microprocessor or other data processing chip, executing program code or processing data stored in memory 11. For example, a video memory processing program 10 based on a convolutional neural network is executed.

The network interface 14 may preferably include a standard wired interface, a wireless interface (eg, a WI-FI interface), and generally establishes a communication connection between the electronic device 1 and other electronic devices. ..

The communication bus 15 realizes connection communication between these components.

Although only electronic device 1 with components 11-15 is shown in FIG. 1, it is understood that more or less components can be implemented in an alternative manner without requiring all indicated components to be implemented. sea bream.

The electronic device 1 may include a user interface, a display, and a touch sensor, and the user interface includes an input unit such as a keyboard, a voice input device such as a device having a voice recognition function such as a microphone, and audio, headphones, and the like. It may include an audio output device. The display may be an LED display, a liquid crystal display, a touch-type liquid crystal display, an organic light-emitting diode (OLED) touch device, or the like. The touch sensor may be a resistance type touch sensor, a capacitive touch sensor, or the like. Further, the touch sensor may include not only a contact type touch sensor but also a proximity type touch sensor and the like. Further, the touch sensor may be a single sensor, or may be, for example, a plurality of sensors arranged in an array.

Preferably, the electronic device 1 may further include a radio frequency (Radio Frequency, RF) circuit, a sensor, an audio circuit, and the like, and the description thereof is omitted here.

In the embodiment of the apparatus shown in FIG. 1, the memory 11 which is a computer recording medium may include an operating system, a video memory processing program 10 based on a convolutional neural network, and the like, and the processor 12 is stored in the memory 11. When the video memory processing program 10 based on the convolutional neural network is executed, step 1 of creating a temporary storage space which is a storage space for temporarily storing input data, output data, input error, and output error, and a type of data to be processed. And, depending on the direction, the temporary storage space corresponding to the processing target data is called, and the processing target data is read into the called temporary storage space. A step 3 of performing a predetermined process and a step 4 of writing the data in the called temporary storage space to the designated external storage space according to the type and direction of the processed data are realized.

In step 1, the temporary storage space is a storage space for temporarily storing input data, output data, input error, and output error, and the corresponding temporary storage space is an input data temporary storage space, an output data temporary storage space, and the like. Includes input error temporary storage space and output error temporary storage space. The temporary storage space may be set in the video memory, the video memory stores the model or data, the larger the video memory, the larger the operable network, and the general video card is mainly. It has several types:

The storage unit of video memory is mainly
1Byte = 8bit,
1K = 1024Byte,
1KB = 1000Byte,
1M = 1024K,
1MB = 1000KB,
1G = 1024M,
1GB = 1000GB,
10K = 10 * 1024Byte,
Includes 10KB = 10000Byte.

Common numerical types and their sizes are shown in the table below.

In the above table, Int is an integer type numerical value, long is a long integer type numerical value, float is a floating point type numerical value (single is a single precision floating point type numerical value, and double is a double precision floating point type numerical value). (Floating point number).

In step 2, if the type of data to be processed is error and the direction is output, the corresponding output error temporary storage space is called according to the output error data, and the output error is read into the output error temporary storage space. Can be processed with.

The step of performing a predetermined process on the process target data includes a step of performing at least one of a convolution process, a superimposition process, a multiplication process, and an integral operation on the process target data.

For example, when performing a convolution process on data, the result of mainly multiplying two variables within a certain range and then adding them is acquired. If the variables of the convolution are the series x (n) and h (n), the convolution result is as shown in the following equation.

In the formula, * indicates convolution. When the time series n = 0, the series h (-i) is the result of the time series i of h (i) being inverted, and the time series inversion causes h (i) to be inverted 180 degrees about the vertical axis. Therefore, such a calculation method of adding after multiplication is called a convolution sum, and is abbreviated as convolution. Further, n is an amount that shifts h (−i), and different n corresponds to different convolution results.

If the variables of the convolution are two functions x (t) and h (t), the calculation of the convolution is performed according to the following equation.

p is an integral variable, integral is also an addition, t is a shifted quantity of the function h (-p), and * indicates convolution.

All of these operations can be performed in temporary storage space to achieve the goal of saving video memory.

In step 4, the step of writing the data in the called temporary storage space to the designated external storage space is to write the processed data in the temporary storage space to the designated external storage space by the set writing method. The writing method includes a writing step, and the writing method includes an Addition mode and a Concat mode.

The type of data includes input data, output data, input error and output error, and the direction of the data includes input and output.

Specifically, data can be written in the designated memory space by a different method according to the writing method (Addition, Concat) set by the user. For example, when the user sets the Addition mode, the data in the corresponding temporary storage space is cumulatively written in the specified storage space, and when the user sets the Concat mode, the data length information set by the user is used. Based on this, the data in the corresponding temporary storage space is sequentially written to the specified storage space at intervals.

Hereinafter, the execution of the video memory processing program based on the convolutional neural network of the present application will be described in detail by taking the convolutional neural network as an example.

In order to obtain the video memory usage status of the output of each layer of the neural network, it is necessary to calculate the shape of the feature map of each layer and save the gradient for backpropagation, and the video memory usage is batch size. Is directly proportional to. The video memory usage of the entire neural network is the model video memory + the video memory usage of each batch size * sample, and when the model is small, it is almost equal to the video memory usage of each batch size * sample.

To save video memory usage in the training process with a convolutional neural network model, for example, merge multiple input data in the corresponding temporary storage space and accumulate multiple input data in the corresponding temporary storage space. Video memory optimization can be performed for the Concat and Addition layers and the like.

For example, FIG. 3 is a partial structure of a conventional CNN network in which video memory optimization processing is not performed.

As shown in FIG. 3, if the back propagation is not considered temporarily and the forward propagation is taken as an example, the size of the input data of the convolution layer is 32 * 32 * 3, and if the batch size is 5, the layer is said. The size of the input data of is 32 * 32 * 3 * 5, and the calculation method of the size of each input / output data is the same as described above. Therefore, when the data is represented by float, the video memory used by the CNN network portion that has not been optimized is 1980 kb.

Video memory optimization is performed on the above portion using the video memory processing method based on the convolutional neural network of the present application, and the configuration diagram after the optimization is as shown in FIG.

Since backpropagation is not considered temporarily, only calling the output data temporary storage space needs to be considered, and the size or size of the temporary storage space is set to the maximum size of the output data of the convolutional layer in the CNN network. In this example, it is 32 * 32 * 16. No actual video memory space is allocated to the output data of the convolution layer in the broken line frame in FIG. 4, and the output data temporary storage space is called.

From the above, when the batch size is 5 and the data is represented by float, the video memory used by the CNN network portion after the video memory optimization is 1340 kb, which saves 32.3% of the video memory. Can be done.

The electronic device 1 according to the above embodiment sets a shared temporary storage space, calls the corresponding temporary storage space according to the type and instruction of the data to be processed, and reads or writes the data into the corresponding temporary storage space. By performing arithmetic processing in, it can be applied to the CNN algorithm, and compared to other frameworks, it can be arbitrarily combined with the Dense, Memory, and Injection modules to form a new CNN structure, which is about half. Video memory can be saved and the degree of parallelism of GPU calculation can be improved.

In another embodiment, the video memory processing program 10 based on the convolutional neural network may further be configured with a shared temporary storage space manager, which may temporarily set input data, output data, input errors, and output errors. Includes temporary storage space to store in. The manager provides several submodules to acquire and manipulate the corresponding temporary storage space, one or more modules stored in memory 11 and executed by processor 12 to complete the present application. .. A module as used herein refers to a set of computer program instruction segments capable of performing a particular function. FIG. 2 is a diagram of a program module of a preferred embodiment of the video memory processing program 10 based on the convolutional neural network in FIG. The video memory processing program 10 based on the convolutional neural network can be divided into the following submodules 210 to 230.

The temporary space acquisition submodule 210 calls and outputs the corresponding temporary storage space according to the type (data or error) and direction (input or output) of the data input to the module.
For example, if "error and output" is input to the temporary space acquisition submodule, the submodule calls and outputs the corresponding output error temporary storage space.

The data read submodule 220 accommodates data in the specified storage space in the corresponding temporary storage space, depending on the type (data or error) and direction (input or output) of the data input to the data read submodule. And output the temporary storage space.

For example, if "error and output" is input to the data reading submodule, the module reads the data in the designated storage space into the output error temporary storage space and outputs the output error temporary storage space.

The above-designated space mainly refers to a storage space in which the processing target data currently exists, and the processing target data is read from the designated space into the temporary storage space and processed. The following is the same.

The data writing submodule 230 is a temporary storage space in which data in the corresponding temporary storage space is designated according to the type (data or error) and direction (input or output) of the data input to the data writing submodule. Write in.

For example, if "error and input" is input to the data writing submodule, the data writing module writes the data in the input error temporary storage space into the designated storage space.

The data writing submodule may also write data in the designated memory space by a different method according to the writing method (Addition \ Concat) set by the user. For example, when the user sets the Addition mode, the data write submodule cumulatively writes the data in the corresponding temporary storage space to the specified storage space, and when the user sets the Concat mode, the data write. The submodule sequentially writes data in the corresponding temporary storage space to the specified storage space at intervals based on the data length information set by the user.

(Example 2)
The present application further provides a video memory processing method based on a convolutional neural network. FIG. 5 is a flowchart of a preferred embodiment of the video memory processing method based on the convolutional neural network according to the present application. The method may be performed by a device, which may be implemented by software and / or hardware.

In this embodiment, the video memory processing method based on the convolutional neural network includes the following S110 to S140.

In S110, a temporary storage space, which is a storage space for temporarily storing input data, output data, input error, and output error, is created.

In this step, the temporary storage space is a storage space that temporarily stores input data, output data, input error, and output error, and the corresponding temporary storage space is an input data temporary storage space, an output data temporary storage space, and the like. Includes input error temporary storage space and output error temporary storage space.

The temporary storage space may be set in the video memory, the video memory stores the model or data, the larger the video memory, the larger the operable network, and the general video card is mainly. It has several types:

The storage unit of video memory is mainly
1Byte = 8bit,
1K = 1024Byte,
1KB = 1000Byte,
1M = 1024K,
1MB = 1000KB,
1G = 1024M,
1GB = 1000GB,
10K = 10 * 1024Byte,
Includes 10KB = 10000Byte.

Common numerical types and their sizes are shown in the table below.

In the above table, Int is an integer type numerical value, long is a long integer type numerical value, float is a floating point type numerical value (single is a single precision floating point type numerical value, and double is a double precision floating point type numerical value). (Floating point number).

In S120, the temporary storage space corresponding to the processing target data is called according to the type and direction of the processing target data, and the processing target data is read into the called temporary storage space.

For example, if the type of data to be processed is error and the direction is output, the corresponding output error temporary storage space is called according to the output error data, and the output error is read into the output error temporary storage space. Can be processed.

In S130, predetermined processing is performed on the processing target data in the called temporary storage space.

The step of performing a predetermined process on the process target data includes a step of performing at least one of a convolution process, a superimposition process, a multiplication process, and an integral operation on the process target data.

For example, when performing a convolution process on data, mainly two variables are multiplied within a certain range and then added to obtain the result. If the variables of the convolution are the series x (n) and h (n), the convolution result is as shown in the following equation.

In the formula, * indicates convolution. When the time series n = 0, the series h (-i) is the result of the time series i of h (i) being inverted, and the time series inversion causes h (i) to be inverted 180 degrees about the vertical axis. Therefore, such a calculation method of adding after multiplication is called a convolution sum, and is abbreviated as convolution. Further, n is an amount that shifts h (−i), and different n corresponds to different convolution results.

If the variables of the convolution are two functions x (t) and h (t), the calculation of the convolution is performed according to the following equation.

p is an integral variable, integral is also an addition, t is a shifted quantity of the function h (-p), and * indicates convolution.

Similarly, any of these operations can be performed within the temporary storage space to achieve the goal of saving video memory.

In S140, the data in the called temporary storage space is written in the designated external storage space according to the type and direction of the processed data.

In the step, the step of writing the data in the called temporary storage space into the designated external storage space is a set writing method, and the processed data in the temporary storage space is designated as external storage. The writing method includes a step of writing in space, and a writing method includes an addition mode and a concat mode.

The type of data includes input data, output data, input error and output error, and the direction of the data includes input and output.

Specifically, data can be written in the designated memory space by a different method according to the writing method (Addition, Concat) set by the user. For example, when the user sets the Addition mode, the data in the corresponding temporary storage space is cumulatively written in the specified storage space, and when the user sets the Concat mode, the data length information set by the user is used. Based on this, the data in the corresponding temporary storage space is sequentially written to the specified storage space at intervals.

Hereinafter, the video memory processing method based on the convolutional neural network according to the present application will be described in detail by taking the convolutional neural network as an example.

In order to obtain the video memory usage status of the output of each layer of the neural network, it is necessary to calculate the shape of the feature map of each layer and save the gradient for backpropagation, and the video memory usage is batch size. Is directly proportional to. The video memory usage of the entire neural network is the model video memory + the video memory usage of each batch size * sample, and when the model is small, it is almost equal to the video memory usage of each batch size * sample.

To save video memory usage in the training process with a convolutional neural network model, for example, merge multiple input data in the corresponding temporary storage space and accumulate to multiple input data in the corresponding temporary storage space. Video memory optimization can be performed for the concat and addition layers and the like so as to perform processing.

For example, FIG. 3 is a partial structure of a conventional CNN network in which video memory optimization processing is not performed.

As shown in FIG. 3, if the back propagation is not considered temporarily and the forward propagation is taken as an example, the size of the input data of the convolution layer is 32 * 32 * 3, and if the batch size is 5, the layer is said. The size of the input data of is 32 * 32 * 3 * 5, and the calculation method of the size of each input / output data is the same as described above. Therefore, when the data is represented by float, the video memory used by the CNN network portion that is not optimized is 1980 kb.

Video memory optimization is performed on the above portion using the video memory processing method based on the convolutional neural network according to the present application, and the configuration diagram after the optimization is as shown in FIG.

Since backpropagation is not considered temporarily, only calling the output data temporary storage space needs to be considered, and the size or size of the temporary storage space is set to the maximum size of the output data of the convolutional layer in the CNN network. In this example, it is 32 * 32 * 16. No actual video memory space is allocated to the output data of the convolution layer in the broken line frame in FIG. 4, and the output data temporary storage space is called.

From the above, when the batch size is 5 and the data is represented by float, the video memory used by the CNN network portion after the video memory optimization is 1340 kb, which saves 32.3% of the video memory. Can be done.

In the video memory processing method based on the convolutional neural according to the above embodiment, a shared temporary storage space is set, the corresponding temporary storage space is called according to the type and instruction of the data to be processed, and the data is stored in the corresponding temporary storage space. It can be applied to the CNN algorithm by reading or writing in space and performing arithmetic processing, and compared with other frameworks, it can be arbitrarily combined with the Dense, Memory, and Injection modules to form a new CNN structure. It can save about half the video memory and improve the degree of parallelism of GPU calculation.

(Example 3)
Corresponding to the video memory processing method based on the convolutional neural network according to the second embodiment, the present application further provides a video memory processing system based on the convolutional neural network. FIG. 6 shows the logical structure of the video memory processing system based on the convolutional neural network according to this embodiment.

As shown in FIG. 6, the video memory processing system 600 based on the convolutional neural network according to the present embodiment includes a space creation unit 610, a data calling unit 620, a preprocessing unit 630, and a data writing unit 640. The functions realized by the space creation unit 610, the data calling unit 620, the preprocessing unit 630, and the data writing unit 640 correspond one-to-one with the corresponding steps in the video memory processing method based on the convolutional neural network in the second embodiment. ..

Specifically, the space creation unit 610 creates a temporary storage space which is a storage space for temporarily storing input data, output data, input error, and output error, and the space creation unit 610 temporarily stores the input data, output data, input error, and output error in the video memory. A storage space can be created, and the video memory stores a model or data, and the larger the video memory, the larger the operable network. The created temporary storage space may include an input data temporary storage space, an output data temporary storage space, an input error temporary storage space, and an output error temporary storage space.

The data calling unit 620 calls the temporary storage space corresponding to the processing target data according to the type and direction of the processing target data, and reads the processing target data into the called temporary storage space. For example, if the type of data to be processed is error and the direction is output, the corresponding output error temporary storage space is called according to the output error data, and the output error is read into the output error temporary storage space. Can be processed.

The pre-processing unit 630 performs predetermined processing on the data to be processed in the temporary storage space called by the data calling unit 620. The predetermined process may include at least one of a convolution process, a superimposition process, a multiplication process, or an integral operation performed on the data to be processed.

For example, when the preprocessing unit 630 performs a convolution process on data, it mainly acquires the result of multiplying two variables within a certain range and then adding them. If the variables of the convolution are the series x (n) and h (n), the convolution result is as shown in the following equation.

In the formula, * indicates convolution. When the time series n = 0, the series h (-i) is the result of the time series i of h (i) being inverted, and the time series inversion causes h (i) to be inverted 180 degrees about the vertical axis. Therefore, such a calculation method of adding after multiplication is called a convolution sum, and is abbreviated as convolution. Further, n is an amount that shifts h (−i), and different n corresponds to different convolution results.

If the variables of the convolution are two functions x (t) and h (t), the calculation of the convolution is performed according to the following equation.

p is an integral variable, integral is also an addition, t is a shifted quantity of the function h (-p), and * indicates convolution.

Similarly, any of these operations can be performed within the temporary storage space to achieve the goal of saving video memory.

The data writing unit 640 writes the data in the called temporary storage space into the designated external storage space according to the type and direction of the processed data.

The data writing unit 640 can write the processed data in the temporary storage space to the designated external storage space by the set writing method, and the writing method includes the Addition mode and the Concat mode. When the user sets the Addition mode, the data in the corresponding temporary storage space is cumulatively written to the specified storage space, and when the user sets the Concat mode, it corresponds based on the data length information set by the user. Writes the data in the temporary storage space to the specified storage space in sequence at intervals.

The video memory processing system based on the convolutional neural network according to the above embodiment sets a shared temporary storage space, calls the corresponding temporary storage space according to the type and instruction of the data to be processed, and stores the data in the corresponding temporary storage space. It can be applied to the CNN algorithm by reading or writing to space and performing arithmetic processing, and can be arbitrarily combined with the Dense, Memory, and Injection modules to form a new CNN structure compared to other frameworks. It can save about half the video memory and improve the parallelism of GPU calculation.

(Example 4)
The computer-readable recording medium according to the embodiment of the present application has an operation of creating a temporary storage space, which is a storage space for temporarily storing input data, output data, input error, and output error when executed by a processor. , The operation of calling the temporary storage space corresponding to the processing target data and reading the processing target data into the called temporary storage space according to the type and direction of the processing target data, and the processing target data in the called temporary storage space. A convolutional neural that realizes the operation of performing a predetermined process on the data and the operation of writing the data in the called temporary storage space to the specified external storage space according to the type and direction of the processed data. Includes a video memory processing program based on.

Preferably, the temporary storage space includes an input data temporary storage space, an output data temporary storage space, an input error temporary storage space, and an output error temporary storage space.

Preferably, the step of performing a predetermined process on the data to be processed includes a step of performing at least one of a convolution process, a superimposition process, a multiplication process, or an integral operation on the data to be processed.

Preferably, the step of writing the data in the called temporary storage space to the specified external storage space writes the processed data in the temporary storage space to the specified external storage space by the set writing method. The writing method includes an Addition mode and a Concat mode.

Preferably, the type of data includes input data, output data, input errors and output errors, and the direction of the data includes inputs and outputs.

Since the specific embodiment of the computer-readable recording medium of the present application is almost the same as the specific embodiment of the video memory processing method, the system, and the electronic device based on the convolutional neural network, the description thereof is omitted here. ..

It should be noted that, as used herein, the terms "include", "provide" or any other variation thereof by cover a non-exclusive inclusion, such as a process, device, article or a process, apparatus, article containing a set of elements. The method includes not only those elements, but also other elements not explicitly exemplified, or further elements specific to these processes, devices, articles or methods. Unless further limited, an element limited by the phrase "contains ..." may further have another identical element in the process, device, article or method containing the element.

The numbers in the above embodiments of the present application are for illustration purposes only and do not indicate superiority or inferiority of the embodiments. As can be clearly understood by those skilled in the art by the above description of the embodiment, the method of the above embodiment can be realized by a method using both software and a necessary general-purpose hardware platform, and of course, it can also be realized by hardware. Although possible, the former is often the more preferred embodiment. Based on such an understanding, the technical means of the present application may be embodied in the form of a software product in which an essential part or a part contributing to the prior art may be embodied, and the computer software product may be a recording medium (for example, ROM / ROM / A plurality of devices (which may be a mobile phone, a computer, a server, a network device, or the like) stored in a RAM (RAM, magnetic disk, optical disk), for causing one terminal device (which may be a mobile phone, a computer, a server, a network device, etc.) to execute the method described in each embodiment of the present application. Includes instructions for.

The above is merely a preferred embodiment of the present application, which does not limit the scope of protection of the present application, and is an equivalent structure or equivalent flow conversion made by utilizing the contents of the specification and drawings of the present application, or other related techniques. Both direct and indirect operations in the field are similarly included in the scope of protection of the present application.

Claims (20)

A video memory processing method based on a convolutional neural network applied to electronic devices.
Steps to create a temporary storage space, which is a storage space for temporarily storing input data, output data, input errors, and output errors.
A step of calling a temporary storage space corresponding to the processing target data according to the type and direction of the processing target data and reading the processing target data into the called temporary storage space.
A step of performing predetermined processing on the processing target data in the called temporary storage space, and
Video memory processing based on a convolutional neural network comprising, depending on the type and direction of the processed data, a step of writing the data in the called temporary storage space to a designated external storage space. Method. The video memory based on the convolutional neural network according to claim 1, wherein the temporary storage space includes an input data temporary storage space, an output data temporary storage space, an input error temporary storage space, and an output error temporary storage space. Processing method. The video memory processing method based on a convolutional neural network according to claim 1, wherein the temporary storage space is set in a video memory. The step of performing a predetermined process on the data to be processed is
The video memory processing method based on a convolutional neural network according to claim 1, further comprising a step of performing at least one of a convolutional process, a superimposition process, a multiplication process, and an integral operation on the data to be processed. .. The convolutional neural network according to claim 1, wherein a predetermined process performed on the data to be processed is a convolutional process of multiplying two variables within a certain range and then acquiring the result of addition. Based on video memory processing method. If the variables of the convolution are the series x (n) and h (n), the convolution result is as shown in the following equation.

In the equation, * indicates convolution, n is a quantity that shifts h (-i), different n corresponds to different convolution results, and when the time series n = 0, the series h (-i) is h ( Based on the convolutional neural network according to claim 5, which is the result of inversion of the time series i of i), wherein h (i) is inverted 180 degrees about the vertical axis due to the inversion of the time series. Video memory processing method. If the convolution variables are two functions x (t) and h (t), the convolution result is as shown in the following equation.

The convolutional neural network according to claim 5, wherein * indicates a convolution, t is a shifted quantity of the function h (−p), p is an integral variable, and the integral is an addition. Based on video memory processing method. The step of writing the data in the called temporary storage space into the specified external storage space is
A billing scheme comprising the steps of writing processed data in the temporary storage space into a designated external storage space in a configured write scheme, wherein the write scheme includes an Addition mode and a Concat mode. The video memory processing method based on the convolutional neural network according to Item 1. When the Addition mode is set, the data in the corresponding temporary storage space is cumulatively written to the external storage space.
The eighth aspect of the present invention, wherein when the Concat mode is set, the data in the corresponding temporary storage space is sequentially written to the external storage space at intervals based on the set data length information. Video memory processing method based on convolutional neural network. The type of data includes input data, output data, input error and output error.
The video memory processing method based on a convolutional neural network according to any one of claims 1 to 9, wherein the direction of the data includes an input and an output. A space creation unit that creates a temporary storage space, which is a storage space that temporarily stores input data, output data, input errors, and output errors.
A data calling unit that calls a temporary storage space corresponding to the processing target data and reads the processing target data into the called temporary storage space according to the type and direction of the processing target data.
A pre-processing unit that performs predetermined processing on the processing target data in the called temporary storage space, and
A video based on a convolutional neural network, comprising a data writer that writes data in the called temporary storage space to a designated external storage space, depending on the type and direction of the processed data. Memory processing system. The eleventh aspect of claim 11, wherein the temporary storage space created by the space creation unit includes an input data temporary storage space, an output data temporary storage space, an input error temporary storage space, and an output error temporary storage space. A video memory processing system based on a convolutional neural network. The video memory processing system based on the convolutional neural network according to claim 11, wherein the space creation unit creates the temporary storage space in the video memory. A predetermined process performed by the preprocessing unit on the data to be processed includes at least one of a convolutional process, a superimposition process, a multiplication process, and an integral operation on the data to be processed. Item 4. The video memory processing system based on the convolutional neural network according to Item 11. The predetermined processing performed by the preprocessing unit on the processing target data is a convolution processing for acquiring the result of addition after multiplying two variables within a certain range.
If the variables of the convolution are the series x (n) and h (n), the convolution result is as shown in the following equation.

In the equation, * indicates convolution, n is a quantity that shifts h (-i), different n corresponds to different convolution results, and when the time series n = 0, the series h (-i) is h ( Based on the convolutional neural network according to claim 11, which is the result of inversion of the time series i of i), wherein h (i) is inverted 180 degrees about the vertical axis due to the inversion of the time series. Video memory processing system. The predetermined processing performed by the preprocessing unit on the processing target data is a convolution processing for acquiring the result of addition after multiplying two variables within a certain range.
If the convolution variables are two functions x (t) and h (t), the convolution result is as shown in the following equation.

The convolutional neural network according to claim 11, wherein * indicates a convolution, t is a shifted quantity of the function h (−p), p is an integral variable, and the integral is an integral. Based on video memory processing system. The data writing unit writes the processed data in the temporary storage space to the designated external storage space by the set writing method.
The video memory processing system based on the convolutional neural network according to claim 11, wherein the writing method includes an Addition mode and a Concat mode. When the Addition mode is set, the data writing unit cumulatively writes the data in the corresponding temporary storage space to the external storage space.
When the Concat mode is set, the data writing unit is characterized in that data in the corresponding temporary storage space is sequentially written to the external storage space at intervals based on the set data length information. The video memory processing system based on the convolutional neural network according to claim 11. Including memory and processor
The memory is a video memory processing program based on a convolutional neural network that, when executed by the processor, realizes the steps of the video memory processing method based on the convolutional neural network according to any one of claims 1 to 11. An electronic device characterized by including. It is characterized by including a video memory processing program based on a convolutional neural network, which implements the steps of the video memory processing method based on the convolutional neural network according to any one of claims 1 to 11, when executed by a processor. A computer-readable recording medium.

Images