CN115456858B - Image processing method, device, computer equipment and computer readable storage medium - Google Patents

Image processing method, device, computer equipment and computer readable storage medium Download PDF

Info

Publication number
CN115456858B
CN115456858B CN202211127995.9A CN202211127995A CN115456858B CN 115456858 B CN115456858 B CN 115456858B CN 202211127995 A CN202211127995 A CN 202211127995A CN 115456858 B CN115456858 B CN 115456858B
Authority
CN
China
Prior art keywords
pixel
image
unit
shuffling
processed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211127995.9A
Other languages
Chinese (zh)
Other versions
CN115456858A (en
Inventor
刘钊含
梁猷强
张斌
沈小勇
吕江波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Smartmore Technology Co Ltd
Original Assignee
Shenzhen Smartmore Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Smartmore Technology Co Ltd filed Critical Shenzhen Smartmore Technology Co Ltd
Priority to CN202211127995.9A priority Critical patent/CN115456858B/en
Publication of CN115456858A publication Critical patent/CN115456858A/en
Application granted granted Critical
Publication of CN115456858B publication Critical patent/CN115456858B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Image Processing (AREA)

Abstract

The application provides an image processing method, an image processing device, computer equipment and a computer readable storage medium, and relates to the technical field of computers. The method comprises the following steps: reading image pixel data corresponding to pixel units in an image to be processed from external storage equipment; performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to external storage equipment; taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as a pixel unit; returning to the step of reading image pixel data corresponding to pixel units in the image to be processed from the external storage equipment until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed. By adopting the method and the device, the efficiency of pixel shuffling can be improved on the computing equipment with limited computing resources.

Description

Image processing method, device, computer equipment and computer readable storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an image processing method, an image processing apparatus, a computer device, and a computer readable storage medium.
Background
With the development of artificial intelligence technology, a deep neural network has emerged, and in the structure of the deep neural network, it is a very common operation to amplify a feature map. Wherein, pixel shuffling, or referred to as depth-to-space operator, is one implementation of sub-pixel convolution, which is accomplished by rearranging matrix elements during implementation.
However, the existing pixel shuffling or depth-to-spatial block operators directly perform corresponding dimension conversion on the complete matrix data or image data. Therefore, after the complete matrix or image data is fetched from the external storage device, a large amount of data operations are needed to realize pixel shuffling, that is, data rearrangement of the complete matrix or image, which is not beneficial to the deployment of the neural network on the computing device with limited computing resources.
Disclosure of Invention
The application provides an image processing method, an image processing device, computer equipment and a computer readable storage medium, which can improve the efficiency of pixel shuffling on computing equipment with limited computing resources.
In a first aspect, the present application provides an image processing method applied to a pixel shuffling unit connected to an external storage device, where the external storage device stores an image to be processed, the method including:
reading image pixel data corresponding to pixel units in an image to be processed from external storage equipment; the pixel unit consists of at least one pixel point;
performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to external storage equipment; the pixel shuffling is to convolve the image pixel data;
taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as a pixel unit;
returning to the step of reading image pixel data corresponding to pixel units in the image to be processed from the external storage equipment until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
In a second aspect, the present application further provides an image processing apparatus, applied to a pixel shuffling unit connected to an external storage device, where the external storage device stores an image to be processed, the apparatus including:
The reading module is used for reading image pixel data corresponding to the pixel units in the image to be processed from the external storage equipment; the pixel unit consists of at least one pixel point;
the shuffling module is used for carrying out pixel shuffling on the image pixel data corresponding to the pixel units, obtaining the feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to the external storage equipment; the pixel shuffling is to convolve the image pixel data;
the replacing module is used for taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as a pixel unit;
the circulation module is used for returning to the step of reading the image pixel data corresponding to the pixel units in the image to be processed from the external storage equipment until the feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
In a third aspect, the present application also provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of the above-mentioned image processing method when executing the computer program.
In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the above-described image processing method.
In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the above image processing method.
According to the method, the pixel shuffling units are arranged in the server, pixel shuffling can be directly carried out on any pixel unit of the picture to be processed, the pixel unit which does not need to be processed can be temporarily stored in the external storage device by using the newly proposed address operation, and only the pixel unit which is being processed needs to be taken out from the external storage device, so that volatile storage resources of heterogeneous computing equipment are saved. And meanwhile, the data in the volatile storage resources of the heterogeneous computing equipment do not need to be subjected to a large amount of shift or similar operation, and the pixel shuffling can be completed directly at the stage of data extraction, so that the data processing efficiency is greatly improved.
Drawings
Fig. 1 is an application environment diagram of an image processing method according to an embodiment of the present application;
Fig. 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;
fig. 3 is a flowchart of a method for obtaining a feature map corresponding to a pixel unit according to an embodiment of the present application;
fig. 4 is a flowchart of a method for obtaining image pixel data corresponding to a pixel point according to an embodiment of the present application;
fig. 5 is a flowchart of a method for reading image pixel data corresponding to a pixel unit according to an embodiment of the present application;
fig. 6 is a flowchart of a method for obtaining a pixel address corresponding to any pixel according to an embodiment of the present application;
fig. 7 is a flow chart of a multi-thread computing and judging method according to an embodiment of the present application;
fig. 8 is a block diagram of an image processing apparatus according to an embodiment of the present application;
FIG. 9 is an internal block diagram of a computer device according to an embodiment of the present application;
fig. 10 is a block diagram of a computer readable storage medium according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The image processing method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the external storage device 102 obtains data; the server 104 receives data from the external storage device 102 in response to the instruction, and calculates the acquired data; the server 104 transmits the calculation result of the data back to the external storage device 102, and the external storage device 102 displays the calculation result. Wherein the external storage device 102 communicates with the server 104 through a communication network. The server 104 reads image pixel data corresponding to pixel units in the image to be processed from the external storage device 102; the pixel unit consists of at least one pixel point; performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to external storage equipment; the pixel shuffling is to convolve the image pixel data; taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as a pixel unit; returning to the step of reading image pixel data corresponding to pixel units in the image to be processed from the external storage equipment until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed. The external storage device 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In some embodiments, as shown in fig. 2, an image processing method is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
step 202, reading image pixel data corresponding to pixel units in an image to be processed from an external storage device.
The external storage device may be a device for storing images other than the computing unit, and may be a hard disk, a mobile storage device, or the like.
The image to be processed can be stored in an external storage device and is not subjected to pixel shuffling, the image to be processed comprises at least one pixel unit, and the image to be processed is cut into a plurality of pixel units according to the principle of pixel point arrangement sequence.
The pixel unit can be one of cutting fragments of the image to be processed after cutting, and the pixel unit consists of at least one pixel point.
The image pixel data may be data contained in each pixel point in the pixel unit, and for any one pixel point, at least one channel correspondingly expresses the image pixel data corresponding to the pixel point.
Specifically, the pixel shuffling unit reads image pixel data corresponding to pixel units in an image to be processed from an external storage device through a communication network, wherein for any one pixel unit, the pixel shuffling unit consists of at least one pixel point, and for any one pixel point, the pixel shuffling unit correspondingly expresses the image pixel data corresponding to the pixel point through at least one channel.
For example, the pixel shuffling unit a reads image pixel data corresponding to a pixel unit in the image to be processed from the external storage device B, and the read pixel unit has a quantized form.
And 204, performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature maps corresponding to the pixel units, and outputting the feature maps corresponding to the pixel units to external storage equipment.
The pixel shuffling can be an action of processing the picture by using sub-pixel convolution (sub-pixel convolution), wherein the sub-pixel convolution is a smart image and feature map upscale method, is a simplified form of normal convolution, and has the strong assumption that a large number of convolution operations are eliminated. The result of the sub-pixel convolution is typically a larger picture that can be used in super-resolution calculations.
The feature map corresponding to the pixel unit may be an image obtained by shuffling pixels of the pixel unit, where the feature map corresponding to the pixel unit is a larger image, and may represent information expressed by a plurality of channels corresponding to pixel points in the pixel unit.
Specifically, each pixel unit is integrally input to a pixel shuffling unit, pixel shuffling is performed on image pixel data in the pixel unit, that is, sub-pixel convolution is performed, a feature map corresponding to the pixel unit is obtained after pixel shuffling, and the feature map corresponding to the pixel unit is output to an external storage device. Typically, the feature map after pixel shuffling is a larger image that can express the information contained in each channel in the pixel shuffling.
For example, the pixel units 1-10 are respectively input into the pixel shuffling unit a, the pixel shuffling unit a performs pixel shuffling on the input pixel units 1-10, that is, performs sub-pixel convolution to obtain feature maps corresponding to the pixel units 1-10, and outputs the feature maps corresponding to the pixel units 1-10 to the external storage device B.
In step 206, a pixel unit composed of the next pixel point of each pixel point in the pixel units is used as the pixel unit.
The pixel arrangement sequence may be a preset sequence, which is used to cut the image to be processed into each pixel unit, and is also used to splice the feature images after pixel shuffling, where for the segmented pixel unit, the sequence is represented by the address of the first pixel in each pixel unit, and for the feature images after pixel shuffling, the sequence is represented by the address of the first image pixel data in each feature image.
Specifically, after the previous pixel unit runs the pixel shuffling, the corresponding feature map is output to the external storage device, and meanwhile, the pixel shuffling unit reads the next pixel unit as the pixel unit to be processed based on the preset pixel point arrangement sequence.
For example, after the pixel unit No. 5 runs the pixel shuffling, the corresponding feature map is output to the external storage device B, and the pixel shuffling unit a reads the pixel unit No. 6 as the pixel unit to be processed based on the preset pixel point arrangement sequence.
And step 208, returning to the step of reading the image pixel data corresponding to the pixel units in the image to be processed from the external storage device until the feature maps corresponding to all the pixel units in the image to be processed are obtained.
Specifically, after the previous pixel unit finishes the pixel shuffling operation, inputting the next pixel unit to the pixel shuffling unit, and repeatedly operating the pixel shuffling unit to input each pixel unit to the pixel shuffling unit as a whole, performing pixel shuffling on the image pixel data in the pixel unit, namely performing sub-pixel convolution, obtaining a feature map corresponding to the pixel unit after the pixel shuffling, and outputting the feature map corresponding to the pixel unit to an external storage device. In general, the feature map obtained by pixel shuffling is a larger image, and information contained in each channel in the pixel shuffling can be expressed until all pixel units corresponding to the image to be processed are subjected to pixel shuffling to obtain a corresponding feature map. And the feature maps corresponding to all the pixel units are used for forming the pixel shuffled image corresponding to the image to be processed.
For example, after the 5 th pixel unit completes the pixel shuffling operation, the 6 th pixel unit is input to the pixel shuffling unit, the pixel shuffling unit repeatedly operates to input each pixel unit to the pixel shuffling unit as a whole, pixel shuffling is performed on the image pixel data in the pixel unit, that is, sub-pixel convolution is performed, a feature map corresponding to the pixel unit is obtained after the pixel shuffling, and the feature map corresponding to the pixel unit is output to an external storage device. In general, the feature map obtained by pixel shuffling is a larger image, and information contained in each channel in the pixel shuffling can be expressed until all 100 pixel units corresponding to the image to be processed are subjected to pixel shuffling to obtain a corresponding feature map. The feature map corresponding to the 100 pixel units is used for forming a pixel shuffled image corresponding to the image to be processed.
In the image processing method, image pixel data corresponding to pixel units in an image to be processed are read from external storage equipment; the pixel unit consists of at least one pixel point; performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to external storage equipment; the pixel shuffling is to convolve the image pixel data; taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as a pixel unit; returning to the step of reading image pixel data corresponding to pixel units in the image to be processed from the external storage equipment until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
By using a method of arranging pixel shuffling units in a server, any pixel unit of a picture to be processed can be directly subjected to pixel shuffling, and the pixel units which do not need to be processed can be temporarily stored in external storage equipment by using the newly proposed address operation, so that only the pixel units which are being processed need to be taken out from the external storage equipment, and the volatile storage resources of heterogeneous computing equipment are saved. And meanwhile, the pixel shuffling can be completed directly at the stage of data fetching without carrying out a large amount of shift or similar operation on the data in the volatile storage resources of the heterogeneous computing equipment, so that the data processing efficiency is greatly improved.
In some embodiments, as shown in fig. 3, when a pixel unit is composed of at least two pixel points, pixel shuffling is performed on image pixel data corresponding to the pixel unit, to obtain a feature map corresponding to the pixel unit, including:
step 302, pixel shuffling is performed on the image pixel data corresponding to the pixel points in the pixel unit, so as to obtain a feature map corresponding to the pixel points.
Specifically, the pixel shuffling unit reads image pixel data corresponding to pixel points in pixel units in an image to be processed from an external storage device through a communication network, wherein the pixel shuffling unit consists of at least one pixel point for any one pixel unit and at least one channel for any one pixel point correspondingly expresses the image pixel data corresponding to the pixel point. And inputting the whole pixel units into a pixel shuffling unit, performing pixel shuffling on the image pixel data of the pixel points in the pixel units, namely running sub-pixel convolution, and obtaining a feature map corresponding to the pixel points after the pixel shuffling. Typically, the feature map after pixel shuffling is a larger image that can express the information contained in each channel in the pixel shuffling.
For example, the pixel shuffling unit a reads image pixel data corresponding to the pixel points in the pixel units in the image to be processed from the external storage device B, and the read pixel points in the pixel units have a quantized form. Respectively inputting the pixel points 1-10 into a pixel shuffling unit A, and performing pixel shuffling on the input pixel points 1-10 by the pixel shuffling unit A, namely running sub-pixel convolution to obtain a characteristic diagram corresponding to the pixel points 1-10
Step 304, using the next pixel point of the pixel points in the pixel unit as the pixel point; and returning to the step of carrying out pixel shuffling on the image pixel data corresponding to the pixel points in the pixel unit to obtain the feature images corresponding to the pixel points until the feature images corresponding to all the pixel points in the pixel unit are obtained.
Specifically, after the pixel shuffling of the previous pixel is completed, the corresponding feature map is output to the external storage device, and meanwhile, the pixel shuffling unit reads the next pixel as the pixel to be processed based on the preset pixel arrangement sequence.
After the previous pixel point finishes the pixel shuffling operation, inputting the next pixel point to the pixel shuffling unit, and repeatedly operating the pixel shuffling unit to integrally input each pixel unit to the pixel shuffling unit, and performing pixel shuffling on the image pixel data of the pixel points in the pixel unit, namely operating sub-pixel convolution, so as to obtain a feature map corresponding to the pixel points after the pixel shuffling. In general, the feature map obtained by pixel shuffling is a larger image, and information contained in each channel in the pixel shuffling can be expressed until all pixel points corresponding to the image to be processed are subjected to pixel shuffling to obtain a corresponding feature map.
For example, after the number 5 pixel point runs the pixel shuffling, the corresponding feature map is output to the external storage device B, and the pixel shuffling unit a reads the number 6 pixel point as the pixel point to be processed based on the preset pixel point arrangement sequence.
After the 5 th pixel point finishes the pixel shuffling operation, inputting the 6 th pixel point to a pixel shuffling unit, repeatedly operating the pixel shuffling unit to integrally input each pixel unit to the pixel shuffling unit, and performing pixel shuffling on the image pixel data of the pixel points in the pixel unit, namely operating sub-pixel convolution, and obtaining a feature map corresponding to the pixel points after the pixel shuffling. In general, the feature map obtained by pixel shuffling is a larger image, and can express information contained in each channel in the pixel shuffling until all 100 pixel points corresponding to the image to be processed are subjected to pixel shuffling to obtain a corresponding feature map.
Step 306, determining the feature map corresponding to the pixel unit according to the feature maps corresponding to all the pixel points in the pixel unit.
Specifically, the feature maps corresponding to all the pixel points are used for forming the pixel shuffled image corresponding to the pixel units.
For example, the feature map corresponding to 100 pixels is used to compose a pixel shuffled image corresponding to the pixel unit.
In this embodiment, the pixel shuffling unit performs pixel shuffling on all the pixel points in the pixel unit, so that a feature map corresponding to all the pixel points in the pixel unit can be obtained, and the definition of the sub-image to be processed corresponding to the pixel unit is improved.
In some embodiments, as shown in fig. 4, the image to be processed has a plurality of channels, and reading image pixel data corresponding to pixel units in the image to be processed from an external storage device includes:
step 402, for each pixel point in the pixel unit, reading image pixel data corresponding to the pixel point in the current channel from the external storage device.
Specifically, the pixel shuffling unit reads image pixel data corresponding to a pixel point in a current channel from an external storage device through a communication network, and for any one pixel unit, the pixel shuffling unit consists of at least one pixel point, and for any one pixel point, the image pixel data corresponding to the pixel point is correspondingly expressed by at least one channel.
For example, the pixel shuffling unit a reads image pixel data corresponding to the current channel from the external storage device B, and the read channel has a quantized form.
Step 404, taking the next channel of the current channels as the current channel; and returning to the step of reading the image pixel data corresponding to the pixel point in the current channel from the external storage device until the image pixel data corresponding to the pixel point in all channels is obtained.
Specifically, after the previous channel finishes the operation of reading the image pixel data corresponding to the pixel point in the current channel, inputting the next channel to the pixel shuffling unit, and repeatedly reading the image pixel data corresponding to the pixel point in the current channel from the external storage device through the communication network by the pixel shuffling unit until the image pixel data corresponding to all channels of the pixel point is read.
For example, after the 5 th channel finishes the operation of reading the image pixel data corresponding to the pixel point in the current channel, inputting the 6 th channel to the pixel shuffling unit, and repeatedly reading the image pixel data corresponding to the pixel point in the current channel from the external storage device through the communication network by the pixel shuffling unit until the image pixel data corresponding to the pixel point in all 100 channels is read.
In step 406, the image pixel data corresponding to the pixel points in all channels is used as the image pixel data corresponding to the pixel points.
Specifically, the image pixel data of all channels corresponding to the pixel point are combined to obtain the image pixel data corresponding to the pixel point.
For example, the image pixel data of 100 channels corresponding to the pixel point is combined to obtain the image pixel data corresponding to the pixel point.
In this embodiment, the pixel shuffling unit extracts image pixel data corresponding to all channels of a pixel, so that image pixel data corresponding to all channels in the pixel can be obtained, and the definition of the sub-image to be processed corresponding to the pixel is improved.
In some embodiments, as shown in fig. 5, before reading image pixel data corresponding to a pixel unit in an image to be processed from an external storage device, the method further includes:
step 502, obtaining a positional relationship between each pixel point and a first pixel point in the pixel unit.
The first pixel may be a pixel with a first address rank within each pixel in each pixel unit.
The positional relationship may be a relationship between an address of each pixel point in the pixel unit and the first pixel point.
Specifically, the pixel shuffling unit reads addresses of all the pixels in the pixel unit from the external storage device, and obtains a positional relationship between any one pixel except the first pixel relative to the first pixel by taking the address of the first pixel as a reference frame.
For example, the pixel shuffling unit a reads addresses 1-100 corresponding to each (100) pixel points in the pixel unit 1 from the external storage device B, and uses the address of the pixel point 1 as a reference frame to obtain the positional relationship between the pixel points 2-100 except the pixel point 1 relative to the pixel point 1.
Step 504, determining the pixel address corresponding to each pixel point according to the position relationship and the pixel address corresponding to the first pixel point.
Specifically, according to the positional relationship between the address of the pixel corresponding to the first pixel point and the addresses of other pixels points and the address of the first pixel point, the address corresponding to any one pixel point in the same pixel unit can be determined. The calculation formula of the position relation is as follows: addr=0x0a00 0000+20 (offset) n (data_width)/8 (1 byte), addr is a positional relationship, and n is a number of pixel points.
For example, according to the positional relationship between the address 1 of the first pixel and the addresses 2-100 of the other pixels and the address 1 of the first pixel, the addresses 2-100 corresponding to the pixels 2-100 in the same pixel unit can be determined.
Step 506, according to the pixel address corresponding to each pixel point, image pixel data corresponding to the pixel unit in the image to be processed is read from the external storage device.
Specifically, after the addresses of the rest pixel points are further determined according to the address of the first pixel point in the pixel unit, the pixel shuffling unit reads image pixel data corresponding to the pixel unit in the image to be processed from the external storage device according to the determined pixel point address.
For example, after further determining the addresses 2-100 of the rest of the pixel points according to the address 1 of the first pixel point in the pixel unit, the pixel shuffling unit reads the image pixel data 1-100 corresponding to the pixel unit in the image to be processed from the external storage device according to the determined pixel point address.
In this embodiment, by using the pixel address corresponding to the first pixel and the positional relationship between each pixel and the first pixel, the accurate address of any one pixel can be accurately located, so that the efficiency of reading the pixels by the pixel shuffling unit is improved.
In some embodiments, as shown in fig. 6, determining the pixel address corresponding to each pixel point according to the position relationship and the pixel address corresponding to the first pixel point includes:
step 602, for any pixel point in the pixel unit, obtaining a position relationship corresponding to the any pixel point.
Specifically, for any one pixel unit in an image to be processed, any one pixel point in a plurality of pixel points contained in the image to be processed, a corresponding relation between the pixel point and a first pixel point is obtained.
For example, for any one pixel unit N in the image to be processed, which includes 100 pixels, for any one pixel m, the corresponding relationship between the pixel m and the first pixel is obtained.
Step 604, determining the pixel address corresponding to any pixel according to the pixel address corresponding to the first pixel and the address offset corresponding to the position relationship corresponding to any pixel.
The address offset may be a distance between any one of the selected pixels and the reference pixel, where the distance is represented by a difference between the address of the selected any one of the pixels and the address of the reference pixel.
Specifically, according to the position relation corresponding to any one pixel point in the pixel unit, the offset of any one pixel point relative to the address of the first pixel point is further determined, and the pixel address corresponding to any one pixel point in the pixel unit is obtained by adding or subtracting the offset based on the address of the first pixel point.
For example, according to the position relationship corresponding to any one pixel point in the pixel unit, the offset 2-100 of the pixel point 2-100 relative to the address of the first pixel point is further determined, and the pixel addresses 2-100 corresponding to the pixel points 2-100 in the pixel unit are obtained by adding or subtracting the offset 2-100 based on the address of the first pixel point.
In this embodiment, by introducing the address offset corresponding to the position relationship corresponding to any pixel point and the pixel address corresponding to the first pixel point, the address of any pixel point can be accurately obtained by adding the address offset and the pixel address corresponding to the first pixel point, and the reading accuracy of the address of any pixel point is improved.
In some embodiments, as shown in fig. 7, the method further comprises:
step 702, obtaining a ratio between a full data processing amount corresponding to the pixel shuffling unit and a data amount corresponding to one pixel unit.
The full data processing amount may be the data processing amount corresponding to the processing data of the pixel shuffling unit by using all resources.
Specifically, the data processing amount which can be processed when the pixel shuffling unit is fully loaded and the processed data amount corresponding to one pixel unit are obtained, and the full-loaded data processing amount corresponding to the pixel shuffling unit is divided by the data amount corresponding to one pixel unit to obtain the ratio between the full-loaded data processing amount and the processed data amount corresponding to one pixel unit.
For example, if the full data throughput corresponding to the pixel shuffling unit is 10 and the data throughput corresponding to one pixel unit is 2, the ratio between the full data throughput corresponding to the pixel shuffling unit and the data throughput corresponding to one pixel unit is 5.
If the ratio is greater than or equal to the threshold, step 704, rounding the ratio as the number of multiple threads, and causing the pixel shuffling unit to perform multi-threaded computations according to the number of multiple threads.
Specifically, if the ratio between the full data processing amount corresponding to the pixel shuffling unit and the data amount corresponding to one pixel unit is greater than or equal to a preset threshold value, rounding the obtained ratio as the number of threads of the multithreading process of the pixel shuffling unit, and using the pixel shuffling unit to calculate according to the number of threads by using multithreading.
For example, if the ratio between the full data processing amount corresponding to the pixel shuffling unit and the data amount corresponding to one pixel unit is 5.5 and is greater than the preset threshold value 2, the obtained ratio is rounded by 5.5, that is, 6 is used as the number of threads 6 of the multithreading of the pixel shuffling unit, and the pixel shuffling unit is used for multithreading calculation according to the number of threads 6.
In this embodiment, the ratio between the full-load data processing amount corresponding to the pixel shuffling unit and the data amount corresponding to one pixel unit is adopted, so that the number of pixel units which can be processed by the pixel shuffling unit at the same time can be known, and the efficiency of pixel shuffling is improved by further adopting multithreading.
In some embodiments, DDR memory and addressing is done as follows:
1. in DDR, the data is stored in sequential order by operators that have not undergone pixel shuffling. In general, when data is read from DDR, data can be obtained by using burst transfer (burst) of AXI4 protocol, only by giving a start address of data and a required data length. After burst transmission obtains the start address and the length of data to be transmitted, the data with the continuous length required by the user is fetched from the start address.
2. The data length to be read is given according to the data amount required each time, if only 9 data are required, and the bit width of each data is 16-bit, the data length rd_data_length=9 (data_number) 16 (data_width)/8 (1 byte) to be read is required, and the output data length is in bytes.
3. For the addresses of individual pixels, since the image is stored sequentially and consecutively in DDR, we first need to know the address of the first pixel of the image, then calculate the address of the pixel according to the data bit width of the pixel and the offset of the pixel from the first pixel. For example, the address of the first pixel is base_addr=0x0a00 0000, the data bit width of a single pixel is 16-bits, and the address of the 20 th pixel is found, and then the address of the 16 th pixel is addr=0x0a00 0000+20 (offset) ×16 (data_width)/8 (1 byte). The calculation process is simply summarized in practice as the base address+offset data length, the base address being preset.
In some embodiments, pixel shuffling is accomplished by addressing. To complete the conversion of this pixel arrangement sequence, the address of the first pixel is given, the first pixel is read in, the first pixel is rewritten into the new address of the DDR, the address of the second pixel is given, the second pixel is read in, the DDR is rewritten, and the address of the DDR is written back to be immediately adjacent to the new address. By analogy, the whole pixel shuffling process can be completed by merely caching the pictures through the control logic of address addressing without separately opening up the memory space of the heterogeneous computing device. The heterogeneous computing device memory space required to be additionally opened is mainly referred to as BRAM or SRAM, and the data fetched each time is mainly temporarily stored by a register/latch used in the control process.
In practical use, only a single pixel is not typically transferred, and thus the data transfer efficiency is greatly reduced. The amount of data actually transferred at one time depends on the model and the role the operator plays in the model. In addition, when the data is read according to the sequence or mode of the pixel shuffling, the data is not required to be restored to the DDR to finish a complete pixel shuffling, and the data can be transmitted to a subsequent logic module for further processing, wherein the pixel shuffling process is considered to be finished in the process of reading the data, and the data is read according to a preset sequence, namely the completion of the pixel shuffling of the current pixel.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an image processing device. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the image processing device provided below may refer to the limitation of the image processing method hereinabove, and will not be repeated here.
In some embodiments, as shown in fig. 8, there is provided an image processing apparatus applied to a pixel shuffling unit connected to an external storage device storing images to be processed, including:
a reading module 802, configured to read image pixel data corresponding to a pixel unit in an image to be processed from an external storage device; the pixel unit consists of at least one pixel point;
the shuffling module 804 is configured to shuffle pixels of image pixel data corresponding to the pixel units to obtain feature maps corresponding to the pixel units, and output the feature maps corresponding to the pixel units to an external storage device; the pixel shuffling is to convolve the image pixel data;
a replacing module 806, configured to take a pixel unit that is formed by a next pixel point of each pixel point in the pixel units as a pixel unit;
a circulation module 808, configured to return to the step of reading, from the external storage device, image pixel data corresponding to pixel units in the image to be processed, until feature maps corresponding to all pixel units in the image to be processed are obtained; and the feature images corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
In some embodiments, when the pixel unit is composed of at least two pixel points, the pixel shuffling is performed on the image pixel data corresponding to the pixel unit, so as to obtain a feature map corresponding to the pixel unit, where the shuffling module is specifically configured to:
carrying out pixel shuffling on image pixel data corresponding to the pixel points in the pixel units to obtain a feature map corresponding to the pixel points;
taking the next pixel point of the pixel points in the pixel unit as the pixel point; returning to the step of performing pixel shuffling on the image pixel data corresponding to the pixel points in the pixel units to obtain feature images corresponding to the pixel points until feature images corresponding to all the pixel points in the pixel units are obtained;
and determining the feature map corresponding to the pixel unit according to the feature maps corresponding to all the pixel points in the pixel unit.
In some embodiments, in an aspect that the image to be processed has a plurality of channels, the reading module is specifically configured to, in reading image pixel data corresponding to a pixel unit in the image to be processed from the external storage device:
for each pixel point in the pixel unit, reading image pixel data corresponding to the pixel point in the current channel from external storage equipment;
taking the next channel of the current channels as the current channel; returning to the step of reading the image pixel data corresponding to the pixel point in the current channel from the external storage device until the image pixel data corresponding to the pixel point in all channels is obtained;
And taking the image pixel data corresponding to the pixel points in all the channels as the image pixel data corresponding to the pixel points.
In some embodiments, before reading image pixel data corresponding to a pixel unit in the image to be processed from the external storage device, the reading module is further configured to:
acquiring the position relation between each pixel point and the first pixel point in the pixel unit;
determining the pixel address corresponding to each pixel point according to the position relation and the pixel address corresponding to the first pixel point;
reading image pixel data corresponding to pixel units in an image to be processed from an external storage device, including:
and reading image pixel data corresponding to the pixel units in the image to be processed from external storage equipment according to the pixel addresses corresponding to the pixel points.
In some embodiments, in determining the pixel address corresponding to each pixel according to the positional relationship and the pixel address corresponding to the first pixel, the reading module is specifically configured to:
for any pixel point in the pixel unit, acquiring a position relation corresponding to the any pixel point;
and determining the pixel address corresponding to any pixel point according to the address offset corresponding to the position relation corresponding to any pixel point and the pixel address corresponding to the first pixel point.
In some embodiments, the shuffling module is specifically further configured to:
acquiring the ratio between the full-load data processing amount corresponding to the pixel shuffling unit and the data amount corresponding to one pixel unit;
if the ratio is greater than or equal to the threshold, rounding the ratio as the number of the multiple threads, and enabling the pixel shuffling unit to execute the multiple thread calculation according to the number of the multiple threads.
The respective modules in the above-described image processing apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 9. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing server data. The communication interface of the computer device is used for communicating with an external terminal through a network connection. Which computer program, when being executed by a processor, carries out the steps of the image processing method described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 9 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In some embodiments, a computer device is also provided, comprising a memory and a processor, the memory storing a computer program, the processor implementing the steps of the image processing method described above when executing the computer program.
In some embodiments, a computer readable storage medium is provided, as shown in fig. 10, which stores a computer program which, when executed by a processor, implements the steps of the image processing method described above.
In some embodiments, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the above-described image processing method.
It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims (10)

1. An image processing method, applied to a pixel shuffling unit connected to an external storage device, the external storage device storing an image to be processed, the method comprising:
reading image pixel data corresponding to pixel units in the image to be processed from the external storage device; wherein the pixel unit consists of at least one pixel point;
Performing pixel shuffling on the image pixel data corresponding to the pixel units to obtain feature images corresponding to the pixel units, and outputting the feature images corresponding to the pixel units to the external storage equipment; the pixel shuffling is to convolve the image pixel data; wherein the convolution process is a sub-pixel convolution process;
taking a pixel unit formed by the next pixel point of each pixel point in the pixel units as the pixel unit;
returning to the step of reading the image pixel data corresponding to the pixel units in the image to be processed from the external storage device until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature graphs corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
2. The method according to claim 1, wherein when the pixel unit is composed of at least two pixel points, the performing pixel shuffling on the image pixel data corresponding to the pixel unit to obtain a feature map corresponding to the pixel unit includes:
performing pixel shuffling on image pixel data corresponding to pixel points in the pixel units to obtain a feature map corresponding to the pixel points;
Taking the next pixel point of the pixel points in the pixel unit as the pixel point; returning to the step of performing pixel shuffling on the image pixel data corresponding to the pixel points in the pixel unit to obtain a feature map corresponding to the pixel points until feature maps corresponding to all the pixel points in the pixel unit are obtained;
and determining the feature map corresponding to the pixel unit according to the feature maps corresponding to all the pixel points in the pixel unit.
3. The method according to claim 2, wherein the image to be processed has a plurality of channels, and the reading image pixel data corresponding to pixel units in the image to be processed from the external storage device includes:
for each pixel point in the pixel unit, reading image pixel data corresponding to the pixel point in a current channel from the external storage device;
taking the next channel of the current channel in the plurality of channels as the current channel; returning to the step of reading the image pixel data corresponding to the pixel point in the current channel from the external storage device until the image pixel data corresponding to all the channels of the pixel point are obtained;
And taking the image pixel data corresponding to the pixel points in all the channels as the image pixel data corresponding to the pixel points.
4. The method of claim 1, wherein before the reading, from the external storage device, image pixel data corresponding to pixel units in the image to be processed, the method further comprises:
acquiring the position relation between each pixel point in the pixel unit and the first pixel point;
determining a pixel address corresponding to each pixel point according to the position relation and the pixel address corresponding to the first pixel point;
the reading the image pixel data corresponding to the pixel unit in the image to be processed from the external storage device includes:
and reading image pixel data corresponding to pixel units in the image to be processed from the external storage equipment according to pixel addresses corresponding to the pixel points.
5. The method of claim 4, wherein determining the pixel address corresponding to each pixel according to the positional relationship and the pixel address corresponding to the first pixel comprises:
for any pixel point in the pixel unit, acquiring a position relation corresponding to the any pixel point;
And determining the pixel address corresponding to any pixel point according to the pixel address corresponding to the first pixel point and the address offset corresponding to the position relation corresponding to any pixel point.
6. The method according to any one of claims 1 to 5, further comprising:
acquiring the ratio between the full-load data processing amount corresponding to the pixel shuffling unit and the data amount corresponding to one pixel unit;
and if the ratio is greater than or equal to a threshold value, rounding the ratio to be the number of the multithreading, and enabling the pixel shuffling unit to execute multithreading calculation according to the number of the multithreading.
7. An image processing apparatus, characterized by being applied to a pixel shuffling unit connected to an external storage device storing an image to be processed, comprising:
the reading module is used for reading image pixel data corresponding to the pixel units in the image to be processed from the external storage equipment; wherein the pixel unit consists of at least one pixel point;
the shuffling module is used for carrying out pixel shuffling on the image pixel data corresponding to the pixel units, obtaining a feature map corresponding to the pixel units, and outputting the feature map corresponding to the pixel units to the external storage equipment; the pixel shuffling is to convolve the image pixel data; wherein the convolution process is a sub-pixel convolution process;
A replacement module, configured to take a pixel unit composed of a next pixel point of each pixel point in the pixel units as the pixel unit;
the circulation module is used for returning to the step of reading the image pixel data corresponding to the pixel units in the image to be processed from the external storage device until feature images corresponding to all the pixel units in the image to be processed are obtained; and the feature graphs corresponding to all the pixel units in the image to be processed are used for forming the image after pixel shuffling corresponding to the image to be processed.
8. The apparatus of claim 7, wherein when the pixel unit is composed of at least two pixel points, the pixel shuffling is performed on the image pixel data corresponding to the pixel unit to obtain a feature map corresponding to the pixel unit, and the shuffling module is specifically configured to:
performing pixel shuffling on image pixel data corresponding to pixel points in the pixel units to obtain a feature map corresponding to the pixel points;
taking the next pixel point of the pixel points in the pixel unit as the pixel point; returning to the step of performing pixel shuffling on the image pixel data corresponding to the pixel points in the pixel unit to obtain a feature map corresponding to the pixel points until feature maps corresponding to all the pixel points in the pixel unit are obtained;
And determining the feature map corresponding to the pixel unit according to the feature maps corresponding to all the pixel points in the pixel unit.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
CN202211127995.9A 2022-09-16 2022-09-16 Image processing method, device, computer equipment and computer readable storage medium Active CN115456858B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211127995.9A CN115456858B (en) 2022-09-16 2022-09-16 Image processing method, device, computer equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211127995.9A CN115456858B (en) 2022-09-16 2022-09-16 Image processing method, device, computer equipment and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN115456858A CN115456858A (en) 2022-12-09
CN115456858B true CN115456858B (en) 2023-07-18

Family

ID=84304727

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211127995.9A Active CN115456858B (en) 2022-09-16 2022-09-16 Image processing method, device, computer equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN115456858B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112749709A (en) * 2019-10-30 2021-05-04 浙江商汤科技开发有限公司 Image processing method and device, electronic equipment and storage medium
CN114758209A (en) * 2022-06-14 2022-07-15 深圳思谋信息科技有限公司 Convolution result obtaining method and device, computer equipment and storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106328045B (en) * 2015-07-06 2019-07-16 西安诺瓦电子科技有限公司 Programmable logic device and its sub-pix Downsapling method and data encoding circuit
CN111915673B (en) * 2020-07-22 2022-01-11 深圳云天励飞技术股份有限公司 Image processing method, image processing device, terminal equipment and storage medium
CN111897579B (en) * 2020-08-18 2024-01-30 腾讯科技（深圳）有限公司 Image data processing method, device, computer equipment and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112749709A (en) * 2019-10-30 2021-05-04 浙江商汤科技开发有限公司 Image processing method and device, electronic equipment and storage medium
CN114758209A (en) * 2022-06-14 2022-07-15 深圳思谋信息科技有限公司 Convolution result obtaining method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN115456858A (en) 2022-12-09

Similar Documents

Publication Publication Date Title
US10936937B2 (en) Convolution operation device and convolution operation method
US11436017B2 (en) Data temporary storage apparatus, data temporary storage method and operation method
CN111311599B (en) Image processing method, device, electronic equipment and storage medium
CN111279384B (en) Compression and decompression of indices in a graphics pipeline
KR20190022237A (en) Method and apparatus for performing convolution operation in neural network
CN107808394B (en) Image processing method based on convolutional neural network and mobile terminal
CN117370488A (en) Data processing method, device, electronic equipment and computer readable storage medium
US20210241408A1 (en) Data processing method and sensor device for performing the same
CN112348182A (en) Neural network maxout layer computing device
CN115456858B (en) Image processing method, device, computer equipment and computer readable storage medium
CN116051345A (en) Image data processing method, device, computer equipment and readable storage medium
US11741349B2 (en) Performing matrix-vector multiply operations for neural networks on electronic devices
CN114648444A (en) Vector up-sampling calculation method and device applied to neural network data processing
CN109375952B (en) Method and apparatus for storing data
CN116527908B (en) Motion field estimation method, motion field estimation device, computer device and storage medium
CN113111891B (en) Image reconstruction method and device, terminal equipment and storage medium
CN115665396B (en) Demura compensation value decompression method and device based on code stream random access
CN116659520B (en) Matching positioning method, device and equipment based on bionic polarization vision enhancement
CN115665335B (en) Image processing method, image processing apparatus, image forming apparatus, and medium
CN116503694B (en) Model training method, image segmentation device and computer equipment
CN111598189B (en) Generative model training method, data generation method, device, medium, and apparatus
CN115908115A (en) Face image processing method, live image processing method and device and electronic equipment
CN118230178A (en) Spatial distribution identification method, device and equipment for construction waste and storage medium
WO2022221982A1 (en) Image reconstruction method and apparatus, terminal device, and storage medium
CN116503241A (en) Image superdivision method, device, equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant