CN115348432B - Data processing method and device, image processing method, electronic equipment and medium - Google Patents
Data processing method and device, image processing method, electronic equipment and medium Download PDFInfo
- Publication number
- CN115348432B CN115348432B CN202210976569.6A CN202210976569A CN115348432B CN 115348432 B CN115348432 B CN 115348432B CN 202210976569 A CN202210976569 A CN 202210976569A CN 115348432 B CN115348432 B CN 115348432B
- Authority
- CN
- China
- Prior art keywords
- data
- component
- processing method
- image
- rows
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 84
- 238000000034 method Methods 0.000 claims abstract description 17
- 230000015654 memory Effects 0.000 claims description 34
- 238000013528 artificial neural network Methods 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 abstract description 6
- 239000011159 matrix material Substances 0.000 description 53
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000010606 normalization Methods 0.000 description 7
- 238000007781 pre-processing Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/67—Circuits for processing colour signals for matrixing
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
A data processing method, a data processing apparatus, an image processing method, an electronic device, and a computer-readable storage medium. The method comprises the following steps: reading a plurality of input data having a first image format; converting the plurality of input data to obtain a plurality of intermediate data with a second image format; and processing the plurality of intermediate data to obtain a plurality of output data, each input data including a first component, the plurality of first components of the plurality of input data arranged to form a plurality of first component rows, the reading the plurality of input data having a first image format comprising: reading a first group of first component lines in the plurality of first component lines and storing the first group of first component lines in a first register; a second set of first component lines of the plurality of first component lines is read and stored in a second register. The method can improve the utilization efficiency of the register, reduce the calculated amount, improve the calculation efficiency and have better flexibility.
Description
Technical Field
Embodiments of the present disclosure relate to a data processing method, a data processing apparatus, an image processing method, an electronic device, and a computer-readable storage medium.
Background
The residual neural network is widely applied to image classification and object recognition, and is characterized by easy optimization and capability of improving accuracy by increasing the depth of the network. The ResNet structure is one of the most successful algorithms of the residual neural network in image recognition in recent years. The image data can be converted into the input of the residual neural network through the preprocessing process, and in the preprocessing process, the required remolded vector can be obtained through operations such as clipping and splitting the image data so as to carry out subsequent convolution operation. In the preprocessing process, the data folding operation is added, so that the convolution operation efficiency can be greatly improved.
Disclosure of Invention
At least one embodiment of the present disclosure provides a data processing method, including: reading a plurality of input data having a first image format; converting the plurality of input data to obtain a plurality of intermediate data with a second image format; and processing the plurality of intermediate data to obtain a plurality of output data, each input data including a first component, the plurality of first components of the plurality of input data being arranged to form a plurality of first component rows, the reading the plurality of input data having a first image format, comprising: reading a first group of first component lines in the plurality of first component lines and storing the first group of first component lines in a first register; a second set of first component lines of the plurality of first component lines is read and stored in a second register.
For example, in a data processing method provided by an embodiment of the present disclosure, a first set of first component rows and a second set of first component rows are read simultaneously.
For example, in a data processing method provided in an embodiment of the present disclosure, a first set of first component rows includes even rows of a plurality of first component rows, and a second set of first component rows includes odd rows of the plurality of first component rows.
For example, in the data processing method provided in an embodiment of the present disclosure, each input data includes a second component and a third component, the second components and the third components of the input data are arranged to form a second component row, and the method further includes: the plurality of second component rows is read and stored in a third register.
For example, in the data processing method provided in an embodiment of the present disclosure, each second component row includes a plurality of second components and a plurality of third components, and the plurality of second components and the plurality of third components in each second component row are alternately arranged.
For example, in the data processing method provided in an embodiment of the present disclosure, the 2n-1 th first component line and the 2 n-th first component line of the plurality of first component lines correspond to the n-th second component line of the plurality of second component lines, and n is a positive integer.
For example, in the data processing method provided in an embodiment of the present disclosure, the first image format is a YUV format, the first component is a Y component, the second component is a U component, and the third component is a V component.
For example, in the data processing method provided in an embodiment of the present disclosure, the first image format is a 4:2:0 sampling format, and the plurality of input data are stored in the memory in a plane format.
For example, in a data processing method provided in an embodiment of the present disclosure, converting a plurality of input data to obtain a plurality of intermediate data having a second image format includes: based on a conversion relation between the first image format and the second image format, the first component stored in the first register and the second component and the third component stored in the third register are calculated to obtain a plurality of first intermediate data, and the first component stored in the second register and the second component and the third component stored in the third register are calculated to obtain a plurality of second intermediate data, wherein the plurality of intermediate data comprises a plurality of first intermediate data and a plurality of second intermediate data.
For example, in a data processing method provided in an embodiment of the present disclosure, processing a plurality of intermediate data includes: and carrying out normalization processing on the plurality of first intermediate data and the plurality of second intermediate data to obtain normalized plurality of first intermediate data and normalized plurality of second intermediate data.
For example, in a data processing method provided in an embodiment of the present disclosure, a normalized plurality of first intermediate data arrangements form a plurality of first data columns, a normalized plurality of second intermediate data arrangements form a plurality of second data columns, and the processing of the plurality of intermediate data to obtain a plurality of output data includes: obtaining odd columns in the first data columns, and performing filling operation and folding operation on the normalized first intermediate data corresponding to the odd columns in the first data columns to obtain a first group of output data; acquiring even columns in the plurality of first data columns, and performing filling operation and folding operation on the normalized first intermediate data corresponding to the even columns in the plurality of first data columns to obtain second group of output data; acquiring odd columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the odd columns in the plurality of second data columns to obtain third group of output data; acquiring even columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the even columns in the plurality of second data columns to obtain fourth group output data; the plurality of output data includes a first set of output data, a second set of output data, a third set of output data, and a fourth set of output data.
For example, in the data processing method provided in an embodiment of the present disclosure, the second image format is an RGB format.
At least one embodiment of the present disclosure also provides an image processing method, including: acquiring an input image, wherein the image format of data in the input image is a first image format; the data in the input image is used as a plurality of input data, and the data in the input image is processed by using the data processing method provided by at least one embodiment of the present disclosure so as to obtain a plurality of output data corresponding to the input image; and processing a plurality of output data corresponding to the input image by using the neural network to obtain the output of the neural network.
For example, in the image processing method provided in an embodiment of the present disclosure, the neural network is a 50-layer residual neural network.
At least one embodiment of the present disclosure also provides a data processing apparatus including: a reading unit configured to read a plurality of input data having a first image format; a conversion unit configured to convert the plurality of input data to a plurality of intermediate data having a second image format; and a processing unit configured to process the plurality of intermediate data to obtain a plurality of output data, each input data including a first component, the plurality of first components of the plurality of input data being arranged to form a plurality of first component rows, the reading unit being configured to, when performing the step of reading the plurality of input data having the first image format: reading a first group of first component lines in the plurality of first component lines and storing the first group of first component lines in a first register; a second set of first component lines of the plurality of first component lines is read and stored in a second register.
At least one embodiment of the present disclosure also provides an electronic device, including: a processor; and a memory storing computer-executable instructions that, when executed by the processor, implement the data processing method provided by at least one embodiment of the present disclosure or the image processing method provided by at least one embodiment of the present disclosure.
At least one embodiment of the present disclosure also provides a computer-readable storage medium for non-transitory storage of computer-executable instructions that, when executed by a processor, implement the data processing method provided by at least one embodiment of the present disclosure or the image processing method provided by at least one embodiment of the present disclosure.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below, and it is apparent that the drawings in the following description relate only to some embodiments of the present disclosure, not to limit the present disclosure.
FIG. 1A shows a flow chart of data processing steps;
FIG. 1B illustrates an exemplary memory layout of an image format;
FIG. 2 shows a schematic flow chart of a data processing method provided by at least one embodiment of the present disclosure;
fig. 3 shows a schematic flow chart of step S201 in the data processing method shown in fig. 2;
Fig. 4 shows a schematic flow chart of step S203 in the data processing method shown in fig. 2;
FIG. 5 illustrates a flow chart of steps of a data processing method provided by at least one embodiment of the present disclosure;
FIGS. 6A and 6B illustrate a flow chart of one example of a data processing method provided by at least one embodiment of the present disclosure;
FIG. 7 illustrates a schematic flow diagram of an image processing method provided by at least one embodiment of the present disclosure;
FIG. 8 illustrates a schematic block diagram of a data processing apparatus provided in accordance with at least one embodiment of the present disclosure;
FIG. 9 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure;
FIG. 10 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure; and
Fig. 11 is a schematic diagram of a storage medium according to some embodiments of the present disclosure.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. It will be apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without the need for inventive faculty, are within the scope of the present disclosure, based on the described embodiments of the present disclosure.
Unless defined otherwise, technical or scientific terms used in this disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The terms "first," "second," and the like, as used in this disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. Likewise, the terms "a," "an," or "the" and similar terms do not denote a limitation of quantity, but rather denote the presence of at least one. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
Fig. 1A shows a flow chart of data processing steps.
As shown in fig. 1A, in the data preprocessing process, the following data processing steps are generally adopted for a plurality of data having a first image format: firstly, converting a plurality of data with a first image format to obtain a plurality of data with a second image format, then, normalizing the plurality of data with the second image format (normalization) to obtain normalized plurality of data, and then, sequentially performing split operation (split), filling operation (pad) and folding operation (fold) on the normalized plurality of data to obtain a plurality of data for subsequent convolution operation.
For example, the first image format is a YUV format, where "Y" represents brightness (luminence or Luma), i.e., gray values; while "U" (blue projection Cb) and "V" (red projection Cr) represent chromaticity (Chrominance or Chroma) to describe color and saturation for a given color. The storage format of YUV is divided into a plane format and a packaging format, and three sampling modes of YUV are mainly adopted: YUV4:4, YUV4:2:2, YUV4:2:0.
The YUV plane format includes YUV420SP (Semi-Planar) format, the Y component is a plane, and the UV component is shared by one plane, i.e., the Y component and the UV component are stored separately. According to U, V storage sequence, the method is divided into NV12 format and NV21 format, for the NV12 format, all Y components are stored first, then UV components are stored in a staggered mode according to the sequence of U and V, for the NV21 format, all Y components are stored first, and then UV components are stored in a staggered mode according to the sequence of V and U. Taking the image format of YUV420SP (NV 12) as an example, the memory layout is shown in fig. 1B. In fig. 1B, the Y component and the UV component are stored separately, the Y component occupies 224×224 units of memory size, the UV component occupies 112×224 units of memory size, and the 2×2Y component corresponds to one U component and one V component. For example, Y (0, 0), Y (0, 1), Y (1, 0), Y (1, 1) correspond to U (0, 0) and V (0, 0); for another example, Y (222,0), Y (222,1), Y (223,0), Y (223,1) correspond to U (111,0) and V (111,0). For example, the data corresponding to each pixel in the image includes a Y component, a U component, and a V component, and the plurality of Y components are stored in the memory according to the positional relationship of the corresponding pixels in the image, e.g., the pixel corresponding to Y (0, 0) is located in the first row and first column in the image, the pixel corresponding to Y (0, 1) is located in the first row and second column in the image, the pixel corresponding to Y (1, 0) is located in the second row and first column in the image, and so on.
In converting YUV data having the format shown in fig. 1B into RGB data, for example, if the data processing step shown in fig. 1A is adopted, the applicant found the following bottleneck: for each pixel in the input image, the corresponding Y, U, V data is required to be taken out from the memory for conversion, the data reading process is complex, and the data reading efficiency is low. And one U component and one V component correspond to 4Y components arranged by 2 x2, sequential processing may result in repeated reading of data, and the multiplexing rate is low.
At least one embodiment of the present disclosure provides a data processing method, including: reading a plurality of input data having a first image format; converting the plurality of input data to obtain a plurality of intermediate data with a second image format; and processing the plurality of intermediate data to obtain a plurality of output data, each input data including a first component, the plurality of first components of the plurality of input data being arranged to form a plurality of first component rows, the reading the plurality of input data having a first image format, comprising: reading a first group of first component lines in the plurality of first component lines and storing the first group of first component lines in a first register; a second set of first component lines of the plurality of first component lines is read and stored in a second register.
The data processing method provided by the embodiment of the disclosure can improve the utilization efficiency of the register, reduce the calculation amount, improve the calculation efficiency and have better flexibility.
At least one embodiment of the present disclosure also provides a data processing apparatus, an image processing method, an electronic device, and a computer-readable storage medium corresponding to the above data processing method.
Embodiments of the present disclosure will be described in detail below with reference to the attached drawings, but the present disclosure is not limited to these specific embodiments.
Fig. 2 shows a schematic flow chart of a data processing method provided in at least one embodiment of the present disclosure.
As shown in fig. 2, the data processing method includes the following steps S201 to S203.
Step S201: a plurality of input data having a first image format is read.
For example, in some embodiments, the first image format is a YUV format, the first image format is a 4:2:0 sampling format, and the plurality of input data is stored in the memory in a flat format.
It should be noted that the first image format may be another image format, and the first image format may also be a sampling format of, for example, 4:2:2, and the plurality of input data may also be stored in another format, which is not limited in this disclosure.
For example, a plurality of input data are stored in the memory, and then in step S201, the plurality of input data are read from the memory.
Step S202: the plurality of input data are converted to a plurality of intermediate data having a second image format.
For example, the first image format and the second image format are not identical. In some embodiments, the second image format may be an RGB format, where R represents a red component, G represents a green component, and B represents a blue component.
It should be noted that the second image format may also be other image formats, such as RGBA, RGBX, etc., which is not limited by the present disclosure. In the embodiment of the present disclosure, the first image format is described as a YUV format, and the second image format is described as an RGB format.
Step S203: the plurality of intermediate data is processed to obtain a plurality of output data.
For example, each input data includes a first component, and a plurality of first components of the plurality of input data are arranged to form a plurality of first component rows.
For example, the plurality of input data may be data in one image. In an image, a plurality of input data are arranged in an array along a first direction and a second direction, wherein the first direction may be a row direction of the image, and the second direction may be a column direction of the image. Each Y component may correspond to one pixel of the image, the data corresponding to each pixel is input data, the size of the image may be h×m, the plurality of pixels corresponding to the plurality of input data are arranged in an array of h×m along the first direction and the second direction, and the plurality of Y components are arranged in an array of h×m along the first direction and the second direction. As shown in fig. 1B, H may be 224 and m may be 224.
For example, the first component is the Y component. As shown in fig. 1B, the first direction may be a lateral direction, and the second direction may be a vertical direction. The plurality of first components are arranged along a first direction to form 224 first component rows.
For example, the pixels in the image corresponding to the first components in each first component row are located in the same row. For example, the first component in the data corresponding to all pixels located in the same line in the image constitutes one first component line, i.e., one first component line corresponds to one pixel line.
For example, the first component rows respectively correspond to the same first component numbers, and each first component number represents the number of first components in the corresponding first component row, for example, the first component number may be M. For example, as shown in fig. 1B, M is 224, and the number of first components in each first component row is 224, e.g., the first component row includes Y (0, 0), Y (0, 1), …, Y (0,223).
As shown in fig. 3, in some embodiments, step S201 may include steps S301-S302.
Step S301: a first set of first component lines of the plurality of first component lines is read and stored in a first register.
Step S302: a second set of first component lines of the plurality of first component lines is read and stored in a second register.
Steps S301 to S302 are first split operations performed on data. The first split operation is a parity split in the second direction (i.e., column direction) for the first component.
For example, in some embodiments, the first set of first component rows includes even rows of the plurality of first component rows and the second set of first component rows includes odd rows of the plurality of first component rows.
For example, in some embodiments, the first set of first component rows and the second set of first component rows are read simultaneously. Thus, in the embodiments of the present disclosure, the first component of the odd-numbered line and the first component of the even-numbered line may be read at the same time, so that the data reading efficiency may be improved.
The first component rows are divided into the first component rows and the second component rows, and the first component rows and the second component rows are read separately and simultaneously.
For example, in some embodiments, each input data further includes a second component and a third component, the plurality of second components and the plurality of third components of the plurality of input data being arranged to form a plurality of second component rows.
For example, the second component is a U component and the third component is a V component. As shown in fig. 1B, a plurality of second components and a plurality of third components of the plurality of input data are arranged to form 112 second component rows.
For example, in some embodiments, each second component row includes a plurality of second components and a plurality of third components, and the plurality of second components and the plurality of third components in each second component row are alternately arranged. The plurality of second component lines respectively correspond to the same plurality of second components, each second component number representing a sum of the number of second components and the number of third components in the corresponding second component line, and when the size of the image may be h×m, the second component number is M. For example, the number of second components and the number of third components in each second component row may be equal, and may be M/2. For example, as shown in fig. 1B, M is 224, the number of second components corresponding to each second component row is 224, and the number of second components and the number of third components in each second component row are 112, for example, the first second component row includes U (0, 0), V (0, 0), U (0, 1), …, V (0,111).
For example, in some embodiments, step S201 may further include: the plurality of second component rows is read and stored in a third register.
For example, as shown in fig. 1B, the U component and the V component are alternately arranged.
For example, in some embodiments, two first component rows correspond to one second component row, a 2n-1 th first component row of the plurality of first component rows and a2 n-th first component row correspond to an n-th second component row of the plurality of second component rows, n being a positive integer.
For example, as shown in fig. 1B, the first and second rows of Y components correspond to the first row of UV components, i.e., the first component row (i.e., Y (0, 0), Y (0, 1), …, Y (0,223)) and the second first component row (i.e., Y (1, 0), Y (1, 1), …, Y (1, 223)) correspond to the first second component row (i.e., U (0, 0), V (0, 0), U (0, 1), …, V (0,111)), the third and fourth rows of Y components correspond to the second row of UV components, i.e., the third first component row (i.e., Y (2, 0), Y (2, 1), …, Y (2,223)) and the fourth first component row (i.e., Y (3, 0), Y (3, 1), …, Y (3,223)) correspond to the second component row (i.e., U (1, 0), V (1, …), V (1,111), and so on.
For example, in some embodiments, step S202 may include: based on the conversion relation between the first image format and the second image format, the first component stored in the first register and the second component and the third component in the third register are calculated to obtain a plurality of first intermediate data, and the first component stored in the second register and the second component and the third component in the third register are calculated to obtain a plurality of second intermediate data.
For example, the plurality of intermediate data includes a plurality of first intermediate data and a plurality of second intermediate data.
For example, the first image format is a YUV format, the storage of each component in the plurality of input data in the memory is shown in fig. 1B, and the second image format is an RGB format. The first register stores the odd-numbered lines of the Y component, the second register stores the even-numbered lines of the Y component, and the third register stores the UV component. Since the 2n-1 row and the 2 n-th row of the Y component correspond to the n-th row of the UV component rows, the corresponding R, G, B values are calculated, respectively. Taking NV12, BT601 (which represents a standard for data conversion between RGB and YUV in common) as an example, the conversion relationship between the first image format and the second image format is represented by the following specific calculation formulas 1 to 3:
R=y+1.402 (V-128) (formula 1)
G=Y-0.34414 (U-128) -0.71414 (V-128) (equation 2)
B=y+1.772 (U-128) (equation 3)
For example, in some embodiments, step S203 may include: and carrying out normalization processing on the plurality of first intermediate data and the plurality of second intermediate data to obtain normalized plurality of first intermediate data and normalized plurality of second intermediate data.
For example, the format of the plurality of intermediate data is an RGB format.
For example, the normalization process includes multiplying each component in each of the plurality of first intermediate data and the plurality of second intermediate data by a corresponding scale, and adding a corresponding bias (bias). The values of scale and bias are both fixed values. For example, for an R component, the ratio is 0.485 and the deviation is 0.229; for the G component, the ratio is 0.456 and the deviation is 0.224; for the B component, the ratio was 0.406 and the deviation was 0.225.
For example, each of the plurality of first intermediate data and the plurality of second intermediate data includes a red component, a green component, and a blue component. The proportion of the red component is scale_R, and the corresponding deviation is bias_R; the proportion of the green component is scale_G, and the corresponding deviation is bias_G; the blue component corresponds to scale_b and the bias corresponds to bias_b. The normalized value of the red component is denoted normed _r=r0×scale_r+bias_r, the normalized value of the green component is denoted normed _g=g0×scale_g+bias_g, the normalized value of the blue component is denoted normed _b=b0×scale_b+bias_b, where R0 represents the initial value of the red component, G0 represents the initial value of the green component, and B0 represents the initial value of the blue component.
For example, in some embodiments, the normalized plurality of first intermediate data arrangements form a plurality of first data columns and the normalized plurality of second intermediate data arrangements form a plurality of second data columns.
For example, the pixels in the image corresponding to the normalized first intermediate data in each first data column are in the same column, and the pixels in the image corresponding to the normalized second intermediate data in each second data column are in the same column. I.e. the normalized first intermediate data in each first data column is arranged in the second direction and the normalized second intermediate data in each second data column is arranged in the second direction.
As shown in fig. 4, in some embodiments, step S203 may include steps S401-S404.
Step S401: and acquiring odd columns in the plurality of first data columns, and performing filling operation and folding operation on the normalized plurality of first intermediate data corresponding to the odd columns in the plurality of first data columns to obtain a first group of output data.
Step S402: and acquiring even columns in the plurality of first data columns, and performing filling operation and folding operation on the normalized plurality of first intermediate data corresponding to the even columns in the plurality of first data columns to obtain a second group of output data.
Step S403: and acquiring odd columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the odd columns in the plurality of second data columns to obtain third group of output data.
Step S404: and acquiring even columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the even columns in the plurality of second data columns to obtain fourth group of output data.
For example, the plurality of output data includes a first set of output data, a second set of output data, a third set of output data, and a fourth set of output data.
Steps S401 to S404 are second splitting operations performed on the data, and four matrices are obtained by splitting a plurality of first data columns into odd columns and even columns and splitting a plurality of second data columns into odd columns and even columns. And then, respectively performing filling operation and folding operation on the four matrixes to finish preprocessing, and then performing subsequent convolution and other operations on the obtained multiple output data. The fill operation is to fill the selected region with zeros, or with a specific value, to meet the desired shape size. The folding operation is to change n channels (channels) of each normalized data (normalized first intermediate data/normalized second intermediate data) into 2n channels, n being a positive integer. The second split operation is a parity split performed in the first direction (i.e., row direction).
In the embodiment of the present disclosure, splitting operation in the data processing step shown in fig. 1A is divided into two splitting operations, the first splitting operation is performed when the first component is read (steps S301 to S302), without adding an additional reading load; the second split operation can be directly implemented by SIMT (Single Instruction Multiple Threads, single instruction multithreading) move instruction, so that the calculation amount can be greatly reduced, and the calculation efficiency can be improved. For example, four adjacent data (x 0, x1, x2, x 3) in BF16 format in the same line are stored by two adjacent data units (32 bits), and are converted from BF16 format to FP32 format by means of a move instruction, the data units occupied by the four adjacent data are changed from 2 to 4, and each data occupies one data unit of 32 bits. The data x1 and x2 are transformed sequentially to obtain (x 0, x2, x1, x 3), then the data is subjected to format conversion by a move instruction, and the data is converted from the FP32 format to the BF16 format, so that the data (x 0, x 2) is placed in a first data unit, and the data (x 1, x 3) is placed in a second data unit, which is equivalent to parity splitting. The same operation is performed on the whole matrix, which is equivalent to performing the second splitting operation.
Since the splitting operation shown in fig. 1A must use 4n registers to store the four matrices obtained by splitting respectively, the splitting operation divides one matrix into a plurality of 2×2 tiles (tile) along two different directions, each tile includes four data, which are respectively data ee (located in the first row and the first column), data eo (located in the first row and the second column), data oe (located in the second row and the first column), data oo (located in the second row and the second column), the data ee in all tiles are spliced to obtain one matrix, the data eo in all tiles are spliced to obtain one matrix, the data oe in all tiles are spliced to obtain one matrix, and the data oo in all tiles are spliced to obtain one matrix. That is, the data at the same position in all the slices constitutes a matrix.
For example, as shown in fig. 1B, one 2×2 tile includes Y (0, 0), Y (0, 1), Y (1, 0), Y (1, 1), Y (0, 0), Y (0, 1), Y (1, 1) may be divided into four matrices after performing the splitting operation, and, as shown in fig. 1B, another 2×2 tile includes Y (222,0), Y (222,1), Y (223,0), Y (223,1), Y (222,0), Y (222,1), Y (223,0), Y (223,1) may be divided into four matrices after performing the splitting operation, Y (0, 0) and Y (222,0) are located in the same matrix, Y (0, 1) and Y (222,1) are located in the same matrix, Y (1, 0) and Y (223,0) are located in the same matrix, and Y (1, 1) and Y (223,1) are located in the same matrix.
In the embodiment of the disclosure, after the split operation is divided into two split operations, the whole processing process has better flexibility.
In this embodiment, the normalization process is performed before steps S401 to S404, or may be performed after steps S401 to S404, which is not limited by the present disclosure.
Fig. 5 shows a flow chart of steps of a data processing method provided by at least one embodiment of the present disclosure.
As shown in fig. 5, the steps of the data processing method provided by the embodiment of the present disclosure are substantially the same as those of the data processing step shown in fig. 1A, except that the splitting operation in fig. 1A is divided into two splitting operations (a first splitting operation and a second splitting operation), the first splitting operation is performed before converting a plurality of input data into a plurality of intermediate data having a second image format, and the second splitting operation is performed between the normalization operation and the filling operation.
It should be noted that the second splitting operation may be performed before the normalizing operation and the filling operation, or may be performed between the normalizing operation and the filling operation, which is not limited in this disclosure.
The splitting operation shown in fig. 1A needs to use 4n registers to store the four matrices obtained by splitting respectively, then the values of the four matrices in the 4n registers are uniformly stored into the shared cache space X0, and then the values are read from X0 and stored into the 4 registers respectively, so that the whole process needs at least 8 cycles. The splitting operation provided in at least one embodiment of the present disclosure, for example, for BF16 data types, 32 threads of data, each register stores 2 data, and only one BF16 format to FP32 format conversion instruction is required to split the 32 x 2 BF16 format data into two 32 x1 FP32 format data, so that one cycle can be completed.
An example of a data processing method provided by at least one embodiment of the present disclosure is described below with reference to fig. 6A and 6B.
As shown in fig. 6A, the first image format is a YUV format, the second image format is an RGB format, the first image format adopts a 4:2:0 sampling format, and a plurality of input data are stored in the memory in a plane format. Each input data includes Y components, and a plurality of Y components are arranged in a first direction and a second direction to form a matrix 601 (corresponding to a plurality of first component rows, one row in the matrix 601 is a first component row). First, the even rows of the Y component are read and all even rows are stored in a first register to form a matrix 602 (corresponding to a first set of first component rows), the odd rows of the Y component are read and all odd rows are stored in a second register to form a matrix 603 (corresponding to a second set of first component rows). The plurality of U-components and V-components are arranged along the first direction and the second direction, and all the U-components and V-components are read and stored in the third register to form a matrix 604 (corresponding to a plurality of second component rows, one row in the matrix 604 is a second component row). Next, based on the conversion relation (formulas 1 to 3) between the first image format and the second image format, the Y component stored in the matrix 602 and the U component and the Y component stored in the matrix 604 are calculated to obtain a first R component matrix, a first G component matrix and a first B component matrix (the first R component matrix corresponds to a matrix formed by R components in the plurality of first intermediate data, the first G component matrix corresponds to a matrix formed by G components in the plurality of first intermediate data, the first B component matrix corresponds to a matrix formed by B components in the plurality of first intermediate data), fig. 6A shows only the first R component matrix 605, and the Y component stored in the matrix 603 and the U component and the Y component stored in the matrix 604 are calculated to obtain a second R component matrix, a second G component matrix and a second B component matrix (the second R component matrix corresponds to an R component formed by R components in the plurality of second intermediate data, the second G component matrix corresponds to a matrix formed by G components in the plurality of second intermediate data, and the second B component matrix corresponds to a matrix formed by B components in the plurality of second intermediate data), and the second B component matrix 606 shows only the second R component matrix.
Next, as shown in fig. 6B, taking the first R component matrix and the second R component matrix as an example, normalization operation is performed on the first R component matrix 605 and the second R component matrix 606 to obtain a normalized first R component matrix 607 (corresponding to a matrix formed by R components in the normalized plurality of first intermediate data) and a normalized second R component matrix 608 (corresponding to a matrix formed by R components in the normalized plurality of second intermediate data). Finally, the odd columns of the first R component matrix 607 are acquired and the corresponding plurality of data are subjected to a filling operation and a folding operation to obtain a first set of output data 609, the even columns of the normalized first R component matrix 607 are acquired and the corresponding plurality of data are subjected to a filling operation and a folding operation to obtain a second set of output data 610, the odd columns of the normalized second R component matrix 608 are acquired and the corresponding plurality of data are subjected to a filling operation and a folding operation to obtain a third set of output data 611, and the even columns of the normalized second R component matrix 608 are acquired and the corresponding plurality of data are subjected to a filling operation and a folding operation to obtain a fourth set of output data 612.
Fig. 7 shows a schematic flow chart of an image processing method provided by at least one embodiment of the present disclosure.
As shown in fig. 7, the image processing method includes steps S701 to S703 as follows.
Step S701: an input image is acquired.
For example, the image format of the data in the input image is the first image format.
Step S702: the data in the input image is used as a plurality of input data, and the data in the input image is processed by using the data processing method provided by at least one embodiment of the present disclosure, so as to obtain a plurality of output data corresponding to the input image.
Step S702 is a preprocessing operation performed on data in an input image.
Step S703: and processing a plurality of output data corresponding to the input image by using the neural network to obtain the output of the neural network.
For example, a plurality of output data are convolved with a neural network.
For example, in some embodiments, the neural network is a 50-layer residual neural network.
The technical effects of the image processing method are the same as those of the data processing method shown in fig. 2, and will not be described again here.
Fig. 8 illustrates a schematic block diagram of a data processing apparatus 800 that may be used to perform the data processing method illustrated in fig. 2, provided in accordance with at least one embodiment of the present disclosure.
As shown in fig. 8, the data processing apparatus 800 includes a reading unit 801, a conversion unit 802, and a processing unit 803.
The reading unit 801 is configured to read a plurality of input data having a first image format.
The conversion unit 802 is configured to convert the plurality of input data into a plurality of intermediate data having a second image format.
The processing unit 803 is configured to process the plurality of intermediate data to obtain a plurality of output data.
For example, each input data includes a first component, and a plurality of first components of the plurality of input data are arranged to form a plurality of first component rows.
In performing the step of reading a plurality of input data having a first image format, the reading unit 801 is configured to: reading a first group of first component lines in the plurality of first component lines and storing the first group of first component lines in a first register; a second set of first component lines of the plurality of first component lines is read and stored in a second register.
The technical effects of the data processing apparatus are the same as those of the data processing method shown in fig. 2, and will not be described herein.
For example, the reading unit 801, the converting unit 802, the processing unit 803 include codes and programs stored in a memory; the processor may execute the code and the program to implement some or all of the functions of the reading unit 801, the converting unit 802, the processing unit 803, as described above. For example, the reading unit 801, the converting unit 802, the processing unit 803 may be dedicated hardware devices for implementing some or all of the functions of the reading unit 801, the converting unit 802, the processing unit 803 as described above. For example, the reading unit 801, the converting unit 802, the processing unit 803 may be one circuit board or a combination of a plurality of circuit boards for realizing the functions as described above. In an embodiment of the present application, the circuit board or the combination of the circuit boards may include: (1) one or more processors; (2) One or more non-transitory memories coupled to the processor; and (3) firmware stored in the memory that is executable by the processor.
At least one embodiment of the present disclosure also provides an electronic device including a processor; and a memory storing computer executable instructions that when executed by the processor implement a data processing method or an image processing method provided by at least one embodiment of the present disclosure.
Fig. 9 is a schematic block diagram of an electronic device 900 provided by some embodiments of the present disclosure. As shown in fig. 9, the electronic device 900 includes a processor 910 and a memory 920. Memory 920 is used to store computer-executable instructions (e.g., one or more computer program modules). The processor 910 is configured to execute computer-executable instructions that, when executed by the processor 910, perform one or more of the steps of the data processing method or the image processing method described above. The memory 920 and the processor 910 may be interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, the processor 910 may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other form of processing unit having data processing capabilities and/or program execution capabilities. For example, the Central Processing Unit (CPU) may be an X86 or ARM architecture, or the like. The processor 910 may be a general-purpose processor or a special-purpose processor that may control other components in the electronic device 900 to perform the desired functions.
For example, memory 920 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, random Access Memory (RAM) and/or cache memory (cache) and the like. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, and the like. One or more computer program modules may be stored on the computer readable storage medium and executed by the processor 910 to implement various functions of the electronic device 900. Various applications and various data, as well as various data used and/or generated by the applications, etc., may also be stored in the computer readable storage medium.
It should be noted that, in the embodiments of the present disclosure, specific functions and technical effects of the electronic device 900 may refer to the above description about the data processing method and the image processing method, which are not repeated herein.
Fig. 10 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure. The electronic device 1000 is, for example, suitable for use in implementing the data processing methods and/or image processing methods provided by embodiments of the present disclosure. The electronic device 1000 may be a terminal device or the like. It should be noted that the electronic device 1000 shown in fig. 10 is merely an example, and does not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure.
As shown in fig. 10, the electronic device 1000 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 1010 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1020 or a program loaded from a storage means 1080 into a Random Access Memory (RAM) 1030. In the RAM 1030, various programs and data required for the operation of the electronic device 1000 are also stored. The processing device 1010, the ROM 1020, and the RAM 1030 are connected to each other by a bus 1040. An input/output (I/O) interface 1050 is also connected to bus 1040.
In general, the following devices may be connected to the I/O interface 1050: input devices 1060 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 1070 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, etc.; storage 1080 including, for example, magnetic tape, hard disk, etc.; and a communication device 1090. The communication device 1090 may allow the electronic device 1000 to communicate wirelessly or by wire with other electronic devices to exchange data. While fig. 10 shows the electronic device 1000 with various means, it is to be understood that not all of the illustrated means are required to be implemented or provided, and that the electronic device 1000 may alternatively be implemented or provided with more or fewer means.
For example, according to an embodiment of the present disclosure, the above-described data processing method or image processing method may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program, carried on a non-transitory computer readable medium, the computer program comprising program code for performing the above-described data processing method or image processing method. In such an embodiment, the computer program may be downloaded and installed from a network via communications device 1090, or from storage device 1080, or from ROM 1020. The functions defined in the data processing method or the image processing method provided by the embodiments of the present disclosure may be implemented when the computer program is executed by the processing device 1010.
At least one embodiment of the present disclosure provides a computer-readable storage medium for non-transitory storage of computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the data processing method or the image processing method provided by at least one embodiment of the present disclosure.
Fig. 11 is a schematic diagram of a storage medium according to some embodiments of the present disclosure. As shown in fig. 11, a storage medium 1100 is used to store computer-executable instructions 1110. For example, the computer-executable instructions 1110, when executed by a computer, may perform one or more steps in accordance with the data processing method or image processing method described above.
The following points need to be described:
(1) The drawings of the embodiments of the present disclosure relate only to the structures to which the embodiments of the present disclosure relate, and reference may be made to the general design for other structures.
(2) The embodiments of the present disclosure and features in the embodiments may be combined with each other to arrive at a new embodiment without conflict.
The foregoing is merely specific embodiments of the disclosure, but the scope of the disclosure is not limited thereto, and the scope of the disclosure should be determined by the claims.
Claims (15)
1. A data processing method, comprising:
Reading a plurality of input data having a first image format;
converting the plurality of input data to obtain a plurality of intermediate data with a second image format; and
Processing the plurality of intermediate data to obtain a plurality of output data,
Wherein each input data of the plurality of input data includes a first component, the plurality of first components of the plurality of input data being arranged to form a plurality of first component rows;
Wherein reading the plurality of input data having the first image format comprises:
Reading a first set of first component lines of the plurality of first component lines and storing the first set of first component lines in a first register,
Reading a second set of first component lines of the plurality of first component lines and storing the second set of first component lines in a second register;
Wherein each input data further comprises a second component and a third component, the second components and the third components of the plurality of input data are arranged to form a plurality of second component rows,
Reading a plurality of input data having the first image format, further comprising:
Reading the plurality of second component rows and storing the plurality of second component rows into a third register;
Wherein converting the plurality of input data to obtain the plurality of intermediate data having the second image format includes:
calculating the first component stored in the first register and the second and third components stored in the third register to obtain a plurality of first intermediate data based on a conversion relation between the first image format and the second image format, calculating the first component stored in the second register and the second and third components stored in the third register to obtain a plurality of second intermediate data,
Wherein the plurality of intermediate data includes the plurality of first intermediate data and the plurality of second intermediate data.
2. The data processing method of claim 1, wherein the first set of first component rows and the second set of first component rows are read simultaneously.
3. The data processing method of claim 2, wherein the first set of first component rows includes even rows of the plurality of first component rows and the second set of first component rows includes odd rows of the plurality of first component rows.
4. The data processing method according to claim 1, wherein each of the second component rows includes a plurality of second components and a plurality of third components, and the plurality of second components and the plurality of third components in each of the second component rows are alternately arranged.
5. The data processing method of claim 4, wherein the 2n-1 th first component row and the 2 n-th first component row of the plurality of first component rows correspond to the n-th second component row of the plurality of second component rows, n being a positive integer.
6. The data processing method of claim 1, wherein the first image format is a YUV format, the first component is a Y component, the second component is a U component, and the third component is a V component.
7. The data processing method according to any one of claims 1 to 6, wherein the first image format is in a 4:2:0 sampling format, and the plurality of input data is stored in a planar format in a memory.
8. The data processing method according to claim 1, wherein processing the plurality of intermediate data includes:
And normalizing the plurality of first intermediate data and the plurality of second intermediate data to obtain normalized plurality of first intermediate data and normalized plurality of second intermediate data.
9. The data processing method according to claim 8, wherein the normalized plurality of first intermediate data arrangements form a plurality of first data columns, the normalized plurality of second intermediate data arrangements form a plurality of second data columns, and the plurality of intermediate data are processed to obtain a plurality of output data, comprising:
acquiring odd columns in the plurality of first data columns, and performing filling operation and folding operation on the normalized plurality of first intermediate data corresponding to the odd columns in the plurality of first data columns to obtain a first group of output data;
acquiring even columns in the plurality of first data columns, and performing filling operation and folding operation on the normalized first intermediate data corresponding to the even columns in the plurality of first data columns to obtain second group output data;
Acquiring odd columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the odd columns in the plurality of second data columns to obtain third group of output data;
Acquiring even columns in the plurality of second data columns, and performing filling operation and folding operation on the normalized plurality of second intermediate data corresponding to the even columns in the plurality of second data columns to obtain fourth group output data;
Wherein the plurality of output data includes the first set of output data, the second set of output data, the third set of output data, and the fourth set of output data.
10. A data processing method according to any of claims 1-6, wherein the second image format is an RGB format.
11. An image processing method, comprising:
Acquiring an input image, wherein the image format of data in the input image is a first image format;
Processing the data in the input image by the data processing method according to claims 1-10 to obtain a plurality of output data corresponding to the input image, wherein the data in the input image is used as a plurality of input data;
And processing the plurality of output data corresponding to the input image by using a neural network to obtain the output of the neural network.
12. The image processing method of claim 11, wherein the neural network is a 50-layer residual neural network.
13. A data processing apparatus comprising:
a reading unit configured to read a plurality of input data having a first image format;
a conversion unit configured to convert the plurality of input data to a plurality of intermediate data having a second image format; and
A processing unit configured to process the plurality of intermediate data to obtain a plurality of output data,
Wherein each input data of the plurality of input data includes a first component, the plurality of first components of the plurality of input data being arranged to form a plurality of first component rows;
Wherein, in performing the step of reading a plurality of input data having a first image format, the reading unit is configured to:
Reading a first set of first component lines of the plurality of first component lines and storing the first set of first component lines in a first register,
Reading a second set of first component lines of the plurality of first component lines and storing the second set of first component lines in a second register;
Wherein each input data further comprises a second component and a third component, the second components and the third components of the plurality of input data are arranged to form a plurality of second component rows,
The reading unit is further configured to:
Reading the plurality of second component rows and storing the plurality of second component rows into a third register;
wherein the conversion unit is further configured to:
calculating the first component stored in the first register and the second and third components stored in the third register to obtain a plurality of first intermediate data based on a conversion relation between the first image format and the second image format, calculating the first component stored in the second register and the second and third components stored in the third register to obtain a plurality of second intermediate data,
Wherein the plurality of intermediate data includes the plurality of first intermediate data and the plurality of second intermediate data.
14. An electronic device, comprising:
A processor; and
A memory storing computer-executable instructions,
Wherein the computer executable instructions, when executed by the processor, implement the data processing method according to any of claims 1-10 or the image processing method according to claim 11 or 12.
15. A computer-readable storage medium for non-transitory storage of computer-executable instructions,
Wherein the computer executable instructions, when executed by a processor, implement the data processing method according to any of claims 1-10 or the image processing method according to claim 11 or 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210976569.6A CN115348432B (en) | 2022-08-15 | 2022-08-15 | Data processing method and device, image processing method, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210976569.6A CN115348432B (en) | 2022-08-15 | 2022-08-15 | Data processing method and device, image processing method, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115348432A CN115348432A (en) | 2022-11-15 |
| CN115348432B true CN115348432B (en) | 2024-05-07 |
Family
ID=83952004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210976569.6A Active CN115348432B (en) | 2022-08-15 | 2022-08-15 | Data processing method and device, image processing method, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115348432B (en) |
Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0447234A2 (en) * | 1990-03-14 | 1991-09-18 | C-Cube Microsystems | Data compression and decompression system and method |
| JP2001069448A (en) * | 1999-08-25 | 2001-03-16 | Canon Inc | Image processor |
| JP2004007474A (en) * | 2002-04-12 | 2004-01-08 | Yamaha Corp | Image data compression method, image data decompression method, and image data decompression circuit |
| US7002627B1 (en) * | 2002-06-19 | 2006-02-21 | Neomagic Corp. | Single-step conversion from RGB Bayer pattern to YUV 4:2:0 format |
| JP2007194959A (en) * | 2006-01-19 | 2007-08-02 | Seiko Epson Corp | Image data conversion device and method, and printer |
| JP2008294574A (en) * | 2007-05-22 | 2008-12-04 | Seiko Epson Corp | Image processing device and control method of image processing device |
| JP2009071616A (en) * | 2007-09-13 | 2009-04-02 | Seiko Epson Corp | Image processor and control method thereof |
| JP2009207195A (en) * | 2009-06-15 | 2009-09-10 | Canon Inc | Image data processing apparatus, imaging system, image data processing method, computer program, and computer-readable storage medium |
| CN101924950A (en) * | 2009-06-17 | 2010-12-22 | 普诚科技股份有限公司 | Data compression method for intelligent accelerating image display and device |
| JP2014241644A (en) * | 2014-09-29 | 2014-12-25 | 京セラドキュメントソリューションズ株式会社 | Image reading device, image forming apparatus, and image processing method |
| CN105872430A (en) * | 2016-04-06 | 2016-08-17 | 豪威科技(上海)有限公司 | Image format converter and image format conversion method |
| CN107729994A (en) * | 2017-11-28 | 2018-02-23 | 北京地平线信息技术有限公司 | The method and apparatus for performing the computing of the convolutional layer in convolutional neural networks |
| JP2018160909A (en) * | 2018-05-23 | 2018-10-11 | 株式会社ニコン | Encoding device, decoding device, encoding method, and decoding method |
| CN112188280A (en) * | 2019-07-03 | 2021-01-05 | 北京嗨动视觉科技有限公司 | Image processing method, device and system and computer readable medium |
| CN112750094A (en) * | 2020-12-30 | 2021-05-04 | 合肥工业大学 | Video processing method and system |
| CN112927174A (en) * | 2019-12-06 | 2021-06-08 | 阿里巴巴集团控股有限公司 | Method and device for image processing and image training to channel shuffling |
| WO2021114904A1 (en) * | 2019-12-09 | 2021-06-17 | 中科寒武纪科技股份有限公司 | Data processing method and apparatus, computer device and storage medium |
| CN113538215A (en) * | 2021-06-11 | 2021-10-22 | 展讯半导体(成都)有限公司 | Image format conversion method, device and system, electronic equipment and storage medium |
| CN113596581A (en) * | 2021-07-30 | 2021-11-02 | 上海商汤临港智能科技有限公司 | Image format conversion method and device, computer equipment and storage medium |
| CN114155137A (en) * | 2021-11-03 | 2022-03-08 | 深圳鲲云信息科技有限公司 | Format conversion method, controller and computer-readable storage medium |
| CN114554171A (en) * | 2022-02-25 | 2022-05-27 | 深圳市洲明科技股份有限公司 | Image format conversion method, device, display screen control equipment and storage medium |
| CN114724595A (en) * | 2022-03-18 | 2022-07-08 | 华中科技大学 | Convolution operation accelerator and convolution operation method |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6674479B2 (en) * | 2000-01-07 | 2004-01-06 | Intel Corporation | Method and apparatus for implementing 4:2:0 to 4:2:2 and 4:2:2 to 4:2:0 color space conversion |
| US6937245B1 (en) * | 2000-08-23 | 2005-08-30 | Nintendo Co., Ltd. | Graphics system with embedded frame buffer having reconfigurable pixel formats |
| JP5108172B2 (en) * | 2000-09-06 | 2012-12-26 | 株式会社ニコン | Image data size conversion processing apparatus, electronic still camera, and image data size conversion processing recording medium |
| JP4200942B2 (en) * | 2004-06-02 | 2008-12-24 | セイコーエプソン株式会社 | Display controller, electronic device, and image data supply method |
| JP2006013701A (en) * | 2004-06-23 | 2006-01-12 | Seiko Epson Corp | Display controller, electronic apparatus, and image data supply method |
| JP2007020112A (en) * | 2005-07-11 | 2007-01-25 | Canon Inc | Image signal processing apparatus, image signal processing method and imaging device |
| US7639263B2 (en) * | 2007-01-26 | 2009-12-29 | Microsoft Corporation | Fast filtered YUV to RGB conversion |
| EP3104608B1 (en) * | 2014-03-06 | 2020-09-09 | Samsung Electronics Co., Ltd. | Image decoding method and device therefor, and image encoding method and device therefor |
| US9760375B2 (en) * | 2014-09-09 | 2017-09-12 | International Business Machines Corporation | Register files for storing data operated on by instructions of multiple widths |
| US11516429B2 (en) * | 2020-04-22 | 2022-11-29 | Grass Valley Limited | System and method for image format conversion using 3D lookup table approximation |
-
2022
- 2022-08-15 CN CN202210976569.6A patent/CN115348432B/en active Active
Patent Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0447234A2 (en) * | 1990-03-14 | 1991-09-18 | C-Cube Microsystems | Data compression and decompression system and method |
| JP2001069448A (en) * | 1999-08-25 | 2001-03-16 | Canon Inc | Image processor |
| JP2004007474A (en) * | 2002-04-12 | 2004-01-08 | Yamaha Corp | Image data compression method, image data decompression method, and image data decompression circuit |
| US7002627B1 (en) * | 2002-06-19 | 2006-02-21 | Neomagic Corp. | Single-step conversion from RGB Bayer pattern to YUV 4:2:0 format |
| JP2007194959A (en) * | 2006-01-19 | 2007-08-02 | Seiko Epson Corp | Image data conversion device and method, and printer |
| JP2008294574A (en) * | 2007-05-22 | 2008-12-04 | Seiko Epson Corp | Image processing device and control method of image processing device |
| JP2009071616A (en) * | 2007-09-13 | 2009-04-02 | Seiko Epson Corp | Image processor and control method thereof |
| JP2009207195A (en) * | 2009-06-15 | 2009-09-10 | Canon Inc | Image data processing apparatus, imaging system, image data processing method, computer program, and computer-readable storage medium |
| CN101924950A (en) * | 2009-06-17 | 2010-12-22 | 普诚科技股份有限公司 | Data compression method for intelligent accelerating image display and device |
| JP2014241644A (en) * | 2014-09-29 | 2014-12-25 | 京セラドキュメントソリューションズ株式会社 | Image reading device, image forming apparatus, and image processing method |
| CN105872430A (en) * | 2016-04-06 | 2016-08-17 | 豪威科技(上海)有限公司 | Image format converter and image format conversion method |
| CN107729994A (en) * | 2017-11-28 | 2018-02-23 | 北京地平线信息技术有限公司 | The method and apparatus for performing the computing of the convolutional layer in convolutional neural networks |
| JP2018160909A (en) * | 2018-05-23 | 2018-10-11 | 株式会社ニコン | Encoding device, decoding device, encoding method, and decoding method |
| CN112188280A (en) * | 2019-07-03 | 2021-01-05 | 北京嗨动视觉科技有限公司 | Image processing method, device and system and computer readable medium |
| CN112927174A (en) * | 2019-12-06 | 2021-06-08 | 阿里巴巴集团控股有限公司 | Method and device for image processing and image training to channel shuffling |
| WO2021114904A1 (en) * | 2019-12-09 | 2021-06-17 | 中科寒武纪科技股份有限公司 | Data processing method and apparatus, computer device and storage medium |
| CN112750094A (en) * | 2020-12-30 | 2021-05-04 | 合肥工业大学 | Video processing method and system |
| CN113538215A (en) * | 2021-06-11 | 2021-10-22 | 展讯半导体(成都)有限公司 | Image format conversion method, device and system, electronic equipment and storage medium |
| CN113596581A (en) * | 2021-07-30 | 2021-11-02 | 上海商汤临港智能科技有限公司 | Image format conversion method and device, computer equipment and storage medium |
| CN114155137A (en) * | 2021-11-03 | 2022-03-08 | 深圳鲲云信息科技有限公司 | Format conversion method, controller and computer-readable storage medium |
| CN114554171A (en) * | 2022-02-25 | 2022-05-27 | 深圳市洲明科技股份有限公司 | Image format conversion method, device, display screen control equipment and storage medium |
| CN114724595A (en) * | 2022-03-18 | 2022-07-08 | 华中科技大学 | Convolution operation accelerator and convolution operation method |
Non-Patent Citations (3)
| Title |
|---|
| 基于MATLAB的图像信息隐藏算法研究与实现;陈小娥;《绍兴文理学院学报 自然科学版》;第38卷(第9期);第86-92页 * |
| 基于跳白块编码和深度神经网络对脉冲星候选体诊断图像的压缩研究;姜家涛等;《天文研究与技术》;第19卷(第5期);第470-478页 * |
| 小尺寸高分辨率的微显示系统设计及FPGA实现;谢博文等;《电子产品世界》;20210304;第27-32页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115348432A (en) | 2022-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110098240B (en) | Pixel structure, display device, pixel driving circuit and display control method | |
| US20230345135A1 (en) | Method, apparatus, and device for processing images, and storage medium | |
| US8519910B2 (en) | Image processing method and display device using the same | |
| JP7195935B2 (en) | IMAGE DISPLAY METHOD, DISPLAY SYSTEM AND COMPUTER-READABLE STORAGE MEDIUM | |
| US9734753B2 (en) | Data acquisition module and method, data processing unit, driver and display device | |
| JP6503061B2 (en) | Image synthesizing method, image chip and image device | |
| US20070230817A1 (en) | Image processing apparatus, image processing method and image processing program | |
| US10504419B2 (en) | Data converting method and apparatus, and computer-readable storage medium | |
| US20190096360A1 (en) | Display method, image processing device, display device, and storage medium | |
| US10096279B2 (en) | Display device having a plurality of sub-display areas comprising a plurality of shared regions | |
| CN113596581B (en) | Image format conversion method, device, computer equipment and storage medium | |
| CN1267813C (en) | System and method of processing chromatic difference signal 4:2:0 plane image data format storage | |
| CN115348432B (en) | Data processing method and device, image processing method, electronic equipment and medium | |
| WO2016095474A1 (en) | Display data transmission method and apparatus, display panel drive method and apparatus | |
| CN103297656A (en) | Extended range color space | |
| KR100726932B1 (en) | Video signal processing circuit, video signal processing method, and computer-readable storage medium | |
| CN100437746C (en) | Character display apparatus, character display method, character display program, and recording medium | |
| CN115359751A (en) | Gamma debugging method and device and computer readable storage medium | |
| CN112261242B (en) | Image data processing method and device | |
| US8681165B2 (en) | Image rotation method and apparatus | |
| CN116109468B (en) | Graphics processing unit, instruction compiling method, storage medium, and terminal device | |
| JP3553376B2 (en) | Parallel image processor | |
| CN116260975B (en) | Video compression method, device and storage medium based on CUDA | |
| US20230127526A1 (en) | Display apparatus and method of driving the same | |
| CN109685731B (en) | Method for accelerating gradual change speed of picture based on SSE4 instruction set |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 201100 room 1302, 13 / F, building 16, No. 2388, Chenhang highway, Minhang District, Shanghai Applicant after: Shanghai Bi Ren Technology Co.,Ltd. Address before: 201100 room 1302, 13 / F, building 16, No. 2388, Chenhang highway, Minhang District, Shanghai Applicant before: Shanghai Bilin Intelligent Technology Co.,Ltd. Country or region before: China |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |