CN112579806A

CN112579806A - Picture processing method

Info

Publication number: CN112579806A
Application number: CN202011179962.XA
Authority: CN
Inventors: 詹俊鲲
Original assignee: Chongqing Seamless Splicing Intelligent Technology Co ltd
Current assignee: Chongqing Seamless Splicing Intelligent Technology Co ltd
Priority date: 2020-10-29
Filing date: 2020-10-29
Publication date: 2021-03-30

Abstract

The application provides a picture processing method, which comprises the following steps: the 5G chip obtains a weight vector of a target object, extracts a plurality of pictures to be recommended, performs feature extraction on the pictures to obtain a plurality of input matrixes, inputs the input matrixes and the weight vector to a main processing circuit of the artificial intelligence chip, and executes picture recommendation operation through the artificial intelligence chip to obtain the recommended pictures. The technical scheme provided by the application has the advantage of high user experience.

Description

Picture processing method

Technical Field

The application relates to the field of artificial intelligence and pictures, in particular to a picture processing method.

Background

The fifth Generation mobile communication technology (english: 5th Generation mobile networks or 5th Generation with less systems, 5th-Generation, 5G or 5G technology for short) is the latest Generation cellular mobile communication technology, and is also an extension following 4G (LTE-A, WiMax), 3G (UMTS, LTE) and 2G (gsm) systems. The performance goals of 5G are high data rates, reduced latency, energy savings, reduced cost, increased system capacity, and large-scale device connectivity.

The artificial intelligence technology becomes the standard configuration of the 5G mobile phone through the development of recent years, but the conventional artificial intelligence is slow in recommending and processing pictures, large in calculation amount and high in chip cost, and the user experience degree is influenced.

Disclosure of Invention

The technical scheme of the invention aims to provide the picture processing, and the sparse data is used for storage and operation, so that the calculation cost is reduced, the power consumption is reduced, and the user experience is improved.

In a first aspect, a method for processing a picture is provided, where the method is performed by a 5G chip and an artificial intelligence chip, and the artificial intelligence chip structure includes: the delta group slave processing circuit comprises a main processing circuit and delta group slave processing circuits, wherein each group of slave processing circuits comprises: the system comprises a plurality of slave processing circuits, 1 broadcast forwarding circuit and 1 multi-way selection switch, wherein the multi-way selection switch is 1P 2T; delta ports of the main processing circuit are respectively connected with each broadcast forwarding circuit of the delta group of slave processing circuits, and each broadcast forwarding circuit is respectively connected with the broadcast ports of a plurality of slave processing circuits of the same group of slave processing circuits;

the other delta ports of the main processing circuit are respectively connected with the P port of each 1P2T of the delta groups of the slave processing circuits, and two T ports of each 1P2T are respectively connected with the adjacent first slave processing circuit and the second slave processing circuit in each group of the slave processing circuits; the slave processing circuit is also connected with other adjacent slave processing circuits in the same group of slave processing circuits through two forwarding ports;

the broadcast forwarding circuit, the first processing circuit, and the second slave processing circuit each include: a sparse conversion module to perform a sparse conversion of data;

the 5G chip acquires a weight vector of a target object, extracts a plurality of pictures to be recommended, performs feature extraction on the plurality of pictures to obtain a plurality of input matrixes, and inputs the plurality of input matrixes and the weight vector to a main processing circuit of the artificial intelligent chip;

the main processing circuit receives a plurality of input matrixes and weight vectors transmitted by the 5G chip; the weight vector is respectively broadcasted to a broadcast forwarding circuit through delta ports, each input matrix in the input matrix is divided into delta groups of cyclic data blocks, and each group of cyclic data blocks comprises: the first cycle sub data block is sent to the first slave processing circuit through a 1P2T switch, and the second cycle sub data block is sent to the second slave processing circuit through a 1P2T switch;

the sparse conversion module of the broadcast forwarding circuit converts the received weight vector into a sparse weight vector and forwards the sparse weight vector to a plurality of slave processing circuits in the same group of slave processing circuits;

when receiving the first cyclic sub data block, 1P2T connects one T port to send to the first slave processing circuit, and when receiving the second cyclic sub data block, connects another T port to send to the second slave processing circuit;

when the first slave processing circuit receives the first cyclic sub-data block, a sparse conversion module in the first slave processing circuit converts floating point data of the first cyclic sub-data block into sparse data to obtain a first sparse cyclic sub-data block, a local cyclic forwarding data block is intercepted from the first sparse cyclic sub-data block, and the residual cyclic forwarding data block is forwarded to other slave processing circuits in a counterclockwise mode;

when the second slave processing circuit receives the second cyclic sub-data block, the sparse conversion module in the second slave processing circuit converts floating point data of the second cyclic sub-data block into sparse data to obtain a second sparse cyclic sub-data block, a local cyclic forwarding data block is intercepted from the second sparse cyclic sub-data block, and the residual cyclic forwarding data block is forwarded to other slave processing circuits clockwise;

the slave processing circuit receives the residual cyclic forwarding data block through one forwarding port, receives the sparse weight vector through the broadcast port, intercepts the local cyclic forwarding data block from the residual cyclic forwarding data block, and sends other cyclic forwarding data blocks to other adjacent slave processing circuits through another forwarding port;

the slave processing circuit executes sparse operation on the local cyclic forwarding data block and the sparse weight vector to obtain a sparse operation result, and the sparse operation result is sent to the broadcast forwarding circuit through the broadcast port;

a sparse conversion module in the broadcast forwarding circuit converts the sparse operation result into an operation result and sends the operation result to a main processing circuit;

the main processing circuit obtains calculation results of the input matrixes and the weight vectors according to the operation results, processes the calculation results to obtain matching degrees of the plurality of input matrixes and the target object, and recommends picture identifications corresponding to the calculation results to the 5G chip if the matching degrees are higher than a matching threshold;

and the 5G chip recommends the picture corresponding to the picture identification.

In a second aspect, an electronic device is provided, which is configured to perform the method provided in the first aspect.

Optionally, the electronic device includes: smart phone, panel computer, VR equipment, smart glasses, smart television, elevator picture terminal or intelligent audio amplifier.

In the method provided by the application, when matrix and vector operation is executed, a 5G-based artificial intelligence chip structure realizes broadcast data lines and cyclic forwarding data through two ports, so that the forwarding data volume of the ports is reduced compared with that of broadcasting and cyclic forwarding at one port, compared with the prior art (such as an H-shaped structure patent of the Cokemartian), the method can reduce the data transmission quantity of a single port of a main processing circuit and also reduce the forwarding data volume of a conversion circuit, in addition, by arranging a 1P2T switch and setting clockwise and anticlockwise different cyclic forwarding directions, the forwarding data volume and the operation volume of a slave processing circuit are the same, the data forwarding can be more balanced, the calculation efficiency is improved, and the relatively balanced forwarding data volume and operation volume can ensure that the hardware configuration of the slave processing circuit is the same, in addition, the slave processing circuit realizes sparse data operation, compared with common data operation, the sparse data storage amount is smaller, the operation is more convenient, the calculation amount is reduced, and the power consumption is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of the connection between a 5G chip and an artificial intelligence chip provided by the present invention.

Fig. 1a is a schematic structural diagram of a sparse conversion module provided in the present invention.

Fig. 2 is a schematic flow chart of an intelligent picture recommendation method provided by the present invention.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The embodiments of the present application will be described below with reference to the drawings.

The term "and/or" in this application is only one kind of association relationship describing the associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in this document indicates that the former and latter related objects are in an "or" relationship.

The "plurality" appearing in the embodiments of the present application means two or more. The descriptions of the first, second, etc. appearing in the embodiments of the present application are only for illustrating and differentiating the objects, and do not represent the order or the particular limitation of the number of the devices in the embodiments of the present application, and do not constitute any limitation to the embodiments of the present application. The term "connect" in the embodiments of the present application refers to various connection manners, such as direct connection or indirect connection, to implement communication between devices, which is not limited in this embodiment of the present application.

In the present application, "|" means an absolute value.

The floating point data is generally calculated by the artificial intelligence chip, the floating point data occupies 32 bits or 64 bits, namely the floating point data is represented by 32 bits or 64 bits no matter the numerical value of the floating point data, for artificial intelligence, the number of element values is large, especially for picture recommendation, when the number of pictures is large, multiple operations are needed, the number of the element values is large, if the floating point data is adopted, the storage space is greatly occupied, and the calculation amount is increased. The 5G chip of the present application may be an existing chip supporting 5G communication.

Referring to fig. 1, fig. 1 provides a schematic structural diagram of a chip based on 5G and an artificial intelligence chip, as shown in fig. 1, the artificial intelligence chip structure includes: a master processing circuit 101, δ sets of slave processing circuits, each set of slave processing circuits comprising: a plurality of slave processing circuits 102, 1 broadcast relay circuit 103, and 1 multiplexer (1P 2T);

wherein, δ ports of the main processing circuit 101 are connected with each broadcast forwarding circuit 102 of δ groups of slave processing circuits, and each broadcast forwarding circuit is connected with broadcast ports of a plurality of slave processing circuits 102 of the same group of slave processing circuits;

the other delta ports of the main processing circuit 101 are connected with the P port of each multi-way selector switch 1P2T of each delta group of slave processing circuits, and two T ports of each multi-way selector switch 1P2T are respectively connected with the adjacent first slave processing circuit and the second slave processing circuit in each group of slave processing circuits; slave processing circuit 102 is also connected to other adjacent slave processing circuits within the same group of slave processing circuits through two forwarding ports;

referring to fig. 1a, as shown in fig. 1a, the broadcast forwarding circuit, the first slave processing circuit, and the second slave processing circuit further include: a sparse conversion module 1010 for performing a sparse conversion of data;

the sparse conversion may include: the sparse data is converted into normal data (i.e. non-sparse data), or the normal data is converted into sparse data.

The specific method of sparse conversion may be: and setting a sparse bitmap, wherein the bitmap is used for representing the position of zero element values in the common data, deleting the element values of 0 element values in the common data to obtain filtered data, and adding the bitmap to the filtered data to obtain sparse data.

Taking 10 values as an example, for example, if the values of the elements 4, 5, 6, and 7 are all 0, then the bitmap may be, 0001111000; deleting the 4 th, 5th, 6 th and 7 th element values in the common data, and then adding the bitmap to obtain sparse data, wherein the data with the element value of 0 is calculated without meaning, so that the quantity of data storage can be reduced in a sparse mode.

And converting the sparse data into the common data, namely, determining the position of a 0 value through the bitmap of the sparse data, and adding the 0 value to the corresponding position to obtain the common data.

Referring to fig. 2, fig. 2 further provides an intelligent picture recommendation method, as shown in fig. 2, including the following steps:

s200, a 5G chip acquires a weight vector of a target object, extracts a plurality of pictures to be recommended, performs feature extraction on the plurality of pictures to obtain a plurality of input matrixes, and inputs the plurality of input matrixes and the weight vector to a main processing circuit of an artificial intelligent chip;

the target object may be a weight vector corresponding to the operator.

The above feature extraction method may be implemented by using an existing feature extraction method, for example, by using a feature extraction network, and the feature extraction method is not limited in the present application.

Step S201, a main processing circuit receives a plurality of input matrixes and weight vectors transmitted by a 5G chip; the weight vector is respectively broadcasted to a broadcast forwarding circuit through delta ports, each input matrix in the input matrix is divided into delta groups of cyclic data blocks, and each group of cyclic data blocks comprises: the first cycle sub data block is sent to the first slave processing circuit through a 1P2T switch, and the second cycle sub data block is sent to the second slave processing circuit through a 1P2T switch;

step S202, a sparse conversion module of the broadcast forwarding circuit converts the received weight vector into a sparse weight vector and forwards the sparse weight vector to a plurality of slave processing circuits in the same group of slave processing circuits; when receiving the first cyclic sub data block, 1P2T connects one T port to send to the first slave processing circuit, and when receiving the second cyclic sub data block, connects another T port to send to the second slave processing circuit;

step S203, when the first slave processing circuit receives the first cyclic sub-data block, a sparse conversion module in the first slave processing circuit converts floating point data of the first cyclic sub-data block into sparse data to obtain a first sparse cyclic sub-data block, a local cyclic forwarding data block is intercepted from the first sparse cyclic sub-data block, and the remaining cyclic forwarding data block is forwarded to other slave processing circuits in a counterclockwise manner;

step S204, when the second slave processing circuit receives the second cyclic sub-data block, a sparse conversion module in the second slave processing circuit converts floating point data of the second cyclic sub-data block into sparse data to obtain a second sparse cyclic sub-data block, a local cyclic forwarding data block is intercepted from the second sparse cyclic sub-data block, and the remaining cyclic forwarding data block is forwarded to other slave processing circuits clockwise;

step S205, the slave processing circuit receives the residual cyclic forwarding data block through one forwarding port, receives the sparse weight vector through the broadcast port, intercepts the local cyclic forwarding data block from the residual cyclic forwarding data block, and sends other cyclic forwarding data blocks to other adjacent slave processing circuits through another forwarding port;

step S206, the slave processing circuit executes sparse operation on the local circulation forwarding data block and the sparse weight vector to obtain a sparse operation result, and the sparse operation result is sent to the broadcast forwarding circuit through the broadcast port; and a sparse conversion module in the broadcast forwarding circuit converts the sparse operation result into an operation result and sends the operation result to the main processing circuit.

Step S207, the main processing circuit obtains calculation results of the input matrixes and the weight vectors according to the operation results, processes the calculation results to obtain matching degrees of a plurality of input matrixes and the target object, and recommends picture identifications corresponding to the calculation results to a 5G chip if the matching degrees are higher than a matching threshold; and the 5G chip recommends the picture corresponding to the picture identification.

There are various implementation methods for processing the calculation result to obtain the matching degrees between the input matrices and the target object, for example, a simpler way is to perform subtraction on the calculation result and a preset result to obtain a difference value, and determine that the reciprocal of the absolute value of the difference value is the matching degree.

If the calculation result is a vector, the difference is the sum of all element values in the difference vector. Of course, the matching degree may also be obtained in other manners, for example, by processing through a softmax function.

In the method provided by the application, when matrix and vector operation is executed, a 5G-based artificial intelligence chip structure realizes broadcast data lines and cyclic forwarding data through two ports, so that the forwarding data volume of the ports is reduced compared with that of broadcasting and cyclic forwarding at one port, compared with the prior art (such as an H-shaped structure patent of the Cokemartian), the method can reduce the data transmission quantity of a single port of a main processing circuit and also reduce the forwarding data volume of a conversion circuit, in addition, by arranging a 1P2T switch and setting clockwise and anticlockwise different cyclic forwarding directions, the forwarding data volume and the operation volume of a slave processing circuit are the same, the data forwarding can be more balanced, the calculation efficiency is improved, and the relatively balanced forwarding data volume and operation volume can ensure that the hardware configuration of the slave processing circuit is the same, in addition, the slave processing circuit realizes sparse data operation, compared with floating point operation, the sparse data storage amount is smaller, the operation is more convenient, the calculation amount is reduced, and the power consumption is reduced.

The above-mentioned multiple slave processing circuits take 6 slave processing circuits as shown in fig. 1, each of the data amounts forwarded from the processing circuits are the same in one counterclockwise forwarding and one clockwise forwarding, and the calculated amount is also about the same, taking the first slave processing circuit as an example, each of the data amounts computed from the processing circuits may be a row element value, then for the first slave processing circuit, it intercepts a row element value from the first cyclic sub-data block, for example, the first cyclic sub-data block includes 6 row element values, intercepts the first row element value, forwards the 5 row element value, and conversely, when forwarding clockwise, the first slave processing circuit receives only a row element value, and does not need to forward, i.e., it forwards a data amount of 5 row element values, and for the second slave processing circuit, the data forwarding data and the operation amount are the same, and the balanced data forwarding data and operation amount can make the hardware configuration of the slave processing circuits more balanced, thereby reducing the cost of the slave processing circuitry.

Optionally, the dividing each input matrix of the plurality of input matrices into δ sets of cyclic data blocks specifically includes:

the main processing circuit extracts an input matrix from a plurality of input matrices, divides the input matrix into delta sets of cyclic data blocks by row values, each set of cyclic data blocks comprising: m/δ row element values, the first cyclic sub data block comprising: m/2 δ row element values, the second cyclic sub data block comprising: the m/2 delta row element values. M is the maximum value of the input matrix row.

Optionally, the slave processing circuit is specifically configured to intercept a row of element values from the first cyclic sub data block and determine the row of element values as the local cyclic forwarding data block.

As shown in fig. 1, the multi-way selector switch includes 2T ports and 1P port, and each T port is fully connected to 1P port.

The technical scheme of the application also provides the electronic equipment, and the electronic equipment executes the method.

Optionally, the electronic device includes: smart phone, panel computer, VR equipment, intelligent glasses, elevator picture terminal, smart television or intelligent audio amplifier.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A picture processing method is characterized in that the method is executed by a 5G chip and an artificial intelligence chip, and the artificial intelligence chip structure comprises: the delta group slave processing circuit comprises a main processing circuit and delta group slave processing circuits, wherein each group of slave processing circuits comprises: the system comprises a plurality of slave processing circuits, 1 broadcast forwarding circuit and 1 multi-way selection switch, wherein the multi-way selection switch is 1P 2T; delta ports of the main processing circuit are respectively connected with each broadcast forwarding circuit of the delta group of slave processing circuits, and each broadcast forwarding circuit is respectively connected with the broadcast ports of a plurality of slave processing circuits of the same group of slave processing circuits; the other delta ports of the main processing circuit are respectively connected with the P port of each 1P2T of the delta groups of the slave processing circuits, and two T ports of each 1P2T are respectively connected with the adjacent first slave processing circuit and the second slave processing circuit in each group of the slave processing circuits; the slave processing circuit is also connected with other adjacent slave processing circuits in the same group of slave processing circuits through two forwarding ports; the broadcast forwarding circuit, the first processing circuit, and the second slave processing circuit each include: a sparse conversion module to perform a sparse conversion of data; the method comprises the following steps:

2. The method of claim 1,

δ is 6, and the plurality of slave processing circuits is 6 slave processing circuits.

3. The method of claim 1, wherein the dividing each of the plurality of input matrices into δ sets of cyclic data blocks specifically comprises:

the main processing circuit extracts an input matrix from a plurality of input matrices, divides the input matrix into delta sets of cyclic data blocks by row values, each set of cyclic data blocks comprising: m/δ row element values, the first cyclic sub data block comprising: m/2 δ row element values, the second cyclic sub data block comprising: the m/2 delta row element values.

4. The method of claim 3,

and the slave processing circuit is specifically used for intercepting a row of element values of the first cycle sub data block to determine the first cycle sub data block as a local cycle forwarding data block.

5. An electronic device, characterized in that the electronic device is configured to perform the picture processing method according to any one of claims 1-4.

6. The electronic device of claim 5,

the electronic device includes: smart phone, panel computer, VR equipment, smart glasses, smart television, elevator picture terminal or intelligent audio amplifier.