CN112991138A

CN112991138A - Arithmetic device based on image processing and related product

Info

Publication number: CN112991138A
Application number: CN201911288518.9A
Authority: CN
Inventors: 尚书林; 卢子威; 赵永刚; 李迅
Original assignee: Bitmain Technologies Inc
Current assignee: Beijing suneng Technology Co.,Ltd.
Priority date: 2019-12-12
Filing date: 2019-12-12
Publication date: 2021-06-18

Abstract

The embodiment of the application provides an arithmetic device based on image processing and a related product, wherein the arithmetic device comprises an image decoding unit, and the image decoding unit comprises an image decoding module and an image conversion module; the image decoding module is used for acquiring an image to be processed, decoding the image to be processed to obtain an image based on a first color space format, and outputting the image based on the first color space format to the image conversion module; the image conversion module is used for converting the color space of the acquired image to obtain an image based on a second color space format, and sending the image based on the second color space format to the memory unit through the bus. Only one interaction is needed between the operation device based on image processing and the memory unit; the method does not need to read and store data with the memory unit for many times, can reduce the occupation of a bus between the operation device and the memory unit, reduces the bandwidth pressure of the bus of the operation device, and accelerates the operation process.

Description

Arithmetic device based on image processing and related product

Technical Field

The application relates to the technical field of artificial intelligence, in particular to an arithmetic device based on image processing and a related product.

Background

With the development of artificial intelligence technology, intelligent computing devices are becoming widely used, and can be applied to the field of image processing. For example, an intelligent computing device is applied to the fields of face recognition, license plate recognition and the like.

In the prior art, each unit in the computing device can be directly connected with one memory unit through a bus respectively; the arithmetic device and the memory unit are two mutually independent logic hardware. Thus, the computing device can process the image to present the image to the memory unit for storage.

However, in the prior art, during the process of processing the image by the computing device, each unit in the computing device needs to read and store data with the memory unit for multiple times. Furthermore, the bus between the computing device and the memory unit needs to be occupied, which increases the bandwidth pressure of the bus of the computing device.

Disclosure of Invention

In view of the above, it is desirable to provide an arithmetic device and related products based on image processing.

In a first aspect, the present application provides an image processing-based arithmetic device, including: an image decoding unit; the image decoding unit comprises an image decoding module and an image conversion module;

the image decoding module is used for acquiring an image to be processed, decoding the image to be processed to obtain an image based on a first color space format, and outputting the image based on the first color space format to the image conversion module;

the image conversion module is configured to perform color space conversion on the image based on the first color space format to obtain an image based on a second color space format, and send the image based on the second color space format to a memory unit connected to the computing device based on image processing through a bus.

In some embodiments, the image decoding unit further comprises a first buffering module;

the image decoding module is specifically configured to output the image based on the first color space format to the first buffering module for buffering;

the first buffer module is configured to send a first trigger signal to the image conversion module when it is determined that the data amount in the first buffer module is greater than a preset threshold, so that the image conversion module acquires an image in the first buffer module.

In some embodiments, a second buffering module is further included in the image decoding unit;

the image conversion module is specifically configured to output the image based on the second color space format to the second buffer module for buffering;

the second buffer module is configured to trigger the memory unit to acquire the image in the second buffer module when it is determined that the data amount in the second buffer module is greater than a preset threshold.

In some embodiments, the image decoding unit further comprises a direct memory access module;

the second buffer module is specifically configured to send a second trigger signal to the direct memory access module when it is determined that the data amount in the second buffer module is greater than a preset threshold;

the direct memory access module is configured to, after receiving the second trigger signal, read the image based on the second color space format from the second buffer module, and send the image based on the second color space format to the memory unit through a bus.

In some embodiments, the image processing-based computing device further comprises an intelligent processing unit;

the intelligent processing unit is configured to read the image based on the second color space format in the memory unit, and perform intelligent processing on the image based on the second color space format.

In some embodiments, the image processing-based computing device further comprises a central processor unit;

the central processing unit is configured to receive a third trigger signal sent by the image decoding unit, where the third trigger signal is used to indicate that the image decoding unit has processed one or more frames of images; and scheduling the intelligent processing unit according to the third trigger signal.

In some embodiments, the central processor unit is further configured to:

acquiring a fourth trigger signal, wherein the fourth trigger signal is used for indicating to start a color space format conversion function of the image conversion module;

and starting the color space format conversion function of the image conversion module according to the fourth trigger signal.

In some embodiments, the central processor unit is further configured to:

acquiring a fifth trigger signal, wherein the fifth trigger signal is used for indicating to close a color space format conversion function of the image conversion module;

according to the fifth trigger signal, closing the color space format conversion function of the image conversion module; or, according to the fifth trigger signal, sending a sixth trigger signal to the image conversion module, so that the image conversion module no longer receives the image output by the image decoding module.

In some embodiments, the first color space format is a YUV color space; the second color space format is an RGB color space, or the second color space format is a gray-scale color space.

In a second aspect, the present application provides an image processing apparatus, including one or more image processing-based computing devices as provided in the first aspect, configured to obtain an image to be processed from another processing device, execute a specified image processing operation, and transmit an execution result to the other processing device through an I/O interface;

when the image processing device comprises a plurality of image processing-based operation devices, the plurality of image processing-based operation devices can be connected through a specific structure and transmit data;

the image processing-based operation devices are interconnected through a Peripheral Component Interface Express (PCIE) bus and transmit data; a plurality of the image processing-based arithmetic devices share the same control system or have respective control systems; the plurality of image processing-based computing devices share a memory unit or have respective memory units.

In a third aspect, the present application provides a combined processing apparatus, including the image processing apparatus provided in the second aspect, a universal interconnection interface, and other processing apparatuses;

and the image processing device interacts with the other processing devices to jointly complete the operation designated by the user.

In a fourth aspect, the present application provides an image processing chip comprising an image processing apparatus as provided in the second aspect or a combined processing apparatus as provided in the third aspect.

In a fifth aspect, the present application provides a board card, which includes the image processing chip provided in the fourth aspect.

In a sixth aspect, the present application provides an electronic device including the image processing chip as provided in the fourth aspect or the board as provided in the fifth aspect.

In a seventh aspect, the present application provides an image processing-based operation method, which is applied to an image processing-based operation device, where the image processing-based operation device includes an image decoding unit, and the image decoding unit includes an image decoding module and an image conversion module; the operation method based on image processing comprises the following steps:

the image decoding module acquires an image to be processed, decodes the image to be processed to obtain an image based on a first color space format, and outputs the image based on the first color space format to the image conversion module;

the image conversion module performs color space conversion on the image based on the first color space format to obtain an image based on a second color space format, and sends the image based on the second color space format to a memory unit connected with the operation device through a bus.

the image decoding module outputs the image based on the first color space format to the first buffer module for buffering;

the first buffer module sends a first trigger signal to the image conversion module when determining that the data volume in the first buffer module is larger than a preset threshold value, so that the image conversion module acquires the image in the first buffer module.

the image conversion module outputs the image based on the second color space format to the second buffer module for caching;

and when the second buffer module determines that the data volume in the second buffer module is larger than a preset threshold value, triggering the memory unit to acquire the image in the second buffer module.

the second buffer module sends a second trigger signal to the direct memory access module when determining that the data volume in the second buffer module is greater than a preset threshold value;

after receiving the second trigger signal, the direct memory access module reads the image based on the second color space format from the second buffer module, and sends the image based on the second color space format to the memory unit through a bus.

In some embodiments, the image processing-based computing device further comprises an intelligent processing unit; the operation method based on image processing further comprises the following steps:

the intelligent processing unit reads the image based on the second color space format in the memory unit and carries out intelligent processing on the image based on the second color space format.

In some embodiments, the image processing-based arithmetic method further comprises a central processor unit; the operation method based on image processing further comprises the following steps:

the central processing unit receives a third trigger signal sent by the image decoding unit, wherein the third trigger signal is used for indicating that the image decoding unit completes processing of one or more frames of images; and scheduling the intelligent processing unit according to the third trigger signal.

In some embodiments, the method for image processing-based operation further includes:

the central processing unit acquires a fourth trigger signal, wherein the fourth trigger signal is used for indicating to start a color space format conversion function of the image conversion module; and starting the color space format conversion function of the image conversion module according to the fourth trigger signal.

the central processing unit acquires a fifth trigger signal, wherein the fifth trigger signal is used for indicating to close the color space format conversion function of the image conversion module; and according to the fifth trigger signal, closing the color space format conversion function of the image conversion module, or according to the fifth trigger signal, sending a sixth trigger signal to the image conversion module, so that the image conversion module does not receive the image output by the image decoding module any more.

As described above, in the solution of the embodiment of the present application, by configuring the image decoding module and the image conversion module in the image decoding unit of the computing device, the image decoding module in the image decoding unit can decode the image to obtain the image based on the first color space format; because the image conversion module is configured in the image decoding unit, the image decoding module in the image decoding unit can directly send the image based on the first color space format to the image conversion module in the image decoding unit for processing, and the image does not need to be sent to the memory unit; and after the image conversion module in the image decoding unit performs color space conversion on the image, sending the obtained image based on the second color space format to the memory unit, so that other units can obtain the image based on the second color space format from the memory unit. In the whole process, only one interaction is needed between the image decoding unit and the memory unit, namely, only one interaction is needed between the operation device based on image processing and the memory unit; the data is not required to be read and stored with the memory unit for multiple times, so that the occupation of a bus between the operation device and the memory unit can be reduced, and the bandwidth pressure of the bus of the operation device is reduced; thereby speeding up the operation process.

Drawings

Fig. 1 is a first schematic view of an application scenario provided in an embodiment of the present application;

fig. 2 is a schematic diagram of an application scenario provided in the embodiment of the present application;

FIG. 3 is a diagram of a computing device according to the prior art;

fig. 4 is a schematic diagram of an image processing-based computing device according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of another image processing-based computing device according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of another image processing-based computing device according to an embodiment of the present disclosure;

fig. 7 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

FIG. 8 is a schematic view of a combined treatment apparatus according to an embodiment of the present disclosure;

fig. 9 is a first schematic diagram of an image processing chip according to an embodiment of the present disclosure;

fig. 10 is a second schematic diagram of an image processing chip according to an embodiment of the present disclosure;

fig. 11 is a third schematic diagram of an image processing chip according to an embodiment of the present disclosure;

fig. 12 is a schematic diagram of a board card provided in the embodiment of the present application;

fig. 13 is a first schematic diagram of an electronic device according to an embodiment of the present disclosure;

fig. 14 is a second schematic diagram of an electronic device according to an embodiment of the present application;

fig. 15 is a third schematic view of an electronic device according to an embodiment of the present application;

fig. 16 is a schematic flowchart of an operation method based on image processing according to an embodiment of the present disclosure;

fig. 17 is a flowchart illustrating another operation method based on image processing according to an embodiment of the present application.

Reference numerals:

11: a prior art image decoding unit;

12: a color conversion unit in the prior art;

13: an intelligent processing unit in the prior art;

14: memory cells in the prior art;

1: an image decoding unit;

2: a memory unit;

3: an intelligent processing unit;

4: a central processing unit;

01: an image decoding module;

02: an image conversion module;

03: a first buffer module;

04: a second buffer module;

05: a direct memory access module.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be understood that the terms "first", "second", etc. in the claims, description, and drawings of the present application are used for distinguishing between different objects and not for describing a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this application, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the application. As used in the specification and claims of this application, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the specification and claims of this application refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

As used in this specification and claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

The technical solution of the present application will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

With the development and application of artificial intelligence, artificial intelligence chips and devices are beginning to be widely used. Among them, an intelligent arithmetic device can be applied to the field of image processing. For example, an intelligent computing device is applied to the fields of face recognition, license plate recognition and the like.

Fig. 1 is a schematic view of an application scenario, as shown in fig. 1, an artificial intelligence chip and an apparatus may be applied to face recognition, and a face image is directly recognized by using the artificial intelligence chip or an arithmetic apparatus, so that a recognition result may be obtained, where the recognition result may be an image or other data.

Fig. 2 is a schematic view of an application scenario ii provided in the embodiment of the present application, and as shown in fig. 2, an artificial intelligence chip and an artificial intelligence device may be applied to license plate recognition, and a vehicle image is directly recognized by using the artificial intelligence chip or an arithmetic device, so that a recognition result may be obtained, where the recognition result may be an image or other data.

For example, an Image may be input into the intelligent computing device, for example, the Image is in a Format such as JPEG, Webp, High Efficiency Image File Format (HEIF); then, the intelligent arithmetic device performs processing such as decoding and conversion on the picture, and converts the image into an image of a format required by the intelligent processing unit.

In the existing artificial intelligence technology, each unit in the arithmetic device can be directly connected with a memory unit through a bus respectively; the arithmetic device and the memory unit are two mutually independent logic hardware. Thus, the computing device can process the image to present the image to the memory unit for storage.

Specifically, fig. 3 is a schematic diagram of a computing device in the prior art, as shown in fig. 3, the computing device includes an image decoding unit 11, a color conversion unit 12, and an intelligent processing unit 13; the image decoding unit 11, the color conversion unit 12, and the intelligent processing unit 13 are connected to the memory unit 14 through buses, respectively. After the image decoding unit 11 decodes the image, an image in a first color space format is obtained, and the image decoding unit 11 sends the image in the first color space format to the memory unit 14 through a bus for storage; the color conversion unit 12 reads the image in the first color space format from the memory unit 14 through the bus; the color conversion unit 12 converts the image in the first color space format into an image in a second color space format, and sends the image in the second color space format to the memory unit 14 through a bus for storage; thus, the intelligent processing unit 13 can read the image in the second color space format from the memory unit 14 to process the image in the second color space format.

However, in the above-described embodiment, since the image decoding unit 11 needs to store the image data in the memory unit 14, and the color conversion unit 12 also needs to read the image data and store the image data in the memory unit 14, each unit in the arithmetic device needs to read and store the data with the memory unit 14 a plurality of times. Thereby occupying the bus between the computing device and the memory unit 14 and thereby increasing the bandwidth pressure on the computing device bus.

In order to solve the problems in the prior art, the present application provides an arithmetic device and related products based on image processing to accelerate the arithmetic process.

Fig. 4 is a schematic diagram of an image processing-based computing device according to an embodiment of the present application, and as shown in fig. 4, the image processing-based computing device includes: an image decoding unit 1; the image decoding unit 1 includes an image decoding module 01 and an image conversion module 02.

The image decoding module 01 is configured to obtain an image to be processed, decode the image to be processed to obtain an image based on the first color space format, and output the image based on the first color space format to the image conversion module 02.

The image conversion module 02 is configured to perform color space conversion on the image based on the first color space format to obtain an image based on the second color space format, and send the image based on the second color space format to the memory unit 2 connected to the computing device based on image processing through the bus.

Illustratively, the image processing-based arithmetic device provided includes an image decoding unit 1, and the image processing-based arithmetic device is a hardware unit. The image decoding unit 1 is a hardware unit in an arithmetic device based on image processing, and the image decoding unit 1 may have an independent hardware configuration or may be a circuit unit.

The image decoding unit 1 is constituted by an image decoding module 01 and an image conversion module 02. The image decoding module 01 and the image conversion module 02 may be connected by a wire in a circuit, or the image decoding module 01 and the image conversion module 02 may be connected by another module or another unit.

The image decoding module 01 may be an independent hardware structure or a circuit unit; the image conversion module 02 may be a separate hardware structure or may be a circuit unit. In this regard, the present embodiment is not limited.

The image processing-based arithmetic device may be provided in the image processing apparatus, and the image processing-based arithmetic device may be connected to other devices in the image processing apparatus. Further, the image processing-based arithmetic device can acquire the image to be processed transmitted from another device. In one example, the image decoding module 01 acquires an image to be processed transmitted by another device. In this regard, the present embodiment is not limited.

The image conversion module 02 may be connected to the memory unit 2 located outside the image processing-based arithmetic device via a bus. Optionally, the image conversion module 02 may also be connected to the memory unit 2 through a bus.

For example, the image processing apparatus has a memory therein, and the image processing-based operation apparatus is connected to the memory, and the image processing-based operation apparatus can acquire the image to be processed from the memory. In one example, the image decoding module 01 retrieves the image to be processed from the memory.

For another example, the image processing apparatus has a memory unit 2, and the image processing-based operation apparatus is connected to the memory unit 2, and the image processing-based operation apparatus can acquire the image to be processed from the memory unit 2. In one example, the image decoding module 01 obtains the image to be processed from the memory unit 2.

For another example, the image processing apparatus has a receiver, and the image processing-based operation device is connected to the receiver, and receives the image to be processed sent from the external device through the receiver. In one example, the image decoding module 01 receives a to-be-processed image transmitted from an external device through a receiver.

After the image decoding module 01 acquires the image to be processed, an image decoding algorithm is configured in the image decoding module 01, so that the image decoding module 01 can decode the image to be processed, and further obtain an image based on the first color space format.

The image decoding algorithm in the image decoding module 01 may be an existing image decoding algorithm. For different image compression formats, the image decoding module 01 decodes the image to be processed by using an image decoding algorithm corresponding to the image compression format. Image compression formats include, but are not limited to, the following: BMP, EPS, GIF, JPG, PDF, PIC, PNG, PSD, TIF.

The image decoding module 01 decodes the image to be processed to obtain an image based on the first color space format. A first color space format, including but not limited to the following color spaces: YUV color space, RGB color space, gray scale color space.

The image decoding module 01 outputs the obtained image based on the first color space format to the image conversion module 02.

Since the image conversion module 02 is located in the image decoding unit 1, the image conversion module 02 in the image decoding unit 1 can directly perform color space conversion on the image based on the first color space format; thus, the image decoding unit 1 no longer needs to transfer the image to the memory unit 2 for storage so that the color conversion unit performs color space conversion on the image. In this embodiment, the image decoding module 01 in the image decoding unit 1 does not need to transmit the image based on the first color space format to the memory unit 2 for storage.

The image conversion module 02 is configured with a color space conversion algorithm, so that the image conversion module 02 can perform color space conversion on the image based on the first color space format by using the color space conversion algorithm to obtain the image based on the second color space format.

For example, when the first color space format is YUV color space, the second color space format is RGB color space, or the second color space format is gray-scale color space.

For another example, when the second color space format is RGB color space, the first color space format is YUV color space; or, when the second color space format is a color space of gray scale, the first color space format is a YUV color space.

Since the image conversion module 02 may be connected to the memory unit 2 through the bus, after the image conversion module 02 obtains the image based on the second color space format, the image conversion module 02 may transmit the image based on the second color space format to the memory unit 2 through the bus. The memory unit 2 may store or otherwise process the image based on the second color space format.

In the above process, the image decoding unit 1 does not need to interact with the memory unit 2 in the process of obtaining the image based on the second color space format; the image decoding unit 1 only needs to send the image based on the second color space format to the memory unit 2 through the bus. In the whole process, only one interaction is needed between the image decoding unit 1 and the memory unit 2, that is, only one interaction is needed between the image processing-based computing device and the memory unit 2. In the prior art, the image decoding unit 11 in the prior art needs to send the image in the first color space format to the memory unit 14 in the prior art through the bus for storage, the color conversion unit 12 in the prior art reads the image in the first color space format from the memory unit 14 in the prior art through the bus, and after performing color space conversion, the color conversion unit 12 in the prior art needs to send the image in the second color space format to the memory unit 14 in the prior art through the bus; the whole process needs to read and store data with the memory unit 2 for many times.

The present embodiment provides an image processing-based arithmetic device comprising an image decoding unit 1, wherein the image decoding unit 1 comprises an image decoding module 01 and an image conversion module 02; the image decoding module 01 is configured to obtain an image to be processed, decode the image to be processed to obtain an image based on a first color space format, and output the image based on the first color space format to the image conversion module 02; the image conversion module 02 is configured to perform color space conversion on the image based on the first color space format to obtain an image based on the second color space format, and send the image based on the second color space format to the memory unit 2 connected to the computing device based on image processing through the bus. By configuring the image decoding module 01 and the image conversion module 02 in the image decoding unit 1 of the arithmetic device, the image decoding module 01 in the image decoding unit 1 can decode the image to obtain an image based on the first color space format; because the image conversion module 02 is configured in the image decoding unit 1, the image decoding module 01 in the image decoding unit 1 can directly send the image based on the first color space format to the image conversion module 02 in the image decoding unit 1 for processing, and does not need to send the image to the memory unit 2; after the image conversion module 02 in the image decoding unit 1 performs color space conversion on the image, the obtained image based on the second color space format is sent to the memory unit 2, so that other units can obtain the image based on the second color space format from the memory unit 2. In the whole process, only one interaction is needed between the image decoding unit 1 and the memory unit 2, namely, only one interaction is needed between the image processing-based computing device and the memory unit 2; the data is not required to be read and stored with the memory unit 2 for multiple times, so that the occupation of a bus between the operation device and the memory unit 2 can be reduced, and the bandwidth pressure of the bus of the operation device is reduced; thereby speeding up the operation process.

Fig. 5 is a schematic diagram of another image processing-based operation device according to an embodiment of the present application, and based on the embodiment shown in fig. 4, as shown in fig. 5, the image decoding unit 1 further includes a first buffer module 03.

The image decoding module 01 is specifically configured to output the image based on the first color space format to the first buffer module 03 for buffering.

The first buffer module 03 is configured to send a first trigger signal to the image conversion module 02 when it is determined that the data amount in the first buffer module 03 is greater than a preset threshold, so that the image conversion module 02 acquires an image in the first buffer module 03.

In one example, the image decoding unit 1 further includes a second buffer module 04.

The image conversion module 02 is specifically configured to output the image based on the second color space format to the second buffer module 04 for buffering.

The second buffer module 04 is configured to trigger the memory unit 2 to obtain the image in the second buffer module 04 when it is determined that the data amount in the second buffer module 04 is greater than the preset threshold.

In one example, the image decoding unit 1 further includes a direct memory access module 05 therein.

The second buffer module 04 is specifically configured to send a second trigger signal to the direct memory access module 05 when it is determined that the data amount in the second buffer module 04 is greater than the preset threshold.

The direct memory access module 05 is configured to, after receiving the second trigger signal, read the image based on the second color space format from the second buffer module 04, and send the image based on the second color space format to the memory unit 2 through the bus.

Illustratively, the provided image processing-based arithmetic device includes an image decoding unit 1, and the image decoding unit 1 is constituted by an image decoding module 01 and an image conversion module 02.

In one example, the image decoding unit 1 further includes a first buffer module 03; the image decoding module 01 is connected with the first buffer module 03, and the image conversion module 02 is connected with the first buffer module 03. For example, the image decoding module 01 is connected to the first buffer module 03 through a wiring in a circuit, and the image conversion module 02 is connected to the first buffer module 03 through a wiring in a circuit.

The first buffer module 03 may be an independent hardware structure or a circuit unit; in this regard, the present embodiment is not limited.

In one example, the image decoding unit 1 further includes a second buffer module 04. The image conversion module 02 is connected to the second buffer module 04, and for example, the image conversion module 02 is connected to the second buffer module 04 through a wiring in a circuit. The memory unit 2 is connected to the second buffer module 04, and in one example, the second buffer module 04 is connected to the memory unit 2 through a bus.

The second buffer module 04 may be an independent hardware structure, or may be a circuit unit; in this regard, the present embodiment is not limited.

The image decoding module 01 outputs the image based on the first color space format to the first buffer module 03; the first buffer module 03 buffers the image based on the first color space format. Accordingly, the image decoding module 01 may continuously transmit the image data based on the first color space format to the first buffer module 03 for buffering. The first buffer module 03 is configured with a first data amount threshold, that is, the amount of data that the first buffer module 03 can buffer is less than or equal to the first data amount threshold.

In the process of caching data by the first buffer module 03, that is, in the process of caching the image data based on the first color space format by the first buffer module 03, the first buffer module 03 may determine whether the data amount in the first buffer module 03 is greater than a preset threshold value in real time. If the first buffer module 03 determines that the data amount in the first buffer module 03 is less than or equal to the preset threshold, the first buffer module 03 continues to buffer the data. If the first buffer module 03 determines that the data amount in the first buffer module 03 is greater than the preset threshold, the first buffer module 03 sends a first trigger signal to the image conversion module 02; after the image conversion module 02 receives the first trigger signal, the image conversion module 02 determines that data can be acquired from the first buffer module 03, and then the image conversion module 02 acquires the image data in the first buffer module 03. In the above process, since the image conversion module 02 and the first buffer module 03 are both located in the image decoding unit 1, the image decoding unit 1 does not need to transmit the image to the memory unit 2 for storage, so that the color conversion unit performs color space conversion on the image; the image conversion module 02 in the image decoding unit 1 can directly perform color space conversion on the image based on the first color space format.

For example, the image decoding module 01 outputs the image based on the first color space format to the first buffer module 03; in the process of outputting, the first buffer module 03 buffers image data; and, after the first buffer module 03 determines that the lines of image data are buffered, the first buffer module 03 triggers the image conversion module 02 to acquire the image data in the first buffer module 03. For example, after the first buffer module 03 determines that 4 lines of YUV422 of image data are buffered, the first buffer module 03 triggers the image conversion module 02 to obtain the image data in the first buffer module 03. For another example, after the first buffer module 03 determines that the image data of 4 lines of RGB is buffered, the first buffer module 03 triggers the image conversion module 02 to obtain the image data in the first buffer module 03.

The image conversion module 02 outputs the image based on the second color space format to the second buffer module 04; the second buffer module 04 buffers the image based on the second color space format. Thus, the image conversion module 02 may continuously transmit the image data based on the second color space format to the second buffer module 04 for buffering. The second buffer module 04 is configured with a second data amount threshold, that is, the amount of data that the second buffer module 04 can buffer is less than or equal to the second data amount threshold.

In the process of caching data by the second buffering module 04, that is, in the process of caching the image data based on the second color space format by the second buffering module 04, the second buffering module 04 may determine whether the data amount in the second buffering module 04 is greater than the preset threshold in real time. If the second buffer module 04 determines that the data amount in the second buffer module 04 is less than or equal to the preset threshold, the second buffer module 04 continues to buffer the data. If the second buffer module 04 determines that the data amount in the second buffer module 04 is greater than the preset threshold, the second buffer module 04 sends a trigger signal to the memory unit 2 through the bus; after the memory unit 2 receives the trigger signal, the memory unit 2 obtains the image data in the second buffer module 04.

For example, in the process of buffering data by the second buffer module 04, after the second buffer module 04 determines that multiple lines of image data are buffered, the memory unit 2 in the second buffer module 04 acquires the image data in the second buffer module 04. For example, after the second buffer module 04 determines that 4 lines of YUV422 of image data are buffered, the second buffer module 04 triggers the memory unit 2 to obtain the image data in the second buffer module 04. For another example, after the second buffer module 04 determines that the image data of 4 lines of RGB is buffered, the second buffer module 04 triggers the memory unit 2 to obtain the image data in the second buffer module 04.

In the above process, the image decoding unit 1 does not need to interact with the memory unit 2 in the process of obtaining the image based on the second color space format; the image decoding unit 1 only needs to send the second buffer module 04 in the image decoding unit 1 to the memory unit 2 through the bus after obtaining the image based on the second color space format. In the whole process, only one interaction is needed between the image decoding unit 1 and the memory unit 2, that is, only one interaction is needed between the image processing-based computing device and the memory unit 2.

In an example, since the Memory unit 2 is independent of the image processing-based computing device provided in this embodiment, and the Memory unit 2 is independent of the image decoding unit 1, in order to facilitate the Memory unit 2 to acquire and store the image based on the second color space format, a Direct Memory Access (DMA) module 05 may be disposed in the image decoding unit 1. The direct memory access module 05 is a hardware logic. In one example, the direct memory access module 05 is connected to the second buffer module 04, for example, the direct memory access module 05 is connected to the second buffer module 04 through a wiring in a circuit; the direct memory access module 05 is connected to the memory unit 2 via a bus.

Also, the direct memory access module 05 may be configured with an original address and a destination address for indicating an address of the read image data.

In the process of caching data by the second buffering module 04, that is, in the process of caching the image data based on the second color space format by the second buffering module 04, the second buffering module 04 may determine whether the data amount in the second buffering module 04 is greater than the preset threshold in real time. If the second buffer module 04 determines that the data amount in the second buffer module 04 is less than or equal to the preset threshold, the second buffer module 04 continues to buffer the data. If the second buffer module 04 determines that the data amount in the second buffer module 04 is greater than the preset threshold, the second buffer module 04 sends a second trigger signal to the direct memory access module 05; the direct memory access module 05 determines that the image data can be stored in the memory unit 2 after receiving the second trigger signal; further, the direct memory access module 05 reads the image based on the second color space format from the second buffer module 04, that is, reads the image data based on the second color space format; the direct memory access module 05 sends the read image data based on the second color space format to the memory unit 2 through the bus. Further, the memory unit 2 stores image data based on the second color space format.

In the present embodiment, in addition to the above-described embodiments, the first buffer module 03 and the second buffer module 04 are arranged in the image decoding unit 1 in the arithmetic device based on image processing; furthermore, in the process that the image decoding module 01 outputs the image data based on the first color space format, the first buffer module 03 may buffer the image data based on the first color space format, thereby relieving the data pressure of the image conversion module 02; in the process of outputting the image data based on the second color space format by the image conversion module 02, the second buffer module 04 may buffer the image data based on the second color space format, thereby relieving the data pressure of the memory unit 2. A direct memory access module 05 may also be configured in the image decoding unit 1, and then the image based on the second color space format in the second buffer module 04 is transmitted to the memory unit 2 for storage through the direct memory access module 05. In the above process, only one interaction is needed between the image decoding unit 1 and the memory unit 2, that is, only one interaction is needed between the image processing-based computing device and the memory unit 2; the data is not required to be read and stored with the memory unit 2 for multiple times, so that the occupation of a bus between the operation device and the memory unit 2 can be reduced, and the bandwidth pressure of the bus of the operation device is reduced; thereby speeding up the operation process.

Fig. 6 is a schematic diagram of another image processing-based computing device according to an embodiment of the present application, and based on the embodiments shown in fig. 4 to fig. 5, as shown in fig. 6, the image processing-based computing device further includes an intelligent processing unit 3.

And the intelligent processing unit 3 is configured to read the image based on the second color space format in the memory unit 2, and perform intelligent processing on the image based on the second color space format.

In one example, the image processing-based arithmetic device further includes a central processing unit 4.

The central processing unit 4 is configured to receive a third trigger signal sent by the image decoding unit 1, where the third trigger signal is used to indicate that the image decoding unit 1 has completed processing one or more frames of images; and dispatching the intelligent processing unit 3 according to the third trigger signal.

In one example, the central processor unit 4 is further configured to: acquiring a fourth trigger signal, wherein the fourth trigger signal is used for indicating to start a color space format conversion function of the image conversion module 02; and according to the fourth trigger signal, starting the color space format conversion function of the image conversion module 02.

In one example, the central processor unit 4 is further configured to:

acquiring a fifth trigger signal, wherein the fifth trigger signal is used for indicating to close the color space format conversion function of the image conversion module 02; according to the fifth trigger signal, the color space format conversion function of the image conversion module 02 is turned off, or according to the fifth trigger signal, a sixth trigger signal is sent to the image conversion module 02, so that the image conversion module 02 does not receive the image output by the image decoding module 01 any more.

Illustratively, on the basis of the above-described embodiment, the intelligent processing unit 3 may also be provided in an image processing-based arithmetic device. The intelligent processing unit 3 may be a separate hardware structure or may be a circuit unit. The intelligent processing unit 3 is connected with the memory unit 2 through a bus.

After the memory unit 2 stores the image data based on the second color space format, the smart processing unit 3 may read the image data based on the second color space format from the memory unit 2. In one example, the intelligent processing unit 3 may actively read the image data based on the second color space format from the memory unit 2 when it is determined that the intelligent processing unit 3 needs to process the image data; in one example, the image decoding unit 1 may trigger the intelligent processing unit 3 to read the image data based on the second color space format from the memory unit 2 after determining that the image data based on the second color space format is stored in the memory unit 2; for example, the direct memory access module 05 triggers the intelligent processing unit 3 to read image data.

The intelligent processing unit 3 intelligently processes the image based on the second color space format according to the current demand. The intelligent processing unit 3 is configured with an intelligent processing algorithm.

In one example, the intelligent processing unit 3 may be a neural network unit, or an inference unit.

In one example, the image processing-based arithmetic device identifies the face image currently, so that the image decoding unit 1 decodes the face image, converts the color space, and stores the image data into the memory unit 2; the intelligent processing unit 3 acquires image data from the memory unit 2, and the image data is subjected to face recognition. For example, the image decoding unit 1 stores the RGB image data in the memory unit 2, and the intelligent processing unit 3 obtains the RGB image data from the memory unit 2 and performs face recognition on the RGB image data by using a face recognition algorithm.

In one example, the image processing-based computing device identifies the license plate image currently, so that the image decoding unit 1 decodes the license plate image, converts the color space of the license plate image, and stores the image data into the memory unit 2; the intelligent processing unit 3 acquires image data from the memory unit 2, and the image data is used for license plate recognition. For example, the image decoding unit 1 stores the RGB image data in the memory unit 2, and the intelligent processing unit 3 obtains the RGB image data from the memory unit 2, and performs license plate recognition on the RGB image data by using a license plate recognition algorithm.

A Central Processing Unit 4 (CPU for short) may also be arranged in the image Processing-based arithmetic device.

The central processing unit 4 may be an independent hardware structure, or may be a circuit unit; in this regard, the present embodiment is not limited. The central processor unit 4 is, for example, an independent processor unit.

The central processing unit 4 is connected to the image decoding unit 1, for example, the central processing unit 4 may be connected to the image decoding unit 1 through a wiring in a circuit. The central processor unit 4 is connected to the intelligent processing unit 3, for example, the central processor unit 4 may be connected to the intelligent processing unit 3 through wiring in a circuit.

When the intelligent processing unit 3 is arranged in the image processing-based computing device, the intelligent processing unit 3 can acquire the image data in the memory unit 2 to perform intelligent processing on the image data; in this process, central processor unit 4 may participate in invoking intelligent processing unit 3. In one example, the image decoding unit 1 may determine whether one or more frames of images are to be stored in the memory during the process of storing the image data based on the second color space format in the memory unit 2, and the like. If the image decoding unit 1 determines that one or more frames of images are to be stored in the memory, it may be determined that the image decoding unit 1 completes processing of one or more frames of images, that is, completes the color space conversion process of one or more frames of images; the image decoding unit 1 sends a third trigger signal to the central processing unit 4; the central processing unit 4 schedules the intelligent processing unit 3 according to the received third trigger signal, so that the intelligent processing unit 3 reads the image based on the second color space format in the memory unit 2. The intelligent processing unit 3 intelligently processes the image based on the second color space format according to the current demand.

For example, the direct memory access module 05 may send a third trigger signal to the central processor unit 4.

In an example, the image processing-based computing device provided in this embodiment of the present application directly configures the image decoding module 01 and the image conversion module 02 in the image decoding unit 1, and further, after the image decoding module 01 decodes an image, the image conversion module 02 may directly perform color space conversion on the decoded image data, so as to obtain an image based on the second color space format; the color space conversion of the image is required because the intelligent processing unit 3 can only perform intelligent processing of the image based on the image in the second color space format, i.e. the intelligent processing unit 3 can only process the image based on the second color space format, e.g. the intelligent processing unit 3 can only process RGB images or grayscale images. Then, when the intelligent processing unit 3 is not required to perform intelligent processing on the image, the color space format conversion function of the image conversion module 02 is not required to be started; when the intelligent processing unit 3 needs to perform intelligent processing on the image, the color space format conversion function of the image conversion module 02 needs to be started.

Therefore, in an initial stage or a scene with non-intelligent processing, the image decoding unit 1 only needs to perform image decoding or other processing on the image to be processed; when the image processing-based operation device determines that the current scene is intelligently processed, determining that the color space format conversion function of the image conversion module 02 needs to be started; at this time, the central processing unit 4 may obtain a fourth trigger signal, where the fourth trigger signal indicates to start the color space format conversion function of the image conversion module 02; the fourth trigger signal may be sent to the central processing unit 4 by another unit in the arithmetic device based on image processing, and the fourth trigger signal may be automatically generated by the central processing unit 4. The central processing unit 4 starts the color space format conversion function of the image conversion module 02 according to the fourth trigger signal. Further, the image decoding and color space conversion processes of any of the above embodiments can be performed.

For example, the image processing-based computing device is located in an image processing device, and the image processing device acquires an image through a camera. The camera sends the acquired image to the image decoding unit 1 for decoding. The image decoding unit 1 transmits the decoded image to the memory unit 2 for storage; for example, in the structure shown in fig. 4, the image decoding module 01 transmits the decoded image to the memory unit 2 for storage through the image conversion module 02, and at this time, the image conversion module 02 does not perform color space conversion on the image; for another example, in the structure shown in fig. 5, the image decoding module 01 transmits the decoded image to the memory unit 2 for storage through the first buffer module 03, the image conversion module 02, the second buffer module 04, and the direct memory access module 05, and at this time, the image conversion module 02 does not perform color space conversion on the image; for another example, the image decoding module 01 transmits the decoded image to the memory unit 2 through the first buffer module 03, the image conversion module 02 and the second buffer module 04 for storage, and at this time, the image conversion module 02 does not perform color space conversion on the image. In the process of acquiring an image, the central processing unit 4 in the image processing-based computing device detects an external fourth trigger signal, for example, the central processing unit 4 receives the fourth trigger signal sent by the controller of the image processing device, and for example, the central processing unit 4 determines that a target object is detected according to the acquired image; the central processing unit 4 starts the color space format conversion function of the image conversion module 02 according to the fourth trigger signal. Further, after the image decoding module 01 decodes the image, the image conversion module 02 performs color space conversion on the decoded image.

In a scene without intelligent processing or after the intelligent processing is completed, the intelligent processing unit 3 does not need to perform intelligent processing on the image by the intelligent processing unit 3, and the intelligent processing unit 3 does not need to acquire the image based on the second color space format from the memory unit 2, so that the image decoding unit 1 does not need to perform color space conversion on the image, and the image conversion module 02 does not need to perform a color space conversion process. Thus, when the intelligent processing unit 3 is not required to perform intelligent processing on the image, the color space format conversion function of the image conversion module 02 does not need to be started.

Therefore, in a scene of non-intelligent processing or after the intelligent processing is completed, the central processing unit 4 may acquire a fifth trigger signal indicating that the color space format conversion function of the image conversion module 02 is turned off; the fifth trigger signal may be sent to the central processing unit 4 by another unit in the image processing-based arithmetic device, or the fifth trigger signal may be automatically generated by the central processing unit 4. The central processing unit 4 turns off the color space format conversion function of the image conversion module 02 according to the fifth trigger signal; or, the central processing unit 4 sends a sixth trigger signal to the image conversion module 02, where the sixth trigger signal is used to indicate that the image conversion module 02 does not need to receive the image output by the image decoding module 01, and at this time, the image decoding module 01 may be directly connected to the memory unit 2 through a bus, so that the image decoding module 01 may send the decoded image to the memory unit 2 through the bus for storage.

For example, in the process of acquiring an image, the central processing unit 4 in the image processing-based computing device detects an external fifth trigger signal, for example, the central processing unit 4 receives the fifth trigger signal sent by the controller of the image processing device, and for example, the central processing unit 4 determines that the target object cannot be detected according to the acquired image; the central processing unit 4 turns off the color space format conversion function of the image conversion module 02 according to the fifth trigger signal. The image decoding module 01 decodes the image sent by the camera.

In the structure shown in fig. 4, the image decoding module 01 transmits the decoded image to the memory unit 2 for storage through the image conversion module 02, and at this time, the image conversion module 02 does not perform color space conversion on the image; for another example, in the structure shown in fig. 5, the image decoding module 01 transmits the decoded image to the memory unit 2 for storage through the first buffer module 03, the image conversion module 02, the second buffer module 04, and the direct memory access module 05, and at this time, the image conversion module 02 does not perform color space conversion on the image; for another example, the image decoding module 01 transmits the decoded image to the memory unit 2 through the first buffer module 03, the image conversion module 02 and the second buffer module 04 for storage, and at this time, the image conversion module 02 does not perform color space conversion on the image. Alternatively, since the image decoding module 01 and the memory unit 2 are connected by a bus, the image decoding module 01 can send the decoded image to the memory unit 2 for storage by the bus.

In the present embodiment, on the basis of the above-mentioned embodiments, the intelligent processing unit 3 is also configured in the arithmetic device based on image processing; the intelligent processing unit 3 may obtain the image based on the second color space format in the memory unit 2, and perform intelligent processing on the image based on the second color space format. In the above process, the image decoding unit 1 in the image processing operation device stores image data into the memory unit 2, the intelligent processing unit 3 reads the image data from the memory unit 2 for processing, and the operation device value of the image processing needs to interact with the memory unit 2 twice; the data is not required to be read and stored with the memory unit 2 for multiple times, so that the occupation of a bus between the operation device and the memory unit 2 can be reduced, and the bandwidth pressure of the bus of the operation device is reduced; thereby speeding up the operation process. Moreover, a central processing unit 4 is also arranged in the arithmetic device based on the image processing, and the central processing unit 4 can control the on and off of the color space format conversion function of the image conversion module 02; when the image conversion module 02 is required to perform color space, the central processing unit 4 starts the color space format conversion function of the image conversion module 02, so that the units and the modules are matched with each other to complete the decoding and color space conversion of the image, and the intelligent processing unit 3 can perform intelligent processing on the image timely and accurately; when the color space of the image conversion module 02 is not required, the central processing unit 4 turns off the color space format conversion function of the image conversion module 02, thereby saving the operation resources of the operation device and improving the processing efficiency of the operation resources.

The units may be hardware circuits including digital circuits, analog circuits, and the like. Physical implementations of hardware circuits include, but are not limited to, physical devices including, but not limited to, transistors, memristors, and the like. The arithmetic unit in the arithmetic device can be any suitable hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, and the like. The memory unit may be any suitable magnetic or magneto-optical storage medium, such as RRAM, DRAM, SRAM, EDRAM, HBM, HMC, etc.

Fig. 7 is a schematic diagram of an image processing apparatus according to an embodiment of the present application, and as shown in fig. 7, the image processing apparatus includes one or more image processing-based computing devices according to any one of the above embodiments, and is configured to obtain an image to be processed from another processing apparatus, execute a specified image processing operation, and transmit an execution result to the other processing apparatus through an I/O interface;

when the image processing device comprises a plurality of arithmetic devices based on image processing, the arithmetic devices based on image processing can be connected through a specific structure and transmit data;

the image processing method comprises the following steps that a plurality of image processing-based operation devices are interconnected through a Peripheral Component Interface Express (PCIE) bus and transmit data; a plurality of arithmetic devices based on image processing share the same control system or own respective control systems; the plurality of image processing-based computing devices share a memory unit or have respective memory units.

The plurality of image processing-based arithmetic devices are connected in an arbitrary connection topology. The image processing device has high compatibility and can be connected with various types of servers through the PCIE interface.

In one example, the memory unit is located in the image processing apparatus.

Fig. 8 is a schematic diagram of a combined processing device according to an embodiment of the present application, and as shown in fig. 8, the combined processing device includes the image processing device, the universal interconnection interface, and other processing devices.

The image processing device interacts with other processing devices to jointly complete the operation designated by the user.

The combined device can be used as a system of intelligent equipment such as a mobile phone, a robot, an unmanned aerial vehicle and the like, the core area of a control part is effectively reduced, the processing speed is increased, and the overall power consumption is reduced.

Fig. 9 is a first schematic diagram of an image processing chip according to an embodiment of the present disclosure, and as shown in fig. 9, the image processing chip includes the image processing apparatus as described above.

In one example, at least one image processing device is included in the image processing chip.

Fig. 10 is a second schematic diagram of an image processing chip according to an embodiment of the present application, and as shown in fig. 10, the image processing chip includes the combined processing device as described above.

In one example, at least one combination processing device is included in the image processing chip.

Fig. 11 is a third schematic diagram of an image processing chip according to an embodiment of the present disclosure, and as shown in fig. 11, the image processing chip includes the image processing apparatus as described above or the combined processing apparatus as described above.

In one example, the image processing chip includes at least one image processing device and at least one combination processing device. The image processing device and the combined processing device may or may not perform data interaction.

Fig. 12 is a schematic view of a board card provided in an embodiment of the present application, and as shown in fig. 12, the board card includes the image processing chip.

In one example, at least one image processing chip is arranged in the board card.

Fig. 13 is a first schematic diagram of an electronic device according to an embodiment of the present disclosure, fig. 14 is a second schematic diagram of an electronic device according to an embodiment of the present disclosure, and fig. 15 is a third schematic diagram of an electronic device according to an embodiment of the present disclosure, where as shown in fig. 13 to fig. 15, the electronic device includes the image processing chip or the board card according to the above.

Wherein, this electron device includes: data processing apparatus, robot, computer, printer, scanner, tablet computer, smart terminal, cell-phone, vehicle event data recorder, navigator, sensor, camera, high in the clouds server, camera, projecting apparatus, wrist-watch, earphone, mobile storage, wearable device, vehicle, domestic appliance, and/or medical equipment.

The vehicles comprise airplanes, ships and/or vehicles; the household appliances comprise a television, an air conditioner, a microwave oven, a refrigerator, an electric cooker, a humidifier, a washing machine, an electric lamp, a gas stove and a range hood; the medical equipment comprises a nuclear magnetic resonance apparatus, a B-ultrasonic apparatus and/or an electrocardiograph.

Fig. 16 is a schematic flowchart of an image processing-based operation method according to an embodiment of the present application, and as shown in fig. 16, the image processing-based operation method is applied to an image processing-based operation device, where the image processing-based operation device includes an image decoding unit, and the image decoding unit includes an image decoding module and an image conversion module; the operation method based on image processing comprises the following steps:

101. the image decoding module acquires an image to be processed, decodes the image to be processed to obtain an image based on the first color space format, and outputs the image based on the first color space format to the image conversion module.

102. The image conversion module performs color space conversion on the image based on the first color space format to obtain an image based on a second color space format, and sends the image based on the second color space format to a memory unit connected with the operation device through a bus.

The principle and technical effect of each step in this embodiment refer to the above embodiments, and are not described again.

Fig. 17 is a schematic flowchart of another image processing-based operation method according to an embodiment of the present disclosure, and as shown in fig. 17, the image processing-based operation method is applied to an image processing-based operation device, where the image processing-based operation device includes an image decoding unit, and the image decoding unit includes an image decoding module and an image conversion module; the operation method based on image processing comprises the following steps:

201. the image decoding module acquires an image to be processed, decodes the image to be processed to obtain an image based on the first color space format, and outputs the image based on the first color space format to the image conversion module.

202. The image decoding unit also comprises a first buffer module; the image decoding module outputs the image based on the first color space format to the first buffer module for buffering.

203. When the data volume in the first buffer module is determined to be larger than the preset threshold value, the first buffer module sends a first trigger signal to the image conversion module, so that the image conversion module acquires the image in the first buffer module.

204. The image conversion module performs color space conversion on the image based on the first color space format to obtain an image based on a second color space format, and sends the image based on the second color space format to a memory unit connected with the operation device through a bus.

In one example, the image decoding unit further includes a second buffering module, and step 204 specifically includes: the image conversion module outputs the image based on the second color space format to a second buffer module for caching; and when the second buffer module determines that the data volume in the second buffer module is larger than a preset threshold value, the second buffer module triggers the memory unit to acquire the image in the second buffer module.

In one example, the image decoding unit further comprises a direct memory access module; the method provided by this embodiment may further include the following steps:

the first step is that the second buffer module sends a second trigger signal to the direct memory access module when determining that the data volume in the second buffer module is larger than a preset threshold value.

And in the second step, after receiving the second trigger signal, the direct memory access module reads the image based on the second color space format from the second buffer module, and sends the image based on the second color space format to the memory unit through the bus.

In one example, the image processing-based arithmetic device further includes an intelligent processing unit; the operation method based on image processing may further include the steps of:

and thirdly, reading the image based on the second color space format in the memory unit by the intelligent processing unit, and intelligently processing the image based on the second color space format.

In one example, the image processing-based arithmetic method further includes a central processor unit; the operation method based on image processing may further include the steps of:

a fourth step, the central processing unit receives a third trigger signal sent by the image decoding unit, wherein the third trigger signal is used for indicating that the image decoding unit completes processing one or more frames of images; and scheduling the intelligent processing unit according to the third trigger signal.

In one example, the operation method based on image processing may further include the following steps:

a fifth step of acquiring a fourth trigger signal by the central processing unit, wherein the fourth trigger signal is used for indicating to start the color space format conversion function of the image conversion module; and starting the color space format conversion function of the image conversion module according to the fourth trigger signal.

In one example, the operation method based on image processing, in one example:

a sixth step of acquiring a fifth trigger signal by the central processing unit, wherein the fifth trigger signal is used for indicating to close the color space format conversion function of the image conversion module; and according to the fifth trigger signal, closing the color space format conversion function of the image conversion module, or according to the fifth trigger signal, sending a sixth trigger signal to the image conversion module so that the image conversion module does not receive the image output by the image decoding module any more.

Reference throughout this application to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An arithmetic device based on image processing, comprising: an image decoding unit; the image decoding unit comprises an image decoding module and an image conversion module;

2. The image-processing-based operation device according to claim 1, wherein the image decoding unit further comprises a first buffer module;

3. The image-processing-based operation device according to claim 1, wherein the image decoding unit further comprises a second buffer module;

4. The image-processing-based arithmetic device according to claim 3, wherein the image decoding unit further includes a direct memory access module;

5. The image-processing-based arithmetic device according to any one of claims 1 to 4, wherein the image-processing-based arithmetic device further comprises an intelligent processing unit;

6. The image-processing-based arithmetic device according to claim 5, further comprising a central processing unit;

7. The image-processing-based arithmetic device of claim 6, wherein the central processor unit is further configured to:

8. The image-processing-based arithmetic device of claim 6, wherein the central processor unit is further configured to:

9. An image processing apparatus, characterized in that the image processing apparatus comprises one or more image processing-based arithmetic devices according to any one of claims 1 to 8, for acquiring the image to be processed from other processing devices, executing the specified image processing operation, and transmitting the execution result to other processing devices through an I/O interface;

10. A combined processing apparatus comprising the image processing apparatus according to claim 9, a universal interconnect interface, and other processing means;

11. An image processing chip, characterized in that it comprises an image processing device according to claim 9 or a combined processing device according to claim 10.

12. A board characterized in that the board comprises the image processing chip of claim 11.

13. An electronic device, characterized in that the electronic device comprises the image processing chip of claim 11 or the board of claim 12.

14. An operation method based on image processing is characterized in that the method is applied to an operation device based on image processing, the operation device based on image processing comprises an image decoding unit, and the image decoding unit comprises an image decoding module and an image conversion module; the operation method based on image processing comprises the following steps:

15. The image-processing-based operation method according to claim 14, wherein the image decoding unit further comprises a first buffer module;

16. The image-processing-based operation method according to claim 14, wherein the image decoding unit further comprises a second buffer module;

17. The image-processing-based operation method according to claim 16, wherein the image decoding unit further includes a direct memory access module;

18. The image-processing-based operation method according to any one of claims 14 to 17, wherein the image-processing-based operation device further includes an intelligent processing unit; the operation method based on image processing further comprises the following steps: