CN117830488A

CN117830488A - Image processing method and device, readable medium and electronic equipment

Info

Publication number: CN117830488A
Application number: CN202410155992.9A
Authority: CN
Inventors: 谢飞; 刘纯; 李佳佳
Original assignee: Beijing Zitiao Network Technology Co Ltd
Current assignee: Beijing Zitiao Network Technology Co Ltd
Priority date: 2024-02-02
Filing date: 2024-02-02
Publication date: 2024-04-05

Abstract

The present disclosure relates to an image processing method, an image processing apparatus, a readable medium, and an electronic device. Image data to be processed can be acquired; performing image preprocessing on the image data to be processed through a pre-configured script to obtain first image data, wherein the script is used for defining execution logic of the image preprocessing; and inputting the first image data into a preset image processing model to obtain second image data.

Description

Image processing method and device, readable medium and electronic equipment

Technical Field

The disclosure relates to the technical field of image processing, and in particular relates to an image processing method, an image processing device, a readable medium and electronic equipment.

Background

In processing the image, a special effect image, such as a mosaic, a sticker, etc., may be added to the picture or video. And the special effect is added to the image, so that various visual algorithms such as a face algorithm, image segmentation and the like are needed.

The input to the visual algorithm is image information, which is typically from an upper layer link. The image is subject to some changes after being processed by the upper layer link, and needs to be restored to the original state before the image is input into the algorithm. That is, before inputting an image to the vision algorithm, an image preprocessing operation needs to be performed on image data acquired from an upper layer link.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, the present disclosure provides an image processing method, the method comprising:

acquiring image data to be processed;

performing image preprocessing on the image data to be processed through a pre-configured script to obtain first image data, wherein the script is used for defining execution logic of the image preprocessing;

and inputting the first image data into a preset image processing model to obtain second image data.

In a second aspect, the present disclosure provides an image processing apparatus, the apparatus comprising:

the acquisition module is used for acquiring the image data to be processed;

the first image processing module is used for carrying out image preprocessing on the image data to be processed through a pre-configured script to obtain first image data, wherein the script is used for defining execution logic of the image preprocessing;

And the second image processing module is used for inputting the first image data into a preset image processing model to obtain second image data.

In a third aspect, the present disclosure provides a computer readable medium having stored thereon a computer program which when executed by a processing device performs the steps of the method of the first aspect of the present disclosure.

In a fourth aspect, the present disclosure provides an electronic device comprising:

a storage device having a computer program stored thereon;

processing means for executing said computer program in said storage means to carry out the steps of the method of the first aspect of the disclosure.

Through the technical scheme, the image data to be processed is obtained; performing image preprocessing on the image data to be processed through a pre-configured script to obtain first image data, wherein the script is used for defining execution logic of the image preprocessing; and inputting the first image data into a preset image processing model to obtain second image data. In this way, the execution logic of the image preprocessing is defined through the script, so that the image preprocessing can be flexibly completed through the script. The subsequent updating operation is converged to the script layer, so that the modification at the algorithm code layer is avoided, the realization is easier, and the subsequent expansion and updating are facilitated.

Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale. In the drawings:

fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment.

Fig. 2 is a flowchart of an image processing method according to the embodiment shown in fig. 1.

Fig. 3 is a flowchart of an image processing method according to the embodiment shown in fig. 1.

Fig. 4 is a flowchart of an image processing method according to the embodiment shown in fig. 3.

Fig. 5 is a block diagram of an image processing apparatus according to an exemplary embodiment.

Fig. 6 is a block diagram of an image processing apparatus according to the embodiment shown in fig. 5.

Fig. 7 is a schematic diagram of an electronic device according to an exemplary embodiment.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.

For example, in response to receiving an active request from a user, a prompt is sent to the user to explicitly prompt the user that the operation it is requesting to perform will require personal information to be obtained and used with the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium for executing the operation of the technical scheme of the present disclosure according to the prompt information.

As an alternative but non-limiting implementation, in response to receiving an active request from a user, the manner in which the prompt information is sent to the user may be, for example, a popup, in which the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.

It will be appreciated that the above-described notification and user authorization process is merely illustrative and not limiting of the implementations of the present disclosure, and that other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.

Meanwhile, it can be understood that the data (including but not limited to the data itself, the acquisition or the use of the data) related to the technical scheme should conform to the requirements of the corresponding laws and regulations and related regulations.

First, an application scenario of the present disclosure will be described, and the present disclosure is mainly applied to a scenario in which an image is subjected to image preprocessing before an image acquired from an upper link is input to a visual algorithm. Since the upper link usually performs processes such as stitching or mirror rotation on the images, but the visual algorithm model is obtained by training based on the original images, in the model application stage of the visual algorithm, the images input into the visual algorithm model also need to be restored (may also be referred to as image preprocessing) on the basis of stitching or mirror rotation, and then input into the visual algorithm.

For example, an input image acquired from a live link may be filled with black edges due to the fact that the image size is not consistent with the aspect ratio of the user terminal screen. But images filled with black edges are not expected for the visual algorithm because the training images given to the algorithm model are typically artwork during the training phase of the visual algorithm model, and these unexpected images will reduce the accuracy of visual algorithm recognition, and therefore image changes due to the upper links need to be restored back before the visual algorithm is entered.

In the related art, the following problems exist by copying image data to be processed from a GPU (Graphics Processing Unit, graphics processor) to a CPU (Central Processing Unit ), and then invoking an algorithm in a visual library on the CPU to perform image preprocessing: the adaptation is needed at the engine level, the algorithm codes are modified, and the modified algorithm codes are compiled to realize different image preprocessing, so that the process is complex, and the subsequent expansion and updating are not facilitated. In addition, the image data to be processed is copied from the GPU to the CPU, and then the CPU invokes the algorithm in the visual library to perform image preprocessing, which is relatively time-consuming, and mainly includes two parts, one part is the time taken for copying from the GPU to the CPU, and the other part is the speed of processing the image on the CPU is usually much lower than the speed of directly performing image processing on the GPU.

In order to solve the above-mentioned problems, the present disclosure provides an image processing method, an apparatus, a readable medium, and an electronic device. The following detailed description of specific embodiments of the present disclosure refers to the accompanying drawings.

Fig. 1 is a flowchart illustrating an image processing method according to an exemplary embodiment, as shown in fig. 1, including the steps of:

In step S101, image data to be processed is acquired.

The image data to be processed herein refers to image data that requires image preprocessing. The image preprocessing may include, for example, cropping and/or mirror-flipping. The image data to be processed may typically be acquired from an upper layer link. Where "upper link" generally refers to a series of image processing steps or data transfer processes prior to the image input algorithm. These steps may include image acquisition devices (e.g., cameras, scanners), image transmission processes (e.g., data transmission, compression, encoding, etc.), and pre-processing (e.g., image rotation, translation, scaling, color space conversion, etc.). These processing steps may cause changes to the image, resulting in the image no longer being in the original input state.

For example, taking the example that the upper link is a live link, the anchor performs online interaction with the user watching live, and the anchor may obtain the head portrait of the user interacting with the live link. Because the proportion of the image size of the head portrait of the interactive user and the screen size of the anchor end is inconsistent, the live link adds black edges to the original image of the head portrait of the interactive user, namely, the head portrait of the interactive user acquired by the anchor end is spliced (namely, the original image and the black background are spliced), if special effects (such as beautifying processing) are added to the head portrait of the interactive user, the head portrait of the interactive user needs to be cut before the image data of the head portrait of the interactive user is input to a visual algorithm, so that the black edges are removed to obtain the original image. Therefore, in this scenario, the image data to be processed is the image data corresponding to the interactive user head portrait, which is acquired by the anchor end and is subjected to the stitching processing, and includes a black edge.

In step S102, image preprocessing is performed on the image data to be processed through a pre-configured script, so as to obtain first image data, where the script is used to define execution logic of the image preprocessing.

A script generally refers to a series of instructions arranged in a particular order for performing a particular task. In the present disclosure, the script of the preset configuration refers to script code written in advance based on execution logic of the image preprocessing process, and the script may include JavaScript, python script or the like, for example.

In this step, the image preprocessing may be performed on the image data to be processed on the GPU through the script configured in advance, to obtain the first image data.

Since scripts are typically programs written in a dynamic language, they do not need to be compiled into binary code, but are executed by an interpreter line by line. Therefore, the image preprocessing operation is realized based on the script, and the image preprocessing operation can be directly executed on the GPU without copying to the CPU, so that the time-consuming problem of the image preprocessing operation on the CPU is solved.

In step S103, the first image data is input into a preset image processing model, and then second image data is obtained.

The preset image processing model may include a visual algorithm model, for example, a face algorithm model, a portrait segmentation model, and the like.

In the disclosure, after the algorithm engine invokes the preset image processing model, the first image data is input into the preset image processing model to obtain the second image data.

In addition, the method can enable the algorithm engine to sense that the input first image data is subjected to the image preprocessing operation by setting the flag bit, wherein different image preprocessing operations can correspond to different flag bits.

By adopting the method, the execution logic of the image preprocessing is defined through the script, so that the image preprocessing is flexibly completed through the script. The subsequent updating operation is converged to the script layer, so that the modification at the algorithm code layer is avoided, the realization is easier, and the subsequent expansion and updating are facilitated. In addition, since the script is usually a program written in a dynamic language, the script is not required to be compiled into binary codes, but is executed by an interpreter line by line, so that the script-based image preprocessing operation can be directly executed on the GPU without copying to the CPU to perform the image preprocessing operation, and the problem of time consumption of performing the image preprocessing operation on the CPU is solved.

Fig. 2 is a flowchart of an image processing method according to the embodiment shown in fig. 1. In the present disclosure, the image data to be processed includes change data relative to an original acquired image. The change data may include image data different from the original acquired image and tag data indicating a type of change. For example, in order to adapt to the screen size of the target terminal, the upper link splices the black frame on the basis of the original acquired image, and then the change data may include the pixel position of each pixel point in the black frame and the first label indicating that the change type is the splice. For another example, if the upper link performs mirror rotation on the original acquired image, the change data may include a rotation center of the mirror rotation and a transformation matrix, and may further include a second tag indicating that the change type is the mirror rotation. This is by way of example only, and the disclosure is not limited thereto in particular.

As shown in fig. 2, step S102 includes the sub-steps of:

in step S1021, the task type of the image preprocessing currently performed is determined according to the change data through the script, and different task types correspond to different image preprocessing operations.

Wherein the task types include: cutting and/or mirror-turning.

In the first mode of this step, the task type of the image preprocessing currently to be performed may be determined according to the tag data for indicating the change type carried in the change data. For example, if the tag data carried in the change data is the first tag, it may be determined that the upper link performs the stitching process on the original acquired image, where the task type of the image preprocessing to be performed currently is cutting; if the tag data carried in the change data is the second tag, it can be determined that the upper link performs mirror rotation processing on the original acquired image, and in this case, the task type of image preprocessing to be performed currently is mirror inversion so as to obtain the original acquired image.

In the second mode, the task type may be determined by the script according to the change data and the model type of the preset image processing model.

It will be appreciated that different types of preset image processing models will also have different requirements on the input image. For example, some algorithms may only require processing of a partial region in the image (e.g., for a face algorithm model, only a face region on the image is typically processed). Therefore, before the image acquired by the upper link is input into the preset image processing model, the region of interest of the model in the image can be cut, so that the image size of an input algorithm is reduced, and the efficiency and the accuracy of image processing are improved.

For example, after determining that the image acquired by the upper layer link needs to be cropped and/or mirror-inverted (based on the first implementation) based on the change data, if it is determined that the model type is a model that only processes a partial region in the image according to the model type of the preset image processing model, the image may be further cropped so as to obtain a target region of interest of the preset image processing model (for example, the target region corresponding to the face algorithm model is a face image region), and then the image of the target region may be input to the preset image processing model.

In step S1022, the image data to be processed is subjected to image preprocessing through the script according to the task type, so as to obtain first image data.

In this step, a first affine transformation matrix corresponding to the task type may be generated by the script according to the image data to be processed; and performing image preprocessing on the image data to be processed according to the first affine transformation matrix by the script-driven rendering engine to obtain the first image data.

Wherein the different task types correspond to different first affine transformation matrices, which are transformation matrices for implementing image preprocessing operations.

In the present disclosure, by executing a pre-configured script, a first affine transformation matrix corresponding to a current task type may be generated according to the image data to be processed, and then the script-driven rendering engine implements an image preprocessing operation of the image data to be processed according to the first affine transformation matrix based on the vertex shader.

By way of example only, the process may be performed,the script layer may drive the rendering engine to perform a corresponding affine transformation on the input image texture, e.g., if the task type is clipping, the first affine transformation matrix used in the vertex shader may be:the examples herein are merely illustrative, and the present disclosure is not limited thereto.

Fig. 3 is a flow chart of an image processing method according to the embodiment shown in fig. 1, the method further comprising the steps of:

in step S104, image rendering is performed according to the second image data.

In one implementation, image rendering may be performed by a rendering engine based on the second image data.

Since the input image of the preset image processing model is an image after image preprocessing, and after the image is preprocessed, the coordinate system corresponding to the image changes, and then the coordinate system corresponding to the algorithm result (i.e., the second image data) output by the preset image processing model is the coordinate system of the image after the preprocessing, in this disclosure, in order to avoid the situation that the image is misplaced when being rendered on the screen, before the algorithm result is input to the rendering engine for rendering, affine inverse transformation may be performed on the algorithm result to achieve mapping of the algorithm result to the first coordinate system (the first coordinate system refers to the coordinate system of the image before the preprocessing (i.e., the image corresponding to the image data to be processed)).

Fig. 4 is a flowchart of an image processing method according to the embodiment shown in fig. 3, and as shown in fig. 4, step S104 includes the following sub-steps:

in step S1041, a second affine transformation matrix is determined according to the first affine transformation matrix, where the second affine transformation matrix is used to map the second image data to a first coordinate system corresponding to the image data to be processed.

After determining the first affine transformation matrix corresponding to the current image preprocessing operation, the script engine can also output the first affine transformation matrix to the algorithm engine, so that the algorithm engine realizes affine inverse transformation of an algorithm result based on the first affine transformation matrix.

The second image data includes dot location data and/or mask, that is, the algorithm result to be restored includes two types, dot location data and mask data.

The second affine transformation matrix comprises a first transformation matrix for mapping the point location data to the first coordinate system and/or a second transformation matrix for mapping the mask to the first coordinate system.

In the present disclosure, a first transformation matrix, i.e., a first affine transformation matrix, is used to map the point location data to a first coordinate system. Thus, mapping the point location data to the first coordinate system may be achieved by the following formula:

Wherein M represents the first transformation matrix, [ x, y, z ]] ^T Representing point location data in a coordinate system in which the algorithm results are located (i.e., a second coordinate system hereinafter) [ x ', y ', z ] '] ^T Represents [ x, y, z ]] ^T Point location data in a first coordinate system.

In addition, this second transformation matrix in the present disclosure is determined by:

acquiring first size data of image data to be processed in the first coordinate system, second size data of the second image data in the second coordinate system and a transformation matrix of the mask outputted by the preset image processing model;

the second coordinate system refers to the coordinate system of the image after the image preprocessing operation, and the second coordinate system is also the coordinate system corresponding to the algorithm result. The first size data may include a length and a width of a corresponding image of the image in the first coordinate system prior to performing the image preprocessing operation. The second size data refers to the length and width of the image corresponding to the algorithm result in the second coordinate system.

And determining the second transformation matrix according to the first affine transformation matrix, the first size data, the second size data and the transformation matrix of the mask output by the preset image processing model.

Illustratively, since the algorithm result output by the visual algorithm is generally non-normalized, and the transformation matrix output by the script engine is normalized, the inverse normalization processing of the first affine transformation matrix output by the script engine is required, which may be specifically implemented by:

where dstW and dstH represent the width and length of the image in the second coordinate system, respectively. srcW and srcH respectively represent the width and length of the image under the first coordinate system, M represents the first affine transformation matrix, and M' is the product of three matrices on the left of the equal sign.

Assuming that the first coordinate system (i.e., the original coordinate system) is the next point v, then there is an equation: t 'M' ^-1 v=t "v, thus, T" =t 'M' ^-1 。

Wherein T 'represents a transformation matrix of a mask outputted by a preset image processing model, M' ^-1 The inverse matrix of matrix M' is represented, T "being the second transformation matrix.

In step S1042, third image data corresponding to the second image data in the first coordinate system is determined according to the second affine transformation matrix.

In this step, the second image data may be mapped to the first coordinate system based on affine transformation theory through the second affine transformation matrix, resulting in the third image data.

In step S1043, image rendering is performed according to the third image data.

In this step, the third image data may be input to a rendering engine, through which image rendering is achieved.

Fig. 5 is a block diagram of an image processing apparatus according to an exemplary embodiment, as shown in fig. 5, the apparatus including:

an acquiring module 501, configured to acquire image data to be processed;

the first image processing module 502 is configured to perform image preprocessing on the image data to be processed through a pre-configured script, so as to obtain first image data, where the script is used to define execution logic of the image preprocessing;

the second image processing module 503 is configured to input the first image data into a preset image processing model, and obtain second image data.

Optionally, the image data to be processed includes change data relative to an original acquired image, and the first image processing module 502 is configured to determine, according to the change data, a task type of image preprocessing to be performed currently through the script, where different task types correspond to different image preprocessing operations; and performing image preprocessing on the image data to be processed through the script according to the task type to obtain first image data.

Optionally, the first image processing module 502 is configured to generate, according to the image data to be processed, a first affine transformation matrix corresponding to the task type through the script; and performing image preprocessing on the image data to be processed according to the first affine transformation matrix by the script-driven rendering engine to obtain the first image data.

Optionally, the task types include: cutting and/or mirror-turning.

Optionally, the first image processing module 502 is configured to determine the task type according to the change data and a model type of the preset image processing model through the script.

Optionally, the first image processing module 502 is configured to perform image preprocessing on the image data to be processed through the pre-configured script on the GPU to obtain the first image data.

Optionally, fig. 6 is a block diagram of an image processing apparatus according to the embodiment shown in fig. 5, and as shown in fig. 6, the apparatus further includes:

and the image rendering module 504 is configured to perform image rendering according to the second image data.

Optionally, the image rendering module 504 is configured to determine a second affine transformation matrix according to the first affine transformation matrix, where the second affine transformation matrix is configured to map the second image data to a first coordinate system corresponding to the image data to be processed; determining third image data corresponding to the second image data in the first coordinate system according to the second affine transformation matrix; and performing image rendering according to the third image data.

Optionally, the second image data comprises point location data and/or a mask, the second affine transformation matrix comprises a first transformation matrix for mapping the point location data to the first coordinate system and/or a second transformation matrix for mapping the mask to the first coordinate system.

Optionally, the first transformation matrix is the first affine transformation matrix.

Optionally, the second transformation matrix is determined by:

Referring now to fig. 7, a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure is shown. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 7, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphic processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some implementations, the clients may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring image data to be processed; performing image preprocessing on the image data to be processed through a pre-configured script to obtain first image data, wherein the script is used for defining execution logic of the image preprocessing; and inputting the first image data into a preset image processing model to obtain second image data.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented in software or hardware. The name of a module is not limited to the module itself in some cases, and for example, the acquisition module may also be described as "a module that acquires image data".

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, example 1 provides an image processing method, including:

acquiring image data to be processed;

inputting the first image data into a preset image processing model to obtain second image data

According to one or more embodiments of the present disclosure, example 2 provides the method of example 1, the image data to be processed includes change data relative to an original acquired image, the image preprocessing the image data to be processed by a pre-configured script, and obtaining the first image data includes:

determining task types of image preprocessing currently to be performed through the script according to the change data, wherein different task types correspond to different image preprocessing operations;

and performing image preprocessing on the image data to be processed through the script according to the task type to obtain first image data.

According to one or more embodiments of the present disclosure, example 3 provides the method of example 2, wherein performing image preprocessing on the image data to be processed through the script according to the task type, to obtain first image data includes:

Generating a first affine transformation matrix corresponding to the task type through the script according to the image data to be processed;

and performing image preprocessing on the image data to be processed according to the first affine transformation matrix by the script-driven rendering engine to obtain the first image data.

Example 4 provides the method of example 2, according to one or more embodiments of the present disclosure, the task types including: cutting and/or mirror-turning.

Example 5 provides the method of example 2, according to one or more embodiments of the present disclosure, the determining, by the script, a task type of image pre-processing currently to be performed according to the change data comprising:

and determining the task type through the script according to the change data and the model type of the preset image processing model.

According to one or more embodiments of the present disclosure, example 6 provides the method of any one of examples 1 to 5, the performing image preprocessing on the image data to be processed through a pre-configured script, to obtain first image data includes:

and performing image preprocessing on the image data to be processed on the GPU through the pre-configured script to obtain the first image data.

Example 7 provides the method of example 3, according to one or more embodiments of the present disclosure, the method further comprising:

and performing image rendering according to the second image data.

According to one or more embodiments of the present disclosure, example 8 provides the method of example 7, the image rendering from the second image data comprising:

determining a second affine transformation matrix according to the first affine transformation matrix, wherein the second affine transformation matrix is used for mapping the second image data to a first coordinate system corresponding to the image data to be processed;

determining third image data corresponding to the second image data in the first coordinate system according to the second affine transformation matrix;

and performing image rendering according to the third image data.

According to one or more embodiments of the present disclosure, example 9 provides the method of example 8, the second image data comprising point location data and/or a mask, the second affine transformation matrix comprising a first transformation matrix for mapping the point location data to the first coordinate system and/or a second transformation matrix for mapping the mask to the first coordinate system.

According to one or more embodiments of the present disclosure, example 10 provides the method of example 9, the first transformation matrix being the first affine transformation matrix.

In accordance with one or more embodiments of the present disclosure, example 11 provides the method of example 9, the second transformation matrix is determined by:

According to one or more embodiments of the present disclosure, example 12 provides an image processing apparatus, the apparatus comprising:

the acquisition module is used for acquiring the image data to be processed;

According to one or more embodiments of the present disclosure, example 13 provides a computer-readable medium having stored thereon a computer program which, when executed by a processing device, implements the steps of the method of any of examples 1-11.

Example 14 provides an electronic device according to one or more embodiments of the present disclosure, comprising:

a storage device having a computer program stored thereon;

processing means for executing the computer program in the storage means to implement the steps of the method of any one of examples 1-11.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims. The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Claims

1. An image processing method, the method comprising:

acquiring image data to be processed;

2. The method of claim 1, wherein the image data to be processed includes change data relative to an original acquired image, wherein the image pre-processing the image data to be processed by a pre-configured script to obtain first image data includes:

3. The method according to claim 2, wherein performing image preprocessing on the image data to be processed through the script according to the task type to obtain first image data includes:

4. The method of claim 2, wherein the task types include: cutting and/or mirror-turning.

5. The method of claim 2, wherein determining the task type of the image pre-processing currently to be performed by the script based on the change data comprises:

6. The method according to any one of claims 1-5, wherein performing image preprocessing on the image data to be processed through a pre-configured script to obtain first image data includes:

7. A method according to claim 3, characterized in that the method further comprises:

And performing image rendering according to the second image data.

8. The method of claim 7, wherein the image rendering from the second image data comprises:

and performing image rendering according to the third image data.

9. The method according to claim 8, wherein the second image data comprises point location data and/or a mask, the second affine transformation matrix comprising a first transformation matrix for mapping the point location data to the first coordinate system and/or a second transformation matrix for mapping the mask to the first coordinate system.

10. The method according to claim 9, wherein the first transformation matrix is the first affine transformation matrix.

11. The method of claim 9, wherein the second transformation matrix is determined by:

12. An image processing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring the image data to be processed;

13. A computer readable medium on which a computer program is stored, characterized in that the program, when being executed by a processing device, carries out the steps of the method according to any one of claims 1-11.

14. An electronic device, comprising:

a storage device having a computer program stored thereon;

processing means for executing said computer program in said storage means to carry out the steps of the method according to any one of claims 1-11.