CN111862107A

CN111862107A - Method and system for processing images

Info

Publication number: CN111862107A
Application number: CN201910361701.0A
Authority: CN
Inventors: 范彦文; 付鹏; 周强; 寇浩锋; 包英泽
Original assignee: Baidu com Times Technology Beijing Co Ltd; Baidu USA LLC
Current assignee: Baidu com Times Technology Beijing Co Ltd; Baidu USA LLC
Priority date: 2019-04-30
Filing date: 2019-04-30
Publication date: 2020-10-30

Abstract

The embodiment of the application discloses a method and a system for processing images. The method is applied to a system for processing the image, the system for processing the image comprises an AI computing module set and a scheduling platform, and AI computing modules in the AI computing module set are connected in parallel; the method comprises the following steps: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result. The implementation mode utilizes the dispatching platform to dispatch the AI computing module set to process the image sequence, and can meet the image processing requirements of various complex scenes.

Description

Method and system for processing images

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a method and a system for processing images.

Background

The AI (Artificial Intelligence) computing module may be hardware integrated by an AI chip, a USB (Universal Serial Bus), a network, a sensor, and the like. The AI computing module can quickly and conveniently increase the AI capacity in a conventional scene, and is an important practical way for AI. However, AI visual scenes are often not single and require multiple deep learning models to perform the overall function. However, the conventional AI computing module usually can only complete a single function, or is computationally inefficient, and cannot meet the image processing requirements of complex scenes.

Disclosure of Invention

The embodiment of the application provides a method and a system for processing an image.

In a first aspect, an embodiment of the present application provides a method for processing an image, which is applied to a system for processing an image, where the system for processing an image includes an AI calculation module set and a scheduling platform, and AI calculation modules in the AI calculation module set are connected in parallel; the method comprises the following steps: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

In some embodiments, the set of AI computing modules includes at least two parallel-processing AI computing modules and/or at least two serial-processing AI computing modules.

In some embodiments, the scheduling platform runs a Software Development Kit (SDK) through which AI computing modules in the set of AI computing modules are scheduled for parallel processing and/or serial processing.

In some embodiments, the scheduling platform groups the computing units included in the AI computing modules in the AI computing module set by SDK, and each group of computing units runs a deep learning model in the image processing process.

In some embodiments, the input information of the at least two parallel-processing AI computing modules is distributed to the at least two parallel-processing AI computing modules after being divided by the scheduling platform, and the output information of an upstream AI computing module of the at least two serial-processing AI computing modules is forwarded to a downstream AI computing module by the scheduling platform.

In some embodiments, the set of AI calculation modules includes a first AI calculation module, a second AI calculation module, a third AI calculation module, and a fourth AI calculation module, the first and second AI calculation modules running the object detection model, the third AI calculation module running the information detection model, and the fourth AI calculation module running the object recognition model.

In some embodiments, the scheduling platform sends the image sequence to a set of AI computation modules; and the AI computing module set processes the image sequence to generate a processing result, which comprises the following steps: the scheduling platform divides the image sequence into a first image subsequence and a second image subsequence, distributes the first image subsequence to the first AI computing module, and distributes the second image subsequence to the second AI computing module; the first AI computing module inputs the first image subsequence to the target detection model, obtains position information of a target in the first image subsequence, and sends the position information to the scheduling platform; the second AI computing module inputs the second image subsequence to the target detection model, obtains the position information of the target in the second image subsequence, and sends the position information to the scheduling platform; the scheduling platform divides a target image area sequence from the image sequence based on the position information of the target in the first image subsequence and the position information of the target in the second image subsequence, and sends the target image area sequence to a third AI computing module; the third AI computing module inputs the target image area sequence into the information detection model, obtains the information of the target in the target image area sequence, and sends the information to the scheduling platform; the scheduling platform selects a target image area from the target image area sequence based on the information of the target in the target image area sequence, and sends the target image area to the fourth AI computing module; and the fourth AI computing module inputs the selected target image area to the target recognition model to obtain a recognition result.

In some embodiments, the information detection model includes a score detection model, a pose detection model, and a keypoint detection model.

In some embodiments, the inputting of the sequence of target image regions into the information detection model by the third AI calculation module to obtain information of the target in the sequence of target image regions includes: and the third AI computing module respectively inputs the target image region sequence into the score detection model, the attitude detection model and the key point detection model to obtain score information, attitude information and key point information of the target in the target image region sequence.

In a second aspect, an embodiment of the present application provides a system for processing an image, including an AI calculation module set and a scheduling platform, where AI calculation modules in the AI calculation module set are connected in parallel; the scheduling platform is configured to send the image sequence to the AI computing module set; and the AI computing module set is configured to process the image sequence and generate a processing result.

In some embodiments, a scheduling platform configured to segment an image sequence into a first image subsequence and a second image subsequence, distribute the first image subsequence to a first AI computation module, and distribute the second image subsequence to a second AI computation module; the first AI computing module is configured to input the first image subsequence to the target detection model, obtain position information of a target in the first image subsequence, and send the position information to the scheduling platform; the second AI computing module is configured to input the second image subsequence to the target detection model, obtain position information of a target in the second image subsequence, and send the position information to the scheduling platform; the scheduling platform is configured to divide the target image area sequence from the image sequence based on the position information of the target in the first image subsequence and the position information of the target in the second image subsequence, and send the target image area sequence to the third AI computing module; the third AI computing module is configured to input the target image area sequence into the information detection model, obtain information of a target in the target image area sequence, and send the information to the scheduling platform; the scheduling platform is configured to select a target image area from the target image area sequence based on the information of the target in the target image area sequence and send the target image area to the fourth AI computing module; and the fourth AI computing module is configured to input the selected target image area into the target recognition model to obtain a recognition result.

In some embodiments, the third AI calculation module is configured to input the sequence of target image regions into the score detection model, the pose detection model, and the keypoint detection model, respectively, to obtain score information, pose information, and keypoint information of the target in the sequence of target image regions.

In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a storage device having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any implementation of the first aspect.

In a fourth aspect, the present application provides a computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements the method as described in any implementation manner of the first aspect.

In a fifth aspect, an embodiment of the present application provides another server, including: an interface; a memory having one or more programs stored thereon; and one or more processors, operatively connected to the interface and the memory, for: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

In a sixth aspect, embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, where the computer program, when executed by one or more processors, causes the one or more processors to: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

According to the method and the system for processing the image, the AI computing module set receives the image sequence sent by the scheduling platform, processes the image sequence and generates a processing result. The scheduling platform is used for scheduling the AI computing module set to process the image sequence, so that the image processing requirements of various complex scenes can be met. And the AI computing module set works simultaneously, thereby improving the processing efficiency of the image sequence.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture of a system for processing images according to the present application;

FIG. 2 is a flow diagram of one embodiment of a method for processing an image according to the present application;

FIG. 3 is a flow diagram of yet another embodiment of a method for processing an image according to the present application;

FIG. 4 is a timing diagram for one embodiment of a system for processing images according to the present application;

FIG. 5 is a timing diagram of yet another embodiment of a system for processing images according to the present application;

FIG. 6 is a schematic block diagram of a computer system suitable for use in implementing an electronic device according to embodiments of the present application.

Detailed Description

The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 shows an exemplary system architecture 100 of a system for processing images according to the present application.

As shown in fig. 1, the system architecture 100 may include a scheduling platform 101 and an AI computation module set 102. The AI calculation module set 102 may include

AI calculation modules

1021, 1022, 1023, and 1024. The scheduling platform 101 is connected to the AI computation module set 102. The

AI computing modules

1021, 1022, 1023 and 1024 are connected in parallel.

The scheduling platform 101 may interact with the set of AI computing modules 102 to receive or send information, etc. For example, the scheduling platform 101 may send the image sequence to the set of AI computation modules 102. The AI computation module set 102 can process the image sequence to generate a processing result.

It should be noted that the method for processing an image provided by the embodiment of the present application is generally performed by a system for processing an image.

It should be understood that the number of scheduling platforms, the set of AI computing modules, and the AI computing modules in FIG. 1 are merely illustrative. There may be any number of scheduling platforms, sets of AI computing modules, and AI computing modules, as desired for an implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method for processing an image according to the present application is shown. The method for processing the image is applied to a system for processing the image. The system for processing the image can comprise an AI computing module set and a scheduling platform, wherein the AI computing modules in the AI computing module set are connected in parallel, and the method comprises the following steps:

step 201, the scheduling platform sends the image sequence to an AI computation module set.

In this embodiment, a scheduling platform (e.g., the scheduling platform 101 shown in fig. 1) may send the image sequence to an AI calculation module set (e.g., the AI calculation module set 102 shown in fig. 1). The image sequence may include multiple video frames in the video, or multiple images obtained by one continuous shooting. Any object may be present in the images in the image sequence including, but not limited to, a human body, a human face, an animal, a plant, an object, a gesture, and the like.

Here, the scheduling platform may be various hardware platforms connected to the set of AI computation modules. Typically, the dispatch platform may be a hardware device running a Linux or Windows operating system. The AI calculation module may be hardware integrated by an AI chip, USB, network, sensors, and the like. Each AI computing module in the AI computing module set can be connected to the dispatching platform through a USB so as to realize the parallel connection of the AI computing modules in the AI computing module set.

Step 202, the AI calculation module set processes the image sequence to generate a processing result.

In this embodiment, the AI calculation module set can process the image sequence to generate a processing result. Generally, the image processing process needs a plurality of deep learning models to complete the whole function, and here, the set of AI computation modules can run all the deep learning models in the image processing process to process the images in the image sequence to generate the processing result.

In some optional implementations of the embodiment, the AI computation module set may include at least two parallel-processing AI computation modules and/or at least two serial-processing AI computation modules, so as to implement parallel processing, serial processing, or serial-parallel hybrid processing of the image computation task in the image processing process. Generally, the scheduling platform can run an SDK (software development Kit) that schedules parallel processing and/or serial processing of AI compute modules in the set of AI compute modules by the SDK. The parallel-processing AI computing modules run the same deep learning model, and the serial-processing AI computing modules run different deep learning models.

In some optional implementations of this embodiment, the input information of the at least two parallel-processing AI computing modules is divided by the scheduling platform and then distributed to the at least two parallel-processing AI computing modules. And the output information of an upstream AI computing module in the at least two serially processed AI computing modules is forwarded to a downstream AI computing module through the scheduling platform. Generally, the upstream AI computing module may send its output information to the scheduling platform, and the scheduling platform may send the received output information directly to the downstream AI computing module as its input information, or may process the image sequence based on the received output information and send the obtained processed information to the downstream AI computing module as its input information.

In some optional implementation manners of this embodiment, each of the AI calculation modules may run all the deep learning models in the image processing process, so as to implement parallel processing of the image calculation task in the image processing process. At this time, the images in the image sequence can be uniformly distributed to each AI calculation module for image processing, and finally, the processing results of each AI calculation module are summarized to obtain the processing results of the image sequence.

In some optional implementation manners of this embodiment, each AI computing module in the set of AI computing modules may run a deep learning model in the image processing process to implement serial processing of the image computing task in the image processing process. At this time, the scheduling platform may input the image sequence to the AI calculation module set, and the images in the image sequence are sequentially processed by the deep learning model operated by each AI calculation module in the AI calculation module set to generate a processing result.

In some optional implementation manners of this embodiment, at least two AI calculation modules in the AI calculation module set run the same deep learning model in the image processing process, and at least two AI calculation modules run different deep learning models in the image processing process, so as to implement serial-parallel hybrid processing of image calculation tasks in the image processing process.

For example, the deep learning model in the image processing process may include an object detection model, an information detection model, and an object recognition model. The AI calculation module set may include a first AI calculation module (e.g., the AI calculation module 1021 shown in fig. 1), a second AI calculation module (e.g., the AI calculation module 1022 shown in fig. 1), a third AI calculation module (e.g., the AI calculation module 1023 shown in fig. 1), and a fourth AI calculation module (e.g., the AI calculation module 1024 shown in fig. 1). The first AI calculation module and the second AI calculation module can run a target detection model, the third AI calculation module can run an information detection model, and the fourth AI calculation module can run a target recognition model. At this time, the first AI calculation module and the second AI calculation module are AI calculation modules for parallel processing. The first AI computing module, the second AI computing module, the third AI computing module and the fourth AI computing module are serially processed AI computing modules. Wherein the object detection model may be used to detect the position of an object in the image. The information detection model may be used to detect information of an object in an image. The object recognition model may be used to recognize objects in an image. The information of the target may include, but is not limited to, a score, a pose, and key points of the target, etc. For example, when the target is a face, the score of the target may be a probability that the target is a face, the pose of the target may be a pose of the face, and the key points of the target may be the five sense organs of the face.

It should be noted that the AI chip in the AI calculation module may include a plurality of different calculation units. In this embodiment, the scheduling architecture in the AI computation module is improved, so that different computation units can run different deep learning models respectively. Thus, the AI computing module has the ability to run multiple deep learning models simultaneously. For example, the scheduling platform may group the computing units included in the AI chip by SDK, and each group of computing units may run a deep learning model in the image processing process.

In the method for processing the image provided by the embodiment of the application, the AI calculation module set receives the image sequence sent by the scheduling platform, processes the image sequence, and generates a processing result. The scheduling platform is used for scheduling the AI computing module set to process the image sequence, so that the image processing requirements of various complex scenes can be met. And the AI computing module set works simultaneously, thereby improving the processing efficiency of the image sequence.

With further reference to FIG. 3, a flow 300 of yet another embodiment of a method for processing an image according to the present application is shown. The method for processing the image is applied to a system for processing the image. The system for processing the image can comprise an artificial intelligence AI computing module set and a scheduling platform, wherein AI computing modules in the AI computing module set are connected in parallel. The set of AI calculation modules may include a first AI calculation module, a second AI calculation module, a third AI calculation module, and a fourth AI calculation module. The first AI computing module and the second AI computing module operate the target detection model, the third AI computing module operates the information detection model, and the fourth AI computing module operates the target identification model, including the following steps:

Step 301, the scheduling platform divides the image sequence into a first image sub-sequence and a second image sub-sequence.

In this embodiment, a scheduling platform (e.g., scheduling platform 101 shown in fig. 1) may first segment the image sequence into a first image sub-sequence and a second image sub-sequence, and then perform steps 302 and 302' simultaneously. For example, the scheduling platform may segment odd frame images in the image sequence into a first image sub-sequence. At the same time, the even frame images in the image sequence are segmented into a second image sub-sequence.

Step 302, the first image subsequence is distributed to the first AI calculation module.

In this embodiment, the scheduling platform may distribute the first image sub-sequence to a first AI calculation module (e.g., AI calculation module 1021 shown in fig. 1).

Step 303, the first AI calculation module inputs the first image subsequence to the target detection model to obtain the position information of the target in the first image subsequence.

In this embodiment, for an image in the first image subsequence, the first AI calculation module may input the image to the object detection model to obtain the position information of the object in the image.

Step 304, the position information of the target in the first image sequence is sent to a dispatching platform.

In this embodiment, for an image in the first image subsequence, the first AI calculation module may send position information of an object in the image to the scheduling platform.

Step 302', the second image subsequence is distributed to the second AI calculation module.

In this embodiment, the scheduling platform may distribute the second image subsequence to a second AI calculation module (e.g., the AI calculation module 1022 shown in fig. 1).

Step 303', the second AI calculation module inputs the second image subsequence to the target detection model to obtain the position information of the target in the second image subsequence.

In this embodiment, for an image in the second image subsequence, the second AI calculation module may input the image to the object detection model to obtain the position information of the object in the image.

Step 304', the position information of the target in the second image sequence is sent to the scheduling platform.

In this embodiment, for an image in the second image subsequence, the second AI calculation module may send the position information of the object in the image to the scheduling platform.

It should be understood that, when both AI computation modules run the target detection model to detect images in the image sequence, twice the real-time processing frame rate can be achieved.

Step 305, the scheduling platform segments a sequence of target image regions from the sequence of images based on the position information of the target in the first image sub-sequence and the position information of the target in the second image sub-sequence.

In this embodiment, the scheduling platform may track the target based on the position information of the target in the first image sub-sequence and the position information of the target in the second image sub-sequence to determine the position information of the target in each frame of image in the image sequence. Subsequently, the scheduling platform may segment the corresponding target image region from each frame of image in the image sequence to obtain a target image region sequence. Wherein the target image area may be an area of a smallest rectangular frame containing the target.

Step 306, the sequence of target image areas is sent to the third AI calculation module.

In this embodiment, the scheduling platform may send the sequence of target image regions to a third AI calculation module (e.g., the AI calculation module 1023 shown in fig. 1).

Step 307, the third AI calculation module inputs the target image region sequence to the information detection model to obtain information of the target in the target image region sequence.

In this embodiment, for a target image area in the sequence of target image areas, the third AI calculation module may input the target image area to the information detection model to obtain information of a target in the target image area.

In some optional implementations of the present embodiment, the information detection model may include, but is not limited to, a score detection model, a pose detection model, a key point detection model, and the like. For example, the information detection model includes a score detection model, a pose detection model and a key point detection model, and the third AI calculation module may input the target image region sequence to the score detection model, the pose detection model and the key point detection model respectively to obtain score information, pose information and key point information of the target in the target image region sequence. At this time, the third AI computation module processes the three image computation tasks of score detection, pose detection, and key point detection in parallel.

And 308, sending the information of the target in the target image area sequence to a dispatching platform.

In this embodiment, the third AI calculation module may send information of the target in the sequence of target image areas to the scheduling platform.

Step 309, the scheduling platform selects a target image area from the target image area sequence based on the information of the target in the target image area sequence.

In this embodiment, the scheduling platform may select the target image region from the sequence of target image regions based on information of the target in the sequence of target image regions. Generally, the scheduling platform may select a target image region from the target image region sequence, where the score, the pose, and the key point all satisfy preset conditions. For example, a target image area with a higher score, a better pose and a better key point position is selected.

In step 310, the selected target image area is sent to the fourth AI calculation module.

In this embodiment, the scheduling platform may send the selected target image area to a fourth AI calculation module (e.g., the AI calculation module 1024 shown in fig. 1).

In step 311, the fourth AI calculation module inputs the selected target image area to the target recognition model to obtain a recognition result.

In this embodiment, the fourth AI calculation module may input the selected target image area to the target recognition model to obtain a recognition result.

As can be seen from fig. 3, the flow 300 of the method for processing an image in the present embodiment highlights image processing steps compared to the embodiment corresponding to fig. 2. Therefore, the scheme described in this embodiment can better meet the image processing requirements of various complex scenes by scheduling the AI computing module set to perform serial-parallel hybrid processing on the image sequence through the scheduling platform.

With further reference to FIG. 4, a timing diagram 400 of one embodiment of a system for processing an image according to the present application is shown.

The system for processing images in this embodiment may include an artificial intelligence AI computation module set and a scheduling platform, where AI computation modules in the AI computation module set are connected in parallel.

As shown in fig. 4, in step 401, the scheduling platform sends the image sequence to the set of AI computation modules.

In step 402, the AI computation module assembly processes the image sequence to generate a processing result.

In the present embodiment, the specific operations of steps 401 and 402 have been described in detail in

step

201 and 202 in the embodiment shown in fig. 2, and are not described herein again.

In some optional implementations of this embodiment, the set of AI computing modules includes at least two parallel-processing AI computing modules and/or at least two serial-processing AI computing modules.

In some optional implementations of this embodiment, the scheduling platform runs a software development kit SDK, and schedules the AI computing modules in the AI computing module set for parallel processing and/or serial processing via the SDK.

In some optional implementation manners of this embodiment, the scheduling platform groups the computing units included in the AI computing modules in the AI computing module set by using the SDK, and each group of computing units runs one deep learning model in the image processing process.

In some optional implementation manners of this embodiment, the input information of the at least two parallel-processing AI computing modules is divided by the scheduling platform and then distributed to the at least two parallel-processing AI computing modules, and the output information of the upstream AI computing module in the at least two serial-processing AI computing modules is forwarded to the downstream AI computing module by the scheduling platform.

In some optional implementation manners of this embodiment, the set of AI calculation modules includes a first AI calculation module, a second AI calculation module, a third AI calculation module, and a fourth AI calculation module, where the first AI calculation module and the second AI calculation module run the target detection model, the third AI calculation module runs the information detection model, and the fourth AI calculation module runs the target identification model.

With further reference to FIG. 5, a timing sequence 500 of yet another embodiment of a system for processing an image according to the present application is shown.

The system for processing images in this embodiment may include an artificial intelligence AI computation module set and a scheduling platform, where AI computation modules in the AI computation module set are connected in parallel. The set of AI calculation modules may include a first AI calculation module, a second AI calculation module, a third AI calculation module, and a fourth AI calculation module. The first AI computing module and the second AI computing module run the target detection model, the third AI computing module runs the information detection model, and the fourth AI computing module runs the target identification model.

As shown in fig. 5, in step 501, the scheduling platform segments the image sequence into a first image sub-sequence and a second image sub-sequence.

In step 502, the scheduling platform distributes the first image subsequence to the first AI computing module.

In step 503, the first AI calculation module inputs the first image subsequence to the target detection model to obtain the position information of the target in the first image subsequence.

In step 504, the first AI calculation module sends the location information of the target in the first image subsequence to the scheduling platform.

In step 502', the scheduling platform distributes the second image subsequence to the second AI computing module.

In step 503', the second AI calculation module inputs the second image subsequence to the target detection model to obtain position information of the target in the second image subsequence.

In step 504', the second AI calculation module sends the location information of the target in the second image subsequence to the scheduling platform.

Step 505, the scheduling platform segments a sequence of target image regions from the image sequence based on the position information of the target in the first image sub-sequence and the position information of the target in the second image sub-sequence.

In step 506, the scheduling platform sends the target image area sequence to the third AI calculation module.

In step 507, the third AI calculation module inputs the target image region sequence to the information detection model to obtain information of the target in the target image region sequence.

In step 508, the third AI calculation module sends the information of the targets in the target image area sequence to the scheduling platform.

In step 509, the scheduling platform selects a target image region from the target image region sequence based on the information of the target in the target image region sequence.

In step 510, the scheduling platform sends the selected target image area to a fourth AI calculation module.

In step 511, the fourth AI calculation module inputs the selected target image area to the target recognition model to obtain a recognition result.

In this embodiment, the specific operations of steps 501-511 and steps 502 '-504' have been described in detail in steps 301-311 and steps 302 '-304' in the embodiment shown in fig. 3, and are not described again here.

In some optional implementations of this embodiment, the information detection model includes a score detection model, a pose detection model, and a keypoint detection model.

In some optional implementation manners of this embodiment, the inputting, by the third AI calculation module, the target image region sequence to the information detection model to obtain information of the target in the target image region sequence includes: and the third AI computing module respectively inputs the target image region sequence into the score detection model, the attitude detection model and the key point detection model to obtain score information, attitude information and key point information of the target in the target image region sequence.

Referring now to FIG. 6, shown is a block diagram of a computer system 600 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or electronic device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

As another aspect, an embodiment of the present application provides another server, including: an interface; a memory having one or more programs stored thereon; and one or more processors, operatively connected to the interface and the memory, for: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

As another aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by one or more processors, the one or more processors are caused to: the scheduling platform sends the image sequence to an AI computing module set; and the AI computing module set processes the image sequence to generate a processing result.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A method for processing images is applied to a system for processing images, the system for processing images comprises an Artificial Intelligence (AI) computation module set and a scheduling platform, wherein AI computation modules in the AI computation module set are connected in parallel;

The method comprises the following steps:

the scheduling platform sends the image sequence to the AI computing module set;

and the AI computing module set processes the image sequence to generate a processing result.

2. The method of claim 1, wherein the set of AI computation modules comprises at least two parallel-processing AI computation modules and/or at least two serial-processing AI computation modules.

3. The method of claim 2, wherein the scheduling platform runs a Software Development Kit (SDK) through which AI computing modules in the set of AI computing modules are scheduled for parallel processing and/or serial processing.

4. The method of claim 3, wherein the scheduling platform groups computing units included in the AI computing modules in the set of AI computing modules by SDK, each group of computing units running a deep learning model in an image processing process.

5. The method according to claim 4, wherein the input information of the at least two parallel-processing AI computing modules is distributed to the at least two parallel-processing AI computing modules after being split by the scheduling platform, and the output information of an upstream AI computing module of the at least two serial-processing AI computing modules is forwarded to a downstream AI computing module by the scheduling platform.

6. The method of claim 5, wherein the set of AI computing modules includes a first AI computing module, a second AI computing module, a third AI computing module, and a fourth AI computing module, the first AI computing module and the second AI computing module running an object detection model, the third AI computing module running an information detection model, the fourth AI computing module running an object identification model.

7. The method of claim 6, wherein the scheduling platform sends the sequence of images to the set of AI computing modules; and the AI computing module set processes the image sequence to generate a processing result, which comprises:

the scheduling platform divides the image sequence into a first image subsequence and a second image subsequence, distributes the first image subsequence to the first AI computing module, and distributes the second image subsequence to the second AI computing module;

the first AI computing module inputs the first image subsequence to the target detection model, obtains position information of a target in the first image subsequence, and sends the position information to the scheduling platform;

the second AI computing module inputs the second image subsequence to the target detection model, obtains position information of a target in the second image subsequence, and sends the position information to the scheduling platform;

The scheduling platform divides a target image area sequence from the image sequence based on the position information of the target in the first image subsequence and the position information of the target in the second image subsequence, and sends the target image area sequence to the third AI computing module;

the third AI computing module inputs the target image area sequence into an information detection model, obtains information of a target in the target image area sequence, and sends the information to the scheduling platform;

the scheduling platform selects a target image area from the target image area sequence based on the information of the target in the target image area sequence, and sends the target image area to the fourth AI computing module;

and the fourth AI computing module inputs the selected target image area to the target recognition model to obtain a recognition result.

8. The method of claim 7, wherein the information detection model comprises a score detection model, a pose detection model, and a keypoint detection model.

9. The method according to claim 8, wherein the third AI computation module inputs the sequence of target image regions into an information detection model, and obtains information of the target in the sequence of target image regions, including:

And the third AI computing module respectively inputs the target image region sequence into the score detection model, the attitude detection model and the key point detection model to obtain score information, attitude information and key point information of the target in the target image region sequence.

10. A system for processing images comprises an artificial intelligence AI computing module set and a scheduling platform, wherein AI computing modules in the AI computing module set are connected in parallel;

the scheduling platform is configured to send the image sequence to the AI computing module set;

and the AI computing module set is configured to process the image sequence and generate a processing result.

11. The system of claim 10, wherein the set of AI computation modules comprises at least two parallel-processing AI computation modules and/or at least two serial-processing AI computation modules.

12. The system of claim 11, wherein the scheduling platform runs a Software Development Kit (SDK) through which AI computing modules in the set of AI computing modules are scheduled for parallel processing and/or serial processing.

13. The system of claim 12, wherein the scheduling platform groups computing units included in the AI computing modules in the set of AI computing modules by SDK, each group of computing units running a deep learning model in an image processing process.

14. The system according to claim 13, wherein the input information of the at least two parallel-processing AI computing modules is distributed to the at least two parallel-processing AI computing modules after being split by the scheduling platform, and the output information of an upstream AI computing module of the at least two serial-processing AI computing modules is forwarded to a downstream AI computing module by the scheduling platform.

15. The system of claim 14, wherein the set of AI computation modules includes a first AI computation module, a second AI computation module, a third AI computation module, and a fourth AI computation module, the first and second AI computation modules running an object detection model, the third AI computation module running an information detection model, and the fourth AI computation module running an object recognition model.

16. The system of claim 15, wherein,

the scheduling platform configured to segment the image sequence into a first image sub-sequence and a second image sub-sequence, distribute the first image sub-sequence to the first AI computation module, and distribute the second image sub-sequence to the second AI computation module;

the first AI computing module is configured to input the first image subsequence to the target detection model, obtain position information of a target in the first image subsequence, and send the position information to the scheduling platform;

The second AI computing module is configured to input the second image subsequence to the target detection model, obtain position information of a target in the second image subsequence, and send the position information to the scheduling platform;

the scheduling platform is configured to segment a target image region sequence from the image sequence based on the position information of the target in the first image subsequence and the position information of the target in the second image subsequence, and send the target image region sequence to the third AI computing module;

the third AI computing module is configured to input the target image region sequence into an information detection model, obtain information of a target in the target image region sequence, and send the information to the scheduling platform;

the scheduling platform is configured to select a target image area from the target image area sequence based on information of a target in the target image area sequence, and send the target image area to the fourth AI computing module;

and the fourth AI computing module is configured to input the selected target image area into the target recognition model to obtain a recognition result.

17. The system of claim 16, wherein the information detection model comprises a score detection model, a pose detection model, and a keypoint detection model.

18. The system of claim 17, wherein,

the third AI calculation module is configured to input the target image region sequence to the score detection model, the pose detection model and the key point detection model respectively, so as to obtain score information, pose information and key point information of a target in the target image region sequence.

19. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.

20. A computer-readable medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, carries out the method according to any one of claims 1-9.