CN109598348B

CN109598348B - Image sample acquisition and model training method and system

Info

Publication number: CN109598348B
Application number: CN201710896985.4A
Authority: CN
Inventors: 马星辰
Original assignee: Beijing Orion Star Technology Co Ltd
Current assignee: Beijing Orion Star Technology Co Ltd
Priority date: 2017-09-28
Filing date: 2017-09-28
Publication date: 2023-06-06
Anticipated expiration: 2037-09-28
Also published as: CN109598348A

Abstract

The embodiment of the invention provides an image sample acquisition and model training method and system, wherein the image sample acquisition method is applied to processing equipment in communication connection with a plurality of mechanical arms, the processing equipment comprises a plurality of first special CPUs for reading image data and a plurality of second special CPUs for processing the image data, and each mechanical arm is provided with a camera, and the method comprises the following steps: acquiring and storing image data acquired by each camera in real time; a plurality of first dedicated CPUs respectively and circularly execute reading processing; and the plurality of second special CPUs perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data. Therefore, the processing equipment adopts a streaming processing mode, can process the image data in real time, acquire the image sample, does not need to retrain a machine learning model when acquiring new image data, and greatly shortens the model training time.

Description

Image sample acquisition and model training method and system

Technical Field

The invention relates to the technical field of model training, in particular to a method and a system for acquiring an image sample and training a model.

Background

Along with the continuous popularization of the data-driven mechanical arm grabbing technology, the requirements on the mechanical arm grabbing precision are higher and higher. The acquisition of image samples is an important component in the data-driven mechanical arm grabbing technology, and only a large number of image samples are acquired, an accurate machine learning model for mechanical arm grabbing can be obtained through training of the image samples.

Currently, the image sample is generally obtained by: image data is acquired through a camera arranged on the mechanical arm, and is processed according to the training requirement of a machine learning model, for example, the image color change, the matrix transformation and the like are processed, so that an image sample is obtained. After a large number of image samples are obtained in the same way, a machine learning model for grabbing the mechanical arm can be obtained through training.

After the machine learning model is trained, if a camera installed on the mechanical arm acquires new image data, the new image data must be processed to obtain corresponding image samples, the corresponding image samples are combined into a large number of image samples, and then training of the machine learning model is performed again, so that model training time is very consumed.

Disclosure of Invention

The embodiment of the invention aims to provide an image sample acquisition and model training method and system so as to shorten the model training time. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides an image sample acquiring method applied to a processing device communicatively connected to a plurality of mechanical arms, where the processing device includes a plurality of first dedicated CPUs for reading image data and a plurality of second dedicated CPUs for processing image data, and each mechanical arm is provided with a camera, and the method includes:

acquiring and storing image data acquired by each camera in real time;

the plurality of first special CPUs respectively and circularly execute reading processing, wherein the reading processing is as follows: reading image data of a first preset image number from the stored image data;

the plurality of second special CPUs perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model.

Optionally, the plurality of second special CPUs are divided into N groups, wherein the first group of CPUs includes a first preset number of second special CPUs, and the nth group of CPUs includes an nth preset number of second special CPUs;

The preset processing mode comprises the following steps: the first group of CPUs processes the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, the N group of CPUs obtains a (N-1) processing result, and the N preset image processing mode is adopted to process the obtained processing result to obtain an image sample corresponding to each image data.

Optionally, the step of processing, by the first group of CPUs, the image data read by the plurality of first special CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data, and the nth group of CPUs obtains the (N-1) th processing result and processes the obtained processing result by adopting the nth preset image processing manner to obtain an image sample corresponding to each image data includes:

the first group of CPUs processes the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, the first processing result is put into a first result queue, the N group of CPUs obtains the (N-1) processing result from the (N-1) result queue, the obtained processing result is processed by adopting the N preset image processing mode to obtain an image sample corresponding to each image data, and the image sample is put into a final result queue.

Optionally, the preset processing mode includes:

the plurality of second special CPUs conduct data processing on the image data read by the plurality of first special CPUs by adopting a preset image preset processing mode to obtain image number samples corresponding to each image data, wherein the preset image preset processing mode comprises the following steps of: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode.

Optionally, the step of acquiring and storing the image data acquired by each camera in real time includes:

acquiring image data acquired by each camera in real time;

and storing the image data, and adjusting the storage sequence of all the image data.

In a first aspect, an embodiment of the present invention provides a model training method applied to a processing device communicatively connected to a plurality of mechanical arms, where the processing device includes a plurality of first dedicated CPUs for reading image data, a plurality of second dedicated CPUs for processing image data, and a GPU, and each mechanical arm is provided with a camera, and the method includes:

acquiring and storing image data acquired by each camera in real time;

The plurality of second special CPUs perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model;

and the GPU acquires the image sample, and performs parameter training on the machine learning model by using the acquired image sample.

Optionally, the number of GPUs is a plurality;

the step of the GPU obtaining the image sample and performing parameter training on the machine learning model by using the obtained image sample comprises the following steps:

each GPU acquires image samples with preset sample numbers, trains the machine learning model by using the acquired image samples, and adjusts parameters of the machine learning model based on training results of other GPUs.

the first group of CPUs process the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, the first processing result is put into a first result queue, the N group of CPUs acquire the (N-1) processing result from the (N-1) result queue, the N preset image processing mode is adopted to process the acquired processing result to obtain an image sample corresponding to each image data, and the image sample is put into a final result queue;

the GPU acquires the image sample from the final result queue;

And carrying out parameter training on the machine learning model by using the acquired image samples.

Optionally, the preset processing mode includes:

acquiring image data acquired by each camera in real time;

In a third aspect, an embodiment of the present invention provides an image sample acquiring system, the system including a storage device and a processing device communicatively connected to a plurality of mechanical arms, the processing device including a plurality of first dedicated CPUs for reading image data and a plurality of second dedicated CPUs for image data processing, each mechanical arm having a camera mounted thereon, wherein:

the storage device is used for acquiring and storing the image data acquired by each camera in real time;

The plurality of first special CPUs are respectively used for circularly executing reading processing, wherein the reading processing is as follows: reading image data of a first preset image number from the stored image data;

the second special CPUs are used for carrying out data processing on the image data read by the first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model.

the first group of CPUs are used for processing the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data;

the N-th group CPU is used for obtaining the (N-1) -th processing result, and processing the obtained processing result by adopting an N-th preset image processing mode to obtain an image sample corresponding to each image data.

Optionally, the first group of CPUs is specifically configured to process, according to a first preset image processing manner, the image data read by the plurality of first special CPUs to obtain a first processing result corresponding to each image data, and put the first processing result into a first result queue;

The N group CPU is specifically configured to obtain the (N-1) processing result from the (N-1) result queue, process the obtained processing result by adopting an N preset image processing mode, obtain an image sample corresponding to each image data, and place the image sample into the final result queue.

Optionally, the plurality of second special CPUs are configured to perform data processing on the image data read by the plurality of first special CPUs by using a preset image preset processing manner, so as to obtain a number of image samples corresponding to each image data, where the preset image preset processing manner includes: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode.

Optionally, the storage device is specifically configured to acquire, in real time, image data acquired by each camera; and storing the image data, and adjusting the storage sequence of all the image data.

In a fourth aspect, an embodiment of the present invention provides a model training system, where the system includes a storage device and a processing device communicatively connected to a plurality of mechanical arms, and the processing device includes a plurality of first dedicated CPUs for reading image data, a plurality of second dedicated CPUs for processing image data, and a GPU, and each mechanical arm is mounted with a camera, where:

the second special CPUs are used for carrying out data processing on the image data read by the first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model;

the GPU is used for acquiring the image sample, and performing parameter training on the machine learning model by using the acquired image sample.

Optionally, the number of GPUs is a plurality;

the GPU is specifically used for each GPU to acquire image samples with preset sample number, the acquired image samples are utilized to train the machine learning model, and parameters of the machine learning model are adjusted based on training results of the rest GPUs.

The first group of CPUs is used for processing the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data

the N group of CPU is specifically used for acquiring an (N-1) processing result from an (N-1) result queue, processing the acquired processing result by adopting an N preset image processing mode to acquire an image sample corresponding to each image data, and placing the image sample into a final result queue;

the GPU is specifically configured to obtain the image sample from the final result queue by using the GPU; and carrying out parameter training on the machine learning model by using the acquired image samples.

Optionally, the plurality of second special CPUs are specifically configured to perform data processing on the image data read by the plurality of first special CPUs by using a preset image preset processing manner, so as to obtain a number of image samples corresponding to each image data, where the preset image preset processing manner includes: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode.

The embodiment of the invention also provides processing equipment, which comprises a processor, a communication interface, a memory, a communication bus, a plurality of first special CPUs and a plurality of second special CPUs, wherein the processor, the communication interface, the memory, the plurality of first special CPUs and the plurality of second special CPUs are in communication with each other through the communication bus, the processing equipment is in communication connection with a plurality of mechanical arms, and each mechanical arm is provided with a camera;

a memory for storing a computer program;

the processor is used for acquiring and storing the image data acquired by each camera in real time when executing the program stored in the memory;

And a plurality of first special CPUs respectively used for circularly executing reading processing, wherein the reading processing is as follows: reading image data of a first preset image number from the stored image data;

and the second special CPUs are used for carrying out data processing on the image data read by the first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of the machine learning model.

The embodiment of the invention also provides another processing device which comprises a processor, a communication interface, a memory, a communication bus, a plurality of first special CPUs, a plurality of second special CPUs and a GPU, wherein the processor, the communication interface, the memory, the plurality of first special CPUs and the plurality of second special CPUs are in communication with each other through the communication bus, the processing device is in communication connection with a plurality of mechanical arms, and each mechanical arm is provided with a camera;

a memory for storing a computer program;

the processor is used for acquiring and storing the image data acquired by each camera when executing the program stored in the memory;

and the GPU is used for acquiring the image sample and performing parameter training on the machine learning model by utilizing the acquired image sample.

The embodiment of the invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program realizes the steps of the image sample acquisition method when being executed by a processor.

The embodiment of the invention also provides another computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program realizes the steps of the model training method when being executed by a processor.

In the scheme provided by the embodiment of the invention, the processing equipment acquires and stores the image data acquired by each camera in real time, the plurality of first special CPUs respectively and circularly execute the reading processing, and the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to acquire the image sample corresponding to each image data. Therefore, the processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquires the image sample for training the machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of an image sample acquisition method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a preset processing mode in the embodiment shown in FIG. 1;

FIG. 3 is a flowchart of a model training method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an image sample acquiring system according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a model training system according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a first processing apparatus according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of a second processing apparatus according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In order to shorten the training time of a machine learning model, the embodiment of the invention provides an image sample acquisition method and system, a model training method and system, a processing device and a computer readable storage medium.

An image sample acquiring method provided by the embodiment of the invention is first described below.

An image sample acquiring method provided by an embodiment of the present invention may be applied to a processing apparatus communicatively connected to a plurality of robot arms, and the processing apparatus may include a plurality of first dedicated CPUs (Central Processing Unit, central processing units) for reading image data and a plurality of second dedicated CPUs for image data processing. Each mechanical arm is provided with a camera so as to collect image data.

As shown in fig. 1, an image sample acquiring method applied to a processing apparatus communicatively connected to a plurality of robot arms, the processing apparatus including a plurality of first dedicated CPUs for reading image data and a plurality of second dedicated CPUs for image data processing, each robot arm having a camera mounted thereon, the method comprising:

s101, acquiring and storing image data acquired by each camera in real time;

s102, the plurality of first dedicated CPUs respectively and circularly execute a reading process, wherein the reading process is as follows: reading image data of a first preset image number from the stored image data;

S103, the plurality of second special CPUs perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model.

In the scheme provided by the embodiment of the invention, the processing equipment acquires and stores the image data acquired by each camera in real time, the plurality of first special CPUs respectively and circularly execute the reading processing, and the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to acquire the image sample corresponding to each image data. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

In the above step S101, the processing device may acquire the image data acquired by each camera in real time, and store the image data, for example, in a hard disk. It will be appreciated that each camera mounted to the robotic arm is capturing image data, and the processing device stores all of the image data captured by the camera.

In the above step S102, the plurality of first dedicated CPUs may respectively perform loop-on-loop reading processing of: image data of a first preset number of images is read from the stored image data. That is, each first dedicated CPU constantly reads image data of a first preset number of images from the stored image data.

The plurality of second special CPUs can perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain image samples corresponding to each image data. The preset data processing manner is preset according to the training requirement of the machine learning model, and may include, for example, image processing manners such as conversion processing of image data into a matrix, pixel value processing, and feature region calibration processing, which are not specifically limited herein.

The number of the first preset images may be determined according to the processing capability of the first special CPU and the memory occupied by the image data, for example, 10, 15, 20, etc., which is not limited herein. In one embodiment, the first special CPU reads the image data of the first preset image number from the stored image data to maximally satisfy the processing capacity of each second special CPU, so that the resources of each second special CPU are maximally utilized.

For example, each second dedicated CPU may process 5 image data simultaneously, and 10 second dedicated CPUs are total, and then the first dedicated CPU reads at least 50 image data of the first preset number of images from the stored image data at a time, so that the processing capacity of each second dedicated CPU can be fully utilized, and the resource utilization rate can be maximized.

It should be noted that, the steps S101-S103 are not necessarily executed according to the sequence of the steps S101-S103, but the steps S101-S103 are executed simultaneously to form a streaming processing mode, so long as each camera collects image data, the processing device can acquire and store the image data collected by each camera in real time, meanwhile, both the steps S102 and S103 are in progress, a dynamic balance state is achieved, the image data collected by each camera can be processed in real time, and image samples for training a machine learning model are continuously acquired, so that the machine learning model can be continuously trained, and the model training time is greatly shortened.

As an implementation manner of the embodiment of the present invention, in order to further improve the resource utilization ratio and improve the data processing speed, the plurality of second dedicated CPUs may be divided into N groups, where the first group of CPUs includes a first preset number of second dedicated CPUs, and the nth group of CPUs includes an nth preset number of second dedicated CPUs.

Correspondingly, the preset processing mode may include: the first group of CPUs processes the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, the N group of CPUs obtains a (N-1) processing result, and the N preset image processing mode is adopted to process the obtained processing result to obtain an image sample corresponding to each image data.

Since the processing modes required for the image data may be a plurality of different processing modes, in order to meet the requirements of rapid and accurate processing, the plurality of second dedicated CPUs may be divided into N groups according to the processing modes required for the image data, and in general, the number of groups of the second dedicated CPUs is the same as the number of kinds of the required processing modes, that is, each group of CPUs may perform one process on the image data. For example, the image data may be processed in 5 ways, and the plurality of second dedicated CPUs may be divided into 5 groups, each group of CPUs performing one process on the image data.

The number of the second dedicated CPUs included in each group of CPUs, that is, the first preset number and the second preset number … nth preset number, may be determined according to the specific types of the first preset image processing mode to the nth preset image processing mode, the required memory, and other factors. If the memory required by a certain preset image processing mode is larger, the number of corresponding second special CPUs can be larger, and if the memory required by a certain preset image processing mode is smaller, the number of corresponding second special CPUs can be smaller.

For example, the processing modes required to perform the image data include 4 types, that is, a first preset image processing mode to a fourth preset image processing mode, the memories required by the 4 types of processing modes are sequentially increased, in order to ensure that the resource utilization rate of all the second special CPUs is the highest when performing the data processing, the number ratio of the second special CPUs required by the four types of processing modes is 1:2:3:4, then, if the number of second dedicated CPUs is 80, the first group of CPUs may include 8 second dedicated CPUs, the second group of CPUs may include 16 second dedicated CPUs, the third group of CPUs may include 24 second dedicated CPUs, and the fourth group of CPUs may include 36 second dedicated CPUs.

It should be noted that, the processing performed by the N sets of CPUs on the processing result obtained by the previous set of CPU processing is also performed simultaneously, that is, the first set of CPUs continuously processes the image data read by the plurality of first dedicated CPUs to obtain the first processing result, and at the same time, the second to nth sets of CPUs also obtain the processing result from the processing result obtained by the previous set of CPU processing, and process the obtained processing result by adopting a preset image processing mode.

Therefore, in this embodiment, the plurality of second special CPUs are divided into N groups, so that stable processing of image data can be ensured, each second special CPU can process image data at the highest processing speed, the resource utilization rate is improved, and the time for subsequent model training can be shortened.

As an implementation manner of the embodiment of the present invention, the step of processing, by the first set of CPUs, the image data read by the plurality of first special CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data, and the nth set of CPUs obtain a (N-1) th processing result and process the obtained processing result by adopting the nth preset image processing manner to obtain an image sample corresponding to each image data may include:

Because the multiple second special CPUs are divided into N groups and perform data processing simultaneously, in order to make the processing flow more smooth and stable, and avoid the occurrence of confusion and processing errors, as shown in fig. 2, the first group of CPUs processes the image data read by the multiple first special CPUs according to a first preset image processing mode, after obtaining a first processing result corresponding to each image data, the first processing result may be put into a first result queue, further, the second group of CPUs may obtain the first processing result from the first result queue, process the obtained first processing result according to a second preset image processing mode, and so on, the nth group of CPUs may obtain the (N-1) processing result from the (N-1) result queue, process the obtained processing result by adopting the nth preset image processing mode, obtain image samples corresponding to each image data, and put the image samples into a final result queue. The training of the machine learning model can then be performed using the image samples in the final result queue.

Therefore, in this embodiment, each group of CPUs may put the processing result into the corresponding processing result queue, and the next group of CPUs may acquire the processing result from the processing result queue for processing, so that the processing flow is smoother and more stable, and the processing error caused by confusion is avoided.

As an implementation manner of the embodiment of the present invention, the foregoing preset processing manner may include:

In one embodiment, the plurality of second special CPUs may perform data processing on the image data read by the plurality of first special CPUs by adopting a preset image preset processing mode, so as to obtain a sample of the number of images corresponding to each image data.

The preset image preset processing mode may include: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode. That is, each of the second special-purpose CPUs may perform N kinds of preset processing on the image data that the first special-purpose CPU has read. For example, the number of preset image processing modes is 5, that is, the first preset image processing mode to the fifth preset image processing mode, so that after each second special CPU obtains the image data read by the first special CPU, the image data can be processed according to the first preset image processing mode, then the obtained processing result is processed according to the second preset image processing mode, and so on, and finally the image sample is obtained by processing according to the fifth preset image processing mode.

After obtaining the image samples, the second special-purpose CPU may place the image samples into a final result queue for use in subsequent machine learning model training.

As an implementation manner of the embodiment of the present invention, the step of acquiring and storing the image data acquired by each camera in real time may include:

acquiring image data acquired by each camera in real time; and storing the image data, and adjusting the storage sequence of all the image data.

The processing device may acquire image data acquired by each camera in real time, then store the acquired image data, and adjust the storage order of all the image data. Specifically, since the number of the mechanical arms is generally larger, the number of image data collected by the camera is also larger, when the first special CPU reads the image data, the processing device also stores the image data which is not read, and meanwhile, the processing device also continuously acquires the image data, and further, after the processing device stores the acquired image data, the processing device can adjust the storage sequence of the acquired image data and the stored image data which is not read.

Thus, when the first special CPU reads image data, the image data stored by the processing equipment just can be read, and the image data stored by the processing equipment before the current moment can also be read, so that if factors such as the acquisition environment of a camera arranged on the mechanical arm change, the image sample obtained by the second special CPU processing can comprise the image sample corresponding to the data acquired after the factors such as the acquisition environment change, and further, the mechanical learning model obtained by training the image samples can meet the requirements of the current mechanical arm use environment and conditions, play the role of mixing new and old image samples, and improve the accuracy of the mechanical learning model obtained by training.

In order to shorten the training time of the machine learning model, the embodiment of the invention also provides a model training method.

The following describes a model training method provided by the embodiment of the invention.

The model training method provided by the embodiment of the invention can be applied to processing equipment in communication connection with a plurality of mechanical arms, and the processing equipment can comprise a plurality of first special CPUs for reading image data, a plurality of second special CPUs for processing the image data and a GPU (Graphics Processing Unit, graphic processor). Each mechanical arm is provided with a camera so as to collect image data.

As shown in fig. 3, a model training method applied to a processing device communicatively connected to a plurality of mechanical arms, the processing device including a plurality of first dedicated CPUs for reading image data, a plurality of second dedicated CPUs for image data processing, and a GPU, each mechanical arm having a camera mounted thereon, the method comprising:

s301, acquiring and storing image data acquired by each camera in real time;

s302, the plurality of first dedicated CPUs respectively and circularly execute a reading process, where the reading process is: reading image data of a first preset image number from the stored image data;

S303, the plurality of second special CPUs perform data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to obtain an image sample corresponding to each image data, wherein the preset data processing mode is preset according to the training requirement of a machine learning model;

s304, the GPU acquires the image sample, and performs parameter training on the machine learning model by using the acquired image sample.

In the scheme provided by the embodiment of the invention, the processing equipment acquires and stores the image data acquired by each camera in real time, the plurality of first special CPUs respectively and circularly execute the reading processing, the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to acquire the image sample corresponding to each image data, the GPU acquires the image sample, and the acquired image sample is used for performing parameter training on the machine learning model. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

In the above step S301, the processing device may acquire the image data acquired by each camera in real time, and store the image data, for example, in a hard disk. It will be appreciated that each camera mounted to the robotic arm is capturing image data, and the processing device stores all of the image data captured by the camera.

In the above step S302, the plurality of first dedicated CPUs may respectively perform the reading processing in a loop, wherein the reading processing is: image data of a first preset number of images is read from the stored image data. That is, each first dedicated CPU constantly reads image data of a first preset number of images from the stored image data.

In the step S304, the GPU may acquire an image sample obtained by processing by the second dedicated CPU, and perform parameter training on the machine learning model constructed in advance by using the acquired image sample, so as to obtain the required machine learning model.

It should be noted that, the steps S301 to S304 are not necessarily executed according to the sequence of the steps S301 to S303, but the steps S301 to S304 are executed simultaneously to form a streaming processing manner, so long as each camera collects image data, the processing device can acquire and store the image data collected by each camera in real time, and meanwhile, the steps S302 to S304 are all performed to achieve a dynamic balance state, so that the image data collected by each camera can be processed in real time, image samples for training a machine learning model are continuously acquired, the machine learning model is continuously trained, and the model training time is greatly shortened.

As an implementation manner of the embodiment of the present invention, the number of GPUs may be plural;

the step of obtaining the image sample by the GPU and performing parameter training on the machine learning model by using the obtained image sample may include:

The processing device may include a plurality of GPUs, in which case each GPU may acquire a preset number of image samples and train the machine learning model using the acquired image samples. The preset number of samples may be determined according to the processing capability of the GPU and the data amount of each image sample, for example, may be 5, 10, 15, etc., which is not limited herein. Of course, the number of image samples acquired by each GPU is optimal to fully utilize the resources of the GPU, so that the resource utilization rate can be further improved, and the model training time is shortened.

Because the plurality of GPUs train the machine learning model at the same time, when each GPU trains the machine learning model, parameters of the machine learning model can be adjusted based on training results of the rest GPUs. The GPUs can communicate through a communication bus, a communication interface and the like to acquire training results of the other GPUs, and parameters of a machine learning model are adjusted based on the training results.

For example, the processing device includes 4 GPUs, and then the 4 GPUs simultaneously acquire image samples, and simultaneously use the acquired image samples to train the machine learning model, and communicate with each other to obtain training results of the remaining GPUs, so as to adjust parameters of the machine learning model.

Therefore, in this embodiment, the processing device includes a plurality of GPUs, and the plurality of GPUs train the machine learning model at the same time, so as to further improve the model training efficiency and shorten the model training time.

Correspondingly, the preset processing mode comprises the following steps: the first group of CPUs processes the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, the N group of CPUs obtains a (N-1) processing result, and the N preset image processing mode is adopted to process the obtained processing result to obtain an image sample corresponding to each image data.

Therefore, in this embodiment, the plurality of second special CPUs are divided into N groups, so that stable processing of image data can be ensured, each second special CPU can process image data at the highest processing speed, the resource utilization rate is improved, and the time of model training is shortened.

Because the plurality of second special CPUs are divided into N groups and simultaneously perform data processing, in order to enable the processing flow to be smooth and stable, processing errors caused by confusion are avoided, the first group of CPUs process the image data read by the plurality of first special CPUs according to a first preset image processing mode, after a first processing result corresponding to each image data is obtained, the first processing result can be put into a first result queue, further, the second group of CPUs can obtain the first processing result from the first result queue, process the obtained first processing result according to a second preset image processing mode, and so on, the Nth group of CPUs can obtain the (N-1) processing result from the (N-1) result queue, process the obtained processing result according to the Nth preset image processing mode, obtain image samples corresponding to each image data, and put the image samples into a final result queue.

Correspondingly, the step of obtaining the image sample by the GPU and performing parameter training on the machine learning model by using the obtained image sample may include:

the GPU acquires the image sample from the final result queue; and carrying out parameter training on the machine learning model by using the acquired image samples.

The N group CPU processes the obtained processing results by adopting an N preset image processing mode, and each obtained image sample is put into a final result queue, so that the GPU can obtain the image samples from the final result queue, and further, the obtained image samples are utilized to carry out parameter training on the machine learning model.

Therefore, in this embodiment, each group of CPUs may put the processing result into the corresponding processing result queue, and the next group of CPUs may acquire the processing result from the processing result queue for processing, and the GPU may acquire the image sample from the final result queue for parameter training of the machine learning model, so that the processing flow is smoother and more stable, and processing errors caused by confusion are avoided.

As an implementation manner of the embodiment of the present invention, the foregoing preset processing manner includes:

After obtaining the image samples, the second special CPU may put the image samples into a final result queue, so that, when training the machine learning model, the GPU may obtain the image samples from the final result queue, and perform parameter adjustment on the machine learning model.

Thus, when the first special CPU reads image data, the image data stored by the processing equipment just can be read, and the image data stored by the processing equipment between the current moments can also be read, so that if factors such as the acquisition environment of a camera arranged on the mechanical arm change, the image samples obtained by the second special CPU processing can comprise image samples corresponding to the acquired data after the factors such as the acquisition environment change, and further, the mechanical learning model obtained by training the image samples can meet the requirements of the current use environment and conditions of the mechanical arm, play the role of mixing new and old image samples, and improve the accuracy of the mechanical learning model obtained by training.

Corresponding to the image sample acquisition method, the embodiment of the invention also provides an image sample acquisition system.

An image sample acquisition system according to an embodiment of the present invention is described below.

As shown in fig. 4, an image sample acquiring system, the system comprises a storage device 410 and a processing device 420 which are communicatively connected with a plurality of mechanical arms, wherein the processing device 420 comprises a plurality of first special-purpose CPUs 4201 for reading image data and a plurality of second special-purpose CPUs 4202 for processing image data, each mechanical arm is provided with a camera, and wherein:

the storage device 410 is configured to acquire and store image data acquired by each camera in real time;

the plurality of first dedicated CPUs 4201 are respectively configured to circularly execute a reading process, wherein the reading process is: reading image data of a first preset image number from the stored image data;

the second special CPUs 4202 are configured to perform data processing on the image data read by the first special CPUs according to a preset data processing manner, so as to obtain an image sample corresponding to each image data, where the preset data processing manner is preset according to the training requirement of the machine learning model.

In the scheme provided by the embodiment of the invention, the storage device acquires and stores the image data acquired by each camera in real time, the plurality of first special CPUs respectively and circularly execute the reading processing, and the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to the preset data processing mode to acquire the image sample corresponding to each image data. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

As one implementation manner of the embodiment of the present invention, the plurality of second dedicated CPUs may be divided into N groups, where the first group of CPUs includes a first preset number of second dedicated CPUs, and the nth group of CPUs includes an nth preset number of second dedicated CPUs;

the nth group of CPUs is configured to obtain the (N-1) th processing result, and process the obtained processing result by using an nth preset image processing manner, so as to obtain an image sample corresponding to each image data.

As an implementation manner of the embodiment of the present invention, the first set of CPUs is specifically configured to process, according to a first preset image processing manner, image data read by the plurality of first special CPUs to obtain a first processing result corresponding to each image data, and put the first processing result into a first result queue;

As an implementation manner of the embodiment of the present invention, the plurality of second special CPUs are configured to perform data processing on image data read by the plurality of first special CPUs by using a preset image preset processing manner, so as to obtain a number of images sample corresponding to each image data, where the preset image preset processing manner includes: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode.

As an implementation manner of the embodiment of the present invention, the storage device is specifically configured to acquire, in real time, image data acquired by each camera; and storing the image data, and adjusting the storage sequence of all the image data.

The storage device can acquire the image data acquired by each camera in real time, then store the acquired image data, and adjust the storage sequence of all the image data. Specifically, since the number of the mechanical arms is generally large, the number of image data collected by the camera is also large, when the first special CPU reads the image data, the storage device also stores the image data which is not read, and meanwhile, the storage device also continuously acquires the image data, and further, after the storage device stores the acquired image data, the storage device can adjust the storage sequence of the acquired image data and the stored image data which is not read.

Thus, when the first special CPU reads image data, the image stored by the storage device just can be read, and the image data stored by the storage device before the current moment can be read, so that if factors such as the acquisition environment of a camera installed on the mechanical arm change, the image sample processed by the second special CPU can comprise the image sample corresponding to the data acquired after the factors such as the acquisition environment change, and further, a mechanical learning model trained by adopting the image samples can meet the requirements of the current mechanical arm use environment and conditions, play a role of mixing new and old image samples, and improve the accuracy of the mechanical learning model trained.

Corresponding to the model training method, the embodiment of the invention also provides a model training system.

The following describes a model training system provided in an embodiment of the present invention.

As shown in fig. 5, a model training system, the system includes a storage device 510 and a processing device 520 communicatively connected to a plurality of mechanical arms, the processing device 520 includes a plurality of first dedicated CPUs 5201 for reading image data, a plurality of second dedicated CPUs 5202 for processing image data, and a GPU5203, each mechanical arm has a camera mounted thereon, wherein:

the storage device 510 is configured to acquire and store image data acquired by each camera in real time;

the plurality of first dedicated CPUs 5201 are respectively configured to circularly execute a reading process, where the reading process is: reading image data of a first preset image number from the stored image data;

the second special CPUs 5202 are configured to perform data processing on the image data read by the first special CPUs according to a preset data processing manner, so as to obtain an image sample corresponding to each image data, where the preset data processing manner is preset according to the training requirement of the machine learning model;

The GPU5203 is configured to acquire the image sample, and perform parameter training on the machine learning model using the acquired image sample.

In the scheme provided by the embodiment of the invention, the storage device acquires and stores the image data acquired by each camera in real time, the plurality of first special CPUs respectively and circularly execute the reading processing, the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to a preset data processing mode to acquire the image sample corresponding to each image data, the GPU acquires the image sample, and the acquired image sample is used for performing parameter training on the machine learning model. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

As one implementation manner of the embodiment of the present invention, the plurality of second dedicated CPUs are divided into N groups, where the first group of CPUs includes a first preset number of second dedicated CPUs, and the nth group of CPUs includes an nth preset number of second dedicated CPUs;

the first set of CPUs is configured to process the image data read by the plurality of first dedicated CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data

As an implementation manner of the embodiment of the present invention, the plurality of second special CPUs are specifically configured to perform data processing on image data read by the plurality of first special CPUs by using a preset image preset processing manner, so as to obtain a number of images sample corresponding to each image data, where the preset image preset processing manner includes: the first preset image processing mode, the second preset image processing mode and the Nth preset image processing mode.

The embodiment of the invention also provides a processing device, as shown in fig. 6, which comprises a processor 601, a communication interface 602, a memory 603, a communication bus 604, a plurality of first special CPUs 605 and a plurality of second special CPUs 606, wherein the processor 601, the communication interface 602, the memory 603, the plurality of first special CPUs 605 and the plurality of second special CPUs 606 complete communication with each other through the communication bus 604, the processing device is in communication connection with a plurality of mechanical arms, and each mechanical arm is provided with a camera;

A memory 603 for storing a computer program;

the processor 601 is configured to acquire image data acquired by each camera in real time and store the image data when executing a program stored in the memory 603;

a plurality of first dedicated CPUs 605 for cyclically executing reading processing, respectively, wherein the reading processing is: reading image data of a first preset image number from the stored image data;

the second special CPUs 606 are configured to perform data processing on the image data read by the first special CPUs according to a preset data processing manner, so as to obtain an image sample corresponding to each image data, where the preset data processing manner is preset according to the training requirement of the machine learning model.

The communication bus mentioned by the processing device may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the processing device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Wherein the plurality of second dedicated CPUs may be divided into N groups, wherein the first group of CPUs includes a first preset number of second dedicated CPUs, and the nth group of CPUs includes an nth preset number of second dedicated CPUs;

The step of processing the image data read by the first group of CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data, obtaining an (N-1) th processing result by the nth group of CPUs, and processing the obtained processing result by adopting the nth preset image processing manner to obtain an image sample corresponding to each image data may include:

The preset processing mode may include:

The step of acquiring and storing the image data acquired by each camera in real time comprises the following steps:

acquiring image data acquired by each camera in real time;

The embodiment of the invention also provides another processing device, as shown in fig. 7, which comprises a processor 701, a communication interface 702, a memory 703, a communication bus 704, a plurality of first special CPUs 705, a plurality of second special CPUs 706 and a GPU707, wherein the processor 701, the communication interface 702, the memory 703, the plurality of first special CPUs 705 and the plurality of second special CPUs 706 are in communication with each other through the communication bus 704, the processing device is in communication connection with a plurality of mechanical arms, and each mechanical arm is provided with a camera;

A memory 703 for storing a computer program;

a processor 701, configured to acquire and store image data acquired by each camera when executing a program stored in a memory 703;

a plurality of first dedicated CPUs 705 for cyclically executing reading processing, respectively, wherein the reading processing is: reading image data of a first preset image number from the stored image data;

the second special CPUs 706 are configured to perform data processing on the image data read by the first special CPUs according to a preset data processing manner, so as to obtain an image sample corresponding to each image data, where the preset data processing manner is preset according to the training requirement of the machine learning model;

and a GPU707 for acquiring the image sample, and performing parameter training on the machine learning model by using the acquired image sample.

The number of GPUs may be plural;

the GPU acquires the image sample from the final result queue;

The preset processing mode may include:

The step of acquiring and storing the image data acquired by each camera in real time may include:

acquiring image data acquired by each camera in real time;

The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the following steps when being executed by a processor:

acquiring and storing image data acquired by each camera in real time;

In the scheme provided by the embodiment of the invention, when the computer program is executed by the processor, the image data acquired by each camera is acquired in real time and stored, the plurality of first special CPUs respectively and circularly execute the reading processing, and the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to the preset data processing mode, so that the image sample corresponding to each image data is obtained. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

The preset processing mode may include:

acquiring image data acquired by each camera in real time;

The embodiment of the invention also provides another computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the following steps when being executed by a processor:

acquiring and storing image data acquired by each camera in real time;

Therefore, in the scheme provided by the embodiment of the invention, when the computer program is executed by the processor, the image data acquired by each camera are acquired in real time and stored, the plurality of first special CPUs respectively and circularly execute the reading processing, the plurality of second special CPUs perform the data processing on the image data read by the plurality of first special CPUs according to the preset data processing mode, the image sample corresponding to each image data is obtained, the GPU acquires the image sample, and the acquired image sample is used for performing parameter training on the machine learning model. The processing equipment adopts a streaming processing mode, can process the image data acquired by each camera in real time, acquire an image sample for training a machine learning model, does not need to retrain the machine learning model when acquiring new image data, and greatly shortens the model training time.

The number of GPUs may be plural;

the GPU acquires the image sample from the final result queue;

The preset processing mode may include:

acquiring image data acquired by each camera in real time;

It should be noted that, with respect to the system, processing device and computer readable storage medium embodiments described above, since they are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the description of the method embodiments for relevant points.

It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An image sample acquisition method, characterized by being applied to a processing device communicatively connected to a plurality of cameras, the processing device including a plurality of first dedicated CPUs for reading image data and a plurality of second dedicated CPUs for image data processing, the method comprising:

acquiring and storing image data acquired by each camera in real time;

the plurality of second special CPUs are divided into N groups, wherein the first group of CPUs comprises a first preset number of second special CPUs, and the N group of CPUs comprises an N preset number of second special CPUs;

2. The method of claim 1, wherein the first group of CPUs processes the image data read by the plurality of first dedicated CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data, the nth group of CPUs obtains a (N-1) th processing result, and processes the obtained processing result by adopting the nth preset image processing manner to obtain an image sample corresponding to each image data, and the step of obtaining the image sample corresponding to each image data comprises:

3. The method according to any one of claims 1-2, wherein the step of acquiring and storing in real time the image data acquired by each camera comprises:

acquiring image data acquired by each camera in real time;

4. A model training method applied to a processing device communicatively connected to a plurality of cameras, the processing device including a plurality of first dedicated CPUs for reading image data, a plurality of second dedicated CPUs for image data processing, and a GPU, the method comprising:

acquiring and storing image data acquired by each camera in real time;

the GPU acquires the image sample, and performs parameter training on the machine learning model by using the acquired image sample;

5. The method of claim 4, wherein the number of GPUs is a plurality;

6. The method of claim 4, wherein the first group of CPUs processes the image data read by the plurality of first dedicated CPUs according to a first preset image processing manner to obtain a first processing result corresponding to each image data, the nth group of CPUs obtains a (N-1) th processing result, and processes the obtained processing result by adopting the nth preset image processing manner to obtain an image sample corresponding to each image data, and the step of obtaining the image sample corresponding to each image data comprises:

the GPU acquires the image sample from the final result queue;

7. The method according to any one of claims 4 to 6, wherein the step of acquiring and storing in real time the image data acquired by each camera comprises:

acquiring image data acquired by each camera in real time;

8. An image sample acquisition system comprising a storage device in communication with a plurality of cameras and a processing device comprising a plurality of first dedicated CPUs for reading image data and a plurality of second dedicated CPUs for image data processing, wherein:

9. The system of claim 8, wherein,

the first group of CPUs are specifically used for processing the image data read by the plurality of first special CPUs according to a first preset image processing mode to obtain a first processing result corresponding to each image data, and the first processing results are put into a first result queue;

10. The system of any one of claim 8 to 9, wherein,

the storage device is specifically used for acquiring the image data acquired by each camera in real time; and storing the image data, and adjusting the storage sequence of all the image data.

11. A model training system comprising a storage device in communication with a plurality of cameras and a processing device comprising a plurality of first dedicated CPUs for reading image data, a plurality of second dedicated CPUs for image data processing and a GPU, wherein:

The GPU is used for acquiring the image sample and performing parameter training on the machine learning model by utilizing the acquired image sample;

12. The system of claim 11, wherein the number of GPUs is a plurality;

13. The system of claim 11, wherein the system comprises a plurality of sensors,

14. The system of any one of claim 11 to 13,

15. The processing device is characterized by comprising a processor, a communication interface, a memory, a communication bus, a plurality of first special CPUs and a plurality of second special CPUs, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus;

A memory for storing a computer program;

16. The processing device is characterized by comprising a processor, a communication interface, a memory, a communication bus, a plurality of first special CPUs, a plurality of second special CPUs and a GPU, wherein the processor, the communication interface, the memory, the plurality of first special CPUs and the plurality of second special CPUs are in communication connection with a plurality of cameras through the communication bus;

a memory for storing a computer program;

the GPU is used for acquiring the image sample, and performing parameter training on the machine learning model by utilizing the acquired image sample;

17. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-3.

18. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 4-7.