CN112416890A - Insect robot mass image data parallel processing platform - Google Patents

Insect robot mass image data parallel processing platform Download PDF

Info

Publication number
CN112416890A
CN112416890A CN202011331446.4A CN202011331446A CN112416890A CN 112416890 A CN112416890 A CN 112416890A CN 202011331446 A CN202011331446 A CN 202011331446A CN 112416890 A CN112416890 A CN 112416890A
Authority
CN
China
Prior art keywords
data
image
parallel
insect
image data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011331446.4A
Other languages
Chinese (zh)
Other versions
CN112416890B (en
Inventor
洪慧
金华华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202011331446.4A priority Critical patent/CN112416890B/en
Publication of CN112416890A publication Critical patent/CN112416890A/en
Application granted granted Critical
Publication of CN112416890B publication Critical patent/CN112416890B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/182Distributed file systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/11File system administration, e.g. details of archiving or snapshots
    • G06F16/116Details of conversion of file system types or formats
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/13File access structures, e.g. distributed indices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0674Disk device
    • G06F3/0676Magnetic disk device
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38Concurrent instruction execution, e.g. pipeline or look ahead
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • G06V20/49Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/04Synchronising
    • H04N5/06Generation of synchronising signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a parallel processing platform for mass image data of an insect robot. The platform comprises a stimulating backpack, an insect robot behavior recording and acquiring platform and an image data parallel preprocessing cluster platform, and is suitable for animal electrical stimulation, stimulated behavior recording and storage and video image distributed parallel preprocessing analysis. Constructing a cluster model based on Hadoop one-master-five-slave nodes, and customizing a MapReduce processing algorithm adaptive to the video field; the method is characterized in that parallel preprocessing analysis is carried out on massive offline behavior record data, and on the aspect of MapReduce framework processing optimization innovative design, a set of novel image parallel processing model is designed, wherein the novel image parallel processing model comprises image data type input and output design and a universal preprocessing algorithm interface. The invention can complete the accurate remote control and remote measurement of the insect robot, the loop control and the data acquisition of the behavior, and has the capability of efficiently preprocessing mass data.

Description

Insect robot mass image data parallel processing platform
Technical Field
The invention belongs to the technical field of brain-computer interfaces, and particularly relates to a parallel processing platform for mass image data of an insect robot.
Background
The animal robot is a micro-hybrid intelligent robot, and combines the movement capability of animals and the control capability of a micro-electro-mechanical system. The robots can solve the complex problem that some biological or artificial intelligence systems in the general environment cannot be processed independently, and have high flexibility, accuracy and efficiency, so that the animal robots have wide application prospects in the fields of public safety, clinical application, engineering operation and the like.
Compared with an animal robot, the insect robot is smaller in size and high in flying speed, three-dimensional feedback state data of insects need to be recorded, a high-performance server is constructed aiming at flying images of the insect robot, data storage and parallel processing analysis are carried out, a traditional data processing system cannot efficiently and quickly process the data, and a universal parallel data processing platform is lacked.
At present, research in the academic world aiming at the field of insect robots is still in a starting stage, experimental scenes or platforms for insect robot experiments are not complete, and meanwhile, the existing experimental platforms can not meet the requirements of image processing analysis and three-dimensional trajectory reconstruction aiming at the quality of mass images which are not processed in the high-speed insect flying process, and can not perform efficient processing analysis on mass large-data image data and optimize a data set. Therefore, how to design a parallel data processing platform which performs mass data processing on image data when the insect robot flies rapidly and comprises computing and storing functions becomes a problem worthy of research.
Disclosure of Invention
In order to overcome the defects of the background technology, the invention designs a multi-view high-flux acquisition and distributed processing platform for an insect robot. The experimental platform provides an experimental scene for the insect neurogenic experiment, and records mass behavior data generated in the high-speed flying process of the insect; the method solves the problems of multi-camera synchronous recording, high-throughput data recording, displaying, acquiring and storing, and can record massive behavior data after the nerve stimulation control of the insect robot in real time.
The platform carries out high-speed parallel processing analysis under the cluster based on the acquired mass behavior data, provides an uploading interface of mass files and a file preprocessing algorithm interface, can carry various types of files and preprocessing algorithms, utilizes the advantages of MapReduce parallel computation and multiple groups of high-efficiency cluster performance, and can effectively improve the processing efficiency of mass insect robot image files.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a. and designing and building a multi-view camera behavior acquisition system facing the insect robot and a mass data distributed parallel processing experiment platform. The platform hardware part comprises an insect load control backpack, six high-frame high-definition cameras, a radio frequency wireless base station, a synchronous controller, a video acquisition card and a distributed Hadoop cluster server, and the platform software part comprises a PC upper computer monitoring interface, a wireless control communication program, a variable frame synchronous control program, a MapReduce parallel image processing interface and the like which jointly form an insect robot experiment platform system;
b. the part of the acquisition system of the a platform is 2*4*2mm3The insect flight experiment space architecture is six Haikang MV-CH050-10CM industrial area-array cameras, the resolution is 2432 multiplied by 2048, and the maximum frame rate is 140 fps. The method solves the problem of synchronous shooting of a plurality of cameras by adopting external triggering hardware triggered by TTL signals, meanwhile, three DALSA camera acquisition cards are used for caching high-throughput video data acquired by the cameras, and the transmission rate is improved and the fault-tolerant capability is provided through data verification by using a high-performance server disk array RAID.
c. And (b) constructing a parallel processing cluster platform on the basis of the acquisition systems a and b, constructing a cluster model by utilizing the acquired mass data based on Hadoop one-master-five slave nodes, uploading the acquired mass unprocessed insect behavior image data to a distributed storage system (HDFS) in an off-line manner, wherein FFmpeg is adopted to cut a video into images, then the images are converted into streaming data which can be identified by javacv, and pre-preparation is carried out on the parallel processing module d.
d. And d adopts a MapReduce parallel computing framework, and the program runs on six nodes in a distributed mode. As Hadoop only defines basic data types facing text data, and the default is that the image data types cannot be supported, the platform designs an image file data type interface which is suitable for being matched under MapReduce, can automatically convert the image file data type interface into a data stream file required by a corresponding program according to the file format type, and simultaneously designs an interface of an image processing algorithm, and can carry different preprocessing algorithms to carry out graying, median filtering, image enhancement, edge detection, clustering algorithm and the like on the image.
The invention has the beneficial effects that: according to the invention, a MapReduce parallel processing technology is introduced in the research process of the image sequence processing efficiency, data acquired by a recording platform is taken as a data source under the framework of a high-throughput data recording platform and a Hadoop parallel computing cluster, the performance and advantages expressed on the mass image processing are analyzed, an improved optimization strategy is provided, and the system structure of an efficient parallel high-speed motion mass image processing system is constructed.
The invention realizes the conversion from image data to data types which can be directly processed by Hadoop, and provides an interface supporting a Mapper end to carry different image processing algorithms to realize the parallel image processing function. Meanwhile, aiming at a large amount of small files in a large data volume, in the cluster framework processing optimization design, a large amount of small files are combined to establish indexes so as to be combined into a large file, and the number of fragments is reduced, so that the cluster processing efficiency is improved.
Drawings
FIG. 1 is a general design diagram of an insect data acquisition and recording system;
FIG. 2 is a Hadoop computer cluster execution process;
FIG. 3 is a MapReduce parallel image processing execution process;
fig. 4 is a single Map task processing procedure.
Detailed Description
In order to make the technical features, objects and effects of the present invention more clearly understood, the present invention will be further described with reference to the accompanying drawings and examples.
The invention is further described below with reference to fig. 1. The insect robot-oriented control neurogenic experimental system builds a distributed parallel processing experimental platform based on behavior data of the insect robot-oriented control neurogenic experimental system, is applied to the field of video processing of insect robot platforms based on a MapReduce distributed parallel processing technology, and improves the high-efficiency processing capacity of the platform on the collected mass image data; the platform solves the problems of synchronous recording of multiple cameras and acquisition and storage of high-throughput data records, can complete experiments such as loop control and behavior data parallel preprocessing of the insect robot in real time, can process massive data aiming at image data of the insect robot during fast flight and comprises parallel data processing with calculation and storage functions, and has the following specific contents:
1. the working process of the insect robot motion attitude and track data acquisition and recording system is as follows:
the behavior of the insect under the electrical stimulation state is shot and recorded through the multi-view high-speed high-definition video camera, the hardware synchronous camera is triggered externally through the signal trigger in the shooting process, the high-performance disk array is used as the high-efficiency recorder to store high-flux data in a lossless mode, and the video data are transmitted to the PC end upper computer to be displayed and stored.
2. The distributed processing cluster platform based on MapReduce comprises the following steps:
fig. 2 shows a MapReduce distributed execution process, where MapReduce is used as a programming mode and adopts a divide and conquer concept. The Hadoop framework realizes distributed parallel computing programming taking MapReduce thought as a main idea, and the components of the Hadoop framework decompose the job task into Map and Reduce which are placed in a TaskTracker child node server for operation. And after receiving the processing request, the child node loads data in the HDFS by using the InputFormat, wherein the data is different from the situation that Block in the HDFS is a physical partition, and logic partition split is performed at the place. The Recordreader reads the data of each fragment from the HDFS, outputs the data as a key value pair to be used as a Map function for inputting, a user program writes the Map function by utilizing self-defined logic, then outputs an intermediate result Shuffle for merging, then transmits the intermediate result Shuffle to the Reduce function, merges a large number of output results from the Map function without using nodes, and finally writes the output results into the HDFS.
3. Frame model modification based on image parallel processing:
in the beginning design, Hadoop mainly aims to meet the requirement of massive internet data management, and when image files are processed in parallel under a MapReduce model, the image files cannot be simply segmented or combined like text files. So in the image domain, the system does not directly support processing images. Meanwhile, Hadoop only defines basic data types facing text data, and the default is that image data types cannot be supported. To implement parallel processing of image data, conversion of image data into a data type that can be directly processed by Hadoop needs to be implemented. The basic principle and the calculation flow of processing the data stored in the HDFS according to the MapReduce model are shown in fig. 3. The invention mainly carries out the following innovation expansion on the current MapReduce framework:
a. image data input design
The data type of the key value pair in MapReduce must support serialization, and the custom data type in Hadoop needs to realize a Writable interface or a Writable Complex < T > interface, wherein the former can only be used as a value type, and the latter can be used as a value type or a key type. The traditional MapReduce processing image data needs to design a corresponding value type to store the image data, an input file of the model adopts a path of a picture in an HDFS, and the path information is stored in a Text file in a character string form, so that key and value types directly adopt a longWritable and Text data type provided by Hadoop default, the longWritable obtains a frame storage position by Text line information to further define a frame unit, and the Text is used for storing image path information recorded in each line. Compared with the traditional method, the model only needs to input the storage path of the image in the input stage, the network transmission amount is less than that of the input image data, and the network transmission amount is reduced when the Map function acquires the data from the fragments. The image data is directly stored in the HDFS after the Map stage is finished, the Reduce stage of the image is omitted, and the transmission quantity of the image data in the model is further reduced.
b. Optimization of file output process
When a large number of small image files are input, the conventional method of the parallel processing model based on the image data type converts the large number of small image files into a large file in advance for processing, and the model adopts a method of transferring parameters required by a Map function by taking a storage path of an image in an HDFS as a value to solve the problem, wherein the acquisition of the image data is completed in the Map function, as shown in FIG. 4.
The model takes the required key-value pairs from the slices generated by the FileInputFormat and passes them to the Map function. When the Map function receives the key-value pair parameters, the Map function is connected to a corresponding path in the HDFS to read the binary data stream of the required image according to the data in the values and the well-defined FSDataInPutStream. The data processing of the model is realized in an OpenCV mode, binary data in a data stream is converted into a Mat type by utilizing a data type Mat provided by OpenCV, then the parallel processing of images in Hadoop is realized by directly calling an image processing algorithm based on OpenCV, and the writing of the image processing algorithm is simplified by utilizing OpenCV. After the image data is processed, a Mat type data is returned, at the end of the Map function, the data is converted into the initial data type (such as JPG format) of the image and stored in the current data node, and finally the image stored locally is uploaded to the HDFS. Parameters received by a Map function in the traditional method from a fragment are also in a key-value pair form, but the value type of the parameters is generally a self-defined image data type, the image data is stored in an object after being serialized, and a processed intermediate result is stored in a current node by using the image data type and waits for being sent to a Reduce function. In the model, image data are directly read into the Map function from the HDFS, and are stored into the HDFS after the processing is finished, so that the data transmission quantity in the model is greatly reduced.

Claims (3)

1. Insect robot massiveness image data parallel processing platform, its characterized in that: the method comprises the following steps:
the system comprises a multi-view high-frame high-definition camera acquisition system and a distributed storage system HDFS;
the system comprises a multi-view high-frame high-definition camera acquisition system, a database, a video acquisition system and a video acquisition system, wherein the multi-view high-frame high-definition camera acquisition system acquires an image frame sequence of insect multi-view flight postures and trajectories to construct the database and provide behavior data for subsequent edge detection and posture analysis;
constructing a master-slave cluster server based on a Hadoop framework, and uploading acquired mass unprocessed insect behavior image data to the HDFS in an off-line manner; the method comprises the following steps that mass image frame data are automatically loaded into a parallel image processing module of a cluster; the parallel image processing module reforms a MapReduce programming mode, so that the parallel image processing module is applied to the field of massive image processing, and the processing efficiency of a platform on massive images is improved.
2. The insect robot massive image data parallel processing platform according to claim 1, wherein:
the multi-view high-frame high-definition camera acquisition system records the flying attitude and the flying trajectory of the insect by using a multi-view synchronous camera;
the system comprises a control window and a multi-camera monitoring window, wherein the control window is used for sending an electric stimulation command to an insect robot from a PC (personal computer) end, the window also comprises a frequency of an electric stimulation signal and a stimulation number selection and adjustment button, and video recording and calibration are carried out;
comprises six Haikang MV-CH050-10CM industrial area-array cameras, the resolution is 2432 multiplied by 2048, and the maximum frame rate is 140 fps;
the system comprises a high-precision TTL synchronous signal controller, a high-resolution image acquisition controller and a high-resolution image acquisition controller, wherein the high-precision TTL synchronous signal controller is used for synchronously triggering six high-definition high-frame industrial cameras;
the system comprises three DALSA camera acquisition cards for caching high-flux video data acquired by a camera;
the system comprises a high-performance server disk array RAID, improves the transmission rate and provides fault-tolerant capability through data verification; uploading the data to a distributed storage system HDFS in an off-line mode, and performing pre-preparation on a parallel processing module.
3. The insect robot massive image data parallel processing platform according to claim 1, wherein: the MapReduce programming mode is specifically modified as follows:
the key and value types in MapReduce directly adopt longWritable and Text data types provided by Hadoop default, the longWritable obtains a frame storage position according to Text line information to further define a frame unit, and the Text is used for storing image path information recorded in each line;
acquiring a required key value pair from a slice generated by the FileInputFormat and transmitting the key value pair to a Map function; when the Map function receives the key value pair parameters, connecting to a binary data stream of a required image under a corresponding path in the HDFS according to data in the values and the well-defined FSDataInPutStream;
the data processing is realized in an OpenCV mode, binary data in a data stream is converted into a Mat type by utilizing a data type Mat provided by OpenCV, and then an image processing algorithm based on OpenCV is directly called to realize the parallel processing of an image in Hadoop
And after the image data is processed, returning a Mat type data, converting the data into the initial data type of the image at the end of the Map function, storing the data in the current data node, and uploading the image stored in the local to the HDFS.
CN202011331446.4A 2020-11-24 2020-11-24 Insect robot mass image data parallel processing platform Active CN112416890B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011331446.4A CN112416890B (en) 2020-11-24 2020-11-24 Insect robot mass image data parallel processing platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011331446.4A CN112416890B (en) 2020-11-24 2020-11-24 Insect robot mass image data parallel processing platform

Publications (2)

Publication Number Publication Date
CN112416890A true CN112416890A (en) 2021-02-26
CN112416890B CN112416890B (en) 2022-10-25

Family

ID=74777046

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011331446.4A Active CN112416890B (en) 2020-11-24 2020-11-24 Insect robot mass image data parallel processing platform

Country Status (1)

Country Link
CN (1) CN112416890B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150125133A1 (en) * 2013-11-06 2015-05-07 Konkuk University Industrial Cooperation Corp. Method for transcoding multimedia, and hadoop-based multimedia transcoding system for performing the method
CN110633733A (en) * 2019-08-14 2019-12-31 中国平安财产保险股份有限公司 Intelligent image matching method and device and computer readable storage medium
CN110689120A (en) * 2019-09-26 2020-01-14 杭州电子科技大学 Loop control neurogenic experiment system for insect robot
CN111251299A (en) * 2020-02-21 2020-06-09 广东工业大学 Hadoop-based cleaning type cloud robot system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150125133A1 (en) * 2013-11-06 2015-05-07 Konkuk University Industrial Cooperation Corp. Method for transcoding multimedia, and hadoop-based multimedia transcoding system for performing the method
CN110633733A (en) * 2019-08-14 2019-12-31 中国平安财产保险股份有限公司 Intelligent image matching method and device and computer readable storage medium
CN110689120A (en) * 2019-09-26 2020-01-14 杭州电子科技大学 Loop control neurogenic experiment system for insect robot
CN111251299A (en) * 2020-02-21 2020-06-09 广东工业大学 Hadoop-based cleaning type cloud robot system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李倩等: "基于Hadoop MapReduce图像处理的数据类型设计", 《软件导刊》 *

Also Published As

Publication number Publication date
CN112416890B (en) 2022-10-25

Similar Documents

Publication Publication Date Title
Kong et al. Mmact: A large-scale dataset for cross modal human action understanding
US9399290B2 (en) Enhancing sensor data by coordinating and/or correlating data attributes
US10818016B2 (en) Systems and methods for predictive/reconstructive visual object tracker
Sun et al. TIB-Net: Drone detection network with tiny iterative backbone
JP2019150911A (en) Robot work system and robot work system control method
EP3445539A1 (en) Methods and apparatus for pruning experience memories for deep neural network-based q-learning
US11594254B2 (en) Event/object-of-interest centric timelapse video generation on camera device with the assistance of neural network input
CN108230309A (en) Field pest and disease monitoring system and method
CN103260007A (en) Intelligent monitoring system based on on-chip multi-port memory controller
Stranieri et al. IRIDIA’s arena tracking system
Zhao et al. Web-based interactive drone control using hand gesture
CN107566785B (en) Big data-oriented video monitoring system and method
CN112416890B (en) Insect robot mass image data parallel processing platform
US10635633B2 (en) Systems and methods for memory architecture
WO2022237061A1 (en) Embedded object cognitive system based on image processing
Sugiyama et al. EEG Channel Optimization for Wireless BMI-based Robot Interaction for Internet of Robotic Things
WO2020207252A1 (en) Data storage method and device, storage medium, and electronic apparatus
CN111401482B (en) Feature point matching method and device, equipment and storage medium
CN114640669A (en) Edge calculation method and device
CN214480701U (en) Cloud robot control system based on Web webpage
CN112926517B (en) Artificial intelligence monitoring method
CN109819044A (en) Remote data acquisition method, apparatus, equipment and storage medium
CN212367393U (en) Video image processing device based on AI chip
CN111654554A (en) Electronic information acquisition method based on wireless communication network
CN115409150A (en) Data compression method, data decompression method and related equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant