CN112804446B - Big data processing method and device based on cloud platform big data - Google Patents
Big data processing method and device based on cloud platform big data Download PDFInfo
- Publication number
- CN112804446B CN112804446B CN202011617203.7A CN202011617203A CN112804446B CN 112804446 B CN112804446 B CN 112804446B CN 202011617203 A CN202011617203 A CN 202011617203A CN 112804446 B CN112804446 B CN 112804446B
- Authority
- CN
- China
- Prior art keywords
- data
- image
- data table
- processing
- related information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a big data processing method based on cloud platform big data, which comprises the following steps: acquiring image data, the image data including at least an original image and image-related information, the original image being an original image acquired by an image sensor; distributing the acquired original images to a server and a client respectively according to a preset strategy for processing; the predetermined policy is controlled by the co-operating unit, the predetermined policy being: the server generates a first data table for storing original images and a second data table for storing image related information; the cooperation unit establishes a third data table for storing original images and a fourth data table for storing image related information; according to the image processing method and the image processing system, the image data are distributed to the server of the cloud platform and the plurality of clients connected with the server for processing through a reasonable strategy, the utilization of image data processing resources is maximized, and the image processing speed can be improved by combining a dual-path image boundary processing method.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a big data processing method based on cloud platform big data.
Background
The cloud computing platform is also called a cloud platform, and is a service based on hardware resources and software resources, and provides computing, network and storage capabilities. Cloud computing platforms can be divided into 3 classes: the cloud computing platform comprises a storage type cloud platform taking data storage as a main part, a computing type cloud platform taking data processing as a main part and a comprehensive cloud computing platform taking computing and data storage processing into consideration.
With the advent of the cloud era, Big data (Big data) has attracted more and more attention. The team of analysts believes that large data (Big data) is often used to describe the large amount of unstructured and semi-structured data created by a company that can take excessive time and money to download to a relational database for analysis. Big data analysis is often tied to cloud computing because real-time large dataset analysis requires a MapReduce-like framework to distribute work to tens, hundreds, or even thousands of computers.
Large data requires special techniques to efficiently process large amounts of data that are tolerant of elapsed time. Technologies applicable to big data include Massively Parallel Processing (MPP) databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.
Big data can not be processed by cloud, and the cloud processing provides elastically expandable basic equipment for the big data, and is one of platforms for generating the big data. Big data technology has begun to be tightly coupled with cloud computing technology, and the relationship between the big data technology and the cloud computing technology is expected to be closer in the future. In addition, emerging computing forms such as the Internet of things and the mobile internet can also help big data revolution together, so that big data marketing can exert greater influence.
Image processing (image processing) techniques that analyze an image with a computer to achieve a desired result. Also known as image processing. Image processing generally refers to digital image processing. Digital images are large two-dimensional arrays of elements called pixels and values called gray-scale values, which are captured by industrial cameras, video cameras, scanners, etc. Image processing techniques generally include image compression, enhancement and restoration, matching, description and identification of 3 parts.
Some useful metric, data or information is extracted from the image. The goal is to get some numerical result, rather than generating another image. The content of image analysis intersects with the research fields of pattern recognition and artificial intelligence, but the image analysis is different from the typical pattern recognition. Image analysis is not limited to classifying specific regions in an image into a fixed number of classes, but primarily provides a description of the image being analyzed. For this purpose, both pattern recognition techniques and knowledge bases on the image content, i.e. on the knowledge representation in artificial intelligence, are used. Image analysis requires extracting the features of an image by an image segmentation method and then symbolizing the image. This description not only answers whether a particular object is present in the image, but also gives a detailed description of the image content.
Digital image processing, analysis and machine vision are exciting and active branches of cognitive science and computer science, and shape is a very important parameter in human perception, recognition and understanding, and chain coding is a shape description method extended by the concept;
chain codes (also called freeman codes) are methods for describing curves or boundaries by coordinates of curve starting points and boundary point direction codes, and are commonly used for representing curves and region boundaries in the fields of image processing, computer graphics, pattern recognition and the like;
for the process of describing the curve or the boundary of the image by using the Freeman chain code, the process of describing the curve boundary and transcoding is required, the data amount required to be processed is large, so that the time consumption is long, and especially the process of describing the curve boundary occupies most of the time consumption;
image data always occupies a main part of a database of a large data platform, and how to optimize the processing of the image data is a problem to be solved.
Disclosure of Invention
The invention provides a cloud platform big data-based big data processing method for optimizing processing of image data, and solves the technical problems in the related art.
According to one aspect of the invention, a big data processing method based on cloud platform big data is provided, which comprises the following steps:
acquiring image data, the image data including at least an original image and image-related information, the original image being an original image acquired by an image sensor;
distributing the acquired original images to a server and a client respectively according to a preset strategy for processing;
the predetermined policy is controlled by the co-operating unit, the predetermined policy being:
the server generates a first data table for storing original images and a second data table for storing image related information;
the cooperation unit establishes a third data table for storing original images and a fourth data table for storing image related information;
the client establishes a fifth data table for storing the original image and a sixth data table for storing image related information;
the physical space sizes of the fifth data table and the sixth data table are fixed;
the first predetermined condition is: the data size of the third data table is larger than the physical space of the fifth data table;
the second predetermined condition is: the data size of the fourth data table is larger than the physical space of the sixth data table;
the third predetermined condition is: the data amount of the third data table is more than 60% of the physical space of the fifth data table and less than 100% of the physical space of the fifth data table;
the fourth predetermined condition is: the data amount of the fourth data table is more than 60% of the physical space of the sixth data table and less than 100% of the physical space of the fifth data table;
the fifth predetermined condition is: the data quantity of the third data table is more than 30% of the physical space of the fifth data table and less than 60% of the physical space of the fifth data table;
the sixth predetermined condition is: the data amount of the fourth data table is more than 30% of the physical space of the sixth data table and less than 60% of the physical space of the fifth data table;
the seventh predetermined condition is: the data size of the third data table is less than 30% of the physical space of the fifth data table;
the eighth predetermined condition is: the data size of the fourth data table is less than 30% of the physical space of the sixth data table.
When the first preset condition and the second preset condition are met at the same time, the original image and the image related information are distributed to a server for processing;
when the first preset condition and the fourth preset condition are met, the original image is distributed to a server for processing, part of image related information is distributed to the server for processing, and part of the image related information is distributed to a client for processing;
when the first preset condition and the sixth preset condition are met, the original image is distributed to a server for processing, and the image related information is distributed to a client for processing;
when the first preset condition and the eighth preset condition are met, part of the original image is distributed to the server for processing, part of the original image is distributed to the client for processing, and image related information is distributed to the client for processing;
when the second preset condition and the third preset condition are met, the image related information is distributed to the server for processing, part of the original image is distributed to the server for processing, and part of the original image is distributed to the client for processing;
when the second preset condition and the fifth preset condition are met, the image related information is distributed to the server for processing, and the original image is distributed to the client for processing;
and when the second preset condition and the seventh preset condition are met, part of the image related information is distributed to the server for processing, part of the image related information is distributed to the client for processing, and the original image is distributed to the client for processing.
Further, the first data table, the second data table, the third data table and the fourth data table are a set of more than one data table, a fifth data table of the client maps one third data table, and a sixth data table maps one fourth data table.
Further, the method for processing the acquired original image by the server and the client comprises the following steps:
placing the acquired original image in a two-dimensional identification area;
connecting pixel points at the edge of the image to form a closed boundary, wherein the central connecting line of two adjacent pixel points is used as a fixed-length line segment, one pixel point of the closed boundary is used as an original point, the original point is used as an initial point, a continuous curve is drawn between the pixel points of the closed boundary, the trend of the fixed-length line segment corresponds to the direction indicator to obtain a digital code, and the digital codes obtained by the fixed-length line segment corresponding to the direction indicator are sequentially arranged to form a chain code;
drawing a continuous curve between pixel points of a closed boundary clockwise by taking an original point as a starting point, and stopping when a first set condition is met;
drawing a continuous curve between the pixel points of the closed boundary along the anticlockwise direction by taking the original point as an initial point, and stopping when a first set condition is met;
the first setting condition is: the method comprises the following steps of (1) carving continuous curves between pixel points of a closed boundary clockwise by taking an original point as an initial point, and carving continuous curves between the pixel points of the closed boundary anticlockwise by taking the original point as the initial point to form a closed curve;
drawing continuous curves corresponding to first pointers between pixel points of a closed boundary clockwise by taking an original point as a starting point to obtain a first chain code;
drawing continuous curves corresponding to the second director among the pixel points of the closed boundary along the counterclockwise direction by taking the original point as a starting point to obtain a second chain code;
and combining the first chain code and the second chain code to obtain the complete chain code of the image boundary.
Furthermore, the first chain code and the second chain code are combined by first reversing the order of the second chain code and then splicing to the tail end of the first chain code.
Further, the first and second pointers are centrosymmetric to each other.
According to an aspect of the present invention, there is provided a cloud platform big data-based big data processing apparatus, including:
a data acquisition unit for acquiring image data including an original image and image-related information;
the cooperation unit is used for distributing the image data to the server and the client side respectively according to a preset strategy for processing;
a server for processing and storing image data;
a big data platform for providing user interaction functionality.
Further, the data acquisition unit at least comprises an original image acquisition module for acquiring an original image and an acquisition module for acquiring image-related information.
Further, the server at least comprises a database for generating a first data table and a second data table for storing image data, a related information processing unit for processing image related information, an original image preprocessing unit for preprocessing an original image, and a chain code generating unit for processing the original image to obtain a chain code.
Furthermore, the big data platform at least comprises a human-computer interaction unit for providing a user interaction function and a data temporary storage unit for sending image data to the client;
the man-machine interaction unit provides a functional module for user operation, wherein the operation comprises retrieval and the like;
the data temporary storage unit temporarily stores the image data obtained by user retrieval and the data processed by the cooperation unit distributed to the client.
The invention has the beneficial effects that:
according to the image processing method and the image processing system, the image data are distributed to the server of the cloud platform and the plurality of clients connected with the server for processing through a reasonable strategy, the utilization of image data processing resources is maximized, and the image processing speed can be improved by combining a dual-path image boundary processing method.
Drawings
Fig. 1 is a first flowchart of a big data processing method based on cloud platform big data according to an embodiment of the present invention;
FIG. 2 is a second flowchart of a big data processing method based on cloud platform big data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a continuous curve drawn between pixels on a closed boundary clockwise and counterclockwise with an origin as a starting point according to an embodiment of the present invention;
FIG. 4 is a diagram of a first pointer according to an embodiment of the present invention;
FIG. 5 is a diagram of a second pointer according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a continuous curve drawn between pixels on a closed boundary clockwise according to an embodiment of the present invention;
FIG. 7 is a schematic block diagram of a cloud platform big data-based big data processing apparatus according to an embodiment of the present invention;
FIG. 8 is a block diagram of a collaboration unit in accordance with an embodiment of the present invention;
FIG. 9 is a block diagram of a server according to an embodiment of the invention;
FIG. 10 is a block diagram of a big data platform according to an embodiment of the present invention.
In the figure: the cloud platform big data based big data processing device 100, the data acquisition unit 110, the cooperation unit 120, the server 130, the database 131, the related information processing unit 132, the original image preprocessing unit 133, the chain code generation unit 134, the big data platform 140, the human-computer interaction unit 141, the data temporary storage unit 142, the index training database 143, and the machine learning unit 144.
Detailed Description
The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand the subject matter described herein and are not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as needed. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with respect to some examples may also be combined in other examples.
In this embodiment, a cloud platform big data-based big data processing method is provided, as shown in fig. 1, a schematic flow chart of the cloud platform big data-based big data processing method according to the present invention is shown, as shown in fig. 1, the cloud platform big data-based big data processing method includes the following steps:
s1, acquiring image data, wherein the image data at least comprises an original image and image related information, and the original image is an original image acquired by an image sensor;
the related information may be a set of information such as a shooting place, a photographer, shooting time, exposure level, and the like;
s2, distributing the collected original images to a server and a client respectively according to a preset strategy for processing;
the predetermined policy is controlled by the co-operating unit, the predetermined policy being:
s3, the server generates a first data table for storing the original image and a second data table for storing the image related information;
s4, the cooperation unit establishes a third data table for storing original images and a fourth data table for storing image related information;
s5, the client establishes a fifth data table for storing the original image and a sixth data table for storing the image related information;
the physical space size of the fifth data table and the sixth data table is fixed. And determining the storage and processing capacity of the integrated client.
The first data table, the second data table, the third data table and the fourth data table may be a set of more than one data table, the fifth data table of the client maps one third data table, and the sixth data table maps one fourth data table.
The first predetermined condition is: the data size of the third data table is larger than the physical space of the fifth data table;
the second predetermined condition is: the data size of the fourth data table is larger than the physical space of the sixth data table;
the third predetermined condition is: the data amount of the third data table is more than 60% of the physical space of the fifth data table and less than 100% of the physical space of the fifth data table;
the fourth predetermined condition is: the data amount of the fourth data table is more than 60% of the physical space of the sixth data table and less than 100% of the physical space of the fifth data table;
the fifth predetermined condition is: the data amount of the third data table is more than 30% of the physical space of the fifth data table and less than 60% of the physical space of the fifth data table;
the sixth predetermined condition is: the data amount of the fourth data table is more than 30% of the physical space of the sixth data table and less than 60% of the physical space of the fifth data table;
the seventh predetermined condition is: the data size of the third data table is less than 30% of the physical space of the fifth data table;
the eighth predetermined condition is: the data size of the fourth data table is less than 30% of the physical space of the sixth data table.
S6, when the first preset condition and the second preset condition are met, the original image and the image related information are distributed to a server for processing;
s7, when the first preset condition and the fourth preset condition are met, the original image is distributed to a server for processing, part of the image related information is distributed to the server for processing, and part of the image related information is distributed to a client for processing;
s8, when the first preset condition and the sixth preset condition are met, the original image is distributed to a server for processing, and the image related information is distributed to a client for processing;
s9, when the first preset condition and the eighth preset condition are met, distributing part of the original image to a server for processing, distributing part of the original image to a client for processing, and distributing the image related information to the client for processing;
s10, when the second preset condition and the third preset condition are met, the image related information is distributed to the server for processing, the original image is partially distributed to the server for processing, and the original image is partially distributed to the client for processing;
s11, when the second preset condition and the fifth preset condition are met, the image related information is distributed to a server for processing, and the original image is distributed to a client for processing;
and S12, when the second preset condition and the seventh preset condition are met, distributing part of the image related information to the server for processing, distributing part of the image related information to the client for processing, and distributing the original image to the client for processing.
Based on the above strategy, the processing capacity of the client is utilized to the maximum extent, the operation load of the server is reduced, and the efficiency of data processing can be ensured.
In order to increase the data processing speed on the basis of the foregoing, this embodiment provides a method for processing an acquired original image by a server and a client, including:
placing the acquired original image in a two-dimensional identification area;
the method comprises the steps that an image is placed in a two-dimensional recognition area for recognition, recognition points corresponding to pixel points are arranged in the recognition area and serve as the background of the image, the recognition points are overlapped with the pixel points of the image during processing, the pixel points of the image cannot be easily recognized due to the change of the image, the corresponding pixel points of the image can be mapped based on the memorized recognition points of the recognition area, and the pixel points of the image are expressed through the expression of the recognition points;
connecting pixel points at the edge of the image to form a closed boundary, wherein the central connecting line of two adjacent pixel points is used as a fixed-length line segment, one pixel point of the closed boundary is used as an original point, the original point is used as an initial point, a continuous curve is drawn between the pixel points of the closed boundary, the trend of the fixed-length line segment corresponds to the direction indicator to obtain a digital code, and the digital codes obtained by the fixed-length line segment corresponding to the direction indicator are sequentially arranged to form a chain code;
drawing a continuous curve between pixel points of a closed boundary clockwise by taking an original point as a starting point, and stopping when a first set condition is met;
drawing a continuous curve between the pixel points of the closed boundary along the counterclockwise direction by taking the original point as a starting point, and stopping when a first set condition is met;
the first setting condition is: the method comprises the following steps of (1) carving continuous curves between pixel points of a closed boundary clockwise by taking an original point as an initial point, and carving continuous curves between the pixel points of the closed boundary anticlockwise by taking the original point as the initial point to form a closed curve;
drawing continuous curves corresponding to first pointers between pixel points of a closed boundary clockwise by taking an original point as a starting point to obtain a first chain code;
drawing continuous curves corresponding to the second director among the pixel points of the closed boundary along the counterclockwise direction by taking the original point as a starting point to obtain a second chain code;
combining the first chain code and the second chain code to obtain an image boundary complete chain code;
the first pointer and the second pointer are mutually centrosymmetric;
in this embodiment, the identification point has fixed coordinates within the identification area.
As shown in fig. 3, a continuous curve corresponding to a pointer is drawn between pixels on a closed boundary clockwise with an origin as a starting point to obtain a first chain code of 434202;
drawing continuous curve corresponding direction indicators among the pixel points of the closed boundary along the anticlockwise direction by taking the original point as a starting point to obtain a second chain code 6760;
the complete chain code combined with the first chain code after the second chain code is inverted is 4342020676;
a first pointer as shown in fig. 4 and a second pointer as shown in fig. 5, the first pointer and the second pointer being centrosymmetric to each other;
from the above process, the combining the first chain code and the second chain code to obtain the image boundary complete chain code includes: the second chain code is inverted and then combined to the end of the first chain code.
As shown in fig. 6, the chain code obtained by plotting the continuous curve corresponding designators between the pixel points of the closed boundary only clockwise is 4342020676, which is completely consistent with the above-mentioned complete chain code;
further, the following table 1 shows the comparison of the chain code results obtained by bi-directional and unidirectional depiction with different starting points S1, S2, S3;
TABLE 1
| Bidirectional clockwise | Bidirectional counter-clockwise | Unidirectional clockwise | |
| S1 | 1234 | 8765 | 12345678 |
| S2 | 8123 | 7654 | 81234567 |
| S3 | 4567 | 3218 | 45678123 |
As can be seen from the above table, the same chain code as the one-way depiction can be obtained from any point, and the encoding time of the chain code can be shortened by nearly 50%, which is particularly suitable for high-frequency image recognition scenes, such as highway vehicle recognition.
The method provides two paths which are not conflicted with each other and are carried out simultaneously for image processing, and boundary chain codes of the images can be obtained more quickly;
as shown in fig. 7-10, based on the big data processing method based on the cloud platform big data of the embodiment, the embodiment provides a big data processing apparatus 100 based on the cloud platform big data, including:
a data acquisition unit 110 for acquiring image data containing an original image and image-related information;
a coordination unit 120, configured to distribute the image data to the server 130 and the client according to a predetermined policy and process the image data;
a server 130 for processing and storing image data;
a big data platform 140 for providing user interaction functionality;
wherein the data acquisition unit 110 at least comprises an original image acquisition module for acquiring an original image and an acquisition module for acquiring image-related information;
the cooperation unit 120 includes at least a cooperation database 121 for generating a third data table and a fourth data table for storing image data and an allocation unit 122 for allocating the image data;
the server 130 includes at least a database 131 for generating a first data table and a second data table for storing image data, a related information processing unit 132 for processing image related information, an original image preprocessing unit 133 for preprocessing an original image, and a chain code generating unit 134 for processing the original image to obtain a chain code;
a big data platform 140, which at least comprises a human-computer interaction unit 141 for providing a user interaction function and a data temporary storage unit 142 for sending image data to a client;
wherein the human-computer interaction unit 141 provides functional modules for user operations, including retrieval, etc.;
the data temporary storage unit 142 temporarily stores the image data retrieved by the user and the data distributed to the client for processing by the cooperation unit 120;
as an optimized option, the big data platform 140 further includes an index training database 143 for storing keyword indexes and a machine learning unit 144 for index training; the index training of the machine learning unit 144 includes recording keywords retrieved by the user and target image data in the selected retrieval result, and establishing mapping between the keywords and the image data according to the probability of the selected image data when retrieving the keywords;
specifically, for the image data a and the keyword a, the number of times of retrieving the keyword a is N, the total number of times of selecting the image data a when retrieving the keyword a is M, the probability of selecting the image data a when retrieving the keyword a is N/M, and if the probability exceeds a threshold value, mapping between the keyword and the image data is established.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present embodiment or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (e.g. a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method of the embodiments.
Although the embodiments of the present invention have been described with reference to the drawings, the present invention is not limited to the above specific embodiments, which are only illustrative and not restrictive, and those skilled in the art can make many forms without departing from the spirit and scope of the present invention and the claims.
Claims (9)
1. A big data processing method based on cloud platform big data is characterized by comprising the following steps:
acquiring image data, the image data including at least an original image and image-related information, the original image being an original image acquired by an image sensor;
distributing the acquired original images to a server and a client respectively according to a preset strategy for processing;
the predetermined policy is controlled by the co-operating unit, the predetermined policy being:
the server generates a first data table for storing original images and a second data table for storing image related information;
the cooperation unit establishes a third data table for storing original images and a fourth data table for storing image related information;
the client establishes a fifth data table for storing the original image and a sixth data table for storing image related information;
the physical space sizes of the fifth data table and the sixth data table are fixed;
the first predetermined condition is: the data size of the third data table is larger than the physical space of the fifth data table;
the second predetermined condition is: the data size of the fourth data table is larger than the physical space of the sixth data table;
the third predetermined condition is: the data amount of the third data table is more than 60% of the physical space of the fifth data table and less than 100% of the physical space of the fifth data table;
the fourth predetermined condition is: the data amount of the fourth data table is more than 60% of the physical space of the sixth data table and less than 100% of the physical space of the fifth data table;
the fifth predetermined condition is: the data amount of the third data table is more than 30% of the physical space of the fifth data table and less than 60% of the physical space of the fifth data table;
the sixth predetermined condition is: the data amount of the fourth data table is more than 30% of the physical space of the sixth data table and less than 60% of the physical space of the fifth data table;
the seventh predetermined condition is: the data size of the third data table is less than 30% of the physical space of the fifth data table;
the eighth predetermined condition is: the data amount of the fourth data table is less than 30% of the physical space of the sixth data table;
when the first preset condition and the second preset condition are met at the same time, the original image and the image related information are distributed to a server for processing;
when the first preset condition and the fourth preset condition are met, the original image is distributed to a server for processing, part of image related information is distributed to the server for processing, and part of the image related information is distributed to a client for processing;
when the first preset condition and the sixth preset condition are met, the original image is distributed to a server for processing, and the image related information is distributed to a client for processing;
when the first preset condition and the eighth preset condition are met, part of the original image is distributed to the server for processing, part of the original image is distributed to the client for processing, and image related information is distributed to the client for processing;
when the second preset condition and the third preset condition are met, the image related information is distributed to the server for processing, part of the original image is distributed to the server for processing, and part of the original image is distributed to the client for processing;
when the second preset condition and the fifth preset condition are met, the image related information is distributed to the server for processing, and the original image is distributed to the client for processing;
and when the second preset condition and the seventh preset condition are met, part of the image related information is distributed to the server for processing, part of the image related information is distributed to the client for processing, and the original image is distributed to the client for processing.
2. The cloud platform big data-based big data processing method according to claim 1, wherein the first data table, the second data table, the third data table, and the fourth data table are a set of more than one data table, a fifth data table of the client maps a third data table, and a sixth data table maps a fourth data table.
3. The big data processing method based on the cloud platform big data according to claim 1, wherein the method for processing the acquired original image by the server and the client comprises the following steps:
placing the acquired original image in a two-dimensional identification area;
connecting pixel points at the edge of the image to form a closed boundary, wherein the central connecting line of two adjacent pixel points is used as a fixed-length line segment, one pixel point of the closed boundary is used as an original point, the original point is used as an initial point, a continuous curve is drawn between the pixel points of the closed boundary, the trend of the fixed-length line segment corresponds to the direction indicator to obtain a digital code, and the digital codes obtained by the fixed-length line segment corresponding to the direction indicator are sequentially arranged to form a chain code;
drawing a continuous curve between pixel points of a closed boundary clockwise by taking an original point as a starting point, and stopping when a first set condition is met;
drawing a continuous curve between the pixel points of the closed boundary along the counterclockwise direction by taking the original point as a starting point, and stopping when a first set condition is met;
the first setting condition is: the method comprises the following steps of (1) carving continuous curves between pixel points of a closed boundary clockwise by taking an original point as an initial point, and carving continuous curves between the pixel points of the closed boundary anticlockwise by taking the original point as the initial point to form a closed curve;
drawing continuous curves corresponding to first pointers between pixel points of a closed boundary clockwise by taking an original point as a starting point to obtain a first chain code;
drawing continuous curves corresponding to the second director among the pixel points of the closed boundary along the anticlockwise direction by taking the original point as an initial point to obtain a second chain code;
and combining the first chain code and the second chain code to obtain the complete chain code of the image boundary.
4. The cloud platform big data-based big data processing method according to claim 3, wherein the first chain code and the second chain code are combined by first reversing the order of the second chain code and then splicing the second chain code to the end of the first chain code.
5. The cloud platform big data-based big data processing method according to claim 3, wherein the first pointer and the second pointer are centrosymmetric to each other.
6. A cloud platform big data-based big data processing apparatus, which is configured to perform the cloud platform big data-based big data processing method according to any one of claims 1 to 5, and the cloud platform big data-based big data processing apparatus includes:
a data acquisition unit for acquiring image data including an original image and image-related information;
the cooperation unit is used for distributing the image data to the server and the client side respectively according to a preset strategy for processing;
a server for processing and storing image data;
a big data platform for providing user interaction functionality.
7. The cloud platform big data-based big data processing device according to claim 6, wherein the data acquisition unit at least comprises an original image acquisition module for acquiring an original image and an acquisition module for acquiring image-related information.
8. The cloud platform big data-based big data processing device according to claim 6, wherein the server at least comprises a database for generating the first data table and the second data table for storing image data, a related information processing unit for processing image related information, an original image preprocessing unit for preprocessing an original image, and a chain code generating unit for processing the original image to obtain a chain code.
9. The big data processing device based on the cloud platform big data is characterized in that the big data platform at least comprises a human-computer interaction unit for providing a user interaction function and a data temporary storage unit for sending image data to a client;
the human-computer interaction unit provides a functional module for user operation;
the data temporary storage unit temporarily stores the image data obtained by user retrieval and the data processed by the cooperation unit distributed to the client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011617203.7A CN112804446B (en) | 2020-12-30 | 2020-12-30 | Big data processing method and device based on cloud platform big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011617203.7A CN112804446B (en) | 2020-12-30 | 2020-12-30 | Big data processing method and device based on cloud platform big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112804446A CN112804446A (en) | 2021-05-14 |
| CN112804446B true CN112804446B (en) | 2022-08-12 |
Family
ID=75804762
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011617203.7A Active CN112804446B (en) | 2020-12-30 | 2020-12-30 | Big data processing method and device based on cloud platform big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112804446B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115828220A (en) * | 2022-12-30 | 2023-03-21 | 南京叁月半信息科技有限公司 | Method and system for identifying identity of communication receiver |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101399832A (en) * | 2007-09-27 | 2009-04-01 | 国际商业机器公司 | Method and system for collaboration application in a client-server network |
| CN107087019A (en) * | 2017-03-14 | 2017-08-22 | 西安电子科技大学 | A kind of end cloud cooperated computing framework and task scheduling apparatus and method |
| CN109857549A (en) * | 2019-01-04 | 2019-06-07 | 平安科技（深圳）有限公司 | Image processing method, system, equipment and medium based on load balancing |
| CN111984364A (en) * | 2019-05-21 | 2020-11-24 | 江苏艾蒂娜互联网科技有限公司 | Artificial intelligence cloud platform for 5G era |
| CN112035481A (en) * | 2020-08-31 | 2020-12-04 | 中国平安财产保险股份有限公司 | Data processing method, data processing device, computer equipment and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7818530B2 (en) * | 2003-11-26 | 2010-10-19 | Hewlett-Packard Development Company, L.P. | Data management systems, articles of manufacture, and data storage methods |
-
2020
- 2020-12-30 CN CN202011617203.7A patent/CN112804446B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101399832A (en) * | 2007-09-27 | 2009-04-01 | 国际商业机器公司 | Method and system for collaboration application in a client-server network |
| CN107087019A (en) * | 2017-03-14 | 2017-08-22 | 西安电子科技大学 | A kind of end cloud cooperated computing framework and task scheduling apparatus and method |
| CN109857549A (en) * | 2019-01-04 | 2019-06-07 | 平安科技（深圳）有限公司 | Image processing method, system, equipment and medium based on load balancing |
| CN111984364A (en) * | 2019-05-21 | 2020-11-24 | 江苏艾蒂娜互联网科技有限公司 | Artificial intelligence cloud platform for 5G era |
| CN112035481A (en) * | 2020-08-31 | 2020-12-04 | 中国平安财产保险股份有限公司 | Data processing method, data processing device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112804446A (en) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210256320A1 (en) | Machine learning artificialintelligence system for identifying vehicles | |
| CN110147722A (en) | A kind of method for processing video frequency, video process apparatus and terminal device | |
| CN111950723A (en) | Neural network model training method, image processing method, device and terminal equipment | |
| CN113704531A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| EP4404148A1 (en) | Image processing method and apparatus, and computer-readable storage medium | |
| US11983903B2 (en) | Processing images using self-attention based neural networks | |
| CN114169381A (en) | Image annotation method and device, terminal equipment and storage medium | |
| CN111767915A (en) | License plate detection method, device, equipment and storage medium | |
| CN110780965A (en) | Vision-based process automation method, device and readable storage medium | |
| CN111179270A (en) | Image co-segmentation method and device based on attention mechanism | |
| CN110717405B (en) | Face feature point positioning method, device, medium and electronic equipment | |
| CN112804446B (en) | Big data processing method and device based on cloud platform big data | |
| CN116662839A (en) | Associated big data cluster analysis method and device based on multidimensional intelligent acquisition | |
| CN115223662A (en) | Data processing method, device, equipment and storage medium | |
| CN114462582A (en) | Data processing method, device and equipment based on convolutional neural network model | |
| CN115797291B (en) | Loop terminal identification method, loop terminal identification device, computer equipment and storage medium | |
| US20240135679A1 (en) | Method for classifying images and electronic device | |
| CN111191065A (en) | Homologous image determining method and device | |
| CN115909009A (en) | Image recognition method, image recognition device, storage medium and electronic equipment | |
| CN115858695A (en) | Information processing method and device and storage medium | |
| CN115661472A (en) | Image duplicate checking method and device, computer equipment and storage medium | |
| CN115115513A (en) | Image processing method, device, equipment and storage medium | |
| CN112487943B (en) | Key frame de-duplication method and device and electronic equipment | |
| CN114329024A (en) | Icon searching method and system | |
| US12125247B2 (en) | Processing images using self-attention based neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Big data processing methods and devices based on cloud platform big data Effective date of registration: 20230106 Granted publication date: 20220812 Pledgee: Nanjing Zidong sub branch of Bank of Nanjing Co.,Ltd. Pledgor: JIANGSU DATATECH INFORMATION TECHNOLOGY CO.,LTD. Registration number: Y2022980029812 |