CN109101646B - Data processing method, device, system and computer readable medium - Google Patents
Data processing method, device, system and computer readable medium Download PDFInfo
- Publication number
- CN109101646B CN109101646B CN201810957602.4A CN201810957602A CN109101646B CN 109101646 B CN109101646 B CN 109101646B CN 201810957602 A CN201810957602 A CN 201810957602A CN 109101646 B CN109101646 B CN 109101646B
- Authority
- CN
- China
- Prior art keywords
- image
- rectangular object
- frame
- block diagram
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a data processing method, a device, a system and a computer readable medium, wherein the method is applied to an edge terminal comprising image acquisition equipment and comprises the following steps: acquiring a video stream in a region acquired by image acquisition equipment; extracting object features in each single-frame image of the video stream to obtain a feature map; for each feature map, determining a rectangular object block diagram of each object in a single-frame image corresponding to the feature map by using a preset classification neural network; tracking objects in the plurality of single-frame images, and determining an optimal image of each object according to a rectangular object block diagram of each object in the plurality of single-frame images; the optimal image of each object is sent to the server, the technical problem that the quality of the image received by the server is low in the prior art is solved, and the technical effect of improving the quality of the image received by the server is achieved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method, apparatus, system, and computer readable medium.
Background
Image capturing devices are ubiquitous today, and the amount of video data generated by these image capturing devices per second is very large. These video data are to be sent to the server. However, due to limitations in storage space and transmission bandwidth, video data needs to be heavily compressed before being sent to the server. After the video data is compressed in a large amount, a problem of image blurring in the video data may be caused. Therefore, a problem of low quality of the image received by the server is caused.
Disclosure of Invention
In view of the above, the present invention provides a data processing method, apparatus, system and computer readable medium to alleviate the technical problem of low quality of images received by a server in the prior art.
In a first aspect, an embodiment of the present invention provides a data processing method, where the method is applied to an edge terminal including an image acquisition device, and the method includes:
acquiring a video stream in the area acquired by the image acquisition equipment;
extracting object features in each single-frame image of the video stream to obtain a feature map;
for each feature map, determining a rectangular object block diagram of each object in a single-frame image corresponding to the feature map by using a preset classification neural network;
tracking objects in the single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the single-frame images;
and sending the optimal image of each object to a server.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the determining, by using a preset classification neural network, a rectangular object block diagram of each object in a single frame image corresponding to the feature map includes:
generating a feature expression function corresponding to the preset classification neural network;
acquiring a pixel value on each pixel point of the characteristic diagram;
substituting the pixel value into the characteristic expression function for each pixel point to obtain the coordinate positions of four points of the object in the single-frame image;
and generating the rectangular object block diagram according to the four-point coordinate positions.
With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the four-point coordinate positions include: the method comprises the following steps of generating a rectangular object block diagram according to four-point coordinate positions, wherein the four-point coordinate positions comprise a first sub coordinate, a second sub coordinate, a third sub coordinate and a fourth sub coordinate, the first sub coordinate and the second sub coordinate are on the same horizontal line, the third sub coordinate and the fourth sub coordinate are on the same horizontal line, the first sub coordinate and the third sub coordinate are on the same vertical line, the second sub coordinate and the fourth sub coordinate are on the same vertical line, and the method comprises the following steps:
connecting the point on the first sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively;
connecting the point of the fourth sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively;
and obtaining a block diagram of the rectangular object.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the tracking objects in the multiple single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the multiple single-frame images, includes:
in any two adjacent single-frame images, setting the same serial number for the same object in different single-frame images;
respectively extracting the rectangular object block diagrams of the objects with the same number from the single-frame images to obtain a rectangular object block diagram set corresponding to the number;
and for each rectangular object block diagram set, performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting preset quality requirements.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where in any two adjacent single-frame images, setting the same serial number for the same object in different single-frame images includes:
in any two adjacent single-frame images, acquiring the serial number of an object and the rectangular object frame diagram of each object in the previous single-frame image, determining the acquired serial number as a first serial number, and determining the acquired rectangular object frame diagram as a first rectangular object frame diagram;
for each object in the next single-frame image, acquiring the rectangular object frame diagram of the object, and determining the acquired rectangular object frame diagram as a second rectangular object frame diagram;
respectively determining the overlapping spaces of the first rectangular object frame diagram and the second rectangular object frame diagrams;
judging whether the overlap space with the largest value exceeds a preset overlap threshold value or not;
if the overlapping space with the largest value exceeds the preset overlapping threshold, determining that a second rectangular object frame diagram corresponding to the overlapping space with the largest value is a target rectangular object frame diagram;
and setting the first number as the number of the object corresponding to the target rectangular object frame.
With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where, for each rectangular object block diagram set, performing quality evaluation on a rectangular object block diagram in the rectangular object block diagram set to obtain the optimal image meeting a preset quality requirement, includes:
acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram;
for each rectangular object block diagram, multiplying the image size by a preset image size weight, and calculating to obtain an image size component;
multiplying the image definition by a preset image definition weight, and calculating to obtain an image definition component;
multiplying the image imaging angle by a preset image imaging angle weight, and calculating to obtain an image imaging angle component;
adding the image size component, the image definition component and the image imaging angle component to calculate an image quality value;
and determining the rectangular object block diagram corresponding to the image quality value with the maximum value as the optimal image.
With reference to the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where, for each rectangular object block diagram set, performing quality evaluation on a rectangular object block diagram in the rectangular object block diagram set to obtain the optimal image meeting a preset quality requirement, includes:
acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram;
and performing quality evaluation on the rectangular object block diagram, and selecting the rectangular object block diagram with the image size larger than a preset image size threshold, the image definition larger than a preset image definition threshold and the image imaging angle larger than a preset image imaging angle threshold as an optimal image to be output, so as to obtain the optimal image meeting preset quality requirements.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus, including: the device comprises an acquisition module, an extraction module, a determination module, a tracking module and a sending module;
the acquisition module is used for acquiring the video stream in the area acquired by the image acquisition equipment;
the extraction module is used for extracting object features in each single-frame image of the video stream to obtain a feature map;
the determining module is used for determining a rectangular object block diagram of each object in the single-frame image corresponding to each feature map by using a preset classification neural network for each feature map;
the tracking module is used for tracking objects in the single-frame images and determining the optimal image of each object according to the rectangular object block diagram of each object in the single-frame images;
the sending module is used for sending the optimal image of each object to a server.
In a third aspect, an embodiment of the present invention further provides a data processing system, including: a server and a plurality of edge terminals to which the method according to the first aspect is applied, the server communicating with the plurality of edge terminals, respectively.
In a fourth aspect, the present invention also provides a computer-readable medium having non-volatile program code executable by a processor, where the program code causes the processor to execute the method according to any one of the first aspect.
The embodiment of the invention has the following beneficial effects: the data processing method provided by the embodiment of the invention is applied to an edge terminal comprising image acquisition equipment, and comprises the following steps: acquiring a video stream in the area acquired by the image acquisition equipment; extracting object features in each single-frame image of the video stream to obtain a feature map; for each feature map, determining a rectangular object block diagram of each object in a single-frame image corresponding to the feature map by using a preset classification neural network; tracking objects in the single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the single-frame images; and sending the optimal image of each object to a server.
Therefore, the edge end does not need to compress all the acquired video data and then send the video data to the server, and only needs to send the optimal image of each object to the server, so that the problem of image blurring caused by the fact that a large amount of video data are compressed can be avoided, and the problem of low quality of the image received by the server is further avoided, therefore, the technical problem of low quality of the image received by the server in the prior art is solved, and the technical effect of improving the quality of the image received by the server is achieved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a data processing method according to an embodiment of the present invention;
FIG. 2 is a flowchart of step S104 in FIG. 1;
FIG. 3 is a schematic block diagram of a data processing apparatus provided by an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data processing system according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, image acquisition devices are ubiquitous, and the amount of video data generated by these image acquisition devices per second is very large. These video data are to be sent to the server. However, due to limitations in storage space and transmission bandwidth, video data needs to be heavily compressed before being sent to the server. After the video data is compressed in a large amount, a problem of image blurring in the video data may be caused. Therefore, the problem of low quality of the image received by the server is caused, and based on the data processing method, the data processing device, the data processing system and the computer readable medium provided by the embodiments of the present invention, the technical problem of low quality of the image received by the server in the prior art can be alleviated, so as to achieve the technical effect of improving the quality of the image received by the server.
To facilitate understanding of the present embodiment, a detailed description is first given of a data processing method disclosed in the present embodiment, where the method is applied to an edge terminal including an image capturing device, as shown in fig. 1, and the data processing method may include the following steps.
And step S101, acquiring the video stream in the area acquired by the image acquisition equipment.
For example, the image capturing device may be a high definition camera, a smart camera, or the like.
And S102, extracting the object characteristics in each single-frame image of the video stream to obtain a characteristic diagram.
In the embodiment of the invention, the object characteristics of the single-frame image can be extracted by adopting a neural network algorithm of a plurality of convolutional layers to obtain the characteristic diagram.
And S103, determining a rectangular object block diagram of each object in the single-frame image corresponding to the feature map by using a preset classification neural network for each feature map.
The determining, by using a preset classification neural network, a rectangular object block diagram of each object in a single-frame image corresponding to the feature map may include the following steps:
(1) and generating a feature expression function corresponding to the preset classification neural network.
(2) And acquiring a pixel value on each pixel point of the characteristic diagram.
For example, a sliding window manner may be adopted to obtain a pixel value on each pixel point of the feature map.
(3) And substituting the pixel value into the characteristic expression function for each pixel point to obtain the coordinate positions of four points of the object in the single-frame image.
For example, the four-point coordinate positions may include: the system comprises a first sub-coordinate, a second sub-coordinate, a third sub-coordinate and a fourth sub-coordinate, wherein the first sub-coordinate and the second sub-coordinate are on the same horizontal line, the third sub-coordinate and the fourth sub-coordinate are on the same horizontal line, the first sub-coordinate and the third sub-coordinate are on the same vertical line, and the second sub-coordinate and the fourth sub-coordinate are on the same vertical line.
(4) And generating the rectangular object block diagram according to the four-point coordinate positions.
The generating the rectangular object block diagram according to the four-point coordinate positions may include: connecting the point on the first sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively; connecting the point of the fourth sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively; and obtaining a block diagram of the rectangular object.
In the embodiment of the invention, for each feature map, the object class and the class confidence of each object in the single-frame image corresponding to the feature map can be determined by using a preset classification neural network.
For example, the object categories may be: pedestrians, bicycles, motorcycles, tricycles, cars, vans, trucks, buses or buses etc. The predetermined classification neural network may be a fast R-CNN neural network model, a YOLO (You Only Look one: Unifield, Real-Time Object Detection), a R-FCN neural network model, or an SSD neural network model.
Step S104, tracking the objects in the single-frame images, and determining the optimal image of each object according to the rectangular object block diagram of each object in the single-frame images.
Wherein the optical flow tracking algorithm or SORT (SIMPLE ONLINE ANDREALTIME TRACKING) algorithm can be used to track the objects in the plurality of single frame images.
Further, as shown in fig. 2, step S104 may include the following steps.
Step S201, in any two adjacent single-frame images, setting the same serial number for the same object in different single-frame images.
Wherein, the step S201 may include the following steps:
(1) in any two adjacent single-frame images, for each object in the previous single-frame image, acquiring the number of the object and the rectangular object frame diagram, determining the acquired number as a first number, and determining the acquired rectangular object frame diagram as a first rectangular object frame diagram.
(2) And for each object in the subsequent single-frame image, acquiring the rectangular object frame diagram of the object, and determining the acquired rectangular object frame diagram as a second rectangular object frame diagram.
(3) And respectively determining the overlapping spaces of the first rectangular object frame diagram and the second rectangular object frame diagrams.
(4) And judging whether the overlapping space with the largest value exceeds a preset overlapping threshold value.
The preset overlap threshold may be determined according to actual service requirements.
(5) And if the overlapping space with the maximum value exceeds the preset overlapping threshold value, determining that the second rectangular object frame corresponding to the overlapping space with the maximum value is the target rectangular object frame.
(6) And setting the first number as the number of the object corresponding to the target rectangular object frame.
Step S202, the rectangular object block diagrams of the objects with the same number are respectively extracted from the single-frame images, and a rectangular object block diagram set corresponding to the number is obtained.
Step S203, for each rectangular object block diagram set, performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting preset quality requirements.
One specific implementation of step S203 may be: acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram; for each rectangular object block diagram, multiplying the image size by a preset image size weight, and calculating to obtain an image size component; multiplying the image definition by a preset image definition weight, and calculating to obtain an image definition component; multiplying the image imaging angle by a preset image imaging angle weight, and calculating to obtain an image imaging angle component; adding the image size component, the image definition component and the image imaging angle component to calculate an image quality value; and determining the rectangular object block diagram corresponding to the image quality value with the maximum value as the optimal image.
For example, the formula for calculating the image quality value may be: x is1*ω1+x2*ω2+x3*ω3Where Z may represent an image quality value, x1Can represent the image size, ω1May represent a preset image size weight, x2Can represent image sharpness, ω2May represent a preset image sharpness weight, x3Can represent the imaging angle of the image, omega3A preset image imaging angle weight may be represented. And each rectangular object block diagram in each rectangular object block diagram set can be calculated to obtain an image quality value.
It is to be noted that the preset image size weight, the preset image sharpness weight, and the preset image imaging angle weight are not fixed. In different scenes, the preset image size weights are different, the preset image definition weights are different, and the preset image imaging angle weights are also different.
Further, another specific implementation manner of step S203 may be: acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram; and performing quality evaluation on the rectangular object block diagram, and selecting the rectangular object block diagram with the image size larger than a preset image size threshold, the image definition larger than a preset image definition threshold and the image imaging angle larger than a preset image imaging angle threshold as an optimal image to be output, so as to obtain the optimal image meeting preset quality requirements.
Illustratively, the image definition of each rectangular object block diagram can be obtained by using a Brenner gradient method, a Tenegrad gradient method, a laplace gradient method, a variance method or an energy gradient method. The image imaging angle of each rectangular object block diagram can be obtained by adopting a VGG classification network model, ResNet (Residual Networks), GoogleNet classification network model, MobileNet classification network model or DenseNet classification network model.
The preset image size threshold, the preset image definition threshold and the preset image imaging angle threshold can be determined according to actual service requirements.
In the embodiment of the invention, the data acquisition process and the video structuring process are fused in the edge end, so that the compressed fuzzy image is prevented from being used in the video structuring process, and the quality of the optimal image can be improved. When the server performs object recognition using the optimal image, the error rate of object recognition can be reduced, and the probability of object recognition can be improved.
Step S105, sending the optimal image of each object to a server.
In the embodiment of the invention, because the edge end only sends the optimal image to the server, the interference of redundant data can be reduced, the occupancy rate of bandwidth is reduced, the cost of data transmission is reduced, and the quality of the transmitted image is improved, so that the server utilizes the high-quality optimal image to perform the object identification process, and the object identification probability is improved.
In the embodiment of the invention, the edge terminal can also send the object type, the object number, the image quality value of the optimal image and the image frame number of the single-frame image in which the optimal image is located of each object to the server.
For example, the edge may send the optimal image, the object type, the object number, the image quality value of the optimal image, and the image frame number of the single-frame image where the optimal image is located of each object to the server by using a User Datagram Protocol (UDP).
Unless specifically stated otherwise, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
In an embodiment of the present invention, a data processing method provided in the embodiment of the present invention is applied to an edge terminal including an image capture device, and the method includes: acquiring a video stream in the area acquired by the image acquisition equipment; extracting object features in each single-frame image of the video stream to obtain a feature map; for each feature map, determining a rectangular object block diagram of each object in a single-frame image corresponding to the feature map by using a preset classification neural network; tracking objects in the single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the single-frame images; and sending the optimal image of each object to a server.
Therefore, the edge end does not need to compress all the collected video data and then send the video data to the server, and only needs to send the optimal image of each object to the server, so that the requirement of subsequent analysis can be met, the problem of image blurring caused by the fact that the video data are compressed in large quantities can be avoided, and the problem of low image quality received by the server is further avoided, therefore, the technical problem of low image quality received by the server in the prior art is solved, and the technical effect of improving the quality of the image received by the server is achieved.
In another embodiment of the present invention, a data processing apparatus disclosed in the embodiment of the present invention is described in detail, and as shown in fig. 3, the data processing apparatus may include: an acquisition module 31, an extraction module 32, a determination module 33, a tracking module 34 and a sending module 35.
The acquiring module 31 is configured to acquire a video stream in the area acquired by the image acquisition device.
The extracting module 32 is configured to extract object features in each single frame image of the video stream to obtain a feature map.
The determining module 33 is configured to determine, for each feature map, a rectangular object block diagram of each object in the single frame image corresponding to the feature map by using a preset classification neural network.
The tracking module 34 is configured to track the objects in the plurality of single-frame images, and determine an optimal image of each object according to the rectangular object block diagram of each object in the plurality of single-frame images.
The sending module 35 is configured to send the optimal image of each object to a server.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments.
In another embodiment of the present invention, a data processing system disclosed in the embodiment of the present invention is described in detail, where the data processing system may include: a server and a plurality of edge terminals applying the method according to any of the above embodiments, the server communicating with the plurality of edge terminals respectively.
For example, the data processing system includes a server and four edge terminals. As shown in fig. 4, the data processing system includes a server 41 and four edge terminals, where the four edge terminals are respectively: edge end 42, edge end 43, edge end 44, and edge end 45. Server 41 communicates with edge terminal 42, edge terminal 43, edge terminal 44, and edge terminal 45, respectively.
Wherein each edge end may comprise an image capture device.
In the embodiment of the invention, each edge end only sends the optimal image to the server, so that the interference of redundant data is reduced, and the problem of image blurring caused by the massive compression of video data is avoided, thereby not only reducing the occupancy rate of bandwidth, but also improving the quality of the image. Meanwhile, the overall performance of the data processing system can be improved, the use cost of the data processing system is reduced, and the flexibility and the expandability of the system are ensured.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In a further embodiment of the present invention, a computer-readable medium having a non-volatile program code executable by a processor and causing the processor to perform any one of the methods of the above embodiments is disclosed.
In all examples shown and described herein, any particular value should be construed as merely exemplary, and not as a limitation, and thus other examples of example embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
The computer program product for performing the data processing method provided in the embodiment of the present invention includes a computer-readable storage medium storing a nonvolatile program code executable by a processor, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the method embodiment, and is not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. A data processing method is applied to an edge terminal comprising an image acquisition device, and comprises the following steps:
acquiring a video stream in the area acquired by the image acquisition equipment;
extracting object features in each single-frame image of the video stream to obtain a feature map;
for each feature map, determining a rectangular object block diagram, an object class and a class confidence coefficient of each object in a single-frame image corresponding to the feature map by using a preset classification neural network;
tracking objects in the single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the single-frame images;
sending the optimal image of each object to a server;
the tracking objects in the plurality of single-frame images, and determining an optimal image of each object according to the rectangular object block diagram of each object in the plurality of single-frame images, includes:
in any two adjacent single-frame images, setting the same serial number for the same object in different single-frame images;
respectively extracting the rectangular object block diagrams of the objects with the same number from the single-frame images to obtain a rectangular object block diagram set corresponding to the number;
for each rectangular object block diagram set, performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting preset quality requirements;
the method for determining the rectangular object block diagram of each object in the single-frame image corresponding to the feature diagram by using the preset classification neural network comprises the following steps:
generating a feature expression function corresponding to the preset classification neural network;
acquiring a pixel value on each pixel point of the characteristic diagram in a sliding window mode;
substituting the pixel value into the characteristic expression function for each pixel point to obtain the coordinate positions of four points of the object in the single-frame image;
and generating the rectangular object block diagram according to the four-point coordinate positions.
2. The data processing method of claim 1, wherein the four-point coordinate positions comprise: the method comprises the following steps of generating a rectangular object block diagram according to four-point coordinate positions, wherein the four-point coordinate positions comprise a first sub coordinate, a second sub coordinate, a third sub coordinate and a fourth sub coordinate, the first sub coordinate and the second sub coordinate are on the same horizontal line, the third sub coordinate and the fourth sub coordinate are on the same horizontal line, the first sub coordinate and the third sub coordinate are on the same vertical line, the second sub coordinate and the fourth sub coordinate are on the same vertical line, and the method comprises the following steps:
connecting the point on the first sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively;
connecting the point of the fourth sub-coordinate with the point on the second sub-coordinate and the point on the third sub-coordinate respectively;
and obtaining a block diagram of the rectangular object.
3. The data processing method according to claim 1, wherein in any two adjacent single-frame images, setting the same number for the same object in different single-frame images comprises:
in any two adjacent single-frame images, acquiring the serial number of an object and the rectangular object frame diagram of each object in the previous single-frame image, determining the acquired serial number as a first serial number, and determining the acquired rectangular object frame diagram as a first rectangular object frame diagram;
for each object in the next single-frame image, acquiring the rectangular object frame diagram of the object, and determining the acquired rectangular object frame diagram as a second rectangular object frame diagram;
respectively determining the overlapping spaces of the first rectangular object frame diagram and the second rectangular object frame diagrams;
judging whether the overlap space with the largest value exceeds a preset overlap threshold value or not;
if the overlapping space with the largest value exceeds the preset overlapping threshold, determining that a second rectangular object frame diagram corresponding to the overlapping space with the largest value is a target rectangular object frame diagram;
and setting the first number as the number of the object corresponding to the target rectangular object frame.
4. The data processing method according to claim 1, wherein the performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting a preset quality requirement for each rectangular object block diagram set comprises:
acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram;
for each rectangular object block diagram, multiplying the image size by a preset image size weight, and calculating to obtain an image size component;
multiplying the image definition by a preset image definition weight, and calculating to obtain an image definition component;
multiplying the image imaging angle by a preset image imaging angle weight, and calculating to obtain an image imaging angle component;
adding the image size component, the image definition component and the image imaging angle component to calculate an image quality value;
and determining the rectangular object block diagram corresponding to the image quality value with the maximum value as the optimal image.
5. The data processing method according to claim 1, wherein the performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting a preset quality requirement for each rectangular object block diagram set comprises:
acquiring the image size, the image definition and the image imaging angle of each rectangular object frame diagram;
and performing quality evaluation on the rectangular object block diagram, and selecting the rectangular object block diagram with the image size larger than a preset image size threshold, the image definition larger than a preset image definition threshold and the image imaging angle larger than a preset image imaging angle threshold as an optimal image to be output, so as to obtain the optimal image meeting preset quality requirements.
6. A data processing apparatus, comprising: the device comprises an acquisition module, an extraction module, a determination module, a tracking module and a sending module;
the acquisition module is used for acquiring the video stream in the area acquired by the image acquisition equipment;
the extraction module is used for extracting object features in each single-frame image of the video stream to obtain a feature map;
the determining module is used for determining a rectangular object block diagram, an object category and a category confidence coefficient of each object in the single-frame image corresponding to each feature map by utilizing a preset classification neural network for each feature map;
the tracking module is used for tracking objects in the single-frame images and determining the optimal image of each object according to the rectangular object block diagram of each object in the single-frame images;
the sending module is used for sending the optimal image of each object to a server;
the tracking module is further configured to:
in any two adjacent single-frame images, setting the same serial number for the same object in different single-frame images;
respectively extracting the rectangular object block diagrams of the objects with the same number from the single-frame images to obtain a rectangular object block diagram set corresponding to the number;
for each rectangular object block diagram set, performing quality evaluation on the rectangular object block diagrams in the rectangular object block diagram set to obtain the optimal image meeting preset quality requirements;
the determining module is further configured to:
generating a feature expression function corresponding to the preset classification neural network;
acquiring a pixel value on each pixel point of the characteristic diagram in a sliding window mode;
substituting the pixel value into the characteristic expression function for each pixel point to obtain the coordinate positions of four points of the object in the single-frame image;
and generating the rectangular object block diagram according to the four-point coordinate positions.
7. A data processing system, comprising: a server and a plurality of edge terminals to which the method according to any of claims 1-5 is applied, said server communicating with a plurality of said edge terminals respectively.
8. A computer-readable medium having non-volatile program code executable by a processor, wherein the program code causes the processor to perform the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810957602.4A CN109101646B (en) | 2018-08-21 | 2018-08-21 | Data processing method, device, system and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810957602.4A CN109101646B (en) | 2018-08-21 | 2018-08-21 | Data processing method, device, system and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109101646A CN109101646A (en) | 2018-12-28 |
| CN109101646B true CN109101646B (en) | 2020-12-18 |
Family
ID=64850636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810957602.4A Expired - Fee Related CN109101646B (en) | 2018-08-21 | 2018-08-21 | Data processing method, device, system and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109101646B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114520890B (en) * | 2020-11-19 | 2023-07-11 | 华为技术有限公司 | Image processing method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108230293A (en) * | 2017-05-31 | 2018-06-29 | 深圳市商汤科技有限公司 | Determine method and apparatus, electronic equipment and the computer storage media of quality of human face image |
| CN108269250A (en) * | 2017-12-27 | 2018-07-10 | 武汉烽火众智数字技术有限责任公司 | Method and apparatus based on convolutional neural networks assessment quality of human face image |
| CN108305240A (en) * | 2017-05-22 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Picture quality detection method and device |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101329767B (en) * | 2008-07-11 | 2011-11-16 | 西安交通大学 | Method for automatically detecting obvious object sequence in video based on learning |
| CN103914853A (en) * | 2014-03-19 | 2014-07-09 | 华南理工大学 | Method for processing target adhesion and splitting conditions in multi-vehicle tracking process |
| CN104469547B (en) * | 2014-12-10 | 2017-06-06 | 西安理工大学 | A kind of video abstraction generating method based on tree-shaped movement objective orbit |
| CN105117705B (en) * | 2015-08-26 | 2018-08-24 | 北京无线电计量测试研究所 | A kind of iris image quality tandem type evaluation method |
| CN105740758A (en) * | 2015-12-31 | 2016-07-06 | 上海极链网络科技有限公司 | Internet video face recognition method based on deep learning |
| CN105844665B (en) * | 2016-03-21 | 2018-11-27 | 清华大学 | The video object method for tracing and device |
| US10185884B2 (en) * | 2016-09-07 | 2019-01-22 | Apple Inc. | Multi-dimensional objective metric concentering |
| CN106407978B (en) * | 2016-09-24 | 2020-10-30 | 上海大学 | Method for detecting salient object in unconstrained video by combining similarity degree |
| CN106778645B (en) * | 2016-12-24 | 2018-05-18 | 深圳云天励飞技术有限公司 | A kind of image processing method and device |
| CN107918767B (en) * | 2017-11-27 | 2019-11-08 | 北京旷视科技有限公司 | Object detection method, device, electronic equipment and computer-readable medium |
| CN108040244B (en) * | 2017-12-11 | 2020-11-24 | 深圳云天励飞技术有限公司 | Snapshot method and device based on light field video stream and storage medium |
| CN108280399A (en) * | 2017-12-27 | 2018-07-13 | 武汉普利商用机器有限公司 | A kind of scene adaptive face identification method |
-
2018
- 2018-08-21 CN CN201810957602.4A patent/CN109101646B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108305240A (en) * | 2017-05-22 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Picture quality detection method and device |
| CN108230293A (en) * | 2017-05-31 | 2018-06-29 | 深圳市商汤科技有限公司 | Determine method and apparatus, electronic equipment and the computer storage media of quality of human face image |
| CN108269250A (en) * | 2017-12-27 | 2018-07-10 | 武汉烽火众智数字技术有限责任公司 | Method and apparatus based on convolutional neural networks assessment quality of human face image |
Non-Patent Citations (2)
| Title |
|---|
| "图像质量评价综述";何南南 等;《北京印刷学院学报》;20170426;第25卷(第2期);第47-50页 * |
| "多参数融合的手背静脉图像质量评价";王一丁 等;《小型微型计算机系统》;20160415;第37卷(第4期);第870-874页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109101646A (en) | 2018-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447990B (en) | Image semantic segmentation method and device, electronic equipment and computer readable medium | |
| CN110869937A (en) | Face image duplication removal method and apparatus, electronic device, storage medium, and program | |
| EP3579180A1 (en) | Image processing method and apparatus, electronic device and non-transitory computer-readable recording medium for selective image enhancement | |
| CN107967677B (en) | Image processing method, image processing device, computer-readable storage medium and computer equipment | |
| CN112001274B (en) | Crowd density determining method, device, storage medium and processor | |
| CN107481271B (en) | Stereo matching method, system and mobile terminal | |
| CN109783680B (en) | Image pushing method, image acquisition device and image processing system | |
| CN111131314A (en) | Network behavior detection method and device, computer equipment and storage medium | |
| CN110991231A (en) | Living body detection method and device, server and face recognition equipment | |
| CN111598176B (en) | Image matching processing method and device | |
| CN111445058A (en) | Data analysis method, device, equipment and computer readable storage medium | |
| CN112802076A (en) | Reflection image generation model and training method of reflection removal model | |
| CN107832598B (en) | Unlocking control method and related product | |
| CN109101646B (en) | Data processing method, device, system and computer readable medium | |
| CN112819889B (en) | Method and device for determining position information, storage medium and electronic device | |
| CN113888438A (en) | Image processing method, device and storage medium | |
| CN110992426B (en) | Gesture recognition method and device, electronic equipment and storage medium | |
| CN112597995A (en) | License plate detection model training method, device, equipment and medium | |
| CN111339949A (en) | License plate recognition method and device and inspection vehicle | |
| CN114267021A (en) | Object recognition method and device, storage medium and electronic equipment | |
| CN114564310A (en) | Data processing method and device, electronic equipment and readable storage medium | |
| CN111601108B (en) | Monitoring video processing method and device and video monitoring terminal | |
| CN114723883A (en) | Three-dimensional scene reconstruction method, device, equipment and storage medium | |
| CN110213457B (en) | Image transmission method and device | |
| CN113095313A (en) | Text string recognition method and device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201218 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |