WO2021000418A1

WO2021000418A1 - Image data processing method and image data processing apparatus

Info

Publication number: WO2021000418A1
Application number: PCT/CN2019/103651
Authority: WO
Inventors: 惠慧
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-07-03
Filing date: 2019-08-30
Publication date: 2021-01-07
Also published as: CN110415295A; CN110415295B

Abstract

The image data processing method of the present application uses the following technical solution: determining the acquisition time point from the image data at a predetermined interval by acquiring image data acquired by the shooting apparatus in real time; carrying out difference processing on the previous frame picture and the next frame picture at the acquisition time point, and screening to obtain a difference image; extracting the contour line of the difference image, if the position range of the contour line is located in a specified area, acquiring the previous frame picture and the next frame picture of the difference image as the first target frame picture and the second target frame picture respectively; intercepting the first target frame picture and the second target frame picture according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and returning the first screenshot and the second screenshot to the server for processing. Under the condition that the target object or the observation area is monitored in real time or continuously, the above technical solution can return the changed first screenshot and second screenshot, improve the data processing speed and efficiency, and reduce the storage resource requirement.

Description

Image data processing method and image data processing equipment

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 3, 2019, the application number is 201910595428.8, and the invention title is "Image Data Processing Method and Image Data Processing Equipment", the entire content of which is incorporated herein by reference Applying.

Technical field

This application relates to the technical field of data transmission and image data processing. Specifically, this application relates to an image data processing method and an image data processing device.

Background technique

Image data processing technology includes a technology that processes images to obtain corresponding data. With the development of Internet of Things technology, life is becoming more and more intelligent. Take the physical display as an example to monitor the target object in a certain area or a certain location. This monitoring method relies on image processing technology and can achieve a high degree of automation. Observe the aforementioned relevant information by observing and analyzing surveillance videos.

The inventor realized that the technical solution of the prior art adopts continuous video recording for monitoring, which easily takes up network transmission resources and storage resources, and requires a large volume of image recognition and analysis, which severely restricts obtaining the display status of the target object , The processing speed and efficiency of the missing state.

Summary of the invention

The purpose of this application is to solve at least one of the above-mentioned technical defects, in particular, it is easy to occupy network transmission resources and storage resources, and the amount of analysis and image recognition required is large, which seriously restricts the acquisition of the display status and lack of target objects. Technical defects in state processing speed and efficiency.

This application provides an image data processing method, including:

Acquiring image data collected in real time by the photographing device, and determining collection time points from the image data at predetermined intervals;

Perform difference processing between the previous frame picture and the next frame picture at the acquisition time point, and filter the difference image;

Extract the contour line of the differential image, if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture and the second target frame respectively image;

The first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the first screenshot and the second screenshot are sent back to the server for processing .

The embodiment of the present application also correspondingly provides an image data processing device, including:

The acquisition module is used to acquire the image data collected by the shooting device in real time, and determine the collection time point from the image data at a predetermined interval;

The processing module is used to perform difference processing between the previous frame picture and the next frame picture at the collection time point, and filter the difference image;

The execution module is used to extract the contour line of the differential image, and if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture respectively And the second target frame picture;

The interception module is used to intercept the first target frame picture and the second target frame picture according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and return the first screenshot and the second screenshot Pass to the server for processing.

The embodiments of the present application also correspondingly provide an image data processing device, and the image data processing device applies an image data processing method, including at least:

A processor for executing the steps of the image data processing method;

Wherein, the image data processing method includes the following steps:

This embodiment also provides a non-volatile computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute an image data processing method;

Wherein, the image data processing method includes the following steps:

An image data processing method provided in an embodiment of the application is to obtain real-time image data collected by a photographing device, and determine the collection time point from the image data at a predetermined interval; One frame of pictures is subjected to difference processing to filter to obtain a difference image; the contour line of the difference image is extracted, and if the position range of the contour line is in a designated area, the previous frame picture and the next frame picture of the difference image are obtained , Respectively as the first target frame picture and the second target frame picture; intercept the first target frame picture and the second target frame picture according to the position range of the contour line, obtain the first screenshot and the second screenshot respectively, and The first screenshot and the second screenshot are returned to the technical solution of the server for processing. An image data processing device and a non-volatile computer-readable storage medium are also provided in the embodiments of the present application. The above technical solution can return the first screenshot and the second screenshot with certain changes in the image under the condition of real-time or continuous monitoring of the target object or specific observation area. The above technical solution can reduce the number of transmitted image data and the size of a single image while basically retaining the validity of the image data, effectively reduce the amount of image data transmission and image data processing, increase processing speed, and reduce storage difficulty.

Description of the drawings

FIG. 1 is a method flowchart of an image data processing method according to an embodiment;

FIG. 2 is a schematic diagram of an application scenario of an image data processing method according to an embodiment;

FIG. 3 is a schematic diagram of image data of an embodiment;

Figure 4 is a schematic diagram of a differential image in an embodiment;

Figure 5 is a schematic diagram of the first screenshot of this embodiment;

Fig. 6 is a flowchart of a method for obtaining a differential image of an embodiment;

Fig. 7 is a flowchart of a method for updating a difference threshold in an embodiment;

FIG. 8 is a flowchart of a method for obtaining a differential image without limiting a time interval in an embodiment;

FIG. 9 is a flowchart of a method for moving a target object according to a sorting instruction in an embodiment;

Fig. 10 is a flowchart of a method for obtaining transaction quantity in an embodiment;

Figure 11 is a flowchart of a method for generating alarm information in an embodiment;

Figure 12 is a flowchart of a method for generating order information in an embodiment;

Fig. 13 is a flowchart of a method for sending a logistics route in an embodiment.

Detailed ways

In order to overcome the technical drawbacks of easily occupying network transmission resources and storage resources, the volume required to analyze and perform image recognition is large, which severely restricts the processing speed and efficiency of obtaining the display state and missing state of the target object, this application provides a For the image data processing method, please refer to Figure 1. The above image data processing method includes the steps:

Step S11: Acquire real-time image data collected by the photographing device, and determine collection time points from the image data at predetermined intervals.

Step S12: Perform difference processing on the previous frame picture and the next frame picture at the acquisition time point, and filter to obtain a difference image.

Step S13: Extract the contour line of the difference image. If the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the difference image as the first target frame picture and the second frame picture respectively. 2. Target frame picture.

Step S14: intercept the first target frame picture and the second target frame picture according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and return the first screenshot and the second screenshot to The server performs processing.

Please refer to FIG. 2, this embodiment of the present application also exemplarily shows the application environment of the foregoing image data processing method for this purpose. The application environment includes camera equipment, and data interaction occurs between the camera equipment and the image data processing platform. The server and the image data processing platform are connected through a network. The image data processing platform sends the image data collected by the camera equipment, processed image results and other data or data obtained from third-party systems and databases to the server for further processing . Among them, the above-mentioned image data processing platform is connected with an order system, a transaction database, a warehousing system, a logistics system, and a dispatching system.

The image data processing platform obtains real-time image data collected by the shooting device through the server (refer to FIG. 3), and determines the collection time point from the above-mentioned image data at predetermined intervals. In this embodiment, the predetermined interval may take the use scene into consideration, and the predetermined interval may be set as the time of the corresponding use scene, or the predetermined interval may be determined according to factors such as image data format. For example, in the application scenario shown by taking the target object as an example in this embodiment, the predetermined interval may be 1 second. In the process of image processing, according to different scenes, the collection time point may also be the time point of framing the image data, for example, 20 milliseconds.

The image data processing platform performs difference processing between the previous frame picture and the next frame picture at the extracted collection time point, and filters to obtain the difference image. The difference image can refer to Figure 4.

The image data processing platform extracts the contour lines of the above-mentioned differential image. The purpose of extracting contour lines is to determine whether a specific area is on the difference image. In addition, the contour line can also locate the approximate position of the target object in the difference image, which can reduce the amount of recognition calculation.

The image data processing platform determines that the position range of the contour line of the above-mentioned differential image is in a designated area, and then obtains the previous frame picture and the next frame picture corresponding to the time point or time period of the differential image according to the predetermined interval time, as respectively The first target frame picture and the second target frame picture. In this embodiment, the previous frame picture and the next frame picture are determined according to the time interval of picture framing, or the number of frames per second can be determined according to the video format, and then the time interval of framing is determined . Generally speaking, the more framing times, the higher the processing accuracy, and the processing amount will increase accordingly.

In this embodiment, the first target frame picture and the second target frame picture are adjacent two frames of pictures with a set interval time. The designated area is the area where the camera device aims at the target position of the target object and is finally reflected in the picture. For example, if the camera device is aimed at a container, the area marked by the contour line is the area where the container is located, and what is outside the contour line is the environment around the container. To give another example, the camera device can be aimed at a certain desk, then the outline marks the desk area, and the image display area outside the outline shows the adjacent desk or aisle.

The image data processing platform takes screenshots according to the position range of the contour lines in the first target frame picture and the second target frame picture, obtains the first screenshot and the second screenshot respectively, and sends the first and second screenshots back to the server for processing .

Take the first screenshot taken from the first target frame picture as an example. In this embodiment, please refer to FIG. 4. The image data processing platform recognizes the contour line of the first target frame picture, determines the position specified by the contour line, and compares the contour line After intercepting, the area inside the contour is obtained, which is the contour area. The image data processing platform intercepts the contour area corresponding to the first target frame picture to obtain the first screenshot. The entire display area of the first target frame picture is the imaging area collected by the imaging device. The image data processing platform sends the first screenshot and the second screenshot captured by the contour line to the server through a network connection for further analysis or processing.

In view of the foregoing technical solution, this application also provides an implementation manner that specifically applies the technical solution. For example, the foregoing technical solution can be applied to the use scenario of transmitting a video stream. The processor acquires a video stream collected by a shooting device in real time in real time, and determines the collection time point from the video stream at predetermined intervals (for example, every two seconds). For example, the collection time point can be 14:05:01, 14:05:03, 14:05:05, 14:05:07, 14:05:09, etc. It can be understood as the processor’s response to the real-time video stream every other time. Set an acquisition time point in two seconds for analysis.

In addition, the aforementioned predetermined interval can also be changed according to changes in application scenarios, time, external environment, and the like. For example, when the application scenario has a large change, the predetermined interval can be shortened; when the application scenario has a small change, the predetermined interval can be extended. Of course, changing factors such as application scenarios, time, external environment, and predetermined intervals can also be matched by establishing a matching model through intelligent learning and other methods.

The processor parses the video stream into multiple frames of pictures, and performs difference processing on the previous frame picture and the next frame picture at the determined collection time point, and obtains the difference image through screening. The difference processing here can be the difference processing of pixel values, or the difference processing for the selected target object.

After that, the processor extracts the contour line of the difference image, and if the position range of the contour line is in the designated area, the previous frame picture and the next frame picture of the difference image are obtained as the first target frame picture respectively And the second target frame picture. The aforementioned designated area may be within the image display area, or may be set outside the image display area. In some embodiments, the designated area can also be adjusted according to the acquisition angle of the image acquisition device.

According to the position range of the contour line, the first target frame picture and the second target frame picture are respectively intercepted to obtain the first screenshot and the second screenshot, and the first screenshot and the second screenshot are sent back to the server for processing deal with.

Through the above processing, the amount of data returned to the server for processing can be reduced, the network requirements and bandwidth occupation for data transmission can be reduced, the processing pressure of the server can also be reduced, the processing speed of the server at the peak of the data can be improved, and the speed of data processing can be accelerated. The speed of feedback.

Please refer to FIG. 6, an embodiment of the present application also provides a technical solution for performing differential processing on frame pictures to obtain differential images. Step S12 performs differential processing on the previous frame picture and the next frame picture at the collection time point, and filtering The steps to obtain a differential image include:

Step S61: Perform grayscale processing on the previous picture and the next picture of the frame picture respectively.

Step S62: Perform a pixel point comparison between the previous frame picture and the next frame picture at the acquisition time point after the grayscale processing is performed to generate a pixel difference value.

Step S63: When the pixel difference value is greater than the difference threshold value, a difference image is generated according to the pixel difference value.

In the above process, the image data processing platform performs grayscale processing on the previous frame of the frame picture and the next frame of the picture. Among them, the previous frame of picture and the next frame of picture are two frames of pictures adjacent to each other with a predetermined interval.

The image data processing platform compares the pixels of the previous frame picture and the next frame of the picture after the grayscale processing to generate a pixel difference. In this process, it is possible to pre-identify and align the pixel points of the frame picture according to the positioning objects in the frame picture, so as to avoid the change of the image capturing area caused by the change of the external environment, which ultimately affects the value of the pixel difference.

In addition, in some embodiments, the image data processing platform may also obtain the reference image of the target object in advance. The reference image may be taken by the camera device at the same shooting angle, or it may be a picture before or after a picture from a frame. For example, when the difference processing is performed, the pixel difference between the previous picture and the next picture of the frame picture shows that the difference between the two pictures is not large, the previous picture or the next picture can be used as the reference image. Thereafter, the reference image can also be updated according to changes in the external environment or changes in time (after a certain time interval, for example, every hour). The reference image can be used as the previous picture to participate in differential processing. The technical solution of using the reference image can overcome the influence of the external environment (such as light intensity, light angle, reflected light, etc.) to a certain extent.

The image data processing platform judges that when the pixel difference between the previous picture and the next picture at the acquisition time point is greater than the difference threshold, the difference between the previous picture and the next picture at the acquisition time is regarded as the difference image. The difference image can show the change of the target object between two frames of images, for example, the movement change such as the target object being moved and the quantitative change such as the disappearance of the target object.

In order to obtain a more accurate differential threshold and improve the accuracy of differential image processing, a technical solution is also provided in this embodiment. Please refer to FIG. 7, after the step of extracting the contour line of the difference image in step S13, the method further includes:

Step S71: Perform difference processing on the area outside the contour line in the frame picture to obtain the background pixel difference value of the corresponding area.

Step S72: Adjust the difference threshold value according to the background pixel difference value, and update the difference threshold value.

Among them, the updated difference threshold can be used for the next difference image screening.

In order to better explain the above process, please refer to Figure 5 and take the contour line of the first target frame picture as an example. The image data processing platform separately extracts the first target frame picture and the second target frame picture outside the contour line. area. The area outside the contour line of the first target frame picture mainly presents the environment where the target object is located. The area outside the contour line and the contour area where the target object is located generally have the same environmental change state. The area outside the contour is smaller than the contour area where the target object is located due to the movement of the object.

The image data processing platform performs differential processing on the area outside the contour line of the first target frame picture and the second target frame picture to obtain the background pixel difference of the corresponding area. The aforementioned background pixel difference value may indicate the environmental change of the contour area where the target object is located.

The image data processing platform adjusts the difference threshold according to the background pixel difference and updates the value of the difference threshold. Thus, the image data processing platform can dynamically adjust and update the difference threshold according to changes in the environment and the like.

In some application scenarios, the change of the target object may be aperiodic or the corresponding change cannot be captured in frame pictures between set intervals. This embodiment provides a corresponding technical solution for this. This technical solution can be aimed at a real-time video stream whose video end time is unknown (ie continuous shooting), and there are application scenarios that change continuously for a long period of time, such as a real-time video stream in which an object is continuously moved or taken away in a surveillance video. Referring to FIG. 8, the step of acquiring image data collected by the shooting device in real time in step S11, and determining the collection time point from the image data at predetermined intervals includes:

Step S81: Acquire a real-time video stream collected by the shooting device in real time, parse the real-time video stream to obtain a number of frames of pictures, and determine the collection time point according to the time of the real-time video stream and a predetermined interval.

In step S63, the step of generating a difference image according to the pixel difference value includes:

Step S82: Obtain at least two positioning images according to the pixel difference value, where the two positioning images are a first positioning image and a second positioning image respectively.

Step S83: Obtain and perform differential processing on the previous frame picture of the first positioning image and the next frame picture of the second positioning image to obtain a differential image.

Wherein, the first positioning image is the positioning image prior to the second positioning image in time sequence, and the second positioning image is the last positioning image in time sequence.

In the above process, the image data processing platform obtains the real-time video stream collected by the shooting equipment in real time, and the image data processing platform parses the real-time video stream to obtain several frames of pictures for subsequent processing.

The image data processing platform performs difference processing between the previous frame picture and the next frame picture in the extracted frame pictures, where the difference processing adopts the difference processing of the pixel value difference. Wherein, the previous frame picture and the next frame picture are two adjacent frames of pictures among the multiple frames of pictures. The image data processing platform performs differential processing on the adjacent previous frame of picture and the next frame of picture, which can be done through gray processing, pixel difference processing, etc.

The image data processing platform performs differential detection of two frames of pictures of the previous frame of pictures and the next frame of pictures through the conditional filtering of the difference threshold, and obtains the pixel difference between the two frames of pictures. According to the comparison between the pixel difference value and the difference threshold value, the image data processing platform uses the previous frame picture at this time as the first positioning image (may be called the first positioning image) when the pixel difference value is greater than the difference threshold value. The image data processing platform continues to analyze other frames of pictures according to the passage of time. When the pixel difference of other picture frames after the above-mentioned difference detection is greater than the difference threshold, the next frame picture at this time is extracted as the second positioning picture ( It can be called the second positioning image). Through the above screening process, the image data processing platform obtains at least two positioning images.

The image data processing platform obtains and performs differential processing on the previous frame picture of the first positioning image and the next frame picture of the second positioning image to obtain a differential image.

Of course, in some embodiments, the positioning image may be multiple positioning images. For example, the pixel difference of the target object within a period of time, there are multiple time points greater than the difference threshold, then these time points that satisfy the difference threshold filtering conditions can be used. Acquire multiple positioning images. When there are multiple positioning images, the image data processing platform may use the first positioning image as the first positioning image, and the last positioning image as the second positioning image.

In order to further utilize the results of the processing of the above-mentioned image data processing platform, an embodiment is provided in the embodiments of this application, which can organize the target objects back to their original positions, which can be specifically applied to unmanned vending machines, display racks, showcases, and express delivery areas. , Express delivery area and other scenes that need to be organized. Please refer to FIG. 9, the above step S14 intercepts the first target frame picture and the second target frame picture according to the position range of the contour line to obtain a first screenshot and a second screenshot, respectively, and combine the first and second screenshots. Second, after the screenshot is sent back to the server for processing, it also includes:

Step S91: Recognizing the current position and the original position of the target object according to the contour line or the feature point of the target object in the first screenshot and the second screenshot, respectively.

The current position of the target object is the position of the target object in the second screenshot, and the original position of the target object is the position of the target object in the first screenshot.

Step S92: Generate a moving distance by calculating the position difference of the target object in the first screenshot and the second screenshot.

Step S93: Determine whether the moving distance is greater than a distance threshold.

If yes, execute step S94, if not, execute step S95.

Step S94: Generate a sorting instruction.

Step S95: No processing is performed.

Step S96: Move the target object from the current position to the original position according to the sorting instruction.

In the above process, after the server receives the first screenshot and the second screenshot from the image data processing platform, it processes them accordingly. The server recognizes and obtains the current position and original position of the target object in the first screenshot and the second screenshot according to the first screenshot and the second screenshot that are returned. Among them, the time stamp of the first screenshot shows that the time of the first screenshot is earlier than the second screenshot. Correspondingly, the position of the target object in the first screenshot is the original position of the target object, and the position of the target object in the second screenshot is the current position of the target object.

The server calculates the position difference of the target object in the first screenshot and the second screenshot, and generates the moving distance of the target object based on the position difference. For example, according to the position difference between the first screenshot and the second screenshot on the display shelf, the target object is recognized to have moved 10 cm outward.

The server obtains the distance threshold, and judges whether the moving distance of the target object is greater than the distance threshold. For example, in this embodiment, the distance threshold of the target object is set to 5 cm, then step S94 is executed at this time to generate a sorting instruction. If the moving distance of the target object is only 4 cm, step S95 is executed without processing.

The server may send a sorting instruction to an external device (for example, a device that can move the target object such as a robotic arm, a roller, a moving horizontal spoke), and the external device moves the target object from the current position to the original position according to the received sorting command. The direction, angle, and distance the external device moves to the target object can be adjusted according to the moving distance recognized by the server.

In order to further utilize the results of the above-mentioned image data processing platform processing in scenes such as supermarkets, shopping malls showcases, and vending machines, an embodiment is provided in the embodiments of the present application, and the number of transactions can be obtained. Please refer to FIG. 10, in step S14, the first target frame picture and the second target frame picture are captured according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the first screenshot After the second screenshot is sent back to the server for processing, it also includes:

Step S101: Obtain object information in the area corresponding to the image data collected in real time by the photographing device.

Step S102: Recognize the first screenshot and the second screenshot respectively to obtain the number change value of the target object.

The image data processing platform can obtain transaction data of the target object in the area corresponding to the image data collected in real time by the shooting device. The area corresponding to the image data collected in real time by the photographing device refers to the area displayed in the first screenshot or the second screenshot. Target Object In this embodiment, packaged potato chips can be taken as an example.

Specifically, the image data processing platform can identify the target object according to the first screenshot or the second screenshot. The image data processing platform can recognize in the first screenshot that the target object is a bag of packaged potato chips. Furthermore, the image data processing platform can perform optical character recognition to obtain keywords based on the text on the outer packaging of the potato chips in the display area. The image data processing platform then finds object information such as the name or bar code of the target object (bag potato chips) in the database according to the keywords.

The image data processing platform recognizes the obtained first screenshot and second screenshot respectively, and obtains the object information and quantity change value of the target object. According to the different times determined by the timestamps of the first screenshot and the second screenshot, the change value of the number of target objects between the screenshot times of the two frames is determined. For example, in the first screenshot, there are 10 packets of potato chips in bags, and there are 5 packets of potato chips in bags in the second screenshot, and the quantity change value is 5.

This application further provides a technical solution that can further process the quantity change value. This technical solution can be applied in scenarios such as self-service shopping or commodity storage in unmanned supermarkets. Please refer to FIG. 11, after the step S102 of separately identifying the first screenshot and the second screenshot to obtain the number change value of the target object, the method further includes:

Step S111: Generate a pre-transaction time period according to the time stamps of the first screenshot and the second screenshot.

Step S112: Find the transaction quantity of the target object in the pre-transaction time period in the transaction database according to the object information.

Step S113: Determine whether the transaction quantity and the quantity change value are consistent. If yes, go to step S115; if not, go to step S114.

Step S114: Generate alarm information according to the transaction quantity and quantity change value.

Wherein, the alarm information is used to add the number of transactions to the user or cancel some transactions.

Step S115: No processing is performed.

The image data processing platform can query the transaction database to obtain the number of transactions of the target object during the pre-transaction period according to the object information such as the name of the target object. The pre-transaction time period for querying the transaction quantity may be a time point determined according to the timestamps of the first screenshot and the second screenshot, and then generated according to a certain time range before and after the time point, for example, 10 minutes in this embodiment. A certain time range before and after the time point can be a reasonable transaction time interval obtained after statistics.

The image data processing platform queries the transaction database in the transaction time interval of 10 minutes, and the transaction quantity of the bagged potato chips is 3 packets.

The image data processing platform judges whether the transaction quantity is consistent with the quantity change value. In the example given in this embodiment, the transaction database shows that the transaction volume of bagged potato chips is 3 packs, but the bagged potato chips identified by the image data processing platform has decreased by 5 packs, and the transaction quantity is inconsistent with the quantity change value.

The image data processing platform generates alarm information according to the transaction quantity and quantity change value of the target object. The image data processing platform sends alarm information to the server, and the alarm information is used to add the number of transactions to the user or cancel some transactions. Among them, the user here can obtain user information through login users who enter the camera area, for example, through access card, social software login, etc. In addition, the image data processing platform can also obtain user information by recognizing the user's facial features in the first screenshot and the second screenshot. According to the user information, the difference between the transaction quantity and the quantity change value is added to the user's order, or the number of target objects in the transaction completed by the user is correspondingly cancelled and the corresponding value is returned.

The embodiment of the present application also provides a technical solution, which can generate order information or storage information according to the quantity change value. Please refer to FIG. 12, after the step S112 of searching the transaction number of the target object in the pre-transaction time period in the transaction database according to the object information, the method further includes:

Step S121: Analyze the number of transactions of the target object in multiple pre-trading time periods to obtain a transaction trend.

Step S122: Obtain storage information of the target object in the area corresponding to the image data collected by the photographing device in real time.

Step S123: Generate order information of the target object according to the transaction trend and the storage information.

Wherein, the order information includes object information, quantity information, and location information of the target object.

In the above process, the image data processing platform analyzes the quantity change value of the transaction quantity of the target object in multiple pre-transaction time periods to obtain the transaction trend. The transaction trend can be a period of time in the future; (for example, within a month) changes in the number of target objects.

The image data processing platform obtains the object information of the target object according to the optical character recognition, and obtains the storage information of the target object in the corresponding area of the image data collected by the shooting equipment in real time in the storage system. The storage information can be the number of current target objects or the number of target objects in a certain area nearby.

The image data processing platform generates order information corresponding to the target object based on the above-analyzed transaction trends and the storage information obtained from the storage system. The order information includes the number of target objects, the order generation time, the estimated delivery time, and the geographic location information that needs to be delivered.

In order to use the aforementioned order information for effective application, an embodiment of the present application provides a technical solution, please refer to FIG. 13. Step S123 after the step of generating the order information of the target object according to the transaction trend and the storage information, further includes:

Step S131: Obtain the order information and analyze the location information of the target object.

Wherein, the location information includes display location information and storage location information of the target object.

Step S132: According to the location information, the dispatch system is called to generate a logistics route corresponding to the order information.

Step S133: Send the order information and logistics route to the storage system and the logistics system.

Wherein, the storage system assembles target objects according to the order information, and the logistics system allocates vehicles and delivery time periods corresponding to the number and volume of the target objects according to the order information.

The image data processing platform obtains the order information generated and processed in the order system, and the image data processing platform analyzes the position information corresponding to the target object contained in the order information. For example, the image data processing platform can obtain the delivery address of the target object by parsing the order information, that is, the display location of the target object. The image data processing platform can also obtain the receiving address or warehouse number of the target object by parsing the order information, that is, the storage location information of the target object.

The image data processing platform calls the dispatch system to generate a logistics route corresponding to the order information according to the above-mentioned location information. The dispatch system also generates logistics information for dispatching logistics vehicles and logistics personnel according to the above-mentioned logistics routes. The dispatch system sends the generated logistics information to the logistics system, and the logistics system calls the map application plug-in in the map system to obtain real-time traffic status information, and update or adjust the logistics route. The logistics system obtains the estimated delivery time according to the logistics information and logistics route, and pushes the estimated delivery time to the delivery address of the target object through the image data processing platform.

An embodiment of the present application also provides an image data processing device, including:

In order to overcome the technical drawbacks of easily occupying network transmission resources and storage resources, the volume required to analyze and perform image recognition is large, which severely restricts the processing speed and efficiency of obtaining the display state and missing state of the target object, this application provides a An image data processing device corresponding to an image data processing method, the image data processing device applies an image data processing method, and the above image data processing device at least includes:

A processor for executing the steps of the image data processing method; wherein the image data processing method includes the following steps: acquiring image data collected by a photographing device in real time, and determining collection time points from the image data at predetermined intervals; Perform difference processing between the previous frame picture and the next frame picture at the acquisition time point to filter the difference image; extract the contour line of the difference image, and if the position range of the contour line is in the designated area, obtain the difference image The previous frame picture and the next frame picture are respectively used as the first target frame picture and the second target frame picture; the first target frame picture and the second target frame picture are intercepted according to the position range of the contour line, The first screenshot and the second screenshot are obtained respectively, and the first screenshot and the second screenshot are sent back to the server for processing.

This embodiment also provides a non-volatile computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute an image data processing method; wherein, the The image data processing method includes the following steps: acquiring real-time image data collected by a photographing device, and determining a collection time point from the image data at predetermined intervals; and performing differential processing on the previous frame picture and the next frame picture at the collection time point , Filter to obtain the difference image; extract the contour line of the difference image, if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the difference image, respectively as the first target frame Picture and a second target frame picture; intercept the first target frame picture and the second target frame picture according to the position range of the contour line to obtain a first screenshot and a second screenshot respectively, and combine the first and second screenshots 2. The screenshot is sent back to the server for processing.

An image data processing method provided in an embodiment of the application is to obtain real-time image data collected by a photographing device, and determine the collection time point from the image data at a predetermined interval; One frame of pictures is subjected to difference processing to filter to obtain a difference image; the contour line of the difference image is extracted, and if the position range of the contour line is in a designated area, the previous frame picture and the next frame picture of the difference image are obtained , Respectively as the first target frame picture and the second target frame picture; intercept the first target frame picture and the second target frame picture according to the position range of the contour line, obtain the first screenshot and the second screenshot respectively, and The first screenshot and the second screenshot are returned to the technical solution of the server for processing. An image data processing device and a non-volatile computer-readable storage medium are also provided in the embodiments of the present application. The above technical solution can return the first screenshot and the second screenshot with certain changes in the image under the condition of real-time or continuous monitoring of the target object or specific observation area. The above technical solution can reduce the number of transmitted image data and the size of a single image while basically retaining the validity of the image data, effectively reduce the amount of image data transmission and the volume of image data processing, and improve the display status and lack of acquisition of target objects The processing speed and efficiency of state and other data reduce the demand for storage resources.

Claims

An image data processing method, including:

Acquiring image data collected in real time by the photographing device, and determining collection time points from the image data at predetermined intervals;

Perform difference processing between the previous frame picture and the next frame picture at the acquisition time point, and filter the difference image;

Extract the contour line of the differential image, if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture and the second target frame respectively image;

The first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the first screenshot and the second screenshot are sent back to the server for processing .
The image data processing method according to claim 1, wherein the step of performing difference processing between the previous frame picture and the next frame picture at the collection time point to obtain the difference image by screening comprises:

Perform grayscale processing on the previous frame of picture and the next frame of picture at the collection time point;

Performing a pixel comparison between the previous frame of picture and the next frame of picture at the acquisition time point after the grayscale processing is performed to generate a pixel difference;

When the pixel difference value is greater than the difference threshold value, a difference image is generated according to the pixel difference value.
The image data processing method according to claim 2, wherein the step of acquiring image data collected by a photographing device in real time, and determining collection time points from the image data at predetermined intervals, comprises:

Acquire a real-time video stream collected by a shooting device in real time, parse the real-time video stream to obtain several frames of pictures, and determine a collection time point according to the time and predetermined interval of the real-time video stream;

The step of generating a differential image according to the pixel difference value includes:

Obtaining at least two positioning images according to the pixel difference, wherein the two positioning images are a first positioning image and a second positioning image respectively;

Obtain and perform differential processing on the previous frame picture of the first positioning image and the next frame picture of the second positioning image to obtain a differential image; wherein the first positioning image is prior to the second positioning image in time sequence The positioning image and the second positioning image are the last positioning image in time sequence.
The image data processing method according to claim 2, after the step of extracting the contour line of the difference image, further comprising:

Performing difference processing on the area outside the contour line in the frame picture corresponding to the acquisition time point to obtain the background pixel difference value of the corresponding area;

The difference threshold is adjusted according to the background pixel difference, and the difference threshold is updated; wherein the updated difference threshold is used for the next difference image screening.
The image data processing method according to claim 1, wherein the first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the After describing the steps of returning the first screenshot and the second screenshot to the server for processing, it also includes:

According to the contour lines or feature points of the target object in the first screenshot and the second screenshot, the current position and the original position of the target object are respectively identified; wherein, the current position of the target object is the position of the target object in the second screenshot, The original position of the target object is the position of the target object in the first screenshot;

Generating a moving distance by calculating the position difference of the target object in the first screenshot and the second screenshot;

Generating a sorting instruction when the moving distance is greater than the distance threshold;

The target object is moved from the current position to the original position according to the sorting instruction.
The image data processing method according to claim 1, wherein the first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the After describing the steps of returning the first screenshot and the second screenshot to the server for processing, it also includes:

Obtain the object information of the target object in the area corresponding to the image data collected in real time by the shooting device;

The first screenshot and the second screenshot are respectively identified to obtain the number change value of the target object.
The image data processing method according to claim 6, after the first screenshot and the second screenshot are respectively identified to obtain the number change value of the target object, the method further comprises:

Generate the pre-transaction time period according to the timestamps of the first screenshot and the second screenshot;

Search the transaction database for the number of transactions of the target object during the pre-transaction period according to the object information;

If the transaction quantity is inconsistent with the quantity change value, alarm information is generated according to the transaction quantity and the quantity change value; wherein the alarm information is used to add the transaction quantity to the user or cancel part of the transaction.
The image data processing method according to claim 7, after the step of searching the transaction database of the target object in the pre-transaction time period in the transaction database according to the object information, the method further comprises:

Analyze the number of transactions of the target object in multiple pre-trading time periods to obtain transaction trends;

Obtain the storage information of the target object in the area corresponding to the image data collected in real time by the shooting device;

According to the transaction trend and the storage information, the order information of the target object is generated; wherein the order information includes the object information, quantity information, and location information of the target object.
The image data processing method according to claim 8, after the step of generating the order information of the target object based on the transaction trend and the storage information, further comprising:

Acquiring the order information and analyzing the location information of the target object; wherein the location information includes display location information and storage location information of the target object;

Invoking a dispatch system according to the location information to generate a logistics route corresponding to the order information;

Send the order information and logistics route to the storage system and the logistics system; wherein the storage system assembles the target object according to the order information, and the logistics system allocates the transportation means and the volume of the corresponding target object according to the order information. Delivery time period.
An image data processing device, including:

The acquisition module is used to acquire the image data collected by the shooting device in real time, and determine the collection time point from the image data at a predetermined interval;

The processing module is used to perform difference processing between the previous frame picture and the next frame picture at the collection time point, and filter the difference image;

The execution module is used to extract the contour line of the differential image, and if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture respectively And the second target frame picture;

The interception module is used to intercept the first target frame picture and the second target frame picture according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and return the first screenshot and the second screenshot Pass to the server for processing.
An image data processing device that applies an image data processing method, including at least:

A processor for executing steps of an image data processing method;

Wherein, the image data processing method includes the following steps:

Acquiring image data collected in real time by the photographing device, and determining collection time points from the image data at predetermined intervals;

Perform difference processing between the previous frame picture and the next frame picture at the acquisition time point, and filter the difference image;

Extract the contour line of the differential image, if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture and the second target frame respectively image;

The first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the first screenshot and the second screenshot are sent back to the server for processing .
The image data processing device according to claim 11, wherein the step of performing difference processing between the previous frame picture and the next frame picture at the collection time point to obtain the difference image by filtering comprises:

Perform grayscale processing on the previous frame of picture and the next frame of picture at the collection time point;

Performing a pixel comparison between the previous frame of picture and the next frame of picture at the acquisition time point after the grayscale processing is performed to generate a pixel difference;

When the pixel difference value is greater than the difference threshold value, a difference image is generated according to the pixel difference value.
The image data processing device according to claim 12, the step of acquiring the image data collected by the photographing device in real time, and determining the collection time point from the image data at predetermined intervals, comprises:

Acquire a real-time video stream collected by a shooting device in real time, parse the real-time video stream to obtain several frames of pictures, and determine a collection time point according to the time and predetermined interval of the real-time video stream;

The step of generating a differential image according to the pixel difference value includes:

Obtaining at least two positioning images according to the pixel difference, wherein the two positioning images are a first positioning image and a second positioning image respectively;

Obtain and perform differential processing on the previous frame picture of the first positioning image and the next frame picture of the second positioning image to obtain a differential image; wherein the first positioning image is prior to the second positioning image in time sequence The positioning image and the second positioning image are the last positioning image in time sequence.
The image data processing device according to claim 12, after the step of extracting the contour line of the difference image, further comprising:

Performing difference processing on the area outside the contour line in the frame picture corresponding to the acquisition time point to obtain the background pixel difference value of the corresponding area;

The difference threshold is adjusted according to the background pixel difference, and the difference threshold is updated; wherein the updated difference threshold is used for the next difference image screening.
The image data processing device according to claim 11, wherein the first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the After describing the steps of returning the first screenshot and the second screenshot to the server for processing, it also includes:

According to the contour lines or feature points of the target object in the first screenshot and the second screenshot, the current position and the original position of the target object are respectively identified; wherein, the current position of the target object is the position of the target object in the second screenshot, The original position of the target object is the position of the target object in the first screenshot;

Generating a moving distance by calculating the position difference of the target object in the first screenshot and the second screenshot;

Generating a sorting instruction when the moving distance is greater than the distance threshold;

The target object is moved from the current position to the original position according to the sorting instruction.
The image data processing device according to claim 11, wherein the first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the After describing the steps of returning the first screenshot and the second screenshot to the server for processing, it also includes:

Obtain the object information of the target object in the area corresponding to the image data collected in real time by the shooting device;

The first screenshot and the second screenshot are respectively identified to obtain the number change value of the target object.
The image data processing device according to claim 16, after the first screenshot and the second screenshot are respectively identified to obtain the number change value of the target object, further comprising:

Generate the pre-transaction time period according to the time stamps of the first screenshot and the second screenshot;

Search the transaction database for the number of transactions of the target object during the pre-transaction period according to the object information;

If the transaction quantity is inconsistent with the quantity change value, alarm information is generated according to the transaction quantity and the quantity change value; wherein the alarm information is used to add the transaction quantity to the user or cancel part of the transaction.
The image data processing device according to claim 17, after the step of searching the transaction database for the transaction quantity of the target object in the pre-transaction time period in the transaction database according to the object information, further comprising:

Analyze the number of transactions of the target object in multiple pre-trading time periods to obtain transaction trends;

Obtain the storage information of the target object in the area corresponding to the image data collected in real time by the shooting device;

According to the transaction trend and the storage information, the order information of the target object is generated; wherein the order information includes the object information, quantity information, and location information of the target object.
The image data processing device according to claim 18, after the step of generating the order information of the target object based on the transaction trend and the storage information, further comprising:

Acquiring the order information and analyzing the location information of the target object; wherein the location information includes display location information and storage location information of the target object;

Invoking a dispatch system according to the location information to generate a logistics route corresponding to the order information;

Send the order information and logistics route to the storage system and the logistics system; wherein the storage system assembles the target object according to the order information, and the logistics system allocates the transportation means and the volume of the corresponding target object according to the order information. Delivery time period.
A non-volatile computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute an image data processing method; wherein, the image data processing method includes The following steps:

Acquiring image data collected in real time by the photographing device, and determining collection time points from the image data at predetermined intervals;

Perform difference processing between the previous frame picture and the next frame picture at the acquisition time point, and filter the difference image;

Extract the contour line of the differential image, if the position range of the contour line is in the designated area, obtain the previous frame picture and the next frame picture of the differential image as the first target frame picture and the second target frame respectively image;

The first target frame picture and the second target frame picture are intercepted according to the position range of the contour line to obtain the first screenshot and the second screenshot respectively, and the first screenshot and the second screenshot are sent back to the server for processing .