WO2019200837A1

WO2019200837A1 - Method and system for measuring volume of parcel, and storage medium and mobile terminal

Info

Publication number: WO2019200837A1
Application number: PCT/CN2018/106796
Authority: WO
Inventors: 张帆
Original assignee: 南京阿凡达机器人科技有限公司
Priority date: 2018-04-17
Filing date: 2018-09-20
Publication date: 2019-10-24
Also published as: CN108627092A

Abstract

A method and system for measuring the volume of a parcel, and a storage medium and a mobile terminal. The method for measuring the volume of a parcel comprises: collecting a target image by means of a camera (1003) of a mobile terminal, wherein the target image comprises a plane marker, an upper surface of a parcel to be measured provided with the plane marker, and two side surfaces, adjacent to the upper surface, of the parcel to be measured (S100); carrying out image processing on the target image, so as to recognize the plane marker in the target image (S200); carrying out image edge detection on the target image, so as to identify, in the target image, angle points of the parcel to be measured (S300); and calculating the global coordinates of the angle points according to the plane marker, and a pre-acquired internal reference matrix and angle points of the camera, so as to obtain, according to the global coordinates of the angle points, the volume of the parcel to be measured (S400). By using the camera (1003) built in a mobile terminal to accurately measure, in real time, the volume of a parcel to be measured, the usage scenario is enriched, additional hardware facilities do not need to be added, and costs are saved.

Description

Method, system, storage medium and mobile terminal for measuring package volume

The present application claims the priority of the Chinese patent application filed on Apr. 17, 2018, the application number of which is: 201,810,344, 342, the invention is entitled "a method for measuring the volume of a package, a system, a storage medium and a mobile terminal", the entire contents of which are incorporated in this.

Technical field

The invention relates to the field of computer vision, in particular to a method, a system, a storage medium and a mobile terminal for measuring a package volume.

Background technique

In today's rapid development of the express delivery industry, there are a large number of express parcels to be tested. The billing method of the parcel to be tested includes the method of weight or volume, and the courier will charge the maximum value of the two. Furthermore, the volume size of the package to be tested and the like will follow the single number information entry system of the package to be tested, facilitating subsequent loading and inventory arrangements. Therefore, it is a necessary technique to quickly calculate the volume of the package to be tested.

The invention patent of the application number CN20151088222.6 discloses a volume measurement method based on a depth camera and a system thereof. The patent acquires a depth map containing an object to be measured by a depth camera, and then separates the object from the depth image according to the depth information. The two-dimensional depth image coordinates of the object to be tested are converted into three-dimensional coordinates in the camera coordinate system according to the pre-calibrated camera parameters, and then the size of the object is calculated according to the three-dimensional coordinates of the object to be tested.

This method has the following disadvantages: a. An additional depth camera is required, the depth camera is relatively expensive, and the cost of the courier terminal device (or mobile phone) is relatively low, and if the depth camera is fully configured, the cost is relatively large. b. The depth camera needs to be adapted and connected with the computing module (mobile terminal device such as mobile phone), and the implementation technology is troublesome. c. At present, most of the depth cameras adopt the structure light or TOF scheme. The structure light is easily interfered by strong light and cannot work outdoors. The accuracy of the TOF depth camera is inherently low, and the error is greater under the interference of outdoor strong light.

Summary of the invention

The object of the present invention is to provide a method, a system, a storage medium and a mobile terminal for measuring a package volume, which realizes real-time and accurate measurement of the volume of the package to be tested by using a camera provided by a mobile terminal such as a mobile phone, and enriches the use scenario without adding additional Hardware facilities to save costs.

The technical solution provided by the present invention is as follows:

The invention provides a method for measuring a package volume, which is used for measuring a volume of a package to be tested provided with a planar identifier, comprising the steps of: S100 collecting a target image by a camera of the mobile terminal; the target image comprising the planar identifier, An upper surface of the package to be tested of the planar identifier, and two side surfaces of the package to be tested adjacent to the upper surface; S200 performs image processing on the target image, thereby the target image Identifying the planar identifier; S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image; S400 according to the planar identifier, a pre-acquired camera The inner parameter matrix and the corner point, the world coordinates of the corner point are calculated, so that the volume of the package to be tested is obtained according to the world coordinates of the corner point.

Preferably, before the step S100 collects the target image by the camera of the mobile terminal, the method includes the following steps: S010: calibrating the camera to obtain an internal reference matrix of the camera.

Preferably, the step S200 performs image processing on the target image, so that the identifying the planar identifier in the target image specifically includes the following steps: S210 performs binary segmentation on the target image, and from the segmented Extracting a contour from the binarized image; S220: selecting, according to the common feature of the pre-stored contour identifier, the contour contour having the common feature as the candidate contour; A front view of the candidate contour; S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier.

Preferably, the plane identifier is a square, and the preset contour feature corresponding to the plane identifier pre-stored in the step S220 is a quadrangle; the front view of the candidate contour obtained in the step S230 is specifically: S231 Reading the pixel coordinates of the four vertices of the candidate contour; S232 defining the pixel coordinates of the four vertices after the candidate contour is projectively transformed into a front view as four pre-stored front views of the planar identifier Pixel coordinates of the vertex; S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projection transformation formula (1), respectively, and obtaining the projective transformation matrix M:

Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour.

Preferably, the step S300 performs image edge detection on the target image, so that identifying a corner point of the package to be tested in the target image specifically includes the following steps: S310 performing edge detection processing on the target image to obtain An edge binary image; S320 searches for an edge line corresponding to the edge of the package to be tested from the edge binary image; S330 calculates a position of an intersection of any two edge lines of the edge line to obtain the package to be tested corner.

Preferably, S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, thereby obtaining the volume of the package to be tested according to the world coordinates of the corner point. Specifically include the steps:

S410: calculating, according to the plane identifier and the pre-acquired internal reference matrix of the camera, an outer parameter matrix of a current field of view of the camera; the outer parameter matrix includes a rotation matrix and a translation vector;

S420: acquiring, according to the pixel coordinates of the corner point in the pixel coordinate system, the world coordinates of the corner point in combination with the rotation matrix and the translation vector;

S430 calculates a length, width, and height of the package to be tested according to the world coordinates of the corner point;

S440 calculates the length and width of the volume into a volume formula to obtain the volume of the package to be tested.

Preferably, the step S410 calculates the rotation matrix and the translation vector of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera acquired in advance, and specifically includes the following steps:

S411 establishes a world coordinate system with the midpoint of the planar identifier as an origin;

S412: reading vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

S413, substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotation matrix and translation vector:

Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.

Preferably, the step S420 is performed according to the pixel coordinates of the corner point in the pixel coordinate system, and the world coordinates of the corner point are obtained by combining the rotation matrix and the translation vector.

S421 reads the pixel coordinates of the corner point of the corner point in the pixel coordinate system of the camera;

S422, substituting the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4) to obtain the world coordinates of the corner point; (u, v, 1) represents the homogeneous coordinate of the pixel coordinates of the corner point, (Xw, Yw, Zw) is the world coordinate of the corner point, and (Xw, Yw, Zw, 1) represents the angle Homogeneous coordinates of the world coordinates of the point;

The step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the following steps:

S431 reads the world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively connected with the first corner point to generate a straight line Parallel to the XYZ axis of the world coordinate system, respectively;

S432: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculating a length of the package to be tested; the second corner point and the first The line generated by the corner point connection is parallel to the X axis of the world coordinate system;

S433: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculating a width of the package to be tested; the second corner point and the first corner The line generated by the point connection is parallel to the Y axis of the world coordinate system;

S434, the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point are combined with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector, and are obtained by substituting the formula (4). a height of the package to be tested; a line generated by the connection of the fourth corner point and the first corner point is parallel to a Z axis of the world coordinate system.

In a second aspect, the present invention also provides a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar identifier, the measurement system of the package volume comprising: an image acquisition module, configured to pass through the mobile terminal The camera captures a target image; the target image includes the planar identifier, an upper surface of the package to be tested provided with the planar identifier, and two side surfaces of the package to be tested adjacent to the upper surface;

An image recognition module, configured to perform image processing on the target image acquired by the image acquisition module, thereby identifying the planar identifier in the target image;

An image processing module, configured to perform image edge detection on the target image acquired by the image acquiring module, so as to identify a corner point of the package to be tested in the target image;

a volume measuring module, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module, an internal parameter matrix of a pre-acquired camera, and the corner point, thereby, according to the corner point The world coordinates get the volume of the package to be tested.

Preferably, the measurement system of the package volume further includes: a calibration module, configured to calibrate the camera before acquiring the target image by the image acquisition module to obtain an internal parameter matrix of the camera.

Preferably, the image recognition module includes: a binary segmentation sub-module for performing binary segmentation on the target image acquired by the extraction module; and an outline extraction sub-module for use in the segmented binarized image And extracting a profile; a storage submodule, configured to store a common feature of the outline corresponding to the planar identifier, the planar identifier template; and a determination processing submodule, configured to be configured according to the storage in the storage submodule a common feature of the contour corresponding to the planar identifier, and selecting, from the contour extracted by the shape extraction sub-module, an outline having the common feature as an alternative contour; and a front view obtaining sub-module for acquiring the preparation Selecting a front view of the contour; the identifying submodule, configured to identify, when the front view of the candidate contour is consistent with the planar identifier template prestored by the storage submodule, the image corresponding to the candidate contour is An image of a flat marker.

Preferably, the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle; the front view obtaining sub-module includes: a reading unit for reading Taking pixel coordinates of four vertices of the candidate contour; defining a unit for defining pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as a pre-existing front view of the planar identifier a pixel coordinate of the four vertices; a calculation unit for substituting the pixel coordinates of the four vertices read by the reading unit and the pixel coordinates of the four vertices defined by the defining unit into the following planar projective transformation formula ( 1), find the projective transformation matrix M:

a transforming unit, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit, to obtain an area included in the candidate contour Corresponding front view.

Preferably, the image processing module includes: an edge detection sub-module, configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image; and a line search sub-module for using the edge Searching for the edge line corresponding to the edge of the package to be tested in the edge binary image obtained by the detection sub-module; the corner point acquisition sub-module is configured to calculate any two edges of the edge line found by the line search sub-module The position of the intersection formed by the straight line obtains the corner point of the package to be tested.

Preferably, the volume measurement module includes: an outer parameter matrix acquisition submodule, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera acquired in advance; The outer parameter matrix includes a rotation matrix and a translation vector;

a corner coordinate acquisition submodule, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in a pixel coordinate system, in combination with the rotation matrix and the translation vector;

a length, width and height acquisition submodule, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

a parcel volume acquisition sub-module, configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.

Preferably, the outer parameter matrix obtaining submodule comprises:

a coordinate system determining unit configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

a vertex coordinate reading unit, configured to read vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

An arithmetic unit, configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and obtain the The rotation matrix and translation vector of the camera's current field of view:

Preferably, the corner coordinate acquisition submodule comprises:

a corner pixel coordinate acquiring unit, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

a corner point world coordinate acquiring unit, configured to substitute pixel coordinates of the corner point, an internal parameter matrix of the pre-acquired camera, and a rotation matrix and a translation vector of a current field of view of the camera into the formula (4) The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw, Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

The long and wide height acquisition submodule includes:

a corner point world coordinate reading unit, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points respectively correspond to the first A straight line generated by a pair of corner points is parallel to the XYZ axis of the world coordinate system, respectively;

a length operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the X axis of the world coordinate system;

a width operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point and a line generated by the first corner connection is parallel to a Y axis of the world coordinate system;

a height operation unit, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point, in combination with the pre-acquired internal parameter matrix of the camera, the rotation matrix of the current field of view of the camera, and the translation vector, and substitute the formula into the formula (4) Calculating a height of the package to be tested; a line generated by connecting the fourth corner point to the first corner point is parallel to a Z axis of the world coordinate system.

In a third aspect, the present invention provides a storage medium storing a plurality of instructions that are executed by one or more processors to implement the steps of the method of measuring a package volume of the present invention.

According to a fourth aspect, the present disclosure provides a mobile terminal, including: a processor that implements each instruction; a storage medium that stores a plurality of instructions; wherein: the processor executes an instruction stored by the storage medium to implement a package volume of the present invention The steps of the measurement method.

The method, system, storage medium and mobile terminal for measuring package volume provided by the invention can bring at least one of the following beneficial effects:

1) The present invention provides a planar marker by placing a planar marker on the upper surface of each package to be tested, and then the camera collects a surface including the planar marker surface and any one or more of the surface of the package to be tested adjacent to the surface on which the planar marker is disposed. Video, the target video is provided with a rotation translation between the world coordinate system, the pixel coordinate system and the camera coordinate system to obtain the length, width and height of the package to be tested, thereby calculating the volume of the package to be tested, which is performed by using the camera attached to the mobile terminal. The target video is collected, and the volume of the package to be tested can be measured in real time and accurately.

2) The present invention uses the camera provided by the mobile terminal to acquire the target video, does not require a depth camera for acquisition, and does not require the depth camera to collect the package to be tested, and must be located directly above the package to be tested, so that no complicated arrangement is required, and The camera is protected from strong light and can work outdoors, enriching the use of the scene, eliminating the need for additional hardware and cost savings.

DRAWINGS

The above-described characteristics, technical features, advantages and implementation manners of a package volume measuring method, system, storage medium and mobile terminal will be further described below in a clear and understandable manner with reference to the accompanying drawings.

1 is a flow chart of an embodiment of a method for measuring a package volume of the present invention;

2 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

3 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

4 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

5 is a schematic view showing a corner point of a package to be tested according to a method and system for measuring a package volume according to the present invention;

6 is a schematic diagram of a world coordinate system with a midpoint of a planar marker as an origin of a method and system for measuring a package volume according to the present invention;

Figure 7 is a flow chart showing another embodiment of a method for measuring a package volume of the present invention;

Figure 8 is a flow chart showing another embodiment of a method for measuring a package volume according to the present invention;

9 is a flow chart showing a volume measuring step of a package to be tested in another embodiment of a method for measuring a package volume according to the present invention;

10a is a schematic diagram of a calibration template of a Zhang Zhengyou plane calibration method in another embodiment of a method for measuring a package volume according to the present invention;

FIG. 10b is a schematic diagram showing the detected feature points in the Zhang Zhengyou plane calibration method in another embodiment of the method for measuring the package volume of the present invention; FIG.

11 is a schematic diagram of a camera measuring attitude in another embodiment of a method for measuring a package volume according to the present invention;

Figure 12 is a front elevational view of the rotated quadrilateral profile in another embodiment of the method for measuring the package volume of the present invention;

Figure 13 is a schematic structural view of an embodiment of a package volume measuring system of the present invention;

Figure 14 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

15 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

16 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

17 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

18 is a schematic structural view of another embodiment of a package volume measuring system of the present invention;

Figure 19 is a block diagram showing an embodiment of a mobile terminal for measuring the volume of the package of the present invention.

detailed description

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings according to the drawings without obtaining creative labor, and obtain Other embodiments.

In order to simplify the drawings, only the parts related to the present invention are schematically shown in the drawings, and they do not represent the actual structure of the product. In addition, in order to make the drawings simple and easy to understand, components having the same structure or function in some of the figures are only schematically illustrated, or only one of them is marked. In the present context, "a" means not only "only one" but also "more than one".

This scheme is applicable to the volume measurement of a package to be tested having a rectangular shape such as a rectangular parallelepiped, a cube, or a regular quadrangular prism. For other complicated shapes and irregular shapes, the package cannot be measured. Since the parcels in the express logistics industry are generally rectangular-shaped cartons, wooden frames, etc., they are generally suitable for measuring the volume size of rectangular shaped parcels in the express logistics industry.

A first embodiment of the present invention, as shown in FIG. 1, is a method for measuring a package volume, which is used to measure the volume of a package to be tested provided with a planar marker, including:

S100 collects a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the plane identifier, and the package to be tested adjacent to the upper surface Two side surfaces;

S200 performing image processing on the target image, thereby identifying the planar identifier in the target image;

S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image;

S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinates of the corner point.

Specifically, in the embodiment, the planar identifier refers to a pre-designed pattern having a specific geometric shape and encoding information, which is usually printed on a piece of paper. The target video can be collected by the front camera or the rear camera that is provided by the mobile terminal. When the mobile terminal is used to collect the target video, the camera lens on the mobile terminal needs to be oriented obliquely above the surface of the planar identifier. It is convenient to collect the upper surface of the package to be tested with the planar identifier and any two side surfaces adjacent to the upper surface of the package to be tested provided with the planar identifier. Here, the upper surface is not necessarily the upper surface away from the ground when the package to be tested is placed, but is defined as the surface that can be collected by any camera with a planar marker in the package to be tested when the target image is captured, ie If the planar identifier is disposed on the upper surface of the package to be tested having a rectangular parallelepiped shape away from the bottom surface, any two or more side surfaces can be obtained from the four side surfaces adjacent to the upper surface; The object is disposed on any one of the side surfaces of the rectangular package to be tested, and then any two or more side surfaces can be obtained from one upper surface and the remaining three side surfaces adjacent to the side surface. Here, only an example in which the shape of the package to be tested is a rectangular parallelepiped is exemplified, and other rectangular shaped packages to be tested are all within the scope of the present invention.

In order to acquire the target image in an all-round and complete manner, the invention can collect video through the camera provided by the mobile terminal, perform image processing on the captured video to obtain a target image, and process the target image to obtain an image of the planar marker. The corner point of the package to be tested is calculated according to the plane identifier and the corner point and the internal reference matrix of the pre-acquired camera to obtain the world coordinates of the corner point of the package to be tested, and the volume of the package to be tested is calculated according to the world coordinates of the corner point. The invention uses the camera provided by the mobile terminal to collect the target video, does not need the depth camera to collect, and does not need the depth camera to collect the package to be tested, and must be located directly above the package to be tested, so no complicated arrangement is needed, which enriches Use the scenario, no need to add additional hardware facilities, and save costs. In addition, the present invention does not need to be equipped with a depth camera and a mobile terminal, and when measuring the volume of the user's package to be tested, not only the depth camera and the mobile terminal but also the depth camera and the mobile terminal are used. The real-time adaptive connection, the invention adopts the camera and the processor provided by the mobile terminal, and has the advantages of low cost and low cost, simplifying the step of measuring the volume of the package to be tested, high measurement efficiency, and the volume of the package to be tested by the mobile terminal The built-in processor is automatically completed, the operation is simple, and the detection result is intuitive, reliable, and the measurement accuracy is high. Therefore, the invention can fully measure the volume of the package, is accurate and efficient, and does not need to install other hardware devices, is simple and convenient, and has high user experience.

In a second embodiment of the present invention, before the S100 collects the target image through the camera of the mobile terminal, the method includes the following steps:

S010 calibrates the camera to obtain an internal reference matrix of the camera.

Specifically, the embodiment is an optimized embodiment of the foregoing first embodiment. Compared with the first embodiment, the main improvement is that the step S010 is added to calibrate the camera, that is, in this embodiment. Before measuring the volume of the package to be tested by using the above solution, the camera for collecting the target video needs to be calibrated. Specifically, by calibrating the camera, the internal reference matrix of the camera is obtained, that is, the camera is determined to be imaged. Geometric model. Camera calibration is the pre-preparation stage of the whole system. When a camera is selected for visual measurement, it must be calibrated to obtain the internal reference matrix composed of parameters such as distortion coefficient and focal length of the camera. In the subsequent implementation phase, the camera's internal reference matrix can be used for corresponding visual measurements and calculations. The calibration operation only needs to be performed once.

The invention uses Zhang Zhengyou's checkerboard calibration method to calibrate the camera. The Zhang Zhengyou calibration method calculates the internal reference coefficient of the camera by using images acquired by different cameras relative to different angles of the chessboard, and finally obtains the internal reference matrix of the camera.

The process of this plane calibration method is as follows:

1 print a plane-calibrated template and affix it to a flat surface;

2 taking several template images from different angles;

3 detecting feature points in the image;

4 According to the coordinate position of the feature point in the world coordinate system and its corresponding pixel coordinates in the image, the internal parameter matrix of the camera is obtained by the simultaneous equations. The internal parameter matrix of the camera is a 3x3 matrix K. The specific form is as follows:

Where fx represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the X-axis direction, fy represents the product of the physical focal length F of the camera lens and the size of each element of the imager in the y-axis direction, cx represents the imager The offset of the center from the optical axis in the X-axis direction, cy represents the offset in the Y-axis direction due to the center of the imager imager and the optical axis.

A third embodiment of the present invention is an optimized embodiment of the first and second embodiments, as shown in FIG. 2, a method for measuring a package volume, which is used for measuring a package to be tested provided with a planar marker. Volume, including:

The step S200 performs image processing on the target image, so that the planar identifier is specifically included in the target image;

S210 performs binary segmentation on the target image, and extracts a contour from the segmented binarized image;

S220: selecting, according to a common feature of the pre-stored contour identifiers of the plane identifiers, an outline contour having the common features as an alternative contour from the extracted contour contours;

S230 acquires a front view of the candidate contour;

S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier;

Specifically, this embodiment is an optimized embodiment of the foregoing first or second embodiment. Compared with the foregoing first or second embodiment, the main improvement is that steps S210-S240 are added to how to The target image is processed to identify the screen plane identifier for further refinement. In this embodiment, the target image is binary-divided, and the binary value is that the gray value of the pixel on the image is set to 0 or 255, that is, the entire image is presented with an obvious black-and-white visual effect, and the image has Different regions of special meaning are separated, and these regions do not intersect each other, and each region satisfies the consistency of a particular region. The image acquired by the camera can also be binary-divided using an adaptive threshold. After performing the binary segmentation processing on the target image, it is necessary to identify the planar identifier, since the camera may be photographed differently from different angles (but both the planar marker and the surface including the planar marker and other surfaces can be captured). The target video, therefore, the planar identifier on the captured image video frame does not have to be the front view. Then, it is necessary to extract the planar marker photographed from various angles, and perform edge detection on the planar marker to obtain the common features of the contour of the planar marker, and then store the common feature in advance to facilitate the video capture. The planar identifier is identified after the frame. Specifically, after the video frame image is binary-divided, the contours of the divided images are extracted, and according to the common features of the contours of the planar identifiers, an alternative contour is found, and the contour is polygon-fitted. Discard those non-convex polygons and those that are not quadrilateral. In addition, some additional constraints can be used to eliminate quadrilateral contours that are not likely to be planar marker images, such as the quadrilateral side is significantly smaller than the remaining edges (the shape is relatively thin), the contour perimeter or area is too small, etc., and the rest An eligible contour is an alternative contour that the planar marker may correspond to. For example, if the screen plane identifier we selected is a square, then after the edge detection, the common feature of the outline can be quadrilateral, then we can exclude those non-convex polygons from the extracted outline. And not the outline of the quadrilateral, etc. After the candidate contour is obtained, the next step is to obtain a front view of each candidate contour, that is, to change the candidate contour image area corresponding to the plane identifier into a front view, thereby further determining whether the candidate contour region is a plane identifier. The image corresponding to the object. Specifically, by comparing the front view of each candidate contour with the pre-stored planar identifier template, when the alignment is consistent, it can be determined that the candidate contour is the shape of the planar identifier. The contour, the image corresponding to this alternative contour is the image of the planar marker, and thus the planar marker is identified. The shape of the planar marker may be a rectangle, a diamond or a square or the like.

Preferably, on the basis of the above embodiment, the planar identifier disposed on any surface of the package to be tested is a square, and the plane pre-stored in the common feature of the contour contour corresponding to the pre-stored planar identifier A common feature of the contour corresponding to the identifier is that the contour corresponding to the planar identifier is a quadrangle. If the set flat markers are different, the common features of the corresponding outlines will be different. When selecting a planar marker, we should try to select a flat marker that is easy to identify and has obvious feature points. For example, a square has four vertices. Since the four sides of the square are the same, adding four vertices is more conducive to subsequent positioning operations, making the operation simpler.

A fourth embodiment of the present invention, as shown in FIG. 3, is an optimized embodiment of the third embodiment, a method for measuring a package volume, which is used to measure the volume of a package to be tested provided with a planar marker, including :

S230 acquires a front view of the candidate contour;

The step S230 obtains a front view of the candidate contour, which is specifically:

S231 reading pixel coordinates of four vertices of the candidate contour;

S232, the pixel coordinates of the four vertices after the projective transformation into the front view are defined as the pixel coordinates of the pre-stored four vertices of the front view of the planar identifier;

S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projective transformation formula (1), respectively, and obtaining the projective transformation matrix M:

S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour. S240, when the front view of the candidate contour is consistent with the pre-stored planar identifier template, identifying an image corresponding to the candidate contour as an image of the planar identifier;

Specifically, this embodiment is an optimized embodiment of the foregoing third embodiment. Compared with the foregoing third embodiment, the main improvement is that steps S231-S234 are added to face how to obtain an alternative contour. The figure is further refined. This embodiment continues the example in which the planar identifier in the above embodiment is square. Continuing to illustrate, according to the square characteristic of the square planar identifier, for example, we can define the four contours after the candidate contour is converted into the front view by the projective transformation. The pixel coordinates, such as the four vertex pixel coordinates of the front view of the square planar identifier are (0, 0), (0, 100), (100, 100), (100, 0), then these four The vertex coordinates are used as the pixel coordinates of the four vertices of the quadrilateral candidate contour. In addition, the pixel coordinates of the original image of the quadrilateral candidate contour can be read from the image. Through the correspondence of the four sets of vertices, the simultaneous equations can be used to solve the photography. Transforming matrix M, by which all pixels of the quadrilateral candidate contour region can be subjected to projective transformation, and a front view corresponding to the region of the quadrilateral candidate contour can be obtained. The size of the front view image is 100×100. , to obtain a front view corresponding to the area of the quadrilateral alternative contour, and then compare the front view with the pre-stored planar identifier template To see if consistent, consistent, then it indicates that this is the outline of the plane quadrilateral outline marker, then the marker will be identified plane out. Here, the front view is selected for comparison because the image of the planar marker photographed by the camera from various angles is different. If the camera is directly above the planar marker, the image of the planar marker is also a rectangle, if the camera is in the plane logo The object is diagonally above, then the image of the rectangular flat marker is an irregular quadrilateral. The purpose of obtaining the front view for comparison is that the front view of the planar marker is viewed from the top directly regardless of the angle from which the camera looks. The image, in this way, facilitates the identification of flat markers.

The fifth embodiment of the present invention, as shown in FIG. 4, is an optimized embodiment of the first embodiment, the second embodiment, the third embodiment, or the fourth embodiment, and includes:

S400 calculates a world coordinate of the corner point according to the plane identifier, an internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain a volume of the package to be tested according to the world coordinate of the corner point;

The S300 performs image edge detection on the target image, so that the corner point of the package to be tested is specifically included in the target image;

S310 performing edge detection processing on the target image to obtain an edge binary image;

S320: searching for the edge line corresponding to the edge of the package to be tested from the edge binary image;

S330 calculates a position of an intersection of any two edge lines in the edge line to obtain a corner point of the package to be tested.

Specifically, this embodiment is an optimized embodiment of any one of the foregoing first to fourth embodiments, and the present embodiment is mainly compared with any one of the foregoing first to fourth embodiments. The improvement is that the corner points of how to detect the package to be tested are further refined by steps S310-S330. In this embodiment, edge detection processing is performed on the target image to obtain an edge binary image, and a Canny edge detection algorithm can be used, that is, a Gaussian filter is used to smooth the target image to detect a significant edge in the target image, and the first-order partial guide is limited. The difference is used to calculate the amplitude and direction of the gradient, the non-maximum suppression of the gradient amplitude is used, and the edge is detected and connected by the double threshold algorithm. Since the edge of the package to be tested is obviously contrasted with the background, the edge detection algorithm can highlight these The apparent edge of the object, the process will get a binary image containing the edge of the package to be tested in the image.

After the edge detection processing of the target image is performed to obtain the edge binary image containing the edge of the package to be tested in the image, the Hough transform can be used to detect the straight line to find all the existing lines from the edge binary image, and the Hough transform detects the edge binary image. The method of the existence of the straight line is that the equation of the straight line in the polar coordinate system is the following formula (5);

ρ=xcosθ+ysinθ (5)

For a certain (xi, yi), a sine curve ρ=xicosθ+yisinθ can be calculated, and for the points in the binarized edge image, different sinusoids can be obtained. According to different θ, a series of calculations can be obtained. ρ, according to the obtained (ρ, θ), establish a two-dimensional array of accumulator cells, wherein the size of the cell is fixed, and if the corresponding (ρ, θ) value falls within a certain unit, the accumulation of the cell The counter is incremented by one, and the corresponding unit whose final accumulator value exceeds a predefined threshold T is considered to represent a straight line. The Hough transform method for detecting straight lines has been discussed extensively and will not be discussed in depth here.

The Hough transform detection line is also a preferred method. Other methods of detecting the line can be used to obtain an edge binary image containing the edge of the package to be tested in the image. For example, the Hough line detection algorithm finds a line in the edge binary image. . The Hough line detection algorithm finds lines based on obvious edges, and small or cluttered edges in binary images are automatically filtered out.

The specific steps of the Hough line detection algorithm for detecting straight lines are as follows:

1. Randomly extract a feature point in the image, that is, an edge point. If the point has been calibrated to be a point on a certain line, continue to randomly extract an edge point in the remaining edge points until all edge points are After the extraction is completed;

2. Perform a Hough transform on the point and perform an accumulation and calculation;

3, select the point with the largest value in the Hough space, if the point is greater than the threshold, proceed to step 4, otherwise return to step 1;

4. According to the maximum value obtained by the Hough transform, from this point, the displacement is along the direction of the straight line, thereby finding the two end points of the straight line;

5. Calculate the length of the line. If it is greater than a certain threshold, it is considered to be a good straight line output. Go back to step 1.

Since the edge of the package to be tested is relatively obvious and long, the edge line corresponding to the edge of the package to be tested can be detected. As shown in FIG. 5 and FIG. 6, the position of the intersection of any two edge lines in the edge line is calculated. The corner points A1, A2, A3, and A4 of the package to be tested may be obtained, and may also include other corner points A5, A6, and A7 of the package to be tested, but only A1, A2, and A3 are calculated based on the volume measurement efficiency of the package to be tested. , A4 can be. Of course, if the calculation of the volume of the package to be tested is correct, you can use A3, A5, A6, A7 to calculate again, and compare the difference between the volume of the package to be tested calculated by A1, A2, A3, and A4. If it is less than the preset difference range, if any volume calculation value can be selected as the volume measurement value of the package to be tested, otherwise, the measurement package needs to be re-tested for video acquisition calculation. The invention adopts edge image acquisition, line detection, determination of candidate polygons and verification of candidate polygons, eliminating the cumbersome neural network training process and sliding operation, and directly detecting the edge line of the package to be tested in the target image, which is simpler and more intuitive. The detection efficiency is improved.

A sixth embodiment of the present invention, as shown in FIG. 7 and FIG. 8, is an optimized embodiment of the first embodiment, the second embodiment, the third embodiment, or the fourth embodiment, including

The S400 calculates the world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the camera acquired in advance, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinate of the corner point. include:

Specifically, the step S410 is to calculate an outer parameter matrix of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera obtained in advance; the outer parameter matrix includes a rotation matrix and a translation vector, and specifically includes steps As shown in Figure 8:

Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. An offset of the center of the imager from the optical axis in the Y-axis direction; R represents a rotation matrix of the world coordinate system to the camera coordinate system, and T represents a translation vector of the world coordinate system to the camera coordinate system;

For example, taking the center O1 of the plane identifier S as the origin of the world coordinate system, and the world coordinate system is also the plane identifier coordinate system of the plane identifier S, then the Z-axis coordinates of the point on the plane where the plane identifier S is located Is 0. If the side length of the plane identifier S is P, the coordinates of the four vertex plane identifiers S coordinate system O1 are (-P/2, -P/2, 0), (P/2, -P/, respectively). 2,0), (P/2, P/2, 0), (-P/2, P/2, 0).

According to the pinhole camera model, if the coordinates of a point P in the camera coordinate system are Pc=(Xc, Yc, Zc), then there is a formula (2, pixel coordinates) (u, v) on the imaging plane of the camera. ) The relationship listed, let P be the world coordinate system Pw = (Xw, Yw, Zw), then P point in the world coordinate system coordinates Pw and camera coordinate system coordinates Pc = (Xc, Yc, Zc) There is a conversion relationship listed in equation (3), where 3x3 matrix R = (r1, r2, r3), matrix R is a rotation matrix from the world coordinate system to the camera coordinate system, and T is from the world coordinate system to the camera coordinates The translation vector of the system, combining the formula (2) with the formula (3), can obtain the conversion equation of the coordinate Pw of the point P in the world coordinate system and the pixel coordinates (u, v) on the corresponding image plane ( 4), since the center point of the selected plane identifier S is the origin to establish a world coordinate system, there is a formula (4) corresponding to any of the four vertices (Xw, Yw, 0) of the plane identifier S. Equation. Since the four vertices have a Z-axis coordinate Zw=0 in the world coordinate system, and the matrix R=(r1, r2, r3), the equation (4) is simplified to obtain the following equation (6):

The above equation can be written by a vertex pixel coordinate of the plane identifier S and its corresponding coordinate in the plane identifier coordinate system. Four equations can be written by the four vertices of the plane identifier S, and then through the direct linear transformation (DLT) algorithm of the equations, r1, r2, r3 and T are obtained. Wherein, the rotation matrix R is an orthogonal matrix, and the column vectors r1, r2, and r3 are mutually orthogonal unit vectors. When r1 and r2 are obtained, r3 can be obtained from the vector product of r1 and r2, thereby vertex pixel coordinates and vertices. When the world coordinates are known, the internal parameter matrix of the camera obtained in advance is substituted into the following formula (2) and formula (3), and the rotation matrix and translation of the current field of view of the camera can be calculated by combining formula (4). vector.

Specifically, the step S420, according to the pixel coordinates of the corner point in the pixel coordinate system, acquiring the world coordinates of the corner point in combination with the rotation matrix and the translation vector, specifically includes the steps, as shown in FIG. 8 :

As shown in FIG. 5 and FIG. 6, the pixel coordinates (u1, v1) of the corner point A1 in the pixel coordinate system are known, and then substituted into the formula (4), since the internal reference matrix K of the camera is known, in addition, S411-S413 has obtained the rotation matrix R and the translation vector T, then the unknowns of the matrix equations are X _w , Y _w , Z _c , and the world of the corner point A1 in the world coordinate system can be obtained by solving the matrix equations. The coordinates (X _w1 , Y _w1 , 0) can be similarly extracted according to the corresponding pixel coordinates (ui, vi) and i∈N(i≧2) of the corner points A2, A3, and A4 in the pixel coordinate system. The respective corner points of the package to be tested correspond to world coordinates (X _wi , Y _wi , 0), i ∈ N (i ≧ 2) on the world coordinate system.

Specifically, the step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the steps, as shown in FIG. 8 :

Specifically, in this embodiment, the world coordinates corresponding to any four corner points are first read, and any one of the four corner points read is used as the first corner point, and the remaining three corner points are respectively associated with the first corner. The straight lines generated by the two-two connection are respectively parallel to the XYZ axis of the world coordinate system, and the line generated by the connection of the second corner point and the first corner point is parallel to the X-axis of the world coordinate system, and the second corner point and the first corner point The line generated by the connection is parallel to the Y-axis of the world coordinate system, and the line formed by the connection of the fourth corner point and the first corner point is parallel to the Z-axis of the world coordinate system. Since the world coordinates of the respective corner points have been calculated through steps S421-S422, as shown in FIG. 5, it is assumed that the first corner point is A1, the second corner point is A3, the third corner point is A2, and the fourth corner point is A4. Then the world coordinates of the first corner point A1 are (X _w1 , Y _w1 , 0), the world coordinates of the second corner point A3 are (X _w3 , Y _w3 , 0), and the world coordinates of the third corner point A2 are (X _w2 , Y _w2 , 0), the world coordinate of the fourth corner A4 is (X _w4 , Y _w4 , 0),

The length of the package to be tested, L=|A ₁ A ₃ |, and the width of the package to be tested, W=|A ₁ A ₂ |, can be obtained by the two-point distance formula in the three-dimensional space as follows.

Since the straight line A ₁ A ₄ is parallel to the Z-axis direction of the world coordinate system, the Z-axis coordinate of the corner point A4 is the height H, and the X-axis coordinate and the Y-axis coordinate of the corner point A4 are the same as the corner point A1, that is, X _w4 =X _W1 , Y _w4 =Y _w1 . Knowing the pixel coordinates (u4, v4) of the corner point A4 in the image, it is substituted into the formula (4). Since the internal parameter matrix of the camera is known, the rotation matrix R and the translation vector have been obtained through the steps S411-S413. T, then the unknowns of the matrix equations are Z _w , Z _c . Obviously, simply substituting X _w4 or Y _w4 into a value yields an unknown by solving the matrix equations. Preferably, Zw is calculated by substituting Xw4 and Yw4 twice, and then the average value of the two is taken as the height H, thereby improving the accuracy and reliability of the measurement of the height of the package to be tested. Through the above calculation of the length, width and height, since the package is a rectangular parallelepiped or a cube, the volume of the package V = L × W × H can be directly calculated. Similarly, corner points A5, A6, and A7 can be calculated in the world coordinate system according to the corresponding pixel coordinates (ui, vi) and i∈N(i≧2) of the corner points A5, A6, and A7 in the pixel coordinate system. In the world coordinates, after calculating the world coordinate system corresponding to the corner points A5, A6, A7, the volume of the package to be tested is calculated according to the corner points A3, A5, A6, A7, according to the corner points A3, A5, A6 The length, width and height calculated by A7 are used to verify whether the calculated volume of the package to be tested is reliable, and will not be repeated again.

In addition, since the line generated by the connection of the corner point A1 and the corner point A4 is parallel to the Z-axis direction of the world coordinate system, the distance between the corner point A1 and the corner point A4 can also calculate the height of the package to be tested, that is:

Below, an example is given to illustrate the volume measurement step of the package to be tested as shown in FIG. 9:

S1, camera calibration:

Camera calibration is the pre-preparation stage of the whole system. When a camera is selected for visual measurement, it must be calibrated to obtain the internal reference matrix composed of parameters such as distortion coefficient and focal length of the camera. In the subsequent execution phase, the camera's internal parameter matrix can be used for corresponding visual measurement and calculation, and the calibration operation only needs to be performed once.

1 print a plane-calibrated template and affix it to a plane, as shown in Figure 10a;

2 taking several template images from different angles;

3 detecting feature points in the image; a schematic diagram is shown in Figure 10b;

S2, obtain a video frame:

The plane marker S is placed on the upper surface of the package to be tested of the express measurement package size, and the center O1 of the plane marker S is taken as the origin of the world coordinate system. Since the thickness of the planar marker is negligible, the upper surface of the package is coplanar with the XY plane of the world coordinate system O1, and the package to be tested is a rectangular parallelepiped or a cube, and the height H direction of the package and the Z axis of the world coordinate system O1. The directions are parallel.

As shown in Figure 11, the courier uses the camera of the handheld terminal (or mobile phone) carried by himself to be diagonally above the package to be tested, ensuring that the upper surface of the package to be tested (including the planar marker) and any two and The two side surfaces adjacent to the surface have a total of three surfaces.

S3, quadrilateral detection:

Plane marker identification: This step is used to identify a specific plane identifier in the image and determine the four corner points of the plane identifier. The specific process is as follows: the image acquired by the camera is binary-divided by the adaptive threshold, and then A quadrilateral contour resembling a planar marker is extracted from the binary image, and the image is transformed into a square front view by projective transformation through four vertices of the quadrilateral contour. By analyzing the encoding in the front view image, the planar identifier corresponding to the quadrilateral image is finally determined.

1 quadrilateral contour detection

The process detects a quadrilateral contour that the planar marker in the image may correspond to. The image is binarized by the adaptive threshold method, and then the outline is extracted from the binary image obtained above. Polygon fitting the contours, discarding those non-convex polygons, and those that are not quadrilateral. Some additional constraints are used to eliminate quadrilateral contours that are not likely to be planar marker images, such as the quadrilateral side being significantly smaller than the remaining edges (the shape is relatively elongated), the contour perimeter or area being too small. The remaining eligible contours are the alternative contours that the planar marker may correspond to.

2Get a front view of the quadrilateral outline area

The process is used to change the quadrilateral image area corresponding to the planar identifier into a square front view, thereby further determining whether the quadrilateral area is an image corresponding to the planar identifier. The four vertex pixel coordinates after the quadrilateral contour is converted into a front view by photographic transformation are (0, 0), (0, 100), (100, 100), (100, 0). The coordinates of the four vertex pixels of the quadrilateral contour are known in the original image, and the projective transformation matrix M is satisfied for each set of points:

X is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

Through the four sets of vertex correspondences, the projective transformation matrix M can be solved by the simultaneous equations, and the projective transformation M can be performed on all the pixels of the original image to obtain a front view corresponding to the quadrilateral contour region, and the front view is a square plane. The marker image, that is, the front view image size is 100×100.

Because the image of the planar marker photographed by the camera from various angles is different, if the camera is directly above the marker, the image of the marker is also a rectangle; if the camera is obliquely viewed, the image of the rectangular planar marker is An irregular quadrilateral. The purpose of obtaining a front view in this step is that no matter which angle the camera looks from, the resulting front view is an image viewed from directly above. It facilitates the identification of subsequent planar markers, that is, the normalization of images.

S4, flat marker identification:

As shown in FIG. 12, the front view image of the acquired planar marker is sequentially rotated by 90°, 180°, and 270° to finally obtain four images. The four images obtained above are respectively matched with the image templates of the planar markers. When the matching with any of the sub-images is successful, the quadrilateral contour corresponds to the planar identifier.

S5: Calculate the outer parameter matrix (rotation matrix R and translation vector T)

Calculating the rotation matrix R1 and the translation vector T1 of the world coordinate system O1 with the midpoint of the planar identifier S relative to the camera coordinate system O2:

Taking the center O1 of the plane identifier S as the origin of the world coordinate system, the point Z-axis coordinate on the plane where the plane identifier S is located is 0. If the side length of the plane identifier S is 80 mm, the coordinates of the four vertex plane identifiers S coordinate system O1 are (-40, -40, 0), (40, -40, 0), (40, 40,0), (-40,40,0).

According to the pinhole camera model, if the coordinates of a point P in the camera coordinate system are Pc=(Xc, Yc, Zc), the following relationship exists with the pixel coordinates (u, v) of the point on the imaging plane of the camera:

Let P behave in the world coordinate system as Pw=(Xw, Yw, Zw), then the P point has the following conversion relationship between the coordinate Pw in the world coordinate system and the coordinate Pc=(Xc, Yc, Zc) in the camera coordinate system:

Combining formula (2) with formula (3), we can get the relationship between the coordinates Pw of point P in the world coordinate system and the pixel coordinates (u, v) on the corresponding image plane:

Substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotate matrix and translation vector.

When the coordinate system O1 of the selected plane identifier S is the world coordinate system Ow, the equation of the formula (4) exists for any of the four vertices (Xw, Yw, 0) of the plane identifier S. Since the Z-axis coordinates Zw=0 of the four vertices in the world coordinate system, the formula (4) is simplified to obtain the following equation (6):

The above can be written by the vertex pixel coordinates of a vertex of the planar identifier S in the pixel coordinate system and the vertex world coordinates of the vertex in the world coordinate system, and the four vertices of the planar identifier S can be written out. Four equations, through the direct linear transformation (DLT) algorithm of the equations, find r1, r2, r3 and T. The rotation matrix R=[r1 r2 r3] is an orthogonal matrix, and the column vectors r1, r2, and r3 are mutually orthogonal unit vectors. When r1 and r2 are obtained, r3 can be obtained from the vector product of r1 and r2.

S6, corner detection of the package to be tested:

Find the corner points of the package to be tested in the target image, read the pixel coordinates corresponding to each corner point, and then use the pixel coordinates corresponding to each corner point to measure the length, width and height.

1 Use the Canny edge detection algorithm on the image to detect significant edges in the image. Since the edge of the package to be tested is clearly contrasted with the background, these distinct object edges can be highlighted by the edge detection algorithm. This process will result in a binary image containing the edges of the wrap in the image.

2 Use the Hough line detection algorithm to find the edge line in the edge binary image. The Hough line detection algorithm looks for edge lines based on distinct and long edges, and small or cluttered edges in the binary image are automatically filtered out. Since the edge of the package to be tested is relatively obvious and relatively long, the edge line corresponding to the edge of the package to be tested can be detected.

3 As shown in FIG. 5 and FIG. 6, the position of the intersection point of each edge line corresponding to the edge of the package to be tested is calculated, and the corner points A1, A2, A3, and A4 are obtained, and the corner point is the corner point corresponding to the package.

S7. Calculating the length and width: As shown in FIG. 5 and FIG. 6, the length L of the package to be tested corresponds to the corner point A1 on the upper surface of the package to be tested, and the distance corresponding to A3, and the width W of the package to be tested corresponds to It is the distance corresponding to the corner points A1 and A2 of the upper surface of the package to be tested.

Knowing the pixel coordinates (u1, v1) of the corner point A1 in the image, it is substituted into the formula (5). Since the internal reference of K matrix camera is known, the above steps S411-S413 have been calculated rotation matrix R and translation vector T, the unknowns of the matrix equation _{_{_{X w, Y w, Z c}}} , can be set by solving the matrix equation The coordinates (X _w1 , Y _w1 , 0) of the corner point A1 in the world coordinate system are obtained.

Similarly, the coordinates (X _w2 , Y _w2 , 0), (X _w3 , Y _w3 , 0) of the corner points A2 and A3 in the world coordinate system can be obtained. The length L=|A ₁ A ₃ | and the width W=|A ₁ A ₂ | can be obtained by the two-point distance formula in the three-dimensional space as follows.

S8. Calculating the height: As shown in FIG. 5 and FIG. 6, the known height H=|A ₁ A ₄ | is parallel to the Z-axis direction of the world coordinate system, so the Z-axis coordinate of the corner point A4 is the height H, and the corner point A4 The X-axis coordinate and the Y-axis coordinate are the same as the corner point A1, that is, X _w4 = X _w1 , Y _w4 = Y _w1 .

Knowing the pixel coordinates (u4, v4) of the corner point A4 in the image, it is substituted into the formula (4). Since the internal parameter matrix of the camera is known, the rotation matrix R and the translation vector T have been obtained in the above steps. The unknowns of the matrix equations are Z _w , Z _c . Obviously, simply substituting X _w4 or Y _w4 into a value yields an unknown by solving the matrix equations. Preferably, Zw is calculated by substituting Xw4 and Yw4 twice, and then the average value of the two is taken as the height H, thereby improving the accuracy and reliability of the measurement of the height of the package to be tested.

S9. Calculating the volume: The length, width and height calculated by the above, since the package is a rectangular parallelepiped or a cube, the volume of the package V=L×W×H can be directly calculated.

S10. Determine whether to exit; if not, return to step S2.

The invention utilizes the camera to accurately calculate the length, width and height of the package to be tested in real time in a complicated scene, and can work normally under outdoor strong light, and calculates the size information of the package under the premise of ensuring accuracy. Since the present invention utilizes a camera attached to the handheld terminal device (or mobile phone), it does not require an additional expensive device such as a depth camera, and the cost is saved. In addition, the courier handheld terminal device (or mobile phone) is integrated with a rear camera, and the camera that is provided by the mobile terminal can work normally under the outdoor strong light, so that the real-time calculation can be performed under the premise of ensuring accuracy. The length, width and height of the package are measured, so that the size information of the package to be tested is conveniently and quickly calculated. Since the present invention utilizes a camera on a handheld terminal device (or a mobile phone), there is no need to add extra expensive equipment, which saves cost. In addition, the user can also use the mobile terminal storing the instruction for measuring the package volume measurement method to verify the volume of the package to be tested estimated by the courier, thereby knowing the size information of the package and avoiding the courier overreporting the volume of the package to be tested. Charges generated, reducing unnecessary mailing costs.

According to a seventh embodiment of the present invention, as shown in FIG. 13, a package volume measuring system is applied to measure the volume of a package to be tested provided with a planar marker, including:

The image acquisition module 1000 is configured to collect a target image by using a camera of the mobile terminal, where the target image includes the planar identifier, an upper surface of the package to be tested provided with the planar identifier, and the adjacent to the upper surface The two side surfaces of the package to be tested;

The image recognition module 2000 is configured to perform image processing on the target image acquired by the image acquisition module 1000, thereby identifying the planar identifier in the target image;

The image processing module 3000 is configured to perform image edge detection on the target image acquired by the image acquiring module 1000, so as to identify a corner point of the package to be tested in the target image;

a volume measurement module 4000, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module 2000, an internal parameter matrix of the camera acquired in advance, and the corner point, thereby The world coordinates of the point get the volume of the package to be tested.

Specifically, the embodiment is a system embodiment based on the same inventive concept. Since the principle of solving the problem of the package volume measurement system is similar to the measurement method of the package volume, the implementation of the system can be referred to the implementation of the measurement method of the package volume, and the repetition is performed. I won't go into details here.

An eighth embodiment of the present invention is an optimized embodiment of the seventh embodiment, a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar marker, preferably, a measurement of a package volume The system also includes:

The calibration module 5000 is configured to calibrate the camera before the image acquisition module acquires the target image through the camera, and acquire an internal reference matrix of the camera.

A ninth embodiment of the present invention, as shown in FIG. 14, is an optimized embodiment of the seventh or eighth embodiment, a measurement system for a package volume, which is used for measuring a package to be tested provided with a planar marker. Volume, the main improvement is that the image recognition module 2000 includes:

a binary segmentation sub-module 2100, configured to perform binary segmentation on the target image acquired by the extraction module;

a shape extraction sub-module 2200, configured to extract a contour from the divided binarized image;

a storage sub-module 2300, configured to store a common feature of the outline of the plane identifier, and the plane identifier template;

a determination processing sub-module 2400, configured to select, according to the common features of the contours corresponding to the planar identifiers stored in the storage sub-module 2300, from the contours extracted by the shape extraction sub-module 2200 The contour of the feature is used as an alternative contour;

a front view obtaining sub-module 2500, configured to obtain a front view of the candidate contour;

The identification sub-module 2600 is configured to: when the front view of the candidate profile is consistent with the planar identifier template pre-stored by the storage sub-module 2300, identify an image corresponding to the candidate profile as the planar identifier image.

A tenth embodiment of the present invention, as shown in FIG. 15, is an optimized embodiment of the ninth embodiment, a measurement system for a package volume, which is used for measuring a volume of a package to be tested provided with a planar marker, mainly The improvement is that the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle;

The front view obtaining submodule 2500 includes:

a reading unit 2510, configured to read pixel coordinates of four vertices of the candidate contour;

a defining unit 2520, configured to define pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as pixel coordinates of four pre-existing front views of the planar identifier;

The calculating unit 2530 is configured to substitute the pixel coordinates of the four vertices read by the reading unit 2510 and the pixel coordinates of the four vertices defined by the defining unit 2520 into the following planar projective transformation formula (1), and obtain the same Projective transformation matrix M:

a transforming unit 2540, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit 2530, to obtain the candidate contour inclusion The area corresponds to the front view.

An eleventh embodiment of the present invention, as shown in FIG. 16, is an optimized embodiment of the seventh, eighth or tenth embodiment, a measurement system for a package volume, the main improvement being that the image processing module 3000 includes:

The edge detection sub-module 3100 is configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image;

a line search sub-module 3200, configured to search for an edge line corresponding to the edge of the package to be tested from the edge binary image obtained by the edge detection sub-module 3100;

The corner point acquisition sub-module 3300 is configured to calculate a position of an intersection of any two edge lines of the edge line found by the line search sub-module 3200, to obtain a corner point of the package to be tested.

According to a twelfth embodiment of the present invention, as shown in FIG. 17 and FIG. 18, a measurement system for a package volume is an optimized embodiment of any one of the seventh embodiment to the eleventh embodiment, mainly The improvement is that the volume measuring module 4000 comprises:

An outer parameter matrix obtaining sub-module 4100, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera acquired in advance; the outer parameter matrix includes a rotation matrix and a translation vector ;

a corner coordinate acquisition sub-module 4200, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in the pixel coordinate system, in combination with the rotation matrix and the translation vector;

a length and width acquisition sub-module 4300, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

The package volume acquisition sub-module 4400 is configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.

As shown in FIG. 18, the outer parameter matrix obtaining submodule 4100 includes:

a coordinate system determining unit 4110, configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

a vertex coordinate reading unit 4120, configured to read vertex pixel coordinates of each vertex of the plane identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

The operation unit 4130 is configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combine the formula (4) to obtain the The rotation matrix and translation vector of the current field of view of the camera:

As shown in FIG. 18, the corner coordinate acquisition sub-module 4200 includes:

a corner pixel coordinate acquiring unit 4210, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

a corner point world coordinate obtaining unit 4220, configured to substitute the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4). The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw) , Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

As shown in FIG. 18, the length, width, and height acquisition submodule 4300 includes:

a corner point world coordinate reading unit 4310, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively associated with the The straight lines generated by the two corners of the first corner point are respectively parallel to the XYZ axis of the world coordinate system;

a length operation unit 4320, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second angle a line formed by connecting the point to the first corner point is parallel to an X axis of the world coordinate system;

a width operation unit 4330, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the Y axis of the world coordinate system;

a height operation unit 4340, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector. Formula (4) calculates the height of the package to be tested; the line generated by the connection of the fourth corner point with the first corner point is parallel to the Z axis of the world coordinate system.

A thirteenth embodiment of the present invention is a storage medium storing a plurality of instructions, the plurality of instructions being executed by one or more processors to implement the following steps:

S100 collects a target video through a camera of the mobile terminal, and acquires a target image from the target video; the target image includes the plane identifier, a target to-be-measured package surface provided with the planar identifier, and the target a plurality of surfaces of the package to be tested adjacent to the surface of the package to be tested;

S200 performs image processing on the target image to identify the planar identifier in the target image;

S300 performing image edge detection on the target image, so that an intersection of the package to be tested on the target image is identified in the target image;

S400 calculates a corner point world coordinate on the package to be tested according to the plane identifier, the pre-acquired internal reference matrix, and the intersection point, so as to obtain the volume of the package to be tested according to the corner point world coordinate.

Preferably, on the basis of the above, another embodiment of the storage medium of the present invention, the storage medium stores a plurality of instructions, and the plurality of instructions are executed by one or more processors to implement any of the present inventions. The steps of the embodiment of the method of measuring the package volume.

The steps of the method for measuring the package volume according to the present invention can be referred to the previous method embodiment part, and the repetition is not repeated here.

According to a fourteenth embodiment of the present invention, a mobile terminal includes: a processor that implements each instruction; a storage medium that stores a plurality of instructions; wherein: the processor executes an instruction stored by the storage medium to implement a package volume The steps of the measurement method.

As shown in FIG. 19, the mobile terminal includes: a memory 1001, one or more (only one shown) processor 1002, and a camera 1003. These components communicate with each other through one or more communication bus signal lines.

It can be understood that the structure shown in FIG. 16 is merely illustrative and does not limit the structure of the mobile terminal. The mobile terminal may further include more or less components than those shown in FIG. 16, or have different functions from those shown in FIG. Device. The components shown in FIG. 16 can be implemented in hardware, software, or a combination thereof.

The memory can be used to store software programs and modules, such as the method for measuring the package volume and the program instructions/modules corresponding to the system embodiment in the embodiment of the present invention, and the processor executes various software programs/modules by allowing the software programs/modules stored in the memory. Functional application and data processing, that is, a method/system for measuring the above-described package volume.

The memory can include high speed random access memory and can also include non-volatile memory such as one or more magnetic storage devices, flash memory or other non-volatile solid state memory. The storage medium may be a magnetic disk, an optical disk, a read only memory (ROM), a random access memory (RAM), or the like.

The processor runs various software within the memory, commands various functions of the mobile terminal, and performs data processing. The camera is used to capture video, which is equivalent to the eyes of a mobile terminal, such as a CCD camera.

In the embodiment of the present invention, the method and system for measuring the package volume, the storage medium and the mobile terminal belong to the same concept. On the mobile terminal, the method for measuring the corresponding package volume can be executed by executing the instruction stored in the storage medium by the processor. For the specific implementation process of the method provided in the above, please refer to the previous embodiment of the measurement method of the package volume, and details are not described herein again.

It should be noted that a general tester in the art can understand all or part of the process of implementing the measurement method of the package volume of the embodiment of the present invention, which can be completed by a computer program to control related hardware, and the computer program can be stored in a computer readable state. The storage medium is stored, for example, in a memory of the mobile terminal and executed by at least one processor in the mobile terminal, and may include a flow of a measurement method embodiment such as a package volume during execution.

It should be noted that the above embodiments can be freely combined as needed. The above description is only a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It should be considered as the scope of protection of the present invention.

Claims

A method for measuring a package volume, which is characterized by being used for measuring a volume of a package to be tested provided with a planar marker, comprising the steps of:

S100 collects a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the plane identifier, and the package to be tested adjacent to the upper surface Two side surfaces;

S200 performing image processing on the target image, thereby identifying the planar identifier in the target image;

S300 performing image edge detection on the target image, thereby identifying a corner point of the package to be tested in the target image;

S400 calculates world coordinates of the corner point according to the plane identifier, the internal parameter matrix of the pre-acquired camera, and the corner point, so as to obtain the volume of the package to be tested according to the world coordinates of the corner point.
The method for measuring a package volume according to claim 1, wherein the step S100 comprises the steps of: before the target image is captured by the camera of the mobile terminal:

S010 calibrates the camera to obtain an internal reference matrix of the camera.
The method for measuring a package volume according to claim 1, wherein the step S200 performs image processing on the target image, so that the identifying the planar identifier in the target image comprises the following steps:

S210 performs binary segmentation on the target image, and extracts a contour from the segmented binarized image;

S220: selecting, according to a common feature of the pre-stored contour identifiers of the plane identifiers, an outline contour having the common features as an alternative contour from the extracted contour contours;

S230 acquires a front view of the candidate contour;

S240: When the front view of the candidate contour is consistent with the pre-stored planar identifier template, the image corresponding to the candidate contour is identified as an image of the planar identifier.
The method for measuring a package volume according to claim 3, wherein the planar identifier is a square, and the preset contour feature corresponding to the planar identifier prestored in step S220 is a quadrangle;

The step S230 obtains a front view of the candidate contour, which is specifically:

S231 reading pixel coordinates of four vertices of the candidate contour;

S232, the pixel coordinates of the four vertices after the projective transformation into the front view are defined as the pixel coordinates of the pre-stored four vertices of the front view of the planar identifier;

S233 substituting the pixel coordinates of the four vertices read above and the pixel coordinates of the defined four vertices into the plane projective transformation formula (1), respectively, and obtaining the projective transformation matrix M:

Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

S234 performs projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M, and obtains a front view corresponding to the region included in the candidate contour.
The method for measuring a package volume according to any one of claims 1 to 4, wherein the step S300 performs image edge detection on the target image, thereby identifying the package to be tested in the target image. The corner points specifically include the steps:

S310 performing edge detection processing on the target image to obtain an edge binary image;

S320: searching for the edge line corresponding to the edge of the package to be tested from the edge binary image;

S330 calculates a position of an intersection of any two edge lines in the edge line to obtain a corner point of the package to be tested.
The method for measuring a package volume according to any one of claims 1 to 4, wherein the step S400 calculates the angle according to the plane identifier, the internal parameter matrix of the camera acquired in advance, and the corner point. The world coordinates of the point, so that the volume of the package to be tested is obtained according to the world coordinates of the corner point, specifically including the steps:

S410: calculating, according to the plane identifier and the pre-acquired internal reference matrix of the camera, an outer parameter matrix of a current field of view of the camera; the outer parameter matrix includes a rotation matrix and a translation vector;

S420: acquiring, according to the pixel coordinates of the corner point in the pixel coordinate system, the world coordinates of the corner point in combination with the rotation matrix and the translation vector;

S430 calculates a length, width, and height of the package to be tested according to the world coordinates of the corner point;

S440 calculates the length and width of the volume into a volume formula to obtain the volume of the package to be tested.
The method for measuring a package volume according to claim 6, wherein the step S410 calculates a rotation matrix and a translation of the current field of view of the camera according to the plane identifier and the internal parameter matrix of the camera acquired in advance. The vector specifically includes the steps:

S411 establishes a world coordinate system with the midpoint of the planar identifier as an origin;

S412: reading vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

S413, substituting the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and combining the formula (4) to obtain the current field of view of the camera. Rotation matrix and translation vector:

Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.
The method for measuring a package volume according to claim 7, wherein the step S420 acquires the world of the corner points according to pixel coordinates of the corner point in a pixel coordinate system in combination with the rotation matrix and the translation vector. The coordinates specifically include the steps:

S421 reads the pixel coordinates of the corner point of the corner point in the pixel coordinate system of the camera;

S422, substituting the pixel coordinates of the corner point, the pre-acquired internal parameter matrix of the camera, and the rotation matrix and the translation vector of the current field of view of the camera into the formula (4) to obtain the world coordinates of the corner point; (u, v, 1) represents the homogeneous coordinate of the pixel coordinates of the corner point, (Xw, Yw, Zw) is the world coordinate of the corner point, and (Xw, Yw, Zw, 1) represents the angle Homogeneous coordinates of the world coordinates of the point;

The step S430 calculates the length, width, and height of the package to be tested according to the world coordinates of the corner point, and specifically includes the following steps:

S431 reads the world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points are respectively connected with the first corner point to generate a straight line Parallel to the XYZ axis of the world coordinate system, respectively;

S432: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculating a length of the package to be tested; the second corner point and the first The line generated by the corner point connection is parallel to the X axis of the world coordinate system;

S433: Substituting the world coordinates corresponding to the first corner point and the world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculating a width of the package to be tested; the second corner point and the first corner The line generated by the point connection is parallel to the Y axis of the world coordinate system;

S434, the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point are combined with the internal parameter matrix of the pre-acquired camera, the rotation matrix of the current field of view of the camera, and the translation vector, and are obtained by substituting the formula (4). a height of the package to be tested; a line generated by the connection of the fourth corner point and the first corner point is parallel to a Z axis of the world coordinate system.
A package volume measuring system is characterized in that it is applied to measuring a volume of a package to be tested provided with a planar marker, the measurement system of the package volume comprising:

An image acquisition module, configured to acquire a target image by using a camera of the mobile terminal; the target image includes the plane identifier, an upper surface of the package to be tested provided with the planar identifier, and the to-before adjacent to the upper surface Measuring the two side surfaces of the package;

An image recognition module, configured to perform image processing on the target image acquired by the image acquisition module, thereby identifying the planar identifier in the target image;

An image processing module, configured to perform image edge detection on the target image acquired by the image acquiring module, so as to identify a corner point of the package to be tested in the target image;

a volume measuring module, configured to calculate a world coordinate of the corner point according to the plane identifier recognized by the image recognition module, an internal parameter matrix of a pre-acquired camera, and the corner point, thereby, according to the corner point The world coordinates get the volume of the package to be tested.
The package volume measuring system according to claim 9, further comprising:

And a calibration module, configured to: after the image acquisition module acquires the target image by using the camera, perform calibration on the camera to obtain an internal parameter matrix of the camera.
The package volume measuring system according to claim 9, wherein the image recognition module comprises:

a binary segmentation submodule, configured to perform binary segmentation on the target image acquired by the extraction module;

a shape extraction sub-module for extracting a contour from the segmented binarized image;

a storage submodule, configured to store a common feature of the outline corresponding to the planar identifier, and the planar identifier template;

a judging processing module, configured to select, according to a common feature of the contour contour corresponding to the plane identifier stored in the storage submodule, an outer shape extracted from the outer shape extracting submodule The contour is used as an alternative contour;

a front view obtaining submodule for obtaining a front view of the candidate contour;

And an identifying sub-module, configured to identify, when the front view of the candidate contour is consistent with the planar identifier template pre-stored by the storage sub-module, an image corresponding to the candidate contour as an image of the planar identifier.
The measurement system of the package volume according to claim 11, wherein the planar identifier is a square, and the common feature of the outline corresponding to the planar identifier stored by the storage sub-module is a quadrangle;

The front view obtaining submodule includes:

a reading unit, configured to read pixel coordinates of four vertices of the candidate contour;

a defining unit, configured to define pixel coordinates of the four vertices after the candidate contour is subjected to projective transformation into a front view as pixel coordinates of four pre-existing front views of the planar identifier;

a calculating unit, configured to substitute the pixel coordinates of the four vertices read by the reading unit and the pixel coordinates of the four vertices defined by the defining unit into the following planar projective transformation formula (1), and obtain a projective transformation matrix M:

Where X is the homogeneous pixel coordinate of the candidate silhouette vertex, and the non-homogeneous pixel coordinates are

X' is the homogeneous coordinate of the vertex after the projective transformation, and its non-homogeneous pixel coordinates are

X1, x2, and x3 are elements in X, x1', x2', and x3' are elements in X'; x and y are non-corresponding non-homogeneous pixel abscissas and ordinates; u and v are projections respectively The transformed non-homogeneous pixel abscissa and ordinate;

a transforming unit, configured to perform a projective transformation on all pixels of the binarized image corresponding to the candidate contour according to the projective transformation matrix M obtained by the calculating unit, to obtain an area included in the candidate contour Corresponding front view.
The package volume measuring system according to any one of claims 9 to 12, wherein the image processing module comprises:

An edge detection sub-module, configured to perform edge detection processing on the target image acquired by the extraction module to obtain an edge binary image;

a line search sub-module, configured to search for an edge line corresponding to the edge of the package to be tested from the edge binary image obtained by the edge detection sub-module;

And a corner point obtaining sub-module, configured to calculate a position of an intersection of any two edge lines of the edge line found by the line search sub-module, to obtain a corner point of the package to be tested.
The package volume measuring system according to claim 9, wherein the volume measuring module comprises:

An outer parameter matrix obtaining submodule, configured to calculate an outer parameter matrix of a current field of view of the camera according to the plane identifier and an internal parameter matrix of the camera obtained in advance; the outer parameter matrix includes a rotation matrix and a translation vector;

a corner coordinate acquisition submodule, configured to acquire world coordinates of the corner point according to the pixel coordinates of the corner point in a pixel coordinate system, in combination with the rotation matrix and the translation vector;

a length, width and height acquisition submodule, configured to calculate a length, a width and a height of the package to be tested according to the world coordinates of the corner point;

a parcel volume acquisition sub-module, configured to calculate the volume of the package to be tested by substituting the length, width, and height into a volume formula.
The package volume measuring system according to claim 14, wherein the outer parameter matrix acquisition submodule comprises:

a coordinate system determining unit configured to establish a world coordinate system with the midpoint of the planar identifier as an origin;

a vertex coordinate reading unit, configured to read vertex pixel coordinates of each vertex of the planar identifier in a pixel coordinate system, and vertex world coordinates of each vertex in a world coordinate system;

An arithmetic unit, configured to substitute the vertex pixel coordinates, the vertex world coordinates, and the pre-acquired internal parameter matrix of the camera into the following formulas (2) and (3), respectively, and obtain the The rotation matrix and translation vector of the camera's current field of view:

Wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the target point in the pixel coordinate system of the camera, and (Xw, Yw, Zw, 1) indicates that the target point is in the The homogeneous coordinates of the world coordinates in the world coordinate system, (Xc, Yc, Zc) represents the camera coordinates of the target point in the camera coordinate system of the camera; fx represents the physical focal length F of the camera lens and the imager in the X-axis direction The product of the size of each unit, fy represents the product of the physical focal length F of the camera lens and the size of each unit of the imager in the y-axis direction, cx represents the offset of the center of the imager from the optical axis in the X-axis direction, and cy represents the imaging due to imaging. The center of the imager and the optical axis are offset in the Y-axis direction; R represents the rotation matrix of the world coordinate system to the camera coordinate system, and T represents the translation vector of the world coordinate system to the camera coordinate system.
The package volume measuring system according to claim 15, wherein the corner coordinate acquisition submodule comprises:

a corner pixel coordinate acquiring unit, configured to read pixel coordinates of the corner point in a pixel coordinate system of the camera;

a corner point world coordinate acquiring unit, configured to substitute pixel coordinates of the corner point, an internal parameter matrix of the pre-acquired camera, and a rotation matrix and a translation vector of a current field of view of the camera into the formula (4) The world coordinates of the corner points; wherein (u, v, 1) represents the homogeneous coordinates of the pixel coordinates of the corner points, and (Xw, Yw, Zw) is the world coordinate of the corner points, (Xw, Yw, Zw, 1) represents the homogeneous coordinates of the world coordinates of the corner points;

The long and wide height acquisition submodule includes:

a corner point world coordinate reading unit, configured to read world coordinates corresponding to any four corner points; any one of the four corner points is a first corner point, and the remaining three corner points respectively correspond to the first A straight line generated by a pair of corner points is parallel to the XYZ axis of the world coordinate system, respectively;

a length operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the second corner point into a coordinate two-point distance formula, and calculate a length of the package to be tested; the second corner point a line generated by the connection with the first corner point is parallel to the X axis of the world coordinate system;

a width operation unit, configured to substitute a world coordinate corresponding to the first corner point and a world coordinate corresponding to the third corner point into a coordinate two-point distance formula, and calculate a width of the package to be tested; the second corner point and a line generated by the first corner connection is parallel to a Y axis of the world coordinate system;

a height operation unit, configured to substitute the world coordinates corresponding to the fourth corner point and the pixel coordinates of the corner point, in combination with the pre-acquired internal parameter matrix of the camera, the rotation matrix of the current field of view of the camera, and the translation vector, and substitute the formula into the formula (4) Calculating a height of the package to be tested; a line generated by connecting the fourth corner point to the first corner point is parallel to a Z axis of the world coordinate system.
A storage medium, characterized in that the storage medium stores a plurality of instructions, the plurality of instructions being executed by one or more processors to implement the package volume of any one of claims 1-8 The steps of the measurement method.
A mobile terminal, comprising:

a processor that implements each instruction;

a storage medium that stores a plurality of instructions;

Wherein the processor executes the instructions stored in the storage medium to implement the steps of the method for measuring a package volume according to any one of claims 1-8.