CN111986251A

CN111986251A - Object volume measuring method, device, measuring equipment and storage medium

Info

Publication number: CN111986251A
Application number: CN201910430630.5A
Authority: CN
Inventors: 陈紫荣; 李元伟
Original assignee: SF Technology Co Ltd; Shenzhen SF Taisen Holding Group Co Ltd
Current assignee: SF Technology Co Ltd; Shenzhen SF Taisen Holding Group Co Ltd; SF Tech Co Ltd
Priority date: 2019-05-22
Filing date: 2019-05-22
Publication date: 2020-11-24

Abstract

The embodiment of the application discloses a method and a device for measuring the volume of an object, measuring equipment and a storage medium, wherein the embodiment of the application can acquire the current object image of the object through the measuring equipment and detect the key point position of the object in the current object image; generating a preset reference model according to the position of the key point; if the preset edge of the object in the current object image is superposed with the preset reference model, acquiring a motion parameter of the measuring equipment moving along the object direction; when the motion parameters meet preset conditions, the object image of the object is acquired through the measuring equipment, the volume of the object is calculated based on the object image, and the measuring accuracy and efficiency are improved.

Description

Object volume measuring method, device, measuring equipment and storage medium

Technical Field

The application relates to the technical field of vision measurement, in particular to a method and a device for measuring the volume of an object, measuring equipment and a storage medium.

Background

With the development of science and technology, the volume measurement of more and more objects can be performed not only by contact but also by non-contact. Taking the volume measurement of the box body as an example, in the prior art, the volume of the box body is generally measured manually, and the manual measurement mode of the volume of the box body has the problems of low measurement efficiency, low measurement accuracy and the like; or, the volume of the box body is measured by hardware equipment such as structured light, a laser sensor and the like, and the measurement method has the problems of high measurement cost and the like caused by adding extra hardware, and the measurement accuracy is low.

Disclosure of Invention

The embodiment of the application provides an object volume measuring method, an object volume measuring device, measuring equipment and a storage medium, and the accuracy and the efficiency of object volume measurement can be improved.

In a first aspect, an embodiment of the present application provides an object volume measurement method, including:

acquiring a current object image of an object through a measuring device, and detecting the position of a key point of the object in the current object image;

generating a preset reference model according to the position of the key point;

if the preset edge of the object in the current object image is superposed with the preset reference model, acquiring a motion parameter of the measuring equipment moving along the object direction;

and when the motion parameters meet preset conditions, acquiring an object image of the object through the measuring equipment, and calculating the volume of the object based on the object image.

In some embodiments, the generating a preset reference model according to the key point position includes:

when the detected key point positions of the object in the current object image are in a preset number, constructing an object three-dimensional structure according to the key point positions;

and generating a preset reference model according to the three-dimensional structure of the object.

In some embodiments, said constructing a three-dimensional structure of the object from said keypoint locations comprises:

calculating the side length and the object pose of the object according to the positions of the key points;

acquiring the device pose and the device parameters of the measuring device when the current object image is acquired;

and constructing a three-dimensional structure of the object according to the side length of the object, the position and posture of the equipment and the parameters of the equipment.

In some embodiments, the generating of the preset reference model according to the three-dimensional structure of the object includes:

according to the three-dimensional structure of the object, constructing three sides with equal side length proportion and preset angle values formed by angles between two adjacent sides to obtain a reference model;

acquiring the display side length of the display screen of the measuring equipment, and determining the side length of the quadrilateral region externally connected with the reference model according to the display side length;

and based on the side length of the external quadrilateral area, carrying out scaling processing on the reference model to obtain a preset reference model.

In some embodiments, the acquiring motion parameters of the movement of the measurement device along the object direction includes:

displaying the preset reference model in a display screen of the measuring equipment;

And adjusting the shooting angle of the measuring equipment based on the displayed preset reference model until the side length of the object in the object image captured by the measuring equipment is coincident with the preset reference model, and acquiring the motion parameter of the measuring equipment moving along the object direction.

In some embodiments, before the acquiring, by the measurement device, the object image of the object when the motion parameter satisfies a preset condition, the method further includes:

respectively carrying out Fourier transform on the motion components of the motion parameters in the three-axis direction to obtain frequency and amplitude values in the three-axis direction;

calculating the standard deviation corresponding to the motion components of the motion parameters in the three-axis direction;

and when the frequency in the three-axis direction meets a preset frequency threshold, the amplitude meets a preset amplitude threshold and the standard deviation meets a preset threshold, determining that the motion parameter meets a preset condition.

In some embodiments, after the obtaining of the motion parameter of the measurement device moving along the object direction, the method further comprises:

when the frequency in the three-axis direction does not meet a preset frequency threshold, outputting prompt information for adjusting the movement frequency of the measuring equipment along the object direction;

When the amplitude in the three-axis direction does not meet a preset amplitude threshold value or the standard deviation in the three-axis direction does not meet a first preset threshold value, outputting prompt information that the movement of the measuring equipment along the Z-axis direction is unstable;

when the standard deviation in the three-axis direction does not meet a second preset threshold value, outputting prompt information that the movement amplitude of the measuring equipment along the object direction is small; the first preset threshold and the second preset threshold are included in a preset threshold.

In a second aspect, an embodiment of the present application further provides an object volume measurement device, including:

the detection module is used for acquiring a current object image of an object through measuring equipment and detecting the position of a key point of the object in the current object image;

the generating module is used for generating a preset reference model according to the key point position;

the acquisition module is used for acquiring a motion parameter of the measuring equipment moving along the object direction if a preset edge of the object in the current object image is superposed with the preset reference model;

and the calculation module is used for acquiring an object image of the object through the measuring equipment when the motion parameters meet preset conditions, and calculating the volume of the object based on the object image.

In some embodiments, the generating module comprises:

the construction unit is used for constructing an object three-dimensional structure according to the key point positions when the preset number of the key point positions of the object in the current object image is detected;

and the generating unit is used for generating a preset reference model according to the three-dimensional structure of the object.

In some embodiments, the construction unit is specifically configured to:

In some embodiments, the generating unit is specifically configured to:

In some embodiments, the obtaining module is specifically configured to:

In some embodiments, the object volume measurement device further comprises:

the transformation module is used for respectively carrying out Fourier transformation on the motion components of the motion parameters in the three-axis direction to obtain the frequency and the amplitude in the three-axis direction;

the standard deviation calculation module is used for calculating the standard deviation corresponding to the motion component of the motion parameter in the three-axis direction;

the determining module is used for determining that the motion parameter meets a preset condition when the frequency in the three-axis direction meets a preset frequency threshold, the amplitude meets a preset amplitude threshold and the standard deviation meets a preset threshold.

In some embodiments, the object volume measurement device further comprises:

the output module is used for outputting prompt information for adjusting the movement frequency of the measuring equipment along the object direction when the frequency in the three-axis direction does not meet a preset frequency threshold; and the number of the first and second groups,

When the amplitude in the three-axis direction does not meet a preset amplitude threshold value or the standard deviation in the three-axis direction does not meet a first preset threshold value, outputting prompt information that the movement of the measuring equipment along the Z-axis direction is unstable; and the number of the first and second groups,

In a third aspect, an embodiment of the present application further provides a measurement device, which includes a memory and a processor, where the memory stores program codes, and the processor executes any one of the object volume measurement methods provided in the embodiments of the present application when calling the program codes in the memory.

In a fourth aspect, the present application further provides a storage medium, where the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to perform any one of the object volume measurement methods provided in the present application.

According to the embodiment of the application, the current object image of the object can be acquired through the measuring equipment, and the position of the key point of the object in the current object image is detected; then generating a preset reference model according to the position of the key point so as to collect motion parameters by taking the preset reference model as a reference, and if a preset edge of an object in the current object image is superposed with the preset reference model, acquiring the motion parameters of the measuring equipment moving along the object direction; and when the motion parameters meet preset conditions, acquiring an object image of the object through the measuring equipment, and calculating the volume of the object based on the object image. The scheme can determine the motion parameters of the measuring equipment based on the preset reference model generated by the object image, and the volume of the object is calculated by the object image acquired when the motion parameters meet the preset conditions, so that the volume of the object can be accurately and quickly calculated, the measuring accuracy and efficiency are improved, hardware such as a laser sensor does not need to be added on the measuring equipment, and the measuring cost is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow chart of an object volume measurement method provided in an embodiment of the present application;

FIG. 2 is another schematic flow chart of a method for measuring a volume of an object according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of object volume measurement provided by embodiments of the present application;

FIG. 4 is another schematic diagram of object volume measurement provided by embodiments of the present application;

FIG. 5 is a schematic structural diagram of an object volume measuring device provided in an embodiment of the present application;

fig. 6 is a schematic structural diagram of a measurement apparatus provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Referring to fig. 1, fig. 1 is a schematic flow chart of an object volume measurement method according to an embodiment of the present application. The execution main body of the object volume measurement method may be the object volume measurement device provided in the embodiment of the present application, or a measurement device integrated with the object volume measurement device, where the object volume measurement device may be implemented in a hardware or software manner, and the measurement device may be a mobile measurement device such as a smart phone, a tablet computer, a palm computer, or a notebook computer. The object volume measuring method may include:

s101, acquiring a current object image of an object through a measuring device, and detecting the position of a key point of the object in the current object image.

The object can be a cabinet, a mainframe box, a desk, a box body and the like, and the measuring equipment can acquire the image of the object through a preset camera to obtain the current object image; alternatively, one or more cameras may be arranged around the object in advance, and the measurement device may send a shooting instruction to the set cameras and receive the current object image sent by the cameras and acquired based on the shooting instruction. The current object image may include an object region photographed from an arbitrary angle. The key point can be flexibly set according to actual needs, for example, the key point can be a vertex or a center point of an object.

In order to improve the accuracy of detecting the positions of the key points, after the current object image is obtained, the measuring equipment can detect the positions of the key points of the object in the current object image through the trained network. For example, the keypoint locations of objects in the current Object image may be detected by a lightweight Supervised depth Object detection network (Tiny-DSOD, Tiny-Learning derived objects Detectors from Scatch) and a Convolutional Pose estimation network (CPM). Specifically, a Tiny-digital-to-analog converter (Tiny-DSOD) network and a continuous processing unit (CPM) network can be trained by using an object sample image with a well-labeled key point position to obtain a trained Tiny-digital-to-analog converter (Tiny-DSOD) network and a trained CPM network, then, the collected object image is subjected to convolution operation through the Tiny-digital-to-analog converter (Tiny-DSOD) network to obtain a feature map, and then the CPM network is used for extracting the key point position of an object based on the feature map, wherein the key point position can be a two-.

For another example, the measuring device may train a Convolutional Neural Network (CNN) and a confidence network by using the object sample image labeled with the position of the keypoint to obtain the trained CNN network and the trained confidence network, perform convolution operation on the acquired object image through the CNN network to obtain a feature map, and detect the position of the keypoint of the object based on the feature map through the confidence network; and so on.

And S102, generating a preset reference model according to the position of the key point.

The preset reference model is used for determining the reference position of the box image, the preset reference model can be a Y-shaped model (which can be called as a Y-shaped clamping groove), a Z-shaped model or a square-shaped model, and the preset reference model can be flexibly arranged according to actual needs.

In some embodiments, generating the preset reference model according to the key point position may include: when the detected key point positions of the object in the current object image are in a preset number, constructing a three-dimensional structure of the object according to the key point positions; and generating a preset reference model according to the three-dimensional structure of the object.

In order to improve the accuracy of generating the preset reference model and ensure that the objects contained in the acquired current object image are visible on multiple sides (for example, three sides are visible), at the moment, the measuring equipment can judge whether the positions of the key points of the objects in the detected current object image are preset numbers, and the preset numbers can be flexibly set according to actual needs. For example, when the object is a box, it may be determined that 7 key points of the box in the detected current object image are located, and when 7 key points of the box in the detected current object image are located, it is determined that three sides of the box in the current object image are visible.

If the detected key point positions of the objects in the current object image are not the preset number, the current object image of the objects is collected again until the key point positions of the objects in the current object image are the preset number. If the detected key point positions of the object in the current object image are preset, the three-dimensional structure of the object can be constructed according to the detected key point positions of the object.

In some embodiments, constructing a three-dimensional structure of an object from keypoint locations may comprise: calculating the side length and the object pose of the object according to the positions of the key points; acquiring the device pose and device parameters of the measuring device when acquiring the current object image; and constructing a three-dimensional structure of the object according to the side length of the object, the position and posture of the equipment and the parameters of the equipment.

In order to improve the reliability of the three-dimensional structure construction of the object, the measuring equipment can construct the three-dimensional structure by using a multi-view geometric three-dimensional reconstruction principle, and firstly, the side length of the object and the object pose can be calculated according to the positions of key points, for example, the side length of the object can be the length, the width and the height of the object, and the object pose can be the rotation amount Rb, the translation amount Tb and the like of the object; and acquiring the device pose and the device parameters of the measuring device when acquiring the current object image, for example, the device pose may be a rotation amount Rc and a translation amount Tc of the measuring device, and the device parameters may be camera intrinsic parameters for acquiring the current object image, such as a parameter of a focal length K or the like, or a parameter of a camera intrinsic parameter or the like.

Then, the measuring device can construct a three-dimensional structure of the object according to the side length of the object, the pose of the device and the parameters of the device. For example, the key point position Ps' of the object under any one of the parameters S, Rb and Tb to be solved can be obtained from Rc, Tc and K according to the camera imaging model and the structural constraints of the object:

Ps'＝fp(S、Rb、Tb、Rc、Tc、K)

wherein the parameters include: the length, width and height dimensions S of the object, the rotation amount Rb and the translation amount Tb of the object; known parameters include: the device parameters K, the detected key point position Ps of the object, the rotation amount Rc and the translation amount Tc of the measuring device.

At this time, an objective function L may be constructed: and L is | | | Ps' -Ps | |, wherein Ps is the key point position of the object in the current object image obtained by detection, and at the moment, L is only related to S, Rb and Tb, and the values of S, Rb and Tb can be obtained by optimizing L to be minimum, so that the three-dimensional structure of the object can be obtained.

After the three-dimensional structure of the object is obtained, the measurement device may generate a preset reference model according to the three-dimensional structure of the object.

In some embodiments, generating the preset reference model from the three-dimensional structure of the object may include: according to the three-dimensional structure of the object, constructing three sides with equal side length proportion and preset angle values formed by angles between two adjacent sides to obtain a reference model; acquiring the display side length of a display screen of the measuring equipment, and determining the side length of a quadrilateral region externally connected with the reference model according to the display side length; and based on the side length of the external quadrilateral area, carrying out scaling processing on the reference model to obtain a preset reference model.

For example, taking the preset reference model as a Y-shaped slot as an example, first, three sides with equal proportions of side lengths and angles between two adjacent sides forming a preset angle value (for example, 120 degrees) are constructed according to the three-dimensional structure of the object, so as to obtain a reference Y-shaped slot. Then, the display side length of the display screen of the measuring equipment is obtained, and the side length of the quadrilateral region circumscribed by the reference Y-shaped clamping groove is determined according to the display side length, for example, the maximum side length of the rectangular region circumscribed by the reference Y-shaped clamping groove is determined according to the display side length. And finally, based on the side length of the circumscribed quadrilateral region, reducing or amplifying the reference model to enable the preset Y-shaped clamping groove to be displayed in a display screen of the measuring equipment according to a preset proportion, so that the preset Y-shaped clamping groove can be obtained, and the preset Y-shaped clamping groove is a better Y-shaped clamping groove.

At this time, the preset Y-shaped slot may be displayed in the display screen of the measuring device to prompt the user to adjust the shooting angle, position, and the like, so as to sufficiently coincide the three edges of the object with the preset Y-shaped slot. The Y-shaped clamping groove is generated according to the three-dimensional structure of the object according to the principle that the equal ratio of the side lengths of the three sides of the object and the three sides form 120 degrees with each other in an image, and the Y-shaped clamping groove is displayed in a display screen of measuring equipment. Wherein the equal ratio of the length of the three sides is the ratio of the length of the pixel displayed on the image by the object box body, and is consistent with the calculated ratio of the three sides of the three-dimensional structure of the object.

S103, if the preset edge of the object in the current object image is overlapped with the preset reference model, obtaining the motion parameter of the measuring equipment moving along the object direction.

After the preset reference model is generated, the measuring device may determine whether a preset edge of the object in the current object image coincides with the preset reference model, for example, the measuring device may display the acquired current object image in a display screen and display the generated preset reference model, and then determine whether the preset reference model coincides with an edge of the object in the current object image, which may be determined when a matching degree between the preset reference model and the edge of the object in the current object image is greater than a preset threshold (e.g., 90%).

If the preset edge of the object in the current object image is not coincident with the preset reference model, outputting relevant non-coincident prompt information to prompt a user to adjust the shooting angle of the measuring device, so that the edge of the object is coincident with the preset reference model.

If the preset edge of the object in the current object image coincides with the preset reference model, the motion parameter of the measuring device moving along the object direction can be obtained, wherein the motion along the object direction can be a push-pull motion along the horizontal direction of the measuring device collecting the current object image, or the measuring device aligns to the object direction to perform a push-pull motion, and the motion along the object direction can be flexibly set according to actual needs.

In some embodiments, obtaining the motion parameter of the movement of the measurement device along the direction of the object may include: displaying a preset reference model in a display screen of the measuring equipment; and adjusting the shooting angle of the measuring equipment based on the displayed preset reference model until the side length of the object in the object image captured by the measuring equipment is coincident with the preset reference model, and acquiring the motion parameters moving along the object direction.

In order to adjust the shooting angle of the measuring device conveniently, the measuring device may display the acquired current object image in a display screen, display a preset reference model in the display screen of the measuring device, and output prompt information based on the displayed preset reference model to prompt a user to adjust the shooting angle of the measuring device, or send an adjustment instruction to the controller so as to control the controller to adjust the shooting angle of the measuring device based on the adjustment instruction, for example, adjust the shooting angle of a camera in the measuring device or the shooting position of the measuring device.

If the preset edge of the object in the current object image is superposed with the preset reference model, setting the current object image as a reference object image so as to calculate the volume of the object subsequently, wherein at the moment, the measuring equipment can output push-pull prompt information to prompt a user to start to push-pull the measuring equipment to move; alternatively, the measuring device may send a control instruction to the controller, so as to control the controller to push or pull the measuring device to generate movement based on the control instruction, so that the measuring device moves along the direction of the object; and so on.

At this time, the motion parameters of the movement of the measuring device in the direction of the object may be acquired, and the motion parameters may include acceleration, a direction of the movement, and the like, for example, acceleration on an axis perpendicular to the display screen of the measuring device may be acquired. If the acceleration is larger than a preset threshold value, judging that the measuring equipment starts to do push-pull movement; if the acceleration is larger than the preset threshold value, prompting a user to push and pull the measuring equipment to move, or controlling a controller connected with the measuring equipment to push and pull the measuring equipment to move. It should be noted that, after the measuring device generates the movement, the movement parameter of the measuring device within a preset time period, for example, the movement parameter of the measuring device within 4 seconds or 6 seconds, may be acquired.

And S104, when the motion parameters meet preset conditions, acquiring an object image of the object through the measuring equipment, and calculating the volume of the object based on the object image.

After the motion parameters are obtained, the motion parameters can be analyzed to judge whether the motion parameters meet preset conditions, and the preset conditions can be flexibly set according to actual needs. For example, the acceleration data may be subjected to spectral analysis, statistical analysis, etc. to determine whether the motion of the measurement device meets the requirements for frequency, amplitude, velocity, and/or vertical stability. If the motion parameters meet the preset conditions, the measuring equipment can acquire an object image of the object and record the time when the object image is acquired, the equipment position of the measuring equipment, and information such as operation parameters (such as acceleration) acquired by a sensor or a gyroscope preset in the measuring equipment. If the motion parameters do not meet the preset conditions, the measuring equipment can output corresponding prompt information to prompt a user to adjust the motion of the measuring equipment, so that the user can be helped to quickly improve and master the push-pull motion of the measuring equipment, and the prompt information can be displayed in a display screen through voice broadcast or characters; or, controlling a controller connected with the measuring equipment to adjust the movement of the measuring equipment so that the movement parameters generated by the measuring equipment meet the preset conditions.

In some embodiments, when the motion parameter satisfies a preset condition, before acquiring an object image of the object by the measurement device, the object volume measurement method may further include: respectively carrying out Fourier transform on the motion components of the motion parameters in the three-axis direction to obtain the frequency and the amplitude in the three-axis direction; calculating the standard deviation corresponding to the motion component of the motion parameter in the three-axis direction; and when the frequency in the three-axis direction meets a preset frequency threshold, the amplitude meets a preset amplitude threshold and the standard deviation meets a preset threshold, determining that the motion parameter meets a preset condition.

For example, the motion parameters include acceleration, Fast Fourier Transform (FFT) may be performed on acceleration components of the acceleration in three axes XYZ, to obtain three axes of frequencies and corresponding amplitudes, and maximum frequencies Fx, Fy, and Fz and corresponding amplitudes Px, Py, and Pz may be screened out from the three axes of frequencies and corresponding amplitudes. And calculating a standard deviation corresponding to the acceleration component of the acceleration in the three-axis direction to obtain Dx, Dy and Dz.

Then, whether the maximum frequency in the three-axis direction meets a preset frequency threshold value, whether the maximum amplitude meets a preset amplitude threshold value, and whether the standard deviation meets a preset threshold value are judged. When the frequency in the three-axis direction meets a preset frequency threshold, the amplitude meets a preset amplitude threshold and the standard deviation meets a preset threshold, determining that the acceleration meets a preset condition; and when one of the maximum frequency, the maximum amplitude and the standard deviation is not satisfied, determining that the acceleration does not satisfy the preset condition.

For example, if Ft1< Fz < Ft2 is satisfied (for example, Ft1 is 0.5, Ft2 is 2), it indicates that the maximum frequency in the three-axis direction satisfies the preset frequency threshold (i.e., the preset frequency threshold is Ft1 to Ft 2); if Pz/Py > Tp, Pz/Px > Tp (for example, Tp is 2), it indicates that the maximum amplitude in the three-axis direction satisfies the preset amplitude threshold (i.e., the preset amplitude threshold is Tp); if Dz/Dy > Td, Dz/Dx > Td (for example, Td equals 5), and Dz > Dt (for example, Dt equals 0.03), it means that the standard deviation in the three-axis direction satisfies the predetermined threshold (i.e., the predetermined threshold is Td and Dt).

In some embodiments, after acquiring the motion parameter of the measuring device moving along the object direction, the object volume measuring method may further include: when the frequency in the three-axis direction does not meet a preset frequency threshold, outputting prompt information for adjusting the movement frequency of the measuring equipment along the object direction; when the amplitude in the three-axis direction does not meet a preset amplitude threshold value or the standard deviation in the three-axis direction does not meet a first preset threshold value, outputting prompt information that the movement of the measuring equipment along the Z-axis direction is unstable; when the standard deviation in the three-axis direction does not meet a second preset threshold value, outputting prompt information that the movement amplitude of the measuring equipment along the object direction is small; the first preset threshold and the second preset threshold are included in the preset threshold.

For example, if Ft1< Fz < Ft2 is not satisfied (for example, Ft1 is 0.5, Ft2 is 2), it indicates that the maximum frequency in the three-axis direction does not satisfy the preset frequency threshold, and a prompt message for adjusting the frequency of the movement of the measuring device along the object direction is output to prompt the user to adjust the push-pull frequency. The prompt message can be displayed in a display screen through voice broadcast or characters.

If the Pz/Py > Tp and the Pz/Px > Tp are not satisfied (for example, Tp is 2), the maximum amplitude in the three-axis direction does not satisfy the preset amplitude threshold value, and at this time, prompt information that the movement of the measuring equipment along the Z-axis direction is unstable is output to prompt a user to push or pull the Z-axis to be unstable.

If the Dz/Dy > Td and the Dz/Dx > Td are not met (for example, the Td is 5), the standard deviation in the three-axis direction does not meet a first preset threshold, and at the moment, a prompt message that the movement of the measuring equipment along the Z-axis direction is unstable is output to prompt a user to push or pull the Z-axis to be unstable; if Dz > Dt is not satisfied (for example, Dt is 0.03), it indicates that the standard deviation in the three-axis direction does not satisfy the second preset threshold, and at this time, a prompt message that the movement amplitude of the measuring device along the object direction is small is output to prompt the user that the push-pull movement amplitude is not enough, so that the user can effectively, i.e., quickly adjust the movement of the measuring device according to the prompt message.

When the motion parameter satisfies the preset condition, the volume of the object can be calculated based on the object image after the measuring device collects the object image of the object. For example, two-dimensional coordinate positions of key points of an object in an object image can be acquired, and the connection relation among the key points can be determined; acquiring the position of the detection equipment when the detection equipment acquires the object image; then determining the three-dimensional coordinate position of the key point of the object according to the two-dimensional coordinate position and the equipment position; calculating the motion parameters of the detection equipment according to the position of the equipment and the time when the detection equipment acquires the object image; and calculating the volume of the object according to the motion parameters, the three-dimensional coordinate position and the connection relation among the key points.

The object volume measuring method according to the above embodiment will be described in further detail below.

Referring to fig. 2, fig. 2 is another schematic flow chart of an object volume measurement method according to an embodiment of the present disclosure. The object volume measuring method can be applied to a measuring device, and the following takes the measuring device to measure a regular box volume as an example, as shown in fig. 2, the flow of the object volume measuring method can be as follows:

s201, the measuring equipment collects a current box body image of the box body and displays the current box body image in a display screen of the measuring equipment.

The measuring equipment can acquire an image of the box body through a camera of the measuring equipment to obtain a current box body image; or one or more cameras can be arranged around the box in advance, the measuring equipment can send a shooting instruction to the set cameras and receive a current box image which is sent by the cameras and acquired based on the shooting instruction, and at the moment, the measuring equipment can display the current box image in a display screen so as to be convenient for a user to view. Wherein, this box can be regular square or cuboid etc. and the shooting angle of box can carry out nimble setting according to actual need, optionally, can adjust shooting angle for box trilateral is visible in the current box image.

S202, detecting the vertex position of the box body in the current box body image by the measuring equipment.

In order to improve the accuracy of vertex position detection, the measuring equipment can detect the vertex position of the box body in the current box body image through a Tiny-DSOD network and a CPM network. For example, a Tiny-DSOD network and a CPM network can be trained by using the box sample image with the marked vertex position to obtain the trained Tiny-DSOD network and the trained CPM network, then, the collected box image is subjected to convolution operation through the Tiny-DSOD network to obtain a feature map, and the vertex position of the box is extracted through the CPM network based on the feature map. For another example, the measurement device may train the CNN network and the confidence level network by using the box sample image with the labeled vertex position to obtain the trained CNN network and the trained confidence level network, then perform convolution operation on the acquired box image through the CNN network to obtain a feature map, and detect the vertex position of the box based on the feature map through the confidence level network; and so on.

For example, as shown in FIG. 3, the measurement device may detect 7 vertex positions of the box in the current box image.

S203, judging whether the number of the detected vertex positions is seven by the measuring equipment; if yes, go to step S204; if not, step S201 is executed.

In order to improve the accuracy of generating the Y-shaped clamping grooves, the measuring equipment can judge whether the number of the detected vertex positions is seven, and when the number of the detected vertex positions of the box in the current box image is 7, the fact that three sides of the box in the current box image are visible is shown. And if the number of the detected vertex positions is less than seven, the measuring equipment acquires the current box body image of the box body again until the number of the vertex positions of the box body in the current box body image is seven. If the number of the detected vertex positions is seven, the measuring device can construct a three-dimensional structure of the box body according to the vertex positions.

And S204, constructing a three-dimensional structure of the box body by the measuring equipment according to the vertex position.

For example, in order to improve the reliability of the three-dimensional structure construction of the box body, the measuring device may construct the three-dimensional structure by using the multi-view geometric three-dimensional reconstruction principle, and first, the side length of the box body and the box body pose may be calculated according to the vertex position, for example, the side length of the box body may be the length, width and height dimension S of the box body, and the box body pose may be the rotation amount Rb, the translation amount Tb, and the like of the box body; and acquiring the device pose and the device parameters of the measuring device when acquiring the current box body image, wherein the device pose can be the rotation amount Rc and the translation amount Tc of the measuring device, and the device parameters can be camera intrinsic parameters for acquiring the current box body image, such as parameters of a focal length K and the like or camera intrinsic parameters and the like.

Then, the measuring equipment can construct a three-dimensional structure of the box body according to the side length of the box body, the position and the posture of the equipment and the parameters of the equipment. For example, according to the camera imaging model and the structural constraints of the box, the vertex positions Ps' of the box under any one of the parameters S, Rb and Tb to be determined can be obtained from Rc, Tc and K, and at this time, the objective function L can be constructed: and L is | | | Ps' -Ps | |, wherein Ps is the vertex position of the box in the current box image obtained by detection, and at the moment, L is only related to S, Rb and Tb, and the values of S, Rb and Tb can be obtained by optimizing the minimum L, so that the three-dimensional structure of the box can be obtained.

S205, the measuring equipment generates a Y-shaped clamping groove according to the three-dimensional structure of the box body.

After the three-dimensional structure of the box body is obtained, the measuring equipment can construct three sides with equal proportion between side lengths and 120-degree angle formed by angles between two adjacent sides according to the three-dimensional structure of the box body, and then the reference Y-shaped clamping groove is obtained. And then, obtaining the display side length of the display screen of the measuring equipment, and determining the maximum side length of the external rectangular area of the reference Y-shaped clamping groove according to the display side length. And finally, based on the maximum side length of the circumscribed rectangular region, reducing or amplifying the reference model to enable the preset Y-shaped clamping groove to be displayed in a display screen of the measuring equipment according to a preset proportion, so that the preset Y-shaped clamping groove can be obtained, and the preset Y-shaped clamping groove is the better Y-shaped clamping groove.

S206, the measuring equipment displays the Y-shaped card slot in a display screen of the measuring equipment.

At this moment, the measuring device can display the preset Y-shaped clamping groove in a display screen of the measuring device to prompt a user to adjust shooting angles, positions and the like, so that the three sides of the box body can be sufficiently coincided with the preset Y-shaped clamping groove. The Y-shaped clamping groove is generated according to the three-dimensional structure of the box body according to the principle that the equal ratio of the side lengths of the three sides of the box body and the three sides form 120 degrees with each other in an image, and the Y-shaped clamping groove is displayed in a display screen of measuring equipment. Wherein the equal ratio of the lengths of the three sides is the ratio of the lengths of the pixels displayed on the image of the box body, and is consistent with the calculated ratio of the three sides of the three-dimensional structure of the box body.

It should be noted that, because a reference Y-shaped slot with an indefinite size is generated, the vertical side of the Y-shaped slot may be perpendicular to the bottom side of the display screen of the measuring device, and the other two sides of the Y-shaped slot form plus and minus 120 degrees with the vertical side, respectively, and the length ratio of the three sides of the Y-shaped slot is the length ratio of the three-dimensional structure of the box body, the size and the position of the Y-shaped slot need to be determined, for example, the maximum side length WHm of the external rectangular region of the Y-shaped slot may be determined according to 80% of the shorter side of the display screen of the measuring device, so that the external rectangular region is filled with the Y-shaped slot to the maximum extent, and thus a better Y-shaped. For example, the length Wy and the width Hy of the circumscribed rectangular region of the reference Y-type card slot are calculated, the scaling ratio of the reference Y-type card slot may be WHm/max (Wy, Hy), and the scaled Y-type card slot is placed in the circumscribed rectangular region and at the center of the display screen of the measuring device.

S207, judging whether three sides of the box body in the current box body image are overlapped with the displayed Y-shaped clamping groove by the measuring equipment; if yes, go to step S208; if not, step S209 is executed.

For example, whether the matching degree between the Y-shaped card slot and three sides of the box in the current box image is greater than a preset threshold (e.g., 90%) may be detected, and if the matching degree is greater than the preset threshold, it is determined that three sides of the box in the current box image coincide with the displayed Y-shaped card slot; and if the matching degree is smaller than or equal to the preset threshold value, determining that the three sides of the box body in the current box body image are not overlapped with the displayed Y-shaped clamping groove.

And S208, acquiring the acceleration of the measuring equipment moving along the direction of the box body.

For example, the acceleration of the push-pull motion along the horizontal direction of the current box image acquired by the measuring device may be acquired, or the acceleration of the push-pull motion of the measuring device aligned with the box direction may be acquired, and the motion along the box direction may be flexibly set according to actual needs.

S209, adjusting the shooting angle by the measuring equipment until the three sides of the box body coincide with the displayed Y-shaped clamping groove in the current box body image, and acquiring the acceleration of the measuring equipment moving along the direction of the box body.

For example, as shown in fig. 4, when three sides of the box in the current box image do not coincide with the Y-shaped slot displayed, a prompt message may be output so that the shooting angle is adjusted based on the prompt message, so that the three sides of the box in the current box image coincide with the Y-shaped slot displayed. After the measuring device generates the motion, the acceleration of the measuring device within a preset time period, for example, 4 seconds or 6 seconds, may be obtained.

S210, judging whether the acceleration meets a preset condition by the measuring equipment; if yes, go to step S211; if not, go to step S212.

For example, the acceleration data may be subjected to spectral analysis, statistical analysis, etc. to determine whether the motion of the measurement device meets the requirements for frequency, amplitude, velocity, and/or vertical stability. Specifically, fast fourier transform may be performed on acceleration components of the acceleration in the directions of three axes XYZ and XYZ, so as to obtain frequencies and corresponding amplitudes in the three axes, and then the maximum frequencies Fx, Fy, and Fz and corresponding amplitudes Px, Py, and Pz are screened out. And calculating a standard deviation corresponding to the acceleration component of the acceleration in the three-axis direction to obtain Dx, Dy and Dz.

If Ft1< Fz < Ft2 (for example, Ft1 is 0.5 and Ft2 is 2), Pz/Py > Tp, Pz/Px > Tp (for example, Tp is 2), Dz/Dy > Td, Dz/Dx > Td (for example, Td is 5), and Dz > Dt (for example, Dt is 0.03) are satisfied, it is determined that the acceleration satisfies the preset condition.

S211, the measuring equipment acquires a box body image of the box body through the measuring equipment, and the volume of the box body is calculated based on the box body image.

For example, two-dimensional coordinate positions of box vertexes in a box image can be obtained, and connection relations among the vertexes can be determined; acquiring the position of the detection equipment when the detection equipment acquires the box body image; then determining the three-dimensional coordinate position of the top point of the box body according to the two-dimensional coordinate position and the equipment position; calculating the acceleration of the detection equipment according to the position of the equipment and the time when the detection equipment acquires the box body image; and calculating the volume of the box body according to the acceleration, the three-dimensional coordinate position and the connection relation among all the vertexes.

The obtaining of the two-dimensional coordinate position of the box vertex in the box image and the determining of the connection relationship between the vertices may include: acquiring a trained Tiny-service digital optical disk (Tiny-DSOD) network and a CPM network, wherein the Tiny-DSOD network and the CPM network are obtained by training sample images based on box vertexes and connection relations thereof labeled in sequence; carrying out convolution operation on the box body image through a Tiny-DSOD network to obtain a characteristic diagram; and extracting the two-dimensional coordinate position of the box body vertex based on the characteristic diagram through the CPM network, and determining the connection relation among the vertexes.

In some embodiments, the box image may include a plurality of box images, and performing a convolution operation on the box image through the target detection network to obtain the feature map may include: converting the multiple box body images into gray level images to obtain multiple gray level images; performing convolution operation on each gray image and a preset Laplace kernel to obtain a plurality of response images; calculating the variance of each response image, and screening out box body images corresponding to the variance greater than or equal to a preset threshold value to obtain effective images; and carrying out convolution operation on the effective image through a target detection network to obtain a characteristic diagram. In order to improve the accuracy of calculating the volume of the box body, the collected box body images can be screened in advance to screen out box body images with higher definition for processing, so that the volume of the box body is calculated, the preset threshold value can be flexibly set according to actual needs, the variance is greater than or equal to the preset threshold value to indicate that the box body images are clearer, and the variance is smaller than the preset threshold value to indicate that the box body images are more fuzzy, so that the fuzzy box body images with the variance smaller than the preset threshold value are required to be rejected. And finally, processing the screened effective images through a Tiny-digital optical network (Tiny-DSOD) network and a Continuous Processing Method (CPM) network.

In some embodiments, determining the three-dimensional coordinate position of the vertex of the box from the two-dimensional coordinate position and the device position may include: selecting one frame of box body image from the multiple box body images as a reference frame image, for example, taking a first frame of box body image as a reference frame image, wherein the first frame of box body image can be a first frame of box body image acquired when the acceleration meets a preset condition; other frame box images except the reference frame in the plurality of box images and the reference frame image form an image group; and determining the three-dimensional coordinate position of the box vertex according to the two-dimensional coordinate position of the box vertex in each box image in the image group and the equipment position when each box image in the image group is shot.

For example, the first frame of acquired box image may be used as a reference frame image. Then, other frame box images except the reference frame in the plurality of box images and the reference frame image form an image group; for example, when a first frame box image is used as a reference frame image, the first frame box image and a second frame box image may be combined into an image group a, the first frame box image and a third frame box image are combined into an image group B, the first frame box image and a fourth frame box image are combined into an image group C, the first frame box image and a fifth frame box image are combined into an image group D, and the first frame box image and a sixth frame box image are combined into an image group E; and so on.

At this time, the three-dimensional coordinate position of the box vertex can be determined based on the two-dimensional coordinate position of the box vertex in each box image in the image group and the device position at the time when each box image in the image group is captured. For example, the side length of the box and the box pose can be calculated according to the vertex position based on the multi-view geometric three-dimensional reconstruction principle, for example, the side length of the box can be the length, width and height dimensions S of the box, and the box pose can be the rotation amount Rb, the translation amount Tb and the like of the box; and acquiring the device position of the measuring device when acquiring the box image, wherein the device position can comprise a device pose, for example, the device pose can be a rotation amount Rc and a translation amount Tc of the measuring device, and acquiring the device parameter of the measuring device when acquiring the box image, and the device parameter can be a camera or camera parameter for acquiring the box image, such as a parameter of a focal length K and the like or a parameter of camera intrinsic parameters and the like. Then, the measuring device can construct a three-dimensional structure of the box body according to the side length, the position and the posture of the box body, the position and the posture of the device and the device parameters of the box body based on the imaging model and the structural constraint of the box body, and finally determine the three-dimensional coordinate position of the vertex of the box body according to the three-dimensional structure of the box body.

In order to improve the accuracy of acquiring the three-dimensional coordinate position, the obtained three-dimensional coordinate position may be corrected, for example, determining the three-dimensional coordinate position of the box vertex according to the two-dimensional coordinate position of the box vertex in each box image in the image group and the device position when each box image in the image group is captured may include: determining the three-dimensional coordinate position of the box vertex according to the two-dimensional coordinate position of the box vertex in each box image in the image group and the equipment position when each box image in the image group is shot, and obtaining the three-dimensional coordinate position of the box vertex corresponding to each group of image groups; projecting the three-dimensional vertex to a two-dimensional plane according to the three-dimensional coordinate position of the box body vertex corresponding to each group of image groups and the corresponding equipment position to obtain a target two-dimensional coordinate position; and based on the target two-dimensional coordinate position, screening out the three-dimensional coordinate position with the highest matching degree with the two-dimensional coordinate position from the three-dimensional coordinate positions of the box body vertexes corresponding to each group of image groups to obtain the three-dimensional coordinate position of the box body vertex.

In some embodiments, calculating the volume of the box according to the acceleration, the three-dimensional coordinate position, and the connection relationship between the vertices may include: acquiring real acceleration of motion generated in the process of acquiring box body images by detection equipment; determining the proportional relation between the pixel size of the box body in the box body image and the actual size according to the acceleration obtained by calculation and the real acceleration; calculating the side length of the box body according to the proportional relation, the three-dimensional coordinate position of the box body vertex and the connection relation among all the vertexes; and calculating the volume of the box body according to the side length of the box body.

Because the measuring equipment can movably collect the box body image in the process of collecting the box body image, the acceleration of the measuring equipment can be calculated based on the box body image collected by the measuring equipment, the equipment position when the box body image is collected, the time when the box body image is collected by the equipment and the like. For example, in the process of acquiring an image of a box body by the measuring device, a user can push and pull the measuring device along the direction of the box body, so that the measuring device moves; or, the measuring device can send a control instruction to the controller so as to control the controller to drive the measuring device to move based on the control instruction, so that the measuring device moves along the direction of the box body. At this time, the measuring device may acquire the acceleration of the measuring device through a preset accelerometer, a sensor, a gyroscope, or the like, so as to obtain the true acceleration.

After obtaining the real acceleration, the measuring device may determine, according to the calculated acceleration and the real acceleration, a proportional relationship between a pixel size of the box in the box image and an actual size of the box in the box image, where the pixel size is a size of the box in the image and the actual size is a size of the box in the real world, and for example, the proportional relationship is pixel size/actual size. At this time, the measuring apparatus may calculate the side length of the case based on the proportional relationship, the three-dimensional coordinate positions of the vertices of the case, and the connection relationship between the vertices. For example, if the box vertex a and the box vertex B are connected to obtain the side AB, the virtual side length R of the side AB in the box image can be calculated according to the three-dimensional coordinate position (x1, y1, z1) of the box vertex a and the three-dimensional coordinate position (x2, y2, z2) of the box vertex B, and then the side length of the box, that is, the side length of the box in the real world, can be calculated according to the virtual side length R and the proportional relationship. Finally, the volume of the bin may be calculated based on the side length of the bin, e.g., the first side length may be calculated as the first side length, the second side length may be calculated as the third side length may be calculated based on the first side length, the second side length, and the third side length of the bin.

S212, the measuring equipment outputs prompt information.

For example, if Ft1< Fz < Ft2 is not satisfied (for example, Ft1 is 0.5 and Ft2 is 2), prompt information for adjusting the frequency of movement of the measuring device in the direction of the case is output to prompt the user to adjust the push-pull frequency. The prompt message can be displayed in a display screen through voice broadcast or characters. If Pz/Py > Tp is not satisfied, Pz/Px > Tp (for example, Tp is 2), outputting unstable prompt information of the movement of the measuring equipment along the Z-axis direction to prompt a user to push or pull the Z-axis to be unstable. If Dz/Dy > Td and Dz/Dx > Td are not satisfied (for example, Td is 5), outputting unstable prompt information of the movement of the measuring equipment along the Z-axis direction to prompt a user to push or pull the Z-axis to be unstable; if Dz > Dt is not satisfied (for example, Dt is 0.03), a prompt message that the movement amplitude of the measuring device along the direction of the box is small is output to prompt the user that the push-pull movement amplitude is not enough, and the like, so that the user can effectively, namely quickly adjust the movement of the measuring device according to the prompt message.

For example, the camera of the measurement device may be aligned to the box (three sides are visible), and the box may be pushed and pulled back and forth for 4 seconds along a direction perpendicular to the display screen of the measurement device at a suitable stable frequency, and the position, size, angle, movement speed, amplitude of the back and forth movement, frequency of the back and forth movement, and the like of the box in the box image collected by the camera of the measurement device may be limited within a certain range in the whole process. In order to ensure that a novice user can quickly master the push-pull method, after the use operation meeting various restrictions is performed, corresponding prompt information is output when the conditions are not met.

In the embodiment of the application, the measuring equipment acquires the current box body image of the box body and detects the vertex position of the box body in the current box body image; then generating a Y-shaped clamping groove according to the vertex position so as to collect the motion acceleration by taking the Y-shaped clamping groove as a reference, and if three sides of the box body in the current box body image are superposed with the Y-shaped clamping groove, acquiring the motion acceleration of the measuring equipment along the direction of the box body; and when the acceleration meets a preset condition, acquiring a box body image of the box body through the measuring equipment, and calculating the volume of the box body based on the box body image. The scheme can determine the motion acceleration of the measuring equipment based on the Y-shaped clamping groove generated by the box body image, and the size of the box body is calculated according to the box body image acquired when the motion acceleration meets the preset condition, so that the measuring accuracy and stability can be ensured, and the measuring accuracy and efficiency are improved.

In order to better implement the object volume measuring method provided by the embodiment of the application, the embodiment of the application also provides a device based on the object volume measuring method. Wherein the terms have the same meanings as those in the above object volume measurement method, and the details of the implementation can be referred to the descriptions in the method embodiments.

Referring to fig. 5, fig. 5 is a schematic structural diagram of an object volume measuring device according to an embodiment of the present disclosure, wherein the object volume measuring device 300 may include a detecting module 301, a generating module 302, an obtaining module 303, a calculating module 304, and the like.

The detecting module 301 is configured to acquire a current object image of an object through a measuring device, and detect a position of a key point of the object in the current object image.

A generating module 302, configured to generate a preset reference model according to the position of the key point.

An obtaining module 303, configured to obtain a motion parameter of the measurement device moving along the object direction if a preset edge of the object in the current object image coincides with the preset reference model.

And the calculating module 304 is used for acquiring an object image of the object through the measuring equipment when the motion parameter meets a preset condition, and calculating the volume of the object based on the object image.

In some implementations, the generating module 302 can include:

the construction unit is used for constructing a three-dimensional structure of the object according to the positions of the key points when the preset number of the key points of the object in the detected current object image is set;

In some embodiments, the construction unit is specifically configured to: calculating the side length and the object pose of the object according to the positions of the key points; acquiring the device pose and device parameters of the measuring device when acquiring the current object image; and constructing a three-dimensional structure of the object according to the side length of the object, the position and posture of the equipment and the parameters of the equipment.

In some embodiments, the generating unit is specifically configured to: according to the three-dimensional structure of the object, constructing three sides with equal side length proportion and preset angle values formed by angles between two adjacent sides to obtain a reference model; acquiring the display side length of a display screen of the measuring equipment, and determining the side length of a quadrilateral region externally connected with the reference model according to the display side length; and based on the side length of the external quadrilateral area, carrying out scaling processing on the reference model to obtain a preset reference model.

In some embodiments, the obtaining module 303 is specifically configured to: displaying a preset reference model in a display screen of the measuring equipment; and adjusting the shooting angle of the measuring equipment based on the displayed preset reference model until the side length of the object in the object image captured by the measuring equipment is coincident with the preset reference model, and acquiring the motion parameters of the measuring equipment moving along the object direction.

In some embodiments, the object volume measurement device 300 may further include:

the determining module is used for determining that the motion parameter meets the preset condition when the frequency in the three-axis direction meets the preset frequency threshold, the amplitude meets the preset amplitude threshold and the standard deviation meets the preset threshold.

when the standard deviation in the three-axis direction does not meet a second preset threshold value, outputting prompt information that the movement amplitude of the measuring equipment along the object direction is small; the first preset threshold and the second preset threshold are included in the preset threshold.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

In the embodiment of the application, the detection module 301 may collect a current object image of an object and detect a position of a key point of the object in the current object image; then, the generating module 302 generates a preset reference model according to the position of the key point, so as to collect motion parameters by using the preset reference model as a reference, and if a preset edge of an object in the current object image coincides with the preset reference model, the acquiring module 303 may acquire the motion parameters of the measuring device moving along the object direction; when the motion parameter satisfies a preset condition, the calculation module 304 may acquire an object image of the object through the measurement device and calculate a volume of the object based on the object image. The scheme can determine the motion parameters of the measuring equipment based on the preset reference model generated by the object image, and the volume of the object is calculated by the object image acquired when the motion parameters meet the preset conditions, so that the volume of the object can be accurately and quickly calculated, the measuring accuracy and efficiency are improved, hardware such as a laser sensor does not need to be added on the measuring equipment, and the measuring cost is reduced.

Correspondingly, an embodiment of the present application further provides a measurement apparatus, as shown in fig. 6, fig. 6 shows a specific structural block diagram of the measurement apparatus provided in the embodiment of the present application, and the measurement apparatus may be used to implement the object volume measurement method provided in the above embodiment. The measuring device can be a smart phone or a tablet computer and the like.

The measurement device may include Radio Frequency (RF) circuitry 401, memory 402 including one or more computer-readable storage media, input unit 403, display unit 404, sensor 405, audio circuitry 406, Wireless Fidelity (WiFi) module 407, processor 408 including one or more processing cores, and power supply 409. Those skilled in the art will appreciate that the measuring device configuration shown in FIG. 6 does not constitute a limitation of the measuring device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:

the RF circuit 401 may be used for receiving and transmitting signals during a message transmission or communication process, and in particular, for receiving downlink information of a base station and then sending the received downlink information to the one or more processors 408 for processing; in addition, data relating to uplink is transmitted to the base station. In general, the RF circuitry 401 includes, but is not limited to, an antenna, at least one Amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the RF circuitry 401 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), and the like.

The memory 402 may be used to store software programs and modules, and the processor 408 executes various functional applications and data processing by operating the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the measuring apparatus, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 408 and the input unit 403 access to the memory 402.

The input unit 403 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, in a particular embodiment, the input unit 403 may include a touch-sensitive surface as well as other input devices. The touch-sensitive surface, also referred to as a touch display screen or a touch pad, may collect touch operations by a user (e.g., operations by a user on or near the touch-sensitive surface using a finger, a stylus, or any other suitable object or attachment) thereon or nearby, and drive the corresponding connection device according to a predetermined program. Alternatively, the touch sensitive surface may comprise two parts, a touch detection means and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts it to touch point coordinates, and sends the touch point coordinates to the processor 408, and can receive and execute commands from the processor 408. In addition, touch sensitive surfaces may be implemented using various types of resistive, capacitive, infrared, and surface acoustic waves. The input unit 403 may include other input devices in addition to the touch-sensitive surface. In particular, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 404 may be used to display information input by or provided to the user as well as various graphical user interfaces of the measurement device, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 404 may include a Display panel, and optionally, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch-sensitive surface may overlay the display panel, and when a touch operation is detected on or near the touch-sensitive surface, the touch operation is transmitted to the processor 408 to determine the type of touch event, and then the processor 408 provides a corresponding visual output on the display panel according to the type of touch event. Although in FIG. 6 the touch-sensitive surface and the display panel are two separate components to implement input and output functions, in some embodiments the touch-sensitive surface may be integrated with the display panel to implement input and output functions.

The measurement device may also include at least one sensor 405, such as a light sensor, motion sensor, and other sensors. In particular, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel according to the brightness of ambient light, and a proximity sensor that may turn off the display panel and/or backlight when the measuring device is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the device is stationary, and can be used for applications of recognizing the posture of a measuring device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured to the measuring device, detailed descriptions thereof are omitted.

Audio circuitry 406, a speaker, and a microphone may provide an audio interface between the user and the measurement device. The audio circuit 406 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into a sound signal for output; on the other hand, the microphone converts the collected sound signals into electrical signals, which are received by the audio circuit 406 and converted into audio data, which are processed by the audio data output processor 408, either via the RF circuit 401 for transmission to another measuring device, for example, or output to the memory 402 for further processing. The audio circuitry 406 may also include an earbud jack to provide communication of peripheral headphones with the measurement device.

WiFi belongs to short distance wireless transmission technology, and the measuring device can help the user send and receive e-mail, browse web page and access streaming media etc. through WiFi module 407, it provides wireless broadband internet access for the user. Although fig. 6 shows the WiFi module 407, it is understood that it does not belong to the essential constitution of the measuring device, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 408 is the control center of the measuring device, connects various parts of the entire measuring device using various interfaces and lines, performs various functions of the measuring device and processes data by running or executing software programs and/or modules stored in the memory 402 and calling up data stored in the memory 402, thereby performing overall monitoring of the measuring device. Optionally, processor 408 may include one or more processing cores; preferably, the processor 408 may integrate an application processor, which handles primarily the operating system, user interface, applications, etc., and a modem processor, which handles primarily the wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 408.

The measurement device also includes a power source 409 (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor 408 via a power management system to manage charging, discharging, and power consumption management functions via the power management system. The power supply 409 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

Although not shown, the measuring device may further include a camera, a bluetooth module, etc., which are not described herein. Specifically, in this embodiment, the processor 408 in the measurement device loads the executable file corresponding to the process of one or more application programs into the memory 402 according to the following instructions, and the processor 408 runs the application program stored in the memory 402, thereby implementing various functions:

acquiring a current object image of an object through measuring equipment, and detecting the position of a key point of the object in the current object image; generating a preset reference model according to the position of the key point; if the preset edge of the object in the current object image is superposed with the preset reference model, acquiring a motion parameter of the measuring equipment moving along the object direction; and when the motion parameters meet preset conditions, acquiring an object image of the object through the measuring equipment, and calculating the volume of the object based on the object image.

In some embodiments, generating the preset reference model according to the key point positions comprises: when the detected key point positions of the object in the current object image are in a preset number, constructing a three-dimensional structure of the object according to the key point positions; and generating a preset reference model according to the three-dimensional structure of the object.

In some embodiments, constructing a three-dimensional structure of the object from the keypoint locations comprises: calculating the side length and the object pose of the object according to the positions of the key points; acquiring the device pose and device parameters of the measuring device when acquiring the current object image; and constructing a three-dimensional structure of the object according to the side length of the object, the position and posture of the equipment and the parameters of the equipment.

In some embodiments, generating the preset reference model from the three-dimensional structure of the object comprises: according to the three-dimensional structure of the object, constructing three sides with equal side length proportion and preset angle values formed by angles between two adjacent sides to obtain a reference model; acquiring the display side length of a display screen of the measuring equipment, and determining the side length of a quadrilateral region externally connected with the reference model according to the display side length; and based on the side length of the external quadrilateral area, carrying out scaling processing on the reference model to obtain a preset reference model.

In some embodiments, obtaining the motion parameter of the movement of the measurement device along the direction of the object comprises: displaying a preset reference model in a display screen of the measuring equipment; and adjusting the shooting angle of the measuring equipment based on the displayed preset reference model until the side length of the object in the object image captured by the measuring equipment is coincident with the preset reference model, and acquiring the motion parameters of the measuring equipment moving along the object direction.

In some embodiments, when the motion parameter satisfies a preset condition, before acquiring an object image of the object by the measurement device, the method includes: respectively carrying out Fourier transform on the motion components of the motion parameters in the three-axis direction to obtain the frequency and the amplitude in the three-axis direction; calculating the standard deviation corresponding to the motion component of the motion parameter in the three-axis direction; and when the frequency in the three-axis direction meets a preset frequency threshold, the amplitude meets a preset amplitude threshold and the standard deviation meets a preset threshold, determining that the motion parameter meets a preset condition.

In some embodiments, after acquiring the motion parameter of the movement of the measuring device along the object direction, the method includes: when the frequency in the three-axis direction does not meet a preset frequency threshold, outputting prompt information for adjusting the movement frequency of the measuring equipment along the object direction; when the amplitude in the three-axis direction does not meet a preset amplitude threshold value or the standard deviation in the three-axis direction does not meet a first preset threshold value, outputting prompt information that the movement of the measuring equipment along the Z-axis direction is unstable; when the standard deviation in the three-axis direction does not meet a second preset threshold value, outputting prompt information that the movement amplitude of the measuring equipment along the object direction is small; the first preset threshold and the second preset threshold are included in the preset threshold.

In the above embodiments, the descriptions of the embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed description of the object volume measurement method, and are not described herein again.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, the present application provides a storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the object volume measurement methods provided in the embodiments of the present application. For example, the instructions may perform the steps of:

Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the storage medium can execute the steps in any object volume measurement method provided in the embodiments of the present application, the beneficial effects that can be achieved by any object volume measurement method provided in the embodiments of the present application can be achieved, and for details, refer to the foregoing embodiments, and are not described herein again.

The method, the device, the measuring equipment and the storage medium for measuring the volume of the object provided by the embodiment of the present application are introduced in detail, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method of measuring the volume of an object, comprising:

generating a preset reference model according to the position of the key point;

2. The object volume measurement method according to claim 1, wherein the generating of the preset reference model according to the key point positions comprises:

3. The object volume measurement method of claim 2, wherein the constructing the three-dimensional structure of the object from the keypoint locations comprises:

4. The object volume measurement method according to claim 2, wherein the generating of the preset reference model from the three-dimensional structure of the object comprises:

5. The object volume measuring method according to claim 1, wherein the acquiring of the motion parameter of the measuring device moving in the object direction includes:

6. The object volume measuring method according to any one of claims 1 to 5, wherein before acquiring an object image of the object by the measuring apparatus when the motion parameter satisfies a preset condition, the method further comprises:

7. The object volume measuring method according to claim 6, wherein after the obtaining of the motion parameter of the movement of the measuring device in the object direction, the method further comprises:

8. An object volume measuring device, comprising:

9. A measuring device comprising a processor and a memory, the memory having program code stored therein, the processor when calling the program code in the memory performing the object volume measuring method according to any one of claims 1 to 7.

10. A storage medium storing instructions adapted to be loaded by a processor to perform a method of measuring a volume of an object according to any one of claims 1 to 7.