CN115310880A - AR interaction method and system for inventory shortage condition - Google Patents
AR interaction method and system for inventory shortage condition Download PDFInfo
- Publication number
- CN115310880A CN115310880A CN202211238368.2A CN202211238368A CN115310880A CN 115310880 A CN115310880 A CN 115310880A CN 202211238368 A CN202211238368 A CN 202211238368A CN 115310880 A CN115310880 A CN 115310880A
- Authority
- CN
- China
- Prior art keywords
- head
- fluctuation range
- normal
- abnormal
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000003993 interaction Effects 0.000 title claims abstract description 58
- 230000002159 abnormal effect Effects 0.000 claims abstract description 65
- 239000013598 vector Substances 0.000 claims abstract description 32
- 230000006870 function Effects 0.000 claims description 21
- 230000007812 deficiency Effects 0.000 claims description 15
- 230000006735 deficit Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 12
- 206010061619 Deformity Diseases 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 abstract description 21
- 239000000463 material Substances 0.000 abstract description 2
- 239000011521 glass Substances 0.000 description 12
- 230000007547 defect Effects 0.000 description 10
- 230000008859 change Effects 0.000 description 9
- 230000001133 acceleration Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000000203 mixture Substances 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 229910001220 stainless steel Inorganic materials 0.000 description 1
- 239000010935 stainless steel Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of material management, in particular to an AR interaction method and an AR interaction system for inventory shortage conditions, wherein the method comprises the steps of obtaining a positioning error sequence of a head-up and shortage event, a quantity sequence of display objects, an experience quality coefficient of the positioning error sequence, a positioning performance index of the quantity sequence and a semantic descriptor of the shortage object to form a state feature vector; clustering a plurality of head-up disc loss events according to the state characteristic vectors to obtain normal clusters and abnormal clusters; calculating the membership degree of each head-up disc loss event to obtain the normal quantity fluctuation range and the abnormal quantity fluctuation range of the display object; the GRU prediction network is trained based on the normal clusters to obtain the prediction number of the display objects, the display strategy of the display objects is confirmed by combining the prediction number, the normal number fluctuation range and the abnormal number fluctuation range, the position and the number of the disqualified objects are determined by a user under the condition of unreliable positioning, and poor experience in the AR interaction process is reduced.
Description
Technical Field
The invention relates to the technical field of material management, in particular to an AR interaction method and an AR interaction system for inventory shortage.
Background
At present, AR glasses are commonly used for inventory check in the industry, but in the check task aiming at high height and dense grids, since the AR glasses positioning technology mainly takes active ToF or structured light as the main part and takes the visual mileage based on key points and IMU as the auxiliary part, the following problems can occur when observing high and far scenes: the shelf is too high and exceeds the AR glasses to actively measure the distance, so that the positioning error is larger; due to the fact that lighting facilities cause backlight, the visual odometer temporarily fails and only the IMU can be relied on for integration, and the problem causes errors on the positioning performance of the stored AR glasses.
Because too much information presents and can shelter from repeatedly, too little information can let the user be difficult to confirm the position of the dish loss object, consequently to the application of inventory dish loss, need carry out real-time supervision to the location state of AR glasses to confirm how much information presents and carry out the judgement of specific position with the assistance people. In the current application of inventory shortage, a user adjusts the amount of information presentation by himself, so that the situation of finding a position by 'black touch' can be relieved to a certain extent, but experience is influenced due to the fact that information presentation is complicated in normal inventory shortage work.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide an AR interaction method and system for inventory shortage condition, wherein the adopted technical solution is as follows:
in a first aspect, an embodiment of the present invention provides an AR interaction method for inventory shortage, where the method includes the following steps:
in the head-up interaction time of a head-up disqualification event, a positioning error sequence and a quantity sequence of display objects are obtained based on a set sampling frequency, the experience quality coefficient of the positioning error sequence and the positioning performance index of the quantity sequence are respectively calculated, and a semantic descriptor of the disqualification object is obtained according to a cargo name label; forming a state feature vector of a head-up disc loss event by using the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor;
acquiring state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up disc loss event and other head-up disc loss events in the cluster to which the head-up disc loss event belongs, and obtaining the membership degree of each head-up disc loss event based on the quantity similarity; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of a display object based on the membership degree of each head-up head disc loss event in the cluster;
training the GRU prediction network by using the number sequence of head-up disc loss events in the normal cluster; and acquiring the predicted number of the display objects in the future new line interaction time by utilizing the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range.
Further, the method for obtaining the quality of experience coefficient includes:
and acquiring a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, acquiring a value of the difference value by using a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient.
Further, the method for obtaining the positioning performance index includes:
the method comprises the steps of obtaining the maximum quantity, the quantity median and the average quantity of a quantity sequence, respectively calculating the absolute values of a first quantity difference value of the maximum quantity and the quantity median and a second quantity difference value of the average quantity, taking the sum of the first quantity difference value and a preset value as a denominator and the absolute value of the second quantity difference value as a numerator to obtain a corresponding first ratio, and taking the product of the first ratio and an experience quality coefficient as a positioning performance index.
Further, the method for calculating the overall state difference index of any two head-up disc loss events according to the state feature vector includes:
calculating the average absolute error of the positioning performance indexes in the feature vectors of the corresponding states of the two head-up disc loss events by using an L1 function, and utilizing
And calculating the similarity of the positioning error sequences in the corresponding state feature vectors of the two head-up head disc deficiency events by using a function, obtaining a corresponding second ratio by using the sum of the similarity and a preset value as a denominator and the average absolute error as a numerator, and using the second ratio as an integral state difference index.
Further, the method for acquiring the number similarity includes:
respectively acquiring pole difference values of the corresponding quantity sequences of the two head-up head disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating the similar distance of the corresponding quantity sequences between the two head-up head disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding third ratio;
and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc deficiency events, and taking the opposite number of the product of the third ratio and the cosine distances as the quantity similarity of the two head-up head disc deficiency events.
Further, the method for obtaining the membership of each head-up disc loss event based on the quantity similarity includes:
and adding the quantity similarity between the current head-up head-disc loss event and each other head-up head-disc loss event in the cluster to which the current head-up head-disc loss event belongs to obtain the membership degree of the current head-up head-disc loss event.
Further, the method for acquiring the normal number fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence from large to small of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form a normal number fluctuation range.
Further, the method for acquiring the fluctuation range of the abnormal number includes:
and selecting head-up head-disc loss events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence from small to large of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up head-disc loss events to form an abnormal number fluctuation range.
Further, the method for confirming the display strategy of the display object by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range comprises the following steps:
in the case where there is no intersection of the normal number fluctuation range and the abnormal number fluctuation range: when the predicted quantity is larger than the upper limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the upper limit of the normal quantity fluctuation range; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the normal quantity fluctuation range; when the predicted number is within the normal number fluctuation range, adjusting the number of the display objects to the predicted number;
in the case where there is an intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is within the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the lower limit of the normal quantity fluctuation range is larger than the lower limit of the abnormal quantity fluctuation range and the predicted quantity is within the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the abnormal quantity fluctuation range.
In a second aspect, an embodiment of the present invention further provides an AR interaction system for inventory shortage situations, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the steps of any one of the above methods when executing the computer program.
The embodiment of the invention at least has the following beneficial effects: according to the scheme, a state feature vector corresponding to the head-up head deficit event is obtained according to the head-up interaction time positioning error of the head-up head deficit event, the number of display objects and a semantic descriptor of the head deficit event, a plurality of head-up head deficit events are clustered based on the state feature vector to obtain a normal cluster and an abnormal cluster, the membership degree of each head-up head deficit event is calculated based on the difference of the state feature vectors of the head-up head deficit events in the same cluster, the normal quantity fluctuation range and the abnormal quantity fluctuation range of the display objects are obtained based on the membership degree, the GRU prediction network is trained by using the number of the display objects in the normal cluster, the training effect of the GRU prediction network is improved, the prediction quantity obtained by the GRU prediction network is more accurate, the display strategies of the display objects are adjusted by combining the prediction quantity, the normal quantity fluctuation range and the abnormal quantity fluctuation range, the head-up head deficit interaction state can be obviously improved, and AR glasses users are helped to determine the positions and the quantity of the head deficit objects under the condition of unreliable positioning, the repeated poor viewing of the head-up head deficit events, the poor positioning and the poor positioning are difficult to wait for the estimation and the poor positioning.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart illustrating steps of an AR interaction method for a stock inventory shortage situation according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention for achieving the predetermined objects, the following detailed description of the embodiments, structures, features and effects of the AR interaction method and system for inventory shortage according to the present invention will be made with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment of the invention aims at the following specific scenes: according to the scheme, hololens is used as actual AR experience equipment to optimize the obvious head-up and head-loss interaction, so that the displayed neighborhood content is increased under the condition of unreliable positioning, a user is helped to determine the position and the number of the head-loss objects, and the bad experiences such as head-up viewing again caused by the fact that the position is difficult to estimate, the positioning convergence is repeatedly waited for, and the positioning convergence fails in the interaction process are avoided.
The following describes a specific scheme of the AR interaction method and system for inventory shortage condition in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of an AR interaction method for inventory shortage is provided, according to an embodiment of the present invention, the method includes the following steps:
step S001, obtaining a positioning error sequence and a quantity sequence of display objects based on a set sampling frequency in the head-up interaction time of a head-up disqualification event, respectively calculating experience quality coefficients of the positioning error sequence and positioning performance indexes of the quantity sequence, and acquiring a semantic descriptor of the disqualification object according to a cargo name label; and forming a state feature vector of the head-up disqualification event by the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor.
Specifically, the camera pose estimation method in the existing AR technology is easily disabled after head-up, and in view of that the output difference of the inertial navigation algorithm such as extended kalman filtering and the visual mileage calculation method becomes large, so that in actual experience, linear drift and jitter of a relatively obvious camera position estimation value are generated, and a rotation parameter of the camera does not change greatly, a head-up event trigger threshold is set, that is, when the head of a person has an upward pitch angle of more than 15 degrees compared with the horizontal ground and exceeds 1s, a head-up disc loss event is regarded to occur.
Because the location state after the new line dish loses probably changes at any time, especially illumination is relatively poor or the condition towards shelf surface stainless steel, consequently can appear the too big condition that and became invalid of positioning error to position between AR stack information and the actual observation information can produce certain error in the experience that leads to new line dish to lose. The relative horizontal displacement is thus evaluated and the quality of experience coefficients for the positioning error are determined.
And when the head-up disc loss event occurs, calculating the linear acceleration of the AR glasses after head-up according to the positioning position, and taking the linear acceleration as the positioning error of the corresponding positioning position. Because the linear acceleration is not very large, but the characteristics of restricting the position of the AR glasses after head-up are less, false large linear acceleration can be generated during continuous positioning correction, the sampling frequency is set to be 1S within the head-up interaction time of a head-up disc loss event, a positioning error is obtained within one second, and then a positioning error sequence T is obtained based on the set sampling frequency.
Determining a quality of experience coefficient H of the positioning error based on the positioning error change in the head-up interaction time: and acquiring a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, acquiring a value of the difference value by using a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient.
As an example, a quality of experience coefficient
The calculation formula of (2) is as follows:
wherein,
is a mean function;
is a median function;
for activating the function, when the value in the middle bracket is less than 0, the value is 0, otherwise, the value is the original value in the middle bracket; 1 is a set constant.
In the head-up head loss and disc loss process, the number of the display objects can assist or influence the position judgment of a user on the head loss and disc loss objects, however, the sampling frequency of the number of the display objects in the head-up head loss and disc loss time is also 1S, in the process of head turning and checking of the user, the number of the display objects is counted based on the picture of the user, and then the number sequence N of the display objects is obtained in the head-up head loss and disc loss event.
And analyzing the positioning performance K of the head-up disc defect event by combining the experience quality coefficient H of the positioning error and the number sequence N of the display objects: the method comprises the steps of obtaining the maximum quantity, the quantity median and the average quantity of a quantity sequence, respectively calculating the absolute values of a first quantity difference value of the maximum quantity and the quantity median, the absolute value of a second quantity difference value of the quantity median and the average quantity, taking the sum of the first quantity difference value and a preset value as a denominator and the absolute value of the second quantity difference value as a numerator to obtain a corresponding ratio, and taking the product of the ratio and an experience quality coefficient as a positioning performance index.
As an example, the calculation formula of the positioning performance index K is:
wherein,
is the maximum number in the number sequence N;
is the median value of the number in the number sequence N;
is the average number in the number sequence N; 1 is a preset value.
Using the number median of the maximum number in the number sequence N to represent whether the number of the display objects in use is greatly changed from the number in historical use; the more proper the positioning error is, the more stable the interaction process of the head-up disc loss event is, the larger the experience quality coefficient H is, the larger the corresponding positioning performance index K is; the smaller the difference between the number median and the maximum number and the average number respectively is, the larger the positioning performance of the interaction process of the head-up disc loss event is considered to be.
It should be noted that, for the process of acquiring the number of the display objects, the present solution does not intervene to control the number of the display objects temporarily, so that the actual number of the display objects is increased or decreased by the user, and how to control the increase or decrease of the number of the display objects around the target object is a more fundamental technical means in the AR experience, which is various and simple, and the present invention is not described in detail again.
Since the goods name tag based on the disc loss object can check the storage environment of the disc loss object, the goods name tag based on the disc loss object can check whether a typical disc loss object appears around. The goods name labels of the discardable objects appearing in the head-up interaction time of the head-up discardable event are utilized to embody the semantic context of the discardable objects in the interaction process of the head-up discardable event, so that the position reference information of some peripheral objects to the discardable objects can be obviously distinguished.
Firstly, performing word segmentation on stock goods name labels of discipline loss objects, constructing a word bag model, then performing word statistics on goods name labels corresponding to disciplines which appear in the head-up interaction time of a head-up discipline loss event, and adding the goods name labels of the discipline loss objects which are displayed in a picture every second in the statistical process until a user heads down to finish discipline loss interaction; after the word frequency is counted based on the word bag model, the word frequency of all words in the head-up interaction time of the head-up disqualification event can be obtained, and besides, common words and words which repeatedly appear in the field also need to be eliminated; and finally, calculating a characteristic value by using a TF-IDF algorithm based on word frequency to obtain a semantic descriptor m of a head loss object in the head-up interaction time of the head-up head loss event, wherein the TF-IDF algorithm is a known technology and is not described in detail in the scheme.
It should be noted that adding the goods name label of the discipline and loss object does not need to be ordered, and only needs to analyze the occurrence frequency of the goods name label and perform word frequency statistics on the goods name label.
And forming a state feature vector { T, N, H, K, m } of the head-up disc defect event by using a positioning error sequence T, a number sequence N, an experience quality coefficient H, a positioning performance index K and a semantic descriptor m in the head-up interaction time of the head-up disc defect event.
Step S002, obtaining state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up head-plate deficiency event and other head-up head-plate deficiency events in the cluster to which the head-up head-plate deficiency event belongs, and obtaining the membership degree of each head-up head-plate deficiency event based on the quantity similarity; and respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of the display object based on the membership of each head-up head-plate defect event in the cluster.
Specifically, the state feature vectors of a plurality of head-up head-disc loss events are obtained by using the method in step S001, and each head-up head-disc loss event is subjected to state detection based on the state feature vectors: and calculating integral state difference indexes of any two head-up and head-down events according to the state characteristic vectors, and clustering the head-up and head-down events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster.
As an example, the difference square of the positioning performance indexes in the state feature vectors corresponding to the two head-up head disc loss events and the similar distance of the positioning error sequence are respectively calculated, the similar distance is optimized by using a constant, the optimized similar distance is used as a denominator, the difference square is used as a numerator to obtain a corresponding ratio, and the opposite number of the ratio is substituted into an exponential function with the constant e as the base to obtain an overall state difference index; taking the integral state difference index as a difference distance, clustering the head-up and head-loss event by using a Gaussian Mixture Model (GMM) based on the difference distance, namely setting two centroids, clustering according to a cluster corresponding to the centroids, wherein the closer the distance to the Gaussian center corresponding to the cluster, the more likely the cluster belongs to, and further obtaining two clusters, wherein the reason for forming the difference of the clusters is mainly because the physical conditions of a viewing scene and the positioning performance of AR glasses are checked, so that the two clusters are divided into a normal cluster and an abnormal cluster; and calculating the average value of the number sequence of the display objects corresponding to the head-up diskout event in each cluster, and taking the cluster with the maximum average value as an abnormal cluster.
The reason why the gaussian mixture model GMM is used is: because the state feature vector of each head-up disc defect event is mainly generated by artificial behaviors, more repeatability and mode exist, data points can be assumed to be in Gaussian distribution, and in high-dimensional spatial distribution, the boundary between two classes can be more accurately distinguished by using a Gaussian mixture model GMM to perform expectation maximization clustering process.
The calculation formula of the overall state difference index R is as follows:
wherein,
the integral state difference index between the head-up disc loss event A and the head-up disc loss event B is shown;
the positioning performance index of the head-up disc loss event A is obtained;
the positioning performance index of the head-up disc loss event B is obtained;
a positioning error sequence of a head-up disc loss event A;
a positioning error sequence of a head-up disc defect event B;
is a similarity function;
as a function of the mean absolute error.
Analyzing the state change in each cluster to determine the critical condition of the quantity change of the display objects, wherein the specific analysis is as follows:
(1) Respectively acquiring pole difference values of the corresponding quantity sequences of the two head-up head disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating the similar distance of the corresponding quantity sequences between the two head-up head disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding ratio; and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc defect events, and taking the opposite number of the product of the ratio and the cosine distances as the quantity similarity of the two head-up head disc defect events.
As an example, the numerical similarity is calculated by:
wherein,
the number similarity of the head-up disqualification event C and the head-up disqualification event D in the same cluster is obtained;
is an exponential function with a constant e as the base;
head-up disc defect event C and head-upThe disc defect event D corresponds to the similarity distance of the number sequences N;
is a function of the range;
is a function of the mean absolute error;
is a cosine similarity function;
a semantic descriptor of a head-up disc loss event C;
is a semantic descriptor of the head-up disqualification event D.
Similar distance using two head-up diskover event number sequences
And the average absolute error value
Characterizing differences between quantitative sequences, the smaller the difference, the greater the corresponding quantitative similarity; the more similar the semantic descriptors of the two head-up disc deficiency events are, the smaller the corresponding cosine distance is, and the larger the quantity similarity is.
(2) And adding the quantity similarity between the current head-up head loss event and each other head-up head loss event in the cluster to which the current head-up head loss event belongs to obtain the membership degree of the current head-up head loss event, and further obtaining the membership degree of each head-up head loss event.
Because the behaviors of the head-up and head-loss events in the interaction process are different, the change of the number of the final display objects is obviously different from the expected reference fluctuation, and the difference may be different in each head-loss process, so that the fluctuation range is determined according to the change of the number of the display objects in the multiple normal reaction processes.
Because behaviors in the interaction process of the head-up disc loss events are different, the number change of the final display objects is obviously different from the expected reference fluctuation, and the difference is possibly different in each head-up disc loss event, so that the normal number fluctuation range and the abnormal number fluctuation range of the display objects are respectively obtained based on the membership degree of each head-up disc loss event in the cluster: selecting head-up disc loss events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence of the membership degree from large to small, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disc loss events to form a normal number fluctuation range; and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence of the membership degree from small to large, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form an abnormal number fluctuation range.
It should be noted that the reason for selecting TOP-5 with membership degree in the normal cluster is that the number of display objects determined in this way is more representative of the better case, so as to determine the appropriate number range of display objects in the interaction process of the head-up diskless event; the abnormal quantity fluctuation range of the display object is obtained according to the abnormal cluster, so that the limitation of the display object in the interaction process of the head-up diskout event is determined, and when the limitation is exceeded, diskout needs to be performed in other modes or is reminded to perform diskout from another viewing angle.
Step S003, training the GRU prediction network by using the number sequence of head-up and head-down events in the normal cluster; and acquiring the predicted number of the display objects in the future new line interaction time by utilizing the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range.
Specifically, the number sequence of each head-up disc loss event in the normal cluster is used as a training sample, the number sequence corresponding to each training sample is labeled and then input into the GRU prediction network, and correspondingly, the training samples are moved 10 detection times in the future, namely 10 seconds in advance, in the scheme, the 10 detection times are moved, and an implementer can perform the detection according to actual performance and performanceA required movement time; constructing a loss function by taking the membership degree of the training sample as weight, so that the loss function
Wherein
the mean square error loss MSE for the ith training sample,
is the membership of the ith training sample.
It should be noted that the training process of the GRU prediction network is a known technology, and detailed description is not repeated in this scheme; the purpose of training the GRU prediction network is to acquire the predicted number of the display objects in the interaction process of the head-up and loss events in real time through the prediction of the neural network, so that the abnormal condition of the number fluctuation of the display objects can be found in time, and the number of the display objects can be controlled by taking corresponding measures.
After the GRU prediction network is trained, when the GRU prediction network is applied to practice, the prediction number of the real objects at the corresponding moment in the future new line interaction time is output according to the trained GRU prediction network, and the prediction number is compared with a normal number fluctuation range and an abnormal number fluctuation range so as to provide a corresponding display strategy for AR glasses.
In the case where there is no intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the predicted number is larger than the upper limit of the fluctuation range of the normal number, the number of the display objects is adjusted to a numerical value corresponding to the upper limit of the fluctuation range of the normal number, which is a common value in the interaction process of the head-up and loss events, and for larger errors, the number of the display objects can be controlled based on previous experience so as to reduce artificial interference; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the normal quantity fluctuation range, a user is reminded that the quantity of the display objects is too small and the problem of positioning failure is possibly caused, the change of the display objects is continuously tracked until the lower limit of the normal quantity fluctuation range is reached, then the prompt is eliminated, and the processing is continuously carried out according to the predicted quantity; when the predicted number is in the normal number fluctuation range, the number of display objects is adjusted to the predicted number.
In the case where there is an intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is within the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range, reminding a user of adjusting the observation position in time, and avoiding ambiguity of errors of AR superposition display; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range, and reminding a user of readjusting the position; when the lower limit of the normal quantity fluctuation range is larger than the lower limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, the quantity of the display objects is adjusted to a value corresponding to the lower limit of the abnormal quantity fluctuation range, and a user is reminded that the quantity of the display objects is too small, but the problem of positioning failure can not be relieved any more, at the moment, the change of the display objects is continuously tracked until the lower limit of the abnormal quantity fluctuation range is reached, a prompt is eliminated, and the user can try to repeat checking and positioning.
In summary, in the embodiment of the present invention, in the head-up interaction time of the head-up disfigurement event, the positioning error sequence and the quantity sequence of the display objects are obtained, and the experience quality coefficient of the positioning error sequence, the positioning performance index of the quantity sequence and the semantic descriptor of the disfigurement object form a state feature vector; acquiring state feature vectors of a plurality of head-up disc loss events, and clustering the head-up disc loss events according to the state feature vectors to obtain normal clusters and abnormal clusters; respectively calculating the quantity similarity of display objects of each head-up disc loss event and other head-up disc loss events in the cluster to which the head-up disc loss event belongs to obtain the membership degree of each head-up disc loss event; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of the display object based on the membership grade; the GRU prediction network is trained by using the number sequence in the normal cluster to obtain the predicted number of the display objects in the future new line interaction time, and the display strategy of the display objects is confirmed by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range, so that AR glasses users are helped to determine the positions and the number of disqualified objects under the condition of unreliable positioning, and bad experience in the AR interaction process is reduced.
Based on the same inventive concept as the above method, an embodiment of the present invention further provides an AR interaction system for an inventory shortage situation, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor, when executing the computer program, implements the steps of any one of the above AR interaction methods for an inventory shortage situation.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And that specific embodiments have been described above. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit of the present invention.
Claims (10)
1. An AR interaction method for inventory deficit conditions, the method comprising the steps of:
in the head-up interaction time of a head-up disqualification event, a positioning error sequence and a quantity sequence of display objects are obtained based on a set sampling frequency, the experience quality coefficient of the positioning error sequence and the positioning performance index of the quantity sequence are respectively calculated, and a semantic descriptor of the disqualification object is obtained according to a cargo name label; forming a state feature vector of a head-up disqualification event by the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor;
acquiring state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up head-plate deficiency event and other head-up head-plate deficiency events in the cluster to which the head-up head-plate deficiency event belongs, and obtaining the membership degree of each head-up head-plate deficiency event based on the quantity similarity; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of a display object based on the membership degree of each head-up disc loss event in the cluster;
training the GRU prediction network by using the number sequence of head-up and head-down events in the normal cluster; and acquiring the predicted number of the display objects in the future new line interaction time by utilizing the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range.
2. The AR interaction method for inventory deficit condition according to claim 1, wherein the method for obtaining the quality of experience coefficient includes:
and acquiring a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, acquiring a value of the difference value by using a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient.
3. The AR interaction method for inventory shortfall situation according to claim 1, wherein the method for obtaining the positioning performance index includes:
the method comprises the steps of obtaining the maximum number, the median number and the average number of the number sequence, respectively calculating a first number difference value of the maximum number and the median number, and an absolute value of a second number difference value of the median number and the average number, taking the sum of the first number difference value and a preset value as a denominator, and the absolute value of the second number difference value as a numerator to obtain a corresponding first ratio, and taking the product of the first ratio and an experience quality coefficient as a positioning performance index.
4. The AR interaction method for inventory diskov conditions according to claim 1, wherein the method for calculating the overall state difference index of any two head-up diskov events according to the state feature vector comprises:
calculating the average absolute error of the positioning performance indexes in the corresponding state feature vectors of the two head-up disc loss events by using an L1 function, and using
And calculating the similarity of the positioning error sequences in the corresponding state feature vectors of the two head-up head disc deficiency events by using a function, obtaining a corresponding second ratio by using the sum of the similarity and a preset value as a denominator and the average absolute error as a numerator, and using the second ratio as an integral state difference index.
5. The AR interaction method for inventory deficit condition according to claim 1, wherein the quantity similarity obtaining method includes:
respectively acquiring pole difference values of corresponding quantity sequences of two head-up disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating a similar distance of the corresponding quantity sequences between the two head-up disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding third ratio;
and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc deficiency events, and taking the opposite number of the product of the third ratio and the cosine distances as the quantity similarity of the two head-up head disc deficiency events.
6. The AR interaction method for inventory head-up disqualification situations as claimed in claim 1, wherein the method of deriving membership for each head-up disqualification event based on quantity similarity comprises:
and adding the quantity similarity between the current head-up head-disc loss event and each other head-up head-disc loss event in the cluster to which the current head-up head-disc loss event belongs to obtain the membership degree of the current head-up head-disc loss event.
7. The AR interaction method for inventory deficit condition according to claim 1, wherein the obtaining method of normal number fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence from large to small of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form a normal number fluctuation range.
8. The AR interaction method for inventory deficit condition according to claim 1, wherein the obtaining method of the abnormal quantity fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence of the membership degree from small to large, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form an abnormal number fluctuation range.
9. The AR interaction method for inventory deficit condition according to claim 1, wherein said method of confirming a display policy of a display object in combination with a predicted quantity, a normal quantity fluctuation range, and an abnormal quantity fluctuation range includes:
in the case where there is no intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the predicted quantity is larger than the upper limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the upper limit of the normal quantity fluctuation range; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the normal quantity fluctuation range; when the predicted number is within the normal number fluctuation range, adjusting the number of the display objects to the predicted number;
in the case where there is an intersection of the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is in the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the lower limit of the normal quantity fluctuation range is larger than the lower limit of the abnormal quantity fluctuation range and the predicted quantity is within the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the abnormal quantity fluctuation range.
10. AR interaction system for inventory shortfall situations, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that the processor, when executing the computer program, implements the steps of an AR interaction method for inventory shortfall situations according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211238368.2A CN115310880B (en) | 2022-10-11 | 2022-10-11 | AR interaction method and system for inventory shortage condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211238368.2A CN115310880B (en) | 2022-10-11 | 2022-10-11 | AR interaction method and system for inventory shortage condition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115310880A true CN115310880A (en) | 2022-11-08 |
| CN115310880B CN115310880B (en) | 2022-12-20 |
Family
ID=83868409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211238368.2A Active CN115310880B (en) | 2022-10-11 | 2022-10-11 | AR interaction method and system for inventory shortage condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115310880B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115713625A (en) * | 2022-11-18 | 2023-02-24 | 盐城众拓视觉创意有限公司 | Method for rapidly combining teaching real-recorded video and courseware background into film |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011043108A1 (en) * | 2009-10-09 | 2011-04-14 | 株式会社日立製作所 | Equipment status monitoring method, monitoring system, and monitoring program |
| US20160217022A1 (en) * | 2015-01-23 | 2016-07-28 | Opsclarity, Inc. | Anomaly detection using circumstance-specific detectors |
| CN113902334A (en) * | 2021-10-28 | 2022-01-07 | 上海众源网络有限公司 | Event abnormal fluctuation detection method and system, electronic equipment and storage medium |
| WO2022057260A1 (en) * | 2020-09-15 | 2022-03-24 | 浙江大学 | Industrial control system communication network anomaly classification method |
| CN114803763A (en) * | 2022-06-30 | 2022-07-29 | 锐创软件技术(启东)有限公司 | Variable speed goods elevator abnormity detection method based on neural network |
| CN115097526A (en) * | 2022-08-22 | 2022-09-23 | 江苏益捷思信息科技有限公司 | Seismic acquisition data quality evaluation method |
-
2022
- 2022-10-11 CN CN202211238368.2A patent/CN115310880B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011043108A1 (en) * | 2009-10-09 | 2011-04-14 | 株式会社日立製作所 | Equipment status monitoring method, monitoring system, and monitoring program |
| US20160217022A1 (en) * | 2015-01-23 | 2016-07-28 | Opsclarity, Inc. | Anomaly detection using circumstance-specific detectors |
| WO2022057260A1 (en) * | 2020-09-15 | 2022-03-24 | 浙江大学 | Industrial control system communication network anomaly classification method |
| CN113902334A (en) * | 2021-10-28 | 2022-01-07 | 上海众源网络有限公司 | Event abnormal fluctuation detection method and system, electronic equipment and storage medium |
| CN114803763A (en) * | 2022-06-30 | 2022-07-29 | 锐创软件技术(启东)有限公司 | Variable speed goods elevator abnormity detection method based on neural network |
| CN115097526A (en) * | 2022-08-22 | 2022-09-23 | 江苏益捷思信息科技有限公司 | Seismic acquisition data quality evaluation method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115713625A (en) * | 2022-11-18 | 2023-02-24 | 盐城众拓视觉创意有限公司 | Method for rapidly combining teaching real-recorded video and courseware background into film |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115310880B (en) | 2022-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180342067A1 (en) | Moving object tracking system and moving object tracking method | |
| CN101464950B (en) | Video human face identification and retrieval method based on on-line learning and Bayesian inference | |
| US9152845B2 (en) | Human attribute estimation system, human attribute estimation apparatus and human attribute estimation method | |
| CN115310880B (en) | AR interaction method and system for inventory shortage condition | |
| EP2418588A1 (en) | Visual localization method | |
| EP4375857A1 (en) | Supervised learning method and apparatus for image features, device, and storage medium | |
| CN113327272B (en) | Robustness long-time tracking method based on correlation filtering | |
| CN110363165B (en) | Multi-target tracking method and device based on TSK fuzzy system and storage medium | |
| Zhang et al. | Fast covariance matching with fuzzy genetic algorithm | |
| CN101739687A (en) | Covariance matrix-based fast maneuvering target tracking method | |
| CN110349188A (en) | Multi-object tracking method, device and storage medium based on TSK fuzzy model | |
| Hamdi et al. | Drotrack: High-speed drone-based object tracking under uncertainty | |
| CN113112151A (en) | Intelligent wind control evaluation method and system based on multidimensional perception and enterprise data quantification | |
| Pope et al. | Learning appearance models for object recognition | |
| CN116580333A (en) | Grain depot vehicle tracking method based on YOLOv5 and improved StrongSORT | |
| CN115731576A (en) | Unsupervised pedestrian re-identification method based on key shielding area | |
| CN114973399A (en) | Human body continuous attitude estimation method based on key point motion estimation | |
| CN114445449A (en) | Target tracking method based on variable structure T-S fuzzy semantic model and related device | |
| CN110660084A (en) | Multi-target tracking method and device | |
| US8457388B2 (en) | Method and system for searching for global minimum | |
| CN117576530A (en) | Multi-target association and tracking method based on motion and appearance characteristic self-adaptive fusion | |
| CN113836980A (en) | Face recognition method, electronic device and storage medium | |
| CN116510272A (en) | Physical stamina event action quality assessment method based on mark distribution learning | |
| US20230177881A1 (en) | Automatic body movement recognition and association system including smoothing, segmentation, similarity, pooling, and dynamic modeling | |
| Ba et al. | Head pose tracking and focus of attention recognition algorithms in meeting rooms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |