CN115310880B - AR interaction method and system for inventory shortage condition - Google Patents

AR interaction method and system for inventory shortage condition Download PDF

Info

Publication number
CN115310880B
CN115310880B CN202211238368.2A CN202211238368A CN115310880B CN 115310880 B CN115310880 B CN 115310880B CN 202211238368 A CN202211238368 A CN 202211238368A CN 115310880 B CN115310880 B CN 115310880B
Authority
CN
China
Prior art keywords
head
fluctuation range
normal
abnormal
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211238368.2A
Other languages
Chinese (zh)
Other versions
CN115310880A (en
Inventor
卢艳华
陈长健
邱超
庄思越
崔刘帅
王光辉
郭浩
孙远
袁穆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CRRC Nanjing Puzhen Co Ltd
Original Assignee
CRRC Nanjing Puzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CRRC Nanjing Puzhen Co Ltd filed Critical CRRC Nanjing Puzhen Co Ltd
Priority to CN202211238368.2A priority Critical patent/CN115310880B/en
Publication of CN115310880A publication Critical patent/CN115310880A/en
Application granted granted Critical
Publication of CN115310880B publication Critical patent/CN115310880B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06395—Quality analysis or management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]

Landscapes

  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Educational Administration (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Marketing (AREA)
  • Theoretical Computer Science (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Operations Research (AREA)
  • Game Theory and Decision Science (AREA)
  • Finance (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Accounting & Taxation (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to the technical field of material management, in particular to an AR interaction method and an AR interaction system for inventory shortage conditions, wherein the method comprises the steps of obtaining a positioning error sequence of a head-up and shortage event, a quantity sequence of display objects, an experience quality coefficient of the positioning error sequence, a positioning performance index of the quantity sequence and a semantic descriptor of the shortage object to form a state feature vector; clustering a plurality of head-up disc loss events according to the state feature vectors to obtain normal clusters and abnormal clusters; calculating the membership degree of each head-up disc loss event to obtain the normal quantity fluctuation range and the abnormal quantity fluctuation range of the display object; the GRU prediction network is trained based on the normal clusters to obtain the prediction number of the display objects, the display strategy of the display objects is confirmed by combining the prediction number, the normal number fluctuation range and the abnormal number fluctuation range, the position and the number of the disqualified objects are determined by a user under the condition of unreliable positioning, and poor experience in the AR interaction process is reduced.

Description

AR interaction method and system for inventory shortage condition
Technical Field
The invention relates to the technical field of material management, in particular to an AR interaction method and system for inventory shortage.
Background
At present, AR glasses are commonly used for inventory check in the industry, but in the check task aiming at high height and dense grids, since the AR glasses positioning technology mainly takes active ToF or structured light as the main part and takes the visual mileage based on key points and IMU as the auxiliary part, the following problems can occur when observing high and far scenes: the shelf is too high and exceeds the AR glasses to actively measure the distance, so that the positioning error is larger; due to the fact that lighting facilities cause backlight, the visual odometer temporarily fails and can only rely on the IMU for integration, and the problem causes errors on the positioning performance of the stored AR glasses.
Because too much information presentation can shelter from repeatedly, too little information can let the user be difficult to confirm the position of dish loss object, consequently to the application of inventory dish loss, needs to carry out real-time supervision to the orientation state of AR glasses to confirm how much information that presents in order to assist the people to carry out the judgement of specific position. In the current application of inventory shortage, a user can adjust the amount of information presentation by himself, the situation of finding a position by touching black can be relieved to a certain extent, but experience is influenced due to the fact that information presentation is complicated in normal inventory shortage work.
Disclosure of Invention
In order to solve the above technical problems, an object of the present invention is to provide an AR interaction method and system for inventory shortage condition, wherein the adopted technical solution is as follows:
in a first aspect, an embodiment of the present invention provides an AR interaction method for inventory shortage, where the method includes the following steps:
in the head-up interaction time of a head-up diskless event, a positioning error sequence and a quantity sequence of display objects are obtained based on a set sampling frequency, an experience quality coefficient of the positioning error sequence and a positioning performance index of the quantity sequence are respectively calculated, and a semantic descriptor of the disdeficient object is obtained according to a goods name label; forming a state feature vector of a head-up disqualification event by the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor;
acquiring state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up head-plate deficiency event and other head-up head-plate deficiency events in the cluster to which the head-up head-plate deficiency event belongs, and obtaining the membership degree of each head-up head-plate deficiency event based on the quantity similarity; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of a display object based on the membership degree of each head-up head disc loss event in the cluster;
training the GRU prediction network by using the number sequence of head-up and head-down events in the normal cluster; and acquiring the predicted number of the display objects in the future new line interaction time by using the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range.
Further, the method for obtaining the quality of experience coefficient includes:
and acquiring a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, acquiring a value of the difference value by using a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient.
Further, the method for obtaining the positioning performance index includes:
the method comprises the steps of obtaining the maximum number, the median number and the average number of the number sequence, respectively calculating a first number difference value of the maximum number and the median number, and an absolute value of a second number difference value of the median number and the average number, taking the sum of the first number difference value and a preset value as a denominator, and the absolute value of the second number difference value as a numerator to obtain a corresponding first ratio, and taking the product of the first ratio and an experience quality coefficient as a positioning performance index.
Further, the method for calculating the overall state difference index of any two head-up disc loss events according to the state feature vector includes:
calculating the average absolute error of the positioning performance indexes in the corresponding state feature vectors of the two head-up disc loss events by using an L1 function, and using
Figure 796178DEST_PATH_IMAGE001
And calculating the similarity of the positioning error sequences in the corresponding state feature vectors of the two head-up head disc deficiency events by using a function, obtaining a corresponding second ratio by using the sum of the similarity and a preset value as a denominator and the average absolute error as a numerator, and using the second ratio as an integral state difference index.
Further, the method for acquiring the number similarity includes:
respectively acquiring pole difference values of the corresponding quantity sequences of the two head-up head disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating the similar distance of the corresponding quantity sequences between the two head-up head disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding third ratio;
and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc deficiency events, and taking the opposite number of the product of the third ratio and the cosine distances as the quantity similarity of the two head-up head disc deficiency events.
Further, the method for obtaining the membership of each head-up disc loss event based on the number similarity includes:
and adding the quantity similarity between the current head-up head-disc loss event and each other head-up head-disc loss event in the cluster to which the current head-up head-disc loss event belongs to obtain the membership degree of the current head-up head-disc loss event.
Further, the method for acquiring the normal number fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence from large to small of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form a normal number fluctuation range.
Further, the method for acquiring the fluctuation range of the abnormal number includes:
and selecting head-up head-disc loss events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence from small to large of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up head-disc loss events to form an abnormal number fluctuation range.
Further, the method for confirming the display strategy of the display object by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range comprises the following steps:
in the case where there is no intersection of the normal number fluctuation range and the abnormal number fluctuation range: when the predicted quantity is larger than the upper limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the upper limit of the normal quantity fluctuation range; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the normal quantity fluctuation range; when the predicted number is within the normal number fluctuation range, adjusting the number of the display objects to the predicted number;
in the case where there is an intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is in the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the lower limit of the normal number fluctuation range is larger than the lower limit of the abnormal number fluctuation range and the predicted number is within the normal number fluctuation range, the number of the display objects is adjusted to a numerical value corresponding to the lower limit of the abnormal number fluctuation range.
In a second aspect, an embodiment of the present invention further provides an AR interaction system for an inventory deficit condition, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor implements the steps of any one of the above methods when executing the computer program.
The embodiment of the invention at least has the following beneficial effects: according to the scheme, a state feature vector corresponding to the head-up head deficit event is obtained according to the head-up interaction time positioning error of the head-up head deficit event, the number of display objects and a semantic descriptor of the head deficit event, a plurality of head-up head deficit events are clustered based on the state feature vector to obtain a normal cluster and an abnormal cluster, the membership degree of each head-up head deficit event is calculated based on the difference of the state feature vectors of the head-up head deficit events in the same cluster, the normal quantity fluctuation range and the abnormal quantity fluctuation range of the display objects are obtained based on the membership degree, the GRU prediction network is trained by using the number of the display objects in the normal cluster, the training effect of the GRU prediction network is improved, the prediction quantity obtained by the GRU prediction network is more accurate, the display strategies of the display objects are adjusted by combining the prediction quantity, the normal quantity fluctuation range and the abnormal quantity fluctuation range, the head-up head deficit interaction state can be obviously improved, and AR glasses users are helped to determine the positions and the quantity of the head deficit objects under the condition of unreliable positioning, the repeated poor viewing of the head-up head deficit events, the poor positioning and the poor positioning are difficult to wait for the estimation and the poor positioning.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart illustrating steps of an AR interaction method for inventory shortage according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined object, the following detailed description of the embodiments, structures, features and effects of the AR interaction method and system for inventory shortage according to the present invention is provided with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The embodiment of the invention aims at the following specific scenes: according to the scheme, hololens is used as actual AR experience equipment to optimize the obvious head-up and head-loss interaction, so that the displayed neighborhood content is increased under the condition of unreliable positioning, a user is helped to determine the position and the number of the head-loss objects, and the bad experiences such as head-up viewing again caused by the fact that the position is difficult to estimate, the positioning convergence is repeatedly waited for, and the positioning convergence fails in the interaction process are avoided.
The following describes a specific scheme of an AR interaction method and system for inventory shortage conditions in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of an AR interaction method for inventory shortage is provided, according to an embodiment of the present invention, the method includes the following steps:
step S001, obtaining a positioning error sequence and a quantity sequence of display objects based on a set sampling frequency in the head-up interaction time of a head-up disqualification event, respectively calculating experience quality coefficients of the positioning error sequence and positioning performance indexes of the quantity sequence, and acquiring a semantic descriptor of the disqualification object according to a goods name label; and forming a state feature vector of the head-up disc loss event by the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor.
Specifically, the camera pose estimation method in the existing AR technology is easily disabled after head-up, and in view of that the output difference between the inertial navigation algorithm such as extended kalman filtering and the visual mileage calculation method becomes large, so that in actual experience, linear drift and jitter of a camera position estimation value are obvious, and the rotation parameter of the camera does not change greatly, a head-up event trigger threshold is set, that is, when the head of a person has an upward pitch angle of more than 15 degrees compared with the horizontal ground and exceeds 1s, the head-up disc loss event is considered to occur.
Because the location state after the new line dish loses probably changes at any time, especially illumination is relatively poor or the condition towards shelf surface stainless steel, consequently can appear the too big condition that and became invalid of positioning error to position between AR stack information and the actual observation information can produce certain error in the experience that leads to new line dish to lose. The relative horizontal displacement is thus evaluated and the quality of experience coefficients for the positioning error determined.
And when the head-up disc loss event occurs, calculating the linear acceleration of the AR glasses after head-up according to the positioning position, wherein the linear acceleration is used for representing the walking characteristics of the person wearing the AR glasses, and the linear acceleration is used as the positioning error of the corresponding positioning position. Because the linear acceleration is not very large, but the characteristics of restricting the position of the AR glasses after head-up are less, false large linear acceleration can be generated during continuous positioning correction, the sampling frequency is set to be 1S within the head-up interaction time of a head-up disc loss event, a positioning error is obtained in one second, and then the positioning error sequence T is obtained based on the set sampling frequency.
Determining a quality of experience coefficient H of the positioning error based on the positioning error change in the new line interaction time: and acquiring a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, acquiring a value of the difference value by using a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient.
As one example, the quality of experience coefficient
Figure 381880DEST_PATH_IMAGE002
The calculation formula of (2) is as follows:
Figure 608462DEST_PATH_IMAGE003
wherein the content of the first and second substances,
Figure 148290DEST_PATH_IMAGE004
is a mean function;
Figure 751309DEST_PATH_IMAGE005
is a median function;
Figure 457097DEST_PATH_IMAGE006
for activating the function, when the value in the middle bracket is less than 0, the value is 0, otherwise, the value is the original value in the middle bracket; 1 is a set constant.
Figure 854581DEST_PATH_IMAGE007
The larger the linear acceleration of the AR glasses after positioning, the larger the average value of the linear acceleration of the AR glasses after positioning, and if the average value is larger than the median value
Figure 147284DEST_PATH_IMAGE008
If the positioning error sequence has a large number of abnormal element values, the positioning error sequence is considered to have a sudden displacement in the positioning process, and the corresponding experience quality coefficient
Figure 288415DEST_PATH_IMAGE002
The smaller.
In the head-up head loss and disc loss process, the number of the display objects can assist or influence the position judgment of a user on the head loss and disc loss objects, however, the sampling frequency of the number of the display objects in the head-up head loss and disc loss time is also 1S, in the process of head turning and checking of the user, the number of the display objects is counted based on the picture of the user, and then the number sequence N of the display objects is obtained in the head-up head loss and disc loss event.
Analyzing the positioning performance K of the head-up and head-down event by combining the experience quality coefficient H of the positioning error and the number sequence N of the display objects: the method comprises the steps of obtaining the maximum number, the number median and the average number of a number sequence, respectively calculating the absolute values of a first number difference value of the maximum number and the number median and a second number difference value of the number median and the average number, taking the sum of the first number difference value and a preset value as a denominator and the absolute value of the second number difference value as a numerator to obtain a corresponding ratio, and taking the product of the ratio and an experience quality coefficient as a positioning performance index.
As an example, the calculation formula of the positioning performance index K is:
Figure 114289DEST_PATH_IMAGE009
wherein the content of the first and second substances,
Figure 417094DEST_PATH_IMAGE010
is the maximum number in the number sequence N;
Figure 197094DEST_PATH_IMAGE011
is the median value of the number in the number sequence N;
Figure 970DEST_PATH_IMAGE012
is the average number in the number sequence N; 1 is a preset value.
Using a number median of the maximum number in the number sequence N to represent whether the number of the display objects in use is greatly changed from the number in historical use; the more proper the positioning error is, the more stable the interaction process of the head-up disc loss event is, the larger the experience quality coefficient H is, the larger the corresponding positioning performance index K is; the smaller the difference between the number median and the maximum number and the average number respectively is, the larger the positioning performance of the interaction process of the head-up disc loss event is considered to be.
It should be noted that, for the process of acquiring the number of the display objects, the scheme does not intervene to control the number of the display objects temporarily, so that the actual number of the display objects is increased or decreased by the user, and how to control the increase or decrease of the number of the display objects around the target object is a more fundamental technical means in the AR experience, which is various and simple, and the invention is not described in detail again.
Since the goods name tag based on the discipline loss object can check the storage environment of the discipline loss object, the goods name tag based on the discipline loss object can check whether a typical discipline loss object appears around. The goods name label of the disc loss object appearing in the head-up interaction time of the head-up disc loss event is utilized to embody the semantic context of the disc loss object in the interaction process of the head-up disc loss event, so that the position reference information of some peripheral objects to the disc loss object can be obviously distinguished.
Firstly, performing word segmentation on stock goods name labels of discipline loss objects, constructing a word bag model, then performing word statistics on goods name labels corresponding to disciplines which appear in the head-up interaction time of a head-up discipline loss event, and adding the goods name labels of the discipline loss objects which are displayed in a picture every second in the statistical process until a user heads down to finish discipline loss interaction; after the word frequency is counted based on the word bag model, the word frequency of all words in the head-up interaction time of the head-up disqualification event can be obtained, and besides, common words and words which repeatedly appear in the field also need to be eliminated; and finally, calculating a characteristic value by using a TF-IDF algorithm based on word frequency to obtain a semantic descriptor m of a head-up loss object in the head-up interaction time of the head-up loss event, wherein the TF-IDF algorithm is a known technology and is not repeated in the scheme.
It should be noted that adding the goods name label of the discipline and loss object does not need to be ordered, and only needs to analyze the occurrence frequency of the goods name label and perform word frequency statistics on the goods name label.
And forming a state feature vector { T, N, H, K, m } of the head-up disc loss event by using a positioning error sequence T, a quantity sequence N, an experience quality coefficient H, a positioning performance index K and a semantic descriptor m in the head-up interaction time of the head-up disc loss event.
Step S002, obtaining state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up head-plate deficiency event and other head-up head-plate deficiency events in the cluster to which the head-up head-plate deficiency event belongs, and obtaining the membership degree of each head-up head-plate deficiency event based on the quantity similarity; and respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of the display object based on the membership of each head-up head-plate defect event in the cluster.
Specifically, the state feature vectors of a plurality of head-up head-disc loss events are obtained by using the method in step S001, and each head-up head-disc loss event is subjected to state detection based on the state feature vectors: and calculating integral state difference indexes of any two head-up and head-down events according to the state characteristic vectors, and clustering the head-up and head-down events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster.
As an example, the difference square of the positioning performance indexes and the similar distance of the positioning error sequence in the state feature vectors corresponding to the two head-up disc loss events are respectively calculated, the similar distances are optimized by using constants, the optimized similar distances are used as denominators, the difference square is used as numerators to obtain corresponding ratios, and the opposite numbers of the ratios are substituted into an exponential function with the constant e as the base to obtain an overall state difference index; taking the integral state difference index as a difference distance, clustering the head-up and head-loss event by using a Gaussian Mixture Model (GMM) based on the difference distance, namely setting two centroids, clustering according to a cluster corresponding to the centroids, wherein the closer the distance to the Gaussian center corresponding to the cluster, the more likely the cluster belongs to, and further obtaining two clusters, wherein the reason for forming the difference of the clusters is mainly because the physical conditions of a viewing scene and the positioning performance of AR glasses are checked, so that the two clusters are divided into a normal cluster and an abnormal cluster; and calculating the average value of the number sequence of the display objects corresponding to the head-up disc defect event in each cluster, and taking the cluster with the maximum average value as an abnormal cluster.
The reason why the gaussian mixture model GMM is used is: because the state feature vector of each head-up disfigurement event is mainly generated by artificial behaviors, more repeatability and model are provided, the data points can be assumed to be in Gaussian distribution, and in high-dimensional spatial distribution, the boundary between two classes can be more accurately distinguished by using a Gaussian mixture model GMM to perform the process of expectation maximization clustering.
The calculation formula of the overall state difference index R is as follows:
Figure 415771DEST_PATH_IMAGE013
wherein, the first and the second end of the pipe are connected with each other,
Figure 656522DEST_PATH_IMAGE014
for raising the headThe overall state difference index between the disc deficit event A and the head-up disc deficit event B;
Figure 156773DEST_PATH_IMAGE015
the positioning performance index of the head-up disc loss event A is obtained;
Figure 374128DEST_PATH_IMAGE016
the positioning performance index of the head-up disc loss event B is obtained;
Figure 643435DEST_PATH_IMAGE017
a positioning error sequence of a head-up disc loss event A;
Figure 55087DEST_PATH_IMAGE018
a positioning error sequence of a head-up disc defect event B;
Figure 42635DEST_PATH_IMAGE019
is a similarity function;
Figure 329260DEST_PATH_IMAGE020
as a function of the mean absolute error.
Figure 718653DEST_PATH_IMAGE021
The larger the value of (A), the smaller the difference of the positioning errors of the head-up head disc loss event A and the head-up head disc loss event B,
Figure 414121DEST_PATH_IMAGE022
the larger the value of (A), the larger the difference of the positioning performance of the head-up head disc loss event A and the head-up head disc loss event B is, the corresponding integral state difference index
Figure 888965DEST_PATH_IMAGE023
The larger.
Analyzing the state change in each cluster to determine the critical condition of the quantity change of the display objects, wherein the specific analysis is as follows:
(1) Respectively acquiring pole difference values of the corresponding quantity sequences of the two head-up head disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating the similar distance of the corresponding quantity sequences between the two head-up head disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding ratio; and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc defect events, and taking the opposite number of the product of the ratio and the cosine distances as the quantity similarity of the two head-up head disc defect events.
As an example, the numerical similarity is calculated by:
Figure 713701DEST_PATH_IMAGE024
wherein the content of the first and second substances,
Figure 459066DEST_PATH_IMAGE025
the number similarity of the head-up disc loss event C and the head-up disc loss event D in the same cluster is determined;
Figure 976635DEST_PATH_IMAGE026
is an exponential function with a constant e as the base;
Figure 938775DEST_PATH_IMAGE027
the similar distance of the number sequences N corresponding to the head-up head disc defect event C and the head-up head disc defect event D is obtained;
Figure 567202DEST_PATH_IMAGE028
as a function of the range;
Figure 167073DEST_PATH_IMAGE020
is a function of the mean absolute error;
Figure 996489DEST_PATH_IMAGE029
is a cosine similarity function;
Figure 445925DEST_PATH_IMAGE030
a semantic descriptor of a head-up disc loss event C;
Figure 471518DEST_PATH_IMAGE031
is a semantic descriptor of the head-up disqualification event D.
Similar distance using two head-up diskout event number sequences
Figure 191475DEST_PATH_IMAGE027
And the mean absolute error value
Figure 50846DEST_PATH_IMAGE032
Characterizing differences between quantitative sequences, the smaller the difference, the greater the corresponding quantitative similarity; the more similar the semantic descriptors of the two head-up disc loss events are, the smaller the corresponding cosine distance is, and the greater the number similarity is.
(2) And adding the quantity similarity between the current head-up head loss event and each other head-up head loss event in the cluster to which the current head-up head loss event belongs to obtain the membership degree of the current head-up head loss event, and further obtaining the membership degree of each head-up head loss event.
Because the behaviors of the head-up disqualification event in the interaction process are different, the change of the final display object number is obviously different from the expected reference fluctuation, and the difference may be different in each disqualification process, so that the fluctuation range is determined according to the change of the display object number in the multiple normal reaction processes.
Because the behaviors in the interaction process of the head-up disqualification event are different, the change of the number of the final display objects is obviously different from the expected reference fluctuation, and the difference may be different in each head-up disqualification event, so that the normal number fluctuation range and the abnormal number fluctuation range of the display objects are respectively obtained based on the membership grade of each head-up disqualification event in the cluster: selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence of the membership degree from large to small, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form a normal number fluctuation range; and selecting head-up head-disc loss events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence from small to large of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up head-disc loss events to form an abnormal number fluctuation range.
It should be noted that the reason for selecting TOP-5 with membership degree in the normal cluster is that the number of display objects determined in this way is more representative of the better case, so as to determine the appropriate number range of display objects in the interaction process of the head-up diskless event; the abnormal quantity fluctuation range of the display object is obtained according to the abnormal cluster, so that the limitation of the display object in the interaction process of the head-up disqualification event is determined, and when the limitation is exceeded, disqualification needs to be performed in other modes or is reminded to perform disqualification from another viewing angle.
Step S003, training the GRU prediction network by using the number sequence of head-up and head-down events in the normal cluster; and acquiring the predicted number of the display objects in the future new line interaction time by utilizing the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range.
Specifically, the number sequence of each head-up disc loss event in the normal cluster is used as a training sample, the number sequence corresponding to each training sample is labeled and then input into the GRU prediction network, and the training samples are correspondingly moved 10 detection times in the future, namely 10 seconds in advance, in the scheme, the 10 detection times are moved, and an implementer can move time according to actual performance and requirements; constructing a loss function by taking the membership degree of the training sample as weight, so that the loss function
Figure 721999DEST_PATH_IMAGE033
Wherein, in the step (A),
Figure 957808DEST_PATH_IMAGE034
the mean square error loss MSE for the ith training sample,
Figure 266692DEST_PATH_IMAGE035
is the membership of the ith training sample.
It should be noted that the training process of the GRU prediction network is a known technology, and detailed description is not repeated in this scheme; the purpose of training the GRU prediction network is to acquire the predicted number of the display objects in the interaction process of the head-up and loss events in real time through the prediction of the neural network, so that the abnormal condition of the number fluctuation of the display objects can be found in time, and the number of the display objects can be controlled by taking corresponding measures.
After the GRU prediction network is trained, when the GRU prediction network is applied to practice, the prediction number of the real objects at the corresponding moment in the future new line interaction time is output according to the trained GRU prediction network, and the prediction number is compared with a normal number fluctuation range and an abnormal number fluctuation range so as to provide a corresponding display strategy for AR glasses.
In the case where there is no intersection of the normal number fluctuation range and the abnormal number fluctuation range: when the predicted number is larger than the upper limit of the fluctuation range of the normal number, the number of the display objects is adjusted to a numerical value corresponding to the upper limit of the fluctuation range of the normal number, which is a common value in the interaction process of the head-up and loss events, and for larger errors, the number of the display objects can be controlled based on previous experience so as to reduce artificial interference; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a value corresponding to the lower limit of the normal quantity fluctuation range, a user is reminded that the quantity of the display objects is too small and the problem of positioning failure is possibly caused, the change of the display objects is continuously tracked until the lower limit of the normal quantity fluctuation range is reached, the prompt is eliminated, and then the processing is continuously carried out according to the predicted quantity; when the predicted number is in the normal number fluctuation range, the number of display objects is adjusted to the predicted number.
In the case where there is an intersection of the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is in the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range, reminding a user of adjusting the observation position in time, and avoiding ambiguity of errors of AR superposition display; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range, and reminding a user of readjusting the position; when the lower limit of the normal quantity fluctuation range is larger than the lower limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, the quantity of the display objects is adjusted to a value corresponding to the lower limit of the abnormal quantity fluctuation range, and a user is reminded that the quantity of the display objects is too small, but the problem of positioning failure can not be relieved any more, the change of the display objects is continuously tracked at the moment, the prompt is eliminated until the lower limit of the abnormal quantity fluctuation range is reached, and the user can try to recover from the loss and position.
In summary, in the embodiment of the present invention, in the head-up interaction time of the head-up disfigurement event, the positioning error sequence and the quantity sequence of the display objects are obtained, and the experience quality coefficient of the positioning error sequence, the positioning performance index of the quantity sequence and the semantic descriptor of the disfigurement object form a state feature vector; acquiring state feature vectors of a plurality of head-up head-disc loss events, and clustering the head-up head-disc loss events according to the state feature vectors to obtain normal clusters and abnormal clusters; respectively calculating the quantity similarity of display objects of each head-up disc loss event and other head-up disc loss events in the cluster to which the head-up disc loss event belongs to obtain the membership degree of each head-up disc loss event; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of the display object based on the membership grade; the GRU prediction network is trained by using the quantity sequence in the normal cluster to obtain the predicted quantity of the display objects in the future new line interaction time, and the display strategy of the display objects is confirmed by combining the predicted quantity, the normal quantity fluctuation range and the abnormal quantity fluctuation range, so that the AR glasses user is helped to determine the position and the quantity of the disqualified objects under the condition of unreliable positioning, and the bad experience in the AR interaction process is reduced.
Based on the same inventive concept as the above method, an embodiment of the present invention further provides an AR interaction system for an inventory shortage situation, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor, when executing the computer program, implements the steps of any one of the above AR interaction methods for an inventory shortage situation.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
All the embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that are within the spirit of the present invention are intended to be included therein.

Claims (8)

1. An AR interaction method for inventory deficit conditions, the method comprising the steps of:
in the head-up interaction time of a head-up disqualification event, a positioning error sequence and a quantity sequence of display objects are obtained based on a set sampling frequency, the experience quality coefficient of the positioning error sequence and the positioning performance index of the quantity sequence are respectively calculated, and a semantic descriptor of the disqualification object is obtained according to a cargo name label; forming a state feature vector of a head-up disqualification event by the positioning error sequence, the quantity sequence, the experience quality coefficient, the positioning performance index and the semantic descriptor;
acquiring state feature vectors of a plurality of head-up head-disc loss events, calculating integral state difference indexes of any two head-up head-disc loss events according to the state feature vectors, and clustering the head-up head-disc loss events based on the integral state difference indexes to obtain two clusters which are respectively a normal cluster and an abnormal cluster; respectively calculating the quantity similarity of display objects of each head-up disc loss event and other head-up disc loss events in the cluster to which the head-up disc loss event belongs, and obtaining the membership degree of each head-up disc loss event based on the quantity similarity; respectively acquiring a normal quantity fluctuation range and an abnormal quantity fluctuation range of a display object based on the membership degree of each head-up disc loss event in the cluster;
training the GRU prediction network by using the number sequence of head-up disc loss events in the normal cluster; acquiring the predicted number of the display objects in the future new line interaction time by utilizing the trained GRU prediction network, and confirming the display strategy of the display objects by combining the predicted number, the normal number fluctuation range and the abnormal number fluctuation range;
the method for acquiring the experience quality coefficient comprises the following steps:
obtaining a median and an average value of the positioning error sequence, calculating a difference value between the median and the average value, obtaining a value of the difference value by utilizing a ReLU function, and taking the reciprocal of the sum of the value and a set constant as a quality of experience coefficient;
the method for acquiring the positioning performance index comprises the following steps:
the method comprises the steps of obtaining the maximum number, the median number and the average number of the number sequence, respectively calculating a first number difference value of the maximum number and the median number, and an absolute value of a second number difference value of the median number and the average number, taking the sum of the first number difference value and a preset value as a denominator, and the absolute value of the second number difference value as a numerator to obtain a corresponding first ratio, and taking the product of the first ratio and an experience quality coefficient as a positioning performance index.
2. The AR interaction method for inventory head-disk deficit condition according to claim 1, wherein the method of calculating an overall state difference index of any two head-up head-disk deficit events according to the state feature vector includes:
calculating the average absolute error of the positioning performance indexes in the feature vectors of the corresponding states of the two head-up disc loss events by using an L1 function, and utilizing
Figure DEST_PATH_IMAGE001
Calculating the similarity of the positioning error sequences in the corresponding state feature vectors of the two head-up head disc deficiency events by using a function, obtaining a corresponding second ratio by using the sum of the similarity and a preset value as a denominator and the average absolute error as a numerator, and calculating the second ratioThe value serves as an overall state difference indicator.
3. The AR interaction method for inventory deficit condition according to claim 1, wherein the quantity similarity obtaining method includes:
respectively acquiring pole difference values of corresponding quantity sequences of two head-up disc loss events based on the state feature vector, calculating an average absolute error value of the pole difference values by using an L1 function, and calculating a similar distance of the corresponding quantity sequences between the two head-up disc loss events; taking the average absolute error value as a denominator and the similar distance as a numerator to obtain a corresponding third ratio;
and calculating cosine distances of the semantic descriptors corresponding to the two head-up head disc deficiency events, and taking the opposite number of the product of the third ratio and the cosine distances as the quantity similarity of the two head-up head disc deficiency events.
4. The AR interaction method for inventory head-up disqualification situations as claimed in claim 1, wherein the method of deriving membership for each head-up disqualification event based on quantity similarity comprises:
and adding the quantity similarity between the current head-up head-disc loss event and each other head-up head-disc loss event in the cluster to which the current head-up head-disc loss event belongs to obtain the membership degree of the current head-up head-disc loss event.
5. The AR interaction method for inventory deficit condition according to claim 1, wherein the obtaining method of the normal number fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the normal cluster based on the sequence from large to small of the membership degree, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form a normal number fluctuation range.
6. The AR interaction method for inventory deficit condition according to claim 1, wherein the obtaining method of the abnormal quantity fluctuation range includes:
and selecting head-up disfigurement events corresponding to TOP-5 with the membership degree in the abnormal cluster based on the sequence of the membership degree from small to large, and acquiring the maximum value and the minimum value in the number sequence corresponding to the head-up disfigurement events to form an abnormal number fluctuation range.
7. The AR interaction method for inventory deficit conditions according to claim 1, wherein the method of confirming a display policy of a display object in combination with a predicted quantity, a normal quantity fluctuation range, and an abnormal quantity fluctuation range includes:
in the case where there is no intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the predicted quantity is larger than the upper limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the upper limit of the normal quantity fluctuation range; when the predicted quantity is smaller than the lower limit of the normal quantity fluctuation range, the quantity of the display objects is adjusted to a numerical value corresponding to the lower limit of the normal quantity fluctuation range; when the predicted number is within the normal number fluctuation range, adjusting the number of the display objects to the predicted number;
in the case where there is an intersection between the normal number fluctuation range and the abnormal number fluctuation range: when the normal quantity fluctuation range is in the abnormal quantity fluctuation range and the predicted quantity is larger than the upper limit of the abnormal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the upper limit of the normal quantity fluctuation range is larger than the upper limit of the abnormal quantity fluctuation range and the predicted quantity is in the normal quantity fluctuation range, adjusting the quantity of the display objects to a numerical value corresponding to the upper limit of the abnormal quantity fluctuation range; when the lower limit of the normal number fluctuation range is larger than the lower limit of the abnormal number fluctuation range and the predicted number is within the normal number fluctuation range, the number of the display objects is adjusted to a numerical value corresponding to the lower limit of the abnormal number fluctuation range.
8. An AR interaction system for inventory shortfall situations, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that the processor, when executing the computer program, implements the steps of an AR interaction method for inventory shortfall situations according to any of claims 1-7.
CN202211238368.2A 2022-10-11 2022-10-11 AR interaction method and system for inventory shortage condition Active CN115310880B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211238368.2A CN115310880B (en) 2022-10-11 2022-10-11 AR interaction method and system for inventory shortage condition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211238368.2A CN115310880B (en) 2022-10-11 2022-10-11 AR interaction method and system for inventory shortage condition

Publications (2)

Publication Number Publication Date
CN115310880A CN115310880A (en) 2022-11-08
CN115310880B true CN115310880B (en) 2022-12-20

Family

ID=83868409

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211238368.2A Active CN115310880B (en) 2022-10-11 2022-10-11 AR interaction method and system for inventory shortage condition

Country Status (1)

Country Link
CN (1) CN115310880B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115713625A (en) * 2022-11-18 2023-02-24 盐城众拓视觉创意有限公司 Method for rapidly combining teaching real-recorded video and courseware background into film

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011043108A1 (en) * 2009-10-09 2011-04-14 株式会社日立製作所 Equipment status monitoring method, monitoring system, and monitoring program
CN113902334A (en) * 2021-10-28 2022-01-07 上海众源网络有限公司 Event abnormal fluctuation detection method and system, electronic equipment and storage medium
WO2022057260A1 (en) * 2020-09-15 2022-03-24 浙江大学 Industrial control system communication network anomaly classification method
CN114803763A (en) * 2022-06-30 2022-07-29 锐创软件技术(启东)有限公司 Variable speed goods elevator abnormity detection method based on neural network
CN115097526A (en) * 2022-08-22 2022-09-23 江苏益捷思信息科技有限公司 Seismic acquisition data quality evaluation method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10261851B2 (en) * 2015-01-23 2019-04-16 Lightbend, Inc. Anomaly detection using circumstance-specific detectors

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011043108A1 (en) * 2009-10-09 2011-04-14 株式会社日立製作所 Equipment status monitoring method, monitoring system, and monitoring program
WO2022057260A1 (en) * 2020-09-15 2022-03-24 浙江大学 Industrial control system communication network anomaly classification method
CN113902334A (en) * 2021-10-28 2022-01-07 上海众源网络有限公司 Event abnormal fluctuation detection method and system, electronic equipment and storage medium
CN114803763A (en) * 2022-06-30 2022-07-29 锐创软件技术(启东)有限公司 Variable speed goods elevator abnormity detection method based on neural network
CN115097526A (en) * 2022-08-22 2022-09-23 江苏益捷思信息科技有限公司 Seismic acquisition data quality evaluation method

Also Published As

Publication number Publication date
CN115310880A (en) 2022-11-08

Similar Documents

Publication Publication Date Title
US20180342067A1 (en) Moving object tracking system and moving object tracking method
CN108416258B (en) Multi-human body tracking method based on human body part model
US9092660B2 (en) Face image registration device and method
KR100647322B1 (en) Apparatus and method of generating shape model of object and apparatus and method of automatically searching feature points of object employing the same
CN115310880B (en) AR interaction method and system for inventory shortage condition
EP2418588A1 (en) Visual localization method
CN113674328A (en) Multi-target vehicle tracking method
CN114972418A (en) Maneuvering multi-target tracking method based on combination of nuclear adaptive filtering and YOLOX detection
US11113597B2 (en) Artificial neural network and method of training an artificial neural network with epigenetic neurogenesis
Zhang et al. Fast covariance matching with fuzzy genetic algorithm
CN110349188B (en) Multi-target tracking method, device and storage medium based on TSK fuzzy model
CN112200021A (en) Target crowd tracking and monitoring method based on limited range scene
CN113112151A (en) Intelligent wind control evaluation method and system based on multidimensional perception and enterprise data quantification
Singh et al. Unsupervised change detection from remote sensing images using hybrid genetic FCM
CN110660084A (en) Multi-target tracking method and device
Pajares et al. Fuzzy cognitive maps for stereovision matching
US8457388B2 (en) Method and system for searching for global minimum
CN116071710B (en) Passenger flow volume statistics method based on intelligent stadium monitoring video
CN113205108A (en) YOLOv 4-based multi-target vehicle detection and tracking method
CN111998853A (en) AGV visual navigation method and system
CN116510272A (en) Physical stamina event action quality assessment method based on mark distribution learning
CN111860097A (en) Abnormal behavior detection method based on fuzzy theory
CN116071570A (en) 3D target detection method under indoor scene
CN115063443A (en) Real-time multi-target tracking method and system
Lanes et al. An analysis of the impact of diversity on stacking supervised classifiers

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant