CN111882365B - Intelligent commodity recommendation system and method for efficient self-service vending machine - Google Patents
Intelligent commodity recommendation system and method for efficient self-service vending machine Download PDFInfo
- Publication number
- CN111882365B CN111882365B CN202010784731.5A CN202010784731A CN111882365B CN 111882365 B CN111882365 B CN 111882365B CN 202010784731 A CN202010784731 A CN 202010784731A CN 111882365 B CN111882365 B CN 111882365B
- Authority
- CN
- China
- Prior art keywords
- user
- commodity
- information
- eye movement
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000010365 information processing Effects 0.000 claims abstract description 65
- 230000007246 mechanism Effects 0.000 claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 15
- 230000004424 eye movement Effects 0.000 claims description 97
- 210000001747 pupil Anatomy 0.000 claims description 52
- 238000012545 processing Methods 0.000 claims description 25
- 230000003993 interaction Effects 0.000 claims description 9
- 230000033001 locomotion Effects 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000000747 cardiac effect Effects 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims 1
- 238000004148 unit process Methods 0.000 claims 1
- 238000012216 screening Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 229940070527 tourmaline Drugs 0.000 description 1
- 229910052613 tourmaline Inorganic materials 0.000 description 1
- 239000011032 tourmaline Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
- G06Q30/0269—Targeted advertisements based on user profile or attribute
- G06Q30/0271—Personalized advertisement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
- G06Q30/0268—Targeted advertisements at point-of-sale [POS]
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07F—COIN-FREED OR LIKE APPARATUS
- G07F9/00—Details other than those peculiar to special kinds or types of apparatus
- G07F9/02—Devices for alarm or indication, e.g. when empty; Advertising arrangements in coin-freed apparatus
- G07F9/023—Arrangements for display, data presentation or advertising
Abstract
The invention relates to the field of self-service vending, and discloses an intelligent recommendation system and method for a high-efficiency self-service vending machine. Comprising the following steps: the system comprises an image acquisition device, a commodity display area, an eye tracker pose adjusting mechanism, an electrocardiograph detection device, a central information processing and control module, a voice prompt component and an execution terminal, wherein the image acquisition device acquires a face image, the central information processing and control module judges whether a potential user exists in a purchase area, if the user decides to purchase a commodity, but can not rapidly select the commodity due to difficulty in selection, the eye tracker pose adjusting mechanism automatically adjusts the position of the eye tracker according to the position of the user, so as to complete eye tracker information acquisition, the central information processing and control module processes eye tracker and heart rate data when the user browses each commodity through a threshold screening method, and outputs recommended commodity information to the execution terminal, so that the user with difficulty in selection can rapidly select the heart commodity, the intelligent degree of auxiliary purchase of the vending machine is improved, and the high-efficiency requirement of the user is met.
Description
Technical Field
The invention relates to the field of self-service vending, in particular to an intelligent commodity recommendation system and method for a high-efficiency self-service vending machine.
Background
With the development of self-service vending service modes in China, the functions of the self-service vending machine are continuously perfected, the commodity vending types are increased increasingly, meanwhile, the requirement on the intelligent degree of the self-service vending machine is higher and higher, and the improvement of the man-machine interaction is an important influence factor for improving the intelligent degree. At present, more self-service vending machines with stronger interactivity realize man-machine interaction through an electronic screen or a mobile phone interface on the front side of the vending machine, and users operate according to page prompt flow to complete the purchasing process. The human-computer interaction method, the device, the storage medium and the vending machine of the patent CN201710859759.9 are used for attracting consumers through a marketing mode that users finish system setting tasks to obtain rewards, so that the human-computer interaction degree of the vending machine is improved, but the purchasing speed is low, and the method and the device are suitable for occasions with low requirements on entertainment places and shopping efficiency; the patent CN201920635528.4 discloses a vending machine shipment mechanism and vending machine adopts spring shipment mechanism, simplifies connection structure, improves convenient to use swiftly, but lacks certain man-machine interaction function, and intelligent degree still needs to promote.
The intelligent commodity recommending system and method for the high-efficiency self-service vending machine are provided in the patent, so that the intelligent commodity recommending system and method for the high-efficiency self-service vending machine is capable of recommending and helping consumers to quickly and easily purchase the heart instrument commodity, and greatly improving purchase efficiency and shopping fun of man-machine interaction.
Disclosure of Invention
The invention aims to provide an intelligent commodity recommending system and method for a high-efficiency self-service vending machine, which are suitable for users with different height ranges, and solve the problem that the selection of the commodity facing the full-scale of the tourmaline is difficult when the users use the vending machine for self-service shopping; the pupil position of the user is automatically captured, the eye movement track and heart rate information change when the user browses commodities are acquired, the data processing method of threshold screening is utilized to obtain the favorite commodity serial numbers of the user and make intelligent recommendation, and the system operation efficiency is improved, so that the user obtains better and faster shopping experience.
The technical scheme adopted for achieving the purpose of the invention comprises the following steps: the intelligent commodity recommending system for the high-efficiency self-service vending machine is composed of an image collecting device (1), a commodity displaying area (2), an eye tracker (3), an eye tracker pose adjusting mechanism (4), an electrocardio detecting device (5), a central information processing and controlling module (6), a voice prompt component (7) and an executing terminal (8), wherein the image collecting device (1) collects image information of a purchasing area, the central information processing and controlling module (6) detects whether potential users exist and judges whether the intelligent commodity recommending system continuously runs or not through face recognition, the voice prompt component (7) guides the users to be positioned right in front of the commodity displaying area (2), palm is placed in an information collecting effective area of the electrocardio detecting device (5), the central information processing and controlling module (6) analyzes face images of the users collected by the image collecting device (1) and pupil calibration position information, calculates the deviation amount of pupil positions of the users and the information collecting effective area of the eye tracker (3), the eye tracker pose adjusting mechanism (4) automatically adjusts pitch angles of the eye tracker (3) and horizontal distance between the eye tracker (3) and the users according to the obtained deviation amount, enables the eye tracker to be positioned in front of the effective area, the information collecting effective area of the eye tracker (3) and the information collecting effective area, and the information of each eye tracker can respectively capture the information of the eye tracker, the eye tracker and each eye tracker, and each eye tracker respectively, and the eye information and the information respectively, the central information processing and control module (6) further analyzes and processes the acquired eye movement track and heart rate information, calculates the cardiac index when the user browses each commodity and controls the vending machine execution terminal (8) to display recommended commodity information.
The image acquisition device (1) is fixedly connected with the vending machine (9), images in a purchase area are acquired in real time, and when a user passes through the purchase area, the central information processing and control module (6) judges whether a potential user exists or not by detecting whether the face image can be identified or not, so that whether the commodity intelligent recommendation system continues to operate or not is controlled; when a potential user exists and the user cannot determine a target commodity, the image acquisition device (1) captures a face image of the user and calibrates pupil position coordinates, and the central information processing and control module (6) analyzes and calculates the deviation amount of the pupil position of the user and the effective acquisition area set by the eye tracker (3), so that preparation is made for adjustment and calibration of the eye tracker pose adjustment mechanism (4) and information acquisition of the eye tracker (3) at the later stage.
The commodity display area (2) is connected with the eye movement instrument (3) and the central information processing and control module (6), the commodity display area (2) displays commodities for users to purchase, and the central information processing and control module (6) divides the interest area according to the commodity positions displayed by the commodity display area (2) of the vending machine and numbers each commodity in sequence.
The eye movement instrument (3) collects eye movement track information of various commodities watched by a user, and transmits the eye movement track information to the central information processing and control module (6).
The eye movement instrument pose adjusting mechanism (4) has two adjustable degrees of freedom, when a user browses commodities in the display area (2), the eye movement instrument pose adjusting mechanism (4) realizes pitch angle adjustment of the eye movement instrument (3) through rotary movement around an x axis, and actively adjusts the angle of the eye movement instrument (3) to enable pupils of the user to be located in an information acquisition effective angle range of the eye movement instrument (3), so that the influence of the difference of heights of the pupils of eyes of users with different heights is avoided; the eye movement instrument pose adjusting mechanism (4) realizes adjustment of the horizontal distance between the eye movement instrument (3) and a user through translational movement along a y axis, actively compensates the change of the horizontal distance between the eye movement instrument (3) and the user, enables a pupil of the user to be positioned in an effective horizontal distance range of information acquisition of the eye movement instrument (3), thereby avoiding influence of front-back distance change of a user station caused by vision height difference, ensuring that the eye movement instrument pose adjusting mechanism (4) can actively adjust a pitch angle and the horizontal distance in real time according to the deviation amount of calibrated pupil position information and an effective information acquisition area of the eye movement instrument (3), and enables the pupil to be positioned in the effective information acquisition area of the eye movement instrument (3) and complete information acquisition work, wherein the y axis is perpendicular to a display area (2) and is in a positive direction along the eye gazing direction of the user, the x axis is parallel to the ground and perpendicular to the y axis, and the right hand direction of the user is in a positive direction, and the right hand direction of the user is known by a right hand rule, and the vertical ground is in a positive direction of the z axis.
The electrocardio detection device (5) comprises a palm touch terminal (51) and an electrocardio information acquisition component (52), a user instructs one hand to conduct interactive browsing operation with the commodity display area (2) according to the voice prompt component (7), the other hand is located at the palm touch terminal (51), further, the palm touch terminal (51) can automatically identify and convert left hand modes and right hand modes, and the electrocardio information acquisition component (52) acquires heart rate of the user when browsing commodities according to acquisition frequency of the synchronous oculograph (3).
The central information processing and controlling module (6) specifically comprises an information acquisition unit (61), an information calculation processing unit (62) and an instruction transmission control unit (63), wherein the information acquisition unit (61) receives image information acquired by the image acquisition device (1) in an identification potential user link, acquires information output by the eye movement instrument (3) and the electrocardio detection device (5) at a specified sampling frequency in an intelligent recommendation link, the information calculation processing unit (62) processes and divides the identification potential user link according to the image acquired by the image acquisition device (1), further completes face recognition and pupil position coordinate calibration, takes pupil position coordinate as a reference, calculates the deviation amount of the pupil position of the user and the information acquisition effective position area of the eye movement instrument (3), analyzes and calculates the acquired eye movement track information and heart rate information in the intelligent recommendation link to obtain the heart indexes of various commodities of the user, obtains the favorite commodity serial numbers of the user by comparing the heart indexes of the user, and the instruction transmission control unit (63) operates each component part in the identification potential user control system, drives the eye movement instrument position and gesture adjusting mechanism (4) to complete active adjustment in an adjustment period, and calculates and outputs corresponding heart movement command deviation amount to the intelligent recommendation terminal 8 or not according to the command.
The voice prompt component (7) guides the user to complete the interaction process with the vending machine (9) so as to help the user to complete the operation steps correctly.
The execution terminal (8) can independently display the recommended commodity information through visual and audio modes such as images and voices, and can also display the recommended commodity information on a commodity display area (2) platform through visual and audio modes such as images and voices.
Based on the intelligent commodity recommendation system of the efficient self-service vending machine, the corresponding method comprises the following steps:
step S100: the image acquisition device (1) acquires images in the purchase area in real time, a central information processing and control module (6) judges whether potential users exist through face recognition, and if the potential users exist, the step S200 is sequentially executed; otherwise, repeatedly executing the step S100;
step S200: judging whether the user determines a target commodity according to whether the user has an operation instruction or not, if the user cannot determine the target commodity, controlling the intelligent recommendation system to continue to execute by the central information processing and control module (6), and sequentially executing the following steps; otherwise, stopping execution by the intelligent recommendation system, and ending the direct jump;
step S300: the eye movement instrument pose adjusting mechanism (4) actively compensates and adjusts the eye movement instrument (3) to the set effective acquisition area according to the deviation amount of the current position of the pupil of the user and the effective acquisition area set by the eye movement instrument (3), and meanwhile, the voice prompt component (7) instructs the user to perform interactive operation with the electrocardiographic detection device (5);
step S400: the user browses the commodity display area (2) in the information acquisition period, the eye movement instrument (3) acquires eye movement track information when the user browses various commodities, and the electrocardiograph detection device (5) acquires corresponding heart rate values;
step S500: the central information processing and control module (6) compares and screens the acquired information to obtain the favorite commodity serial numbers of the users;
step S600: the central information processing and control module (6) drives the execution terminal (8) to display the recommended commodity information.
Further, step S100 includes: the image acquisition device (1) acquires human face image information in a vending machine purchasing area in real time, if no person passes through the area or passes through the area but does not stay in the area, the image acquisition device (1) outputs no identifiable human face front image in the information, the central information processing and control module (6) judges that the image information analysis processing result is no potential user, and meanwhile, the commodity intelligent recommendation system is controlled to repeatedly execute the step; if someone is located in the purchasing area and views the commodity display area (2) to display the commodity, the image acquisition device (1) outputs a face front image with identifiable information, the central information processing and control module (6) judges that the image information analysis processing result is a potential user, and meanwhile controls the intelligent commodity recommendation system to execute the next step;
further, step S200 includes: the voice prompt component (7) instructs the user to browse the commodity display area (2), if the user does not select the commodity within the system setting time, the step S300 is sequentially executed, and if the user browses the commodity within the system setting time, the system displays specific commodity information according to the user selection;
further, step S300 includes: the information calculation processing unit (62) further processes and divides the face image to finish face recognition and marks pupil position coordinates as a reference, calculates pupil position coordinates and analyzes the deviation amount of the user and the information acquisition effective area of the eye tracker (3), so that the position and posture adjustment mechanism (4) of the eye tracker automatically and actively compensates and adjusts according to the current deviation amount until the pupil of the user is positioned in the effective acquisition area of the eye tracker (3); meanwhile, the voice prompt component (7) prompts a user to put the palm in an effective detection area of the palm touch terminal (51), when the electrocardio information acquisition component (52) fails to detect heart rate information of the user, the voice prompt component (7) prompts the user to correct the palm placement position, otherwise, the voice prompt component (7) prompts the user to finish the appointed operation;
further, step S400 includes: the commodity display area (2) displays commodities in the vending machine (9), when a user browses the commodities, the eye movement instrument (3) collects eye movement track data of the user, the electrocardio detection device (5) collects heart rate data of the user at a specified frequency, and the corresponding data are transmitted to the central information processing and control module (6);
further, step S500 includes: the central information processing and control module (6) sequentially divides and numbers the regions of interest of the commodities according to the commodity display positions, obtains the position points of the pupils of the user staying in each region of interest by utilizing the eye movement track information acquired by the eye movement instrument (3), and counts the time length T corresponding to the position points of each region of interest 1 、T 2 、T 3 、…T n At the same time, the electrocardiograph detection device (5) collects heart rate information of the user at the same sampling frequency, the information calculation processing unit (62) compares the gazing duration of each commodity with a set threshold T, and if the gazing duration T of the commodity serial number r r If T is less than or equal to T, the user favors the commodity, and the central information processing and control module (6) does not count the heart rate of the time corresponding to the commodity number r; if the fixation time T of commodity number r r >T, the interest of the user on the commodity is high, and the central information processing and control module (6) counts the heart rate value of the time corresponding to the commodity serial number r of fixation; the electrocardio detection device (5) collects that the heart rate value of the commodity with the user watching time longer than the set threshold T is respectively X u 、X v 、…X z An information calculation processing unit (62) calculates the heart rate value X acquired u 、X v 、…X z And (3) performing descending order arrangement, wherein the commodities corresponding to the first three orders of the heart rate value ordering are recommended commodities, and transmitting a recommended commodity display instruction to the execution terminal (8).
The invention has the beneficial effects that: the intelligent commodity recommending system and method for the high-efficiency self-service vending machine can accurately identify potential users, automatically adjust the pose of the eye movement instrument according to the characteristics and the positions of the different heights of the users, rapidly process the eye movement track and heart rate data when the users browse commodities by adopting a threshold screening method, intelligently recommend the heart instrument commodities, not only help users with difficulty in selecting shopping, improve purchasing efficiency, but also have strong operation visibility, and meet the high-efficiency requirements of the users.
Drawings
FIG. 1 is a block diagram of an intelligent commodity recommendation system for a high-efficiency self-service vending machine;
FIG. 2 is a schematic diagram of efficient and rapid capture of eye movement information;
FIG. 3 is a flow chart of an intelligent commodity recommendation system for a high-efficiency self-service vending machine;
FIG. 4 is a schematic diagram of a central information processing and control module dividing a region of interest;
FIG. 5 is a schematic diagram of deviation compensation of an eye movement device pose adjustment mechanism;
reference numerals illustrate:
1. an image acquisition device; 2. a merchandise display area; 3. an eye movement instrument; 4. an eye movement instrument pose adjusting mechanism; 5. an electrocardiograph detection device; 6. a central information processing and control module; 7. a voice prompt component; 8. an execution terminal; 9. a vending machine;
51. a palm touch terminal; 52. an electrocardiographic information acquisition component; 61. an information acquisition unit; 62. an information calculation processing unit; 63. an instruction transmission control unit.
Detailed Description
The following describes specific embodiments of the present invention in detail with reference to the drawings.
Fig. 1 is a block diagram of a high-efficiency intelligent commodity recommendation system for a self-service vending machine. As shown in fig. 1, the intelligent commodity recommendation system includes: the system comprises an image acquisition device (1), a commodity display area (2), an eye tracker (3), an eye tracker pose adjusting mechanism (4), an electrocardiograph detection device (5), a central information processing and control module (6), a voice prompt component (7) and an execution terminal (8); further, the intelligent recommendation process includes: the image acquisition device (1) acquires image information of a purchase area, the central information processing and control module (6) detects whether potential users exist through face recognition and judges whether the commodity intelligent recommendation system continues to operate, the voice indication assembly (7) guides the users to be located right in front of the commodity display area (2) and places palms in an information acquisition effective area of the electrocardio detection device (5), the central information processing and control module (6) analyzes face images of the users acquired by the image acquisition device (1) and calibrates pupil position information, calculates deviation amounts of pupil positions of the users and information acquisition effective areas of the eye tracker (3), the position posture adjustment mechanism (4) of the eye tracker automatically adjusts pitch angles of the eye tracker (3) and horizontal distances between the eye tracker (3) and the users according to the obtained deviation amounts, enables pupils to be located in the information effective acquisition area, accordingly eye movement track information and heart rate information of all commodities of the users are effectively and rapidly captured, and the central information processing and control module (6) further calculates eye movement track information acquired by the users and browses the heart rate information of commodities, and analyzes heart rate information of the commodity recommendation terminal (8) and displays commodity indexes.
In the embodiment, the image acquisition device (1) adopts a face recognition binocular camera, the face recognition binocular camera is fixedly connected with the vending machine (9), images in a purchase area are acquired in real time, the information acquisition unit (61) receives the images transmitted by the face recognition binocular camera, the information calculation processing unit (62) analyzes whether the images have identifiable face images to judge whether potential users exist or not, pupil centers are marked in the identifiable face images, two groups of pupil center position coordinate measurement values are obtained by the aid of the binocular camera, the information calculation processing unit (62) calculates the two groups of measurement values in a weighting mode to obtain final pupil positions, and deviation amounts of the users and the effective area set by the eye movement instrument (3) are further analyzed and calculated.
The commodity display area (2) is connected with the eye movement instrument (3) and the central information processing and control module (6), the commodity display area (2) displays commodities for users to purchase, and the central information processing and control module (6) divides the interest area according to the positions of the commodities displayed by the commodity display area (2) and numbers each commodity in sequence.
The eye movement instrument (3) collects eye movement track information when a user views various commodities, and transmits the eye movement track information to the central information processing and control module (6).
As shown in fig. 1, the electrocardiograph detection device (5) includes a palm touch terminal (51) and an electrocardiograph information collection component (52), a user performs interactive browsing operation with the commodity display area (2) according to the indication of the voice prompt component (7), the other hand is located at the palm touch terminal (51), further, the palm touch terminal (51) can automatically identify and convert left and right hand modes, and the electrocardiograph information collection component (52) collects the heart rate value when the user browses the commodity.
As shown in fig. 1, the central information processing and controlling module (6) specifically includes an information collecting unit (61), an information calculating and processing unit (62) and an instruction transmission control unit (63), the information collecting unit (61) receives image information collected by the image collecting device (1) in the identification potential user link, collects information output by the eye movement instrument (3) and the electrocardiograph detecting device (5) in the intelligent recommending link at a specified sampling frequency, the information calculating and processing unit (62) processes and divides the information according to images collected by the image collecting device (1) in the identification potential user link, further completes face recognition and pupil position coordinate calibration, takes pupil position coordinate as a reference and calculates the deviation amount of the pupil position of the user and the effective position area of the eye movement instrument (3), analyzes and calculates the collected eye movement track information and heart rate information in the intelligent recommending link to obtain cardiac indexes of various commodities of the user, obtains the favorite commodity serial numbers of the user by comparing the cardiac indexes, the instruction transmission control unit (63) operates in each component part of the identification potential user link control system, drives the eye movement instrument pose adjusting mechanism (4) to complete active adjustment in the adjusting period, calculates the corresponding movement command of the user in the intelligent compensating and outputs the corresponding command of the recommended terminal 8 according to the deviation amount.
The voice prompt component (7) guides the user to complete the interaction process with the vending machine (9) so as to help the user to complete the operation steps correctly.
The execution terminal (8) can independently display the recommended commodity information through visual and audio modes such as images and voices, and can also display the recommended commodity information on a commodity display area (2) platform through visual and audio modes such as images and voices.
FIG. 2 is a schematic view of high-efficiency and rapid capturing of eye movement information, as shown in FIG. 2, the y-axis is perpendicular to the display area (2) and is in a positive direction along the user's eye gaze direction, the x-axis is parallel to the ground and perpendicular to the y-axis, and the user's right-hand direction is in a positive direction, as can be seen from the right-hand rule, the vertical ground is in a positive z-axis direction, L in the figure 1 、L 2 For the lower limit and the upper limit of the set effective information acquisition area in the y direction in the eye tracker (3), theta is the projection angle of the set effective information acquisition area in the eye tracker (3) on the yz plane, when a user a is positioned in the position shown in fig. 2, the eye tracker pose adjusting mechanism (4) is in an initial state and does not need to adjust the position of the eye tracker (3), and when another user b is positioned in the position shown in fig. 2, the eye tracker pose adjusting mechanism (4) realizes the active adjustment of the pitch angle of the eye tracker (3) through the rotation motion around the x axis, so that the user is positioned in the effective angle range theta of information acquisition after the eye tracker (3) rotates by delta theta in the yz plane, thereby avoiding the influence of the difference of the pupil heights of eyes of users with different heights; active adjustment of the horizontal distance between the eye movement device (3) and the user is realized through translational movement along the y axis, so that the eye movement device (3) is translated by delta L in the y direction, and the user is positioned in the effective horizontal range L of information acquisition 1 ～L 2 And the influence of the change of the distance between the front and back of the user station caused by the difference of the vision height is avoided, and the position and posture adjusting mechanism (4) of the eye movement instrument can actively adjust the pitch angle and the horizontal distance according to the deviation of the calibrated pupil position information and the information acquisition effective area of the eye movement instrument (3) in real time, so that the pupil is positioned in the effective information acquisition area of the eye movement instrument (3) and the information acquisition work is completed.
According to the intelligent commodity recommending system of the efficient self-service vending machine, for users with difficult selection, user information is collected in two directions by using the eye movement instrument (3) and the electrocardiograph detection device (5), and the vending machine execution terminal (8) is driven to display recommended commodity information through the central information processing and control module (6), so that the intelligent degree of the self-service vending machine is improved to a certain extent; meanwhile, the system adopts the image acquisition device (1), and judges whether a potential user exists or not to control the operation stop of other components in the system by utilizing the face recognition, so that the operation cost of the vending machine is saved to a certain extent; the eye movement instrument pose adjusting mechanism (4) is combined with the image acquisition device (1) to automatically adjust the eye movement instrument (3) to an information acquisition effective area, so that the operation efficiency of the system is improved.
Based on the intelligent recommendation system for the high-efficiency self-service vending goods in the embodiment, a corresponding intelligent recommendation method is further provided, and fig. 3 is a flow chart of the intelligent recommendation system for the high-efficiency self-service vending goods.
As shown in fig. 3, the intelligent commodity recommendation method of the high-efficiency self-service vending machine comprises the following steps:
step S100: the image acquisition device (1) acquires images in the purchase area in real time, a central information processing and control module (6) judges whether potential users exist through face recognition, and if the potential users exist, the step S200 is sequentially executed; otherwise, repeatedly executing the step S100;
step S200: judging whether the user determines a target commodity according to whether the user has an operation instruction or not, if the user cannot determine the target commodity, controlling the intelligent recommendation system to continue to execute by the central information processing and control module (6), and sequentially executing the following steps; otherwise, stopping execution by the intelligent recommendation system, and ending the direct jump;
step S300: the eye movement instrument pose adjusting mechanism (4) actively compensates and adjusts the eye movement instrument (3) to the set effective acquisition area according to the deviation amount of the current position of the pupil of the user and the effective acquisition area set by the eye movement instrument (3), and meanwhile, the voice prompt component (7) instructs the user to perform interactive operation with the electrocardiographic detection device (5);
step S400: the user browses the commodity display area (2) in the information acquisition period, the eye movement instrument (3) acquires eye movement track information when the user browses various commodities, and the electrocardiograph detection device (5) acquires corresponding heart rate values;
step S500: the central information processing and control module (6) compares and screens the acquired information to obtain the favorite commodity serial numbers of the users;
step S600: the central information processing and control module (6) drives the execution terminal (8) to display the recommended commodity information.
Further, step S100 includes: the face recognition binocular camera collects human face image information of a vending machine purchasing area in real time, if no person passes through the area or passes through the area but does not stay in the area, no face front image exists in the information collected by the face recognition binocular camera, the central information processing and control module (6) judges that an image information analysis processing result is no potential user, and meanwhile controls the commodity intelligent recommendation system to repeatedly execute the steps; if a person is located in the purchasing area and views the commodity displayed in the commodity display area (2), the face front image exists in the information acquired by the face recognition binocular camera, the central information processing and control module (6) judges that the image information analysis processing result is a potential user, and meanwhile controls the intelligent commodity recommendation system to execute the next step;
further, step S200 includes: the voice prompt component (7) instructs the user to browse the commodity display area (2), if the user does not select the commodity within the system setting time, the step S300 is sequentially executed, and if the user browses the commodity within the system setting time, the system displays specific commodity information according to the user selection;
further, step S300 includes: the information calculation processing unit (62) further processes and divides the face image to finish face recognition and marks pupil position coordinates as a reference, calculates pupil position coordinates and analyzes the deviation amount of the user and the information acquisition effective area of the eye tracker (3), so that the position and posture adjustment mechanism (4) of the eye tracker automatically and actively compensates and adjusts according to the current deviation amount until the pupil of the user is positioned in the effective acquisition area of the eye tracker (3); meanwhile, the voice prompt component (7) prompts a user to put the palm in an effective detection area of the palm touch terminal (51), when the electrocardio information acquisition component (52) fails to detect heart rate information of the user, the voice prompt component (7) prompts the user to correct the palm placement position, otherwise, the voice prompt component (7) prompts the user to finish the appointed operation;
further, step S400 includes: the commodity display area (2) displays commodities in the vending machine (9), when a user browses the commodities, the eye movement instrument (3) collects eye movement track data of the user, the electrocardio detection device (5) collects heart rate data of the user at a specified frequency, and the corresponding data are transmitted to the central information processing and control module (6);
further, step S500 includes:the central information processing and control module (6) sequentially divides and numbers the regions of interest of the commodities according to the commodity display positions, obtains the position points of the pupils of the user staying in each region of interest by utilizing the eye movement track information acquired by the eye movement instrument (3), and counts the time length T corresponding to the position points of each region of interest 1 、T 2 、T 3 、…T n At the same time, the electrocardiograph detection device (5) collects heart rate information of the user at the same sampling frequency, the information calculation processing unit (62) compares the gazing duration of each commodity with a set threshold T, and if the gazing duration T of the commodity serial number r r If T is less than or equal to T, the user favors the commodity, and the central information processing and control module (6) does not count the heart rate of the time corresponding to the commodity number r; if the fixation time T of commodity number r r >T, the interest of the user on the commodity is high, and the central information processing and control module (6) counts the heart rate value of the time corresponding to the commodity serial number r of fixation; the electrocardio detection device (5) collects that the heart rate value of the commodity with the user watching time longer than the set threshold T is respectively X u 、X v 、…X z An information calculation processing unit (62) calculates the heart rate value X acquired u 、X v 、…X z And (3) performing descending order arrangement, wherein the commodities corresponding to the first three orders of the heart rate value ordering are recommended commodities, and transmitting a recommended commodity display instruction to the execution terminal (8).
Fig. 4 is a schematic diagram of a central information processing and control module dividing regions of interest, as shown in fig. 4, the system sequentially divides corresponding regions of interest according to commodities displayed in a commodity display area (2), in the figure, a region n is a region where each commodity is located, a dashed line frame is the divided regions of interest, and the commodities are numbered in one-to-one correspondence with positions, so that the control system and commodity information are unified.
Fig. 5 is a schematic diagram of deviation compensation of the position and posture adjustment mechanism of the eye tracker, as shown in fig. 5, the y and z directions of the position and posture adjustment mechanism (4) of the eye tracker relative to the vending machine are kept consistent with the coordinate directions shown in fig. 2, fig. 5 (a) shows an initial position of the position and posture adjustment mechanism (4) of the eye tracker, namely, a position state of the position and posture adjustment mechanism (4) of the eye tracker when the eye tracker (3) does not work or does not need to be adjusted, fig. 5 (b) corresponds to a specific deviation compensation adjustment process of the position and posture adjustment mechanism (4) of the eye tracker when the other user b in fig. 2 is out of the effective information collection range of the eye tracker (3), and the horizontal distance and pitch angle adjustment of the eye tracker (3) are completed by translating delta L in the y direction and rotating around the x axis to an angle delta theta with the z axis.
The intelligent recommendation method of the high-efficiency self-service vending machine can assist in selection according to the purchase demand of the user, and can help the user to select the subjective heart instrument commodity through the physical behavior characteristics when the user with difficulty in selecting the target commodity can not be determined, so that the intelligent demand of the user is met, and the purchase speed of the user is improved; and a threshold screening method is adopted to simplify the signal processing process and improve the system operation efficiency.
Claims (2)
1. An intelligent commodity recommendation system for a high-efficiency self-service vending machine, which is characterized by comprising: the system comprises an image acquisition device, a commodity display area, an eye tracker pose adjusting mechanism, an electrocardiograph detection device, a central information processing and control module, a voice prompt component and an execution terminal, wherein the image acquisition device acquires image information of a purchase area, the central information processing and control module detects whether potential users exist and judges whether a commodity intelligent recommending system continuously operates or not through face recognition, the voice prompt component guides the users to be positioned right in front of the commodity display area and places palms in an information acquisition effective area of the electrocardiograph detection device, the central information processing and control module analyzes face images of the users acquired by the image acquisition device and marks pupil position information, calculates deviation between pupil positions of the users and the information acquisition effective area of the eye tracker, automatically adjusts pitch angle of the eye tracker and horizontal distance between the eye tracker and the users according to the obtained deviation amount, enables pupils to be positioned in the information effective acquisition area, and accordingly captures eye movement information effectively and rapidly, the eye movement track information and heart rate information when the eye tracker and the electrocardiograph detection device respectively record commodities of the users, the central information processing and the control module further analyzes the acquired eye movement track and the information and the heart rate information when the users browse commodities, and controls the display of the terminal to display heart rate information of the commodities;
the image acquisition device is fixedly connected with the vending machine, acquires images in the purchase area in real time, and when a user passes through the purchase area, the central information processing and control module judges whether a potential user exists or not by detecting whether the face image can be identified or not, so that whether the commodity intelligent recommendation system continues to operate or not is controlled; when a potential user exists and the user cannot determine a target commodity, the image acquisition device captures a user face image, calibrates pupil position coordinates, and the central information processing and control module analyzes and calculates the deviation amount of the pupil position of the user and the effective acquisition area set by the eye tracker, so as to prepare for adjustment and calibration of the eye tracker pose adjustment mechanism and later-stage eye tracker information acquisition;
the commodity display area is connected with the eye tracker and the central information processing and control module, the commodity display area displays commodities for users to purchase, and the central information processing and control module divides the interest area according to the positions of the commodities displayed in the commodity display area of the vending machine and numbers each commodity in sequence;
the eye movement instrument collects eye movement track information of various commodities watched by a user and transmits the eye movement track information to the central information processing and control module;
the eye movement instrument pose adjusting mechanism has two adjustable degrees of freedom, and when a user browses commodities in a display area, the eye movement instrument pose adjusting mechanism realizes the adjustment of the pitch angle of the eye movement instrument through rotary motion around an x axis, actively adjusts the angle of the eye movement instrument, and enables the pupil of the user to be positioned in the effective angle range of information acquisition of the eye movement instrument, so that the influence of the difference of the heights of the pupils of eyes of users with different heights is avoided; the eye movement instrument pose adjusting mechanism realizes adjustment of the horizontal distance between the eye movement instrument and a user through translational movement along a y axis, actively compensates the change of the horizontal distance between the eye movement instrument and the user, enables the pupil of the user to be positioned in the effective horizontal distance range of information acquisition of the eye movement instrument, thereby avoiding the influence of the change of the front and back distances of a user station caused by the difference of vision heights, ensuring that the eye movement instrument pose adjusting mechanism can actively adjust the pitch angle and the horizontal distance in real time according to the deviation amount of calibrated pupil position information and the effective information acquisition area of the eye movement instrument, enabling the pupil to be positioned in the effective information acquisition area of the eye movement instrument and completing information acquisition work, wherein the y axis is perpendicular to a display area and is in the positive direction along the eye fixation direction of the user, the x axis is parallel to the ground and perpendicular to the y axis, and the right hand direction of the user is in the positive direction, and the right hand rule indicates that the vertical ground is in the positive direction of the z axis;
the electrocardiograph detection device comprises a palm touch terminal and an electrocardiograph information acquisition component, wherein a user indicates one hand to perform interactive browsing operation with a commodity display area according to the voice prompt component, the other hand is positioned on the palm touch terminal, further, the palm touch terminal can automatically identify and convert left and right hand modes, and the electrocardiograph information acquisition component synchronizes the acquisition frequency of the eye tracker to acquire the heart rate of the user when browsing commodities;
the central information processing and controlling module specifically comprises an information acquisition unit, an information calculation processing unit and an instruction transmission control unit, wherein the information acquisition unit receives image information acquired by an image acquisition device in an identification potential user link, acquires information output by an eye movement instrument and an electrocardio detection device at a designated sampling frequency in an intelligent recommendation link, the information calculation processing unit processes and divides the image acquired by the image acquisition device in the identification potential user link, further completes face identification and pupil position coordinate calibration, takes the pupil position coordinate as a reference and calculates the deviation amount of the pupil position of the user and the effective position area acquired by the eye movement instrument information, analyzes and calculates the acquired eye movement track information and heart rate information in the intelligent recommendation link to obtain the cardiac indexes of various commodities of the user, compares the cardiac indexes to obtain the favorite commodity serial numbers of the user, and instructs the transmission control unit to drive whether each component part in the control system of the identification potential user link operates or not, and drive an eye movement instrument pose adjusting mechanism to complete active compensation adjustment of the displacement deviation amount in an adjusting period and output corresponding control instructions to an executing terminal in the intelligent recommendation link according to the heart calculation result of the user;
the voice prompt component guides the user to complete the interaction process with the vending machine, and helps the user to complete the operation steps correctly;
the execution terminal independently displays the recommended commodity information through an image or voice audio-visual mode or displays the recommended commodity information on a commodity display area platform through an image or voice audio-visual mode.
2. The intelligent commodity recommending system method for the efficient self-service vending machine is characterized by comprising the following steps of:
step S100: the image acquisition device acquires images in the purchase area in real time, judges whether potential users exist or not through face recognition by the central information processing and control module, and if the potential users exist, sequentially executes the step S200; otherwise, repeatedly executing the step S100;
step S200: judging whether the user determines a target commodity according to whether the user has an operation instruction or not, if the user cannot determine the target commodity, controlling the intelligent recommendation system to continue to execute by the central information processing and control module, and sequentially executing the following steps; otherwise, stopping execution by the intelligent recommendation system, and ending the direct jump;
step S300: the eye movement instrument pose adjusting mechanism actively compensates and adjusts the eye movement instrument to the set effective acquisition area according to the deviation amount of the current position of the pupil of the user and the set effective acquisition area of the eye movement instrument, and meanwhile, the voice prompt component instructs the user to perform interactive operation with the electrocardiograph detection device;
step S400: the method comprises the steps that in an information acquisition period, a user browses a commodity display area, an eye movement instrument acquires eye movement track information when the user browses various commodities, and an electrocardiograph detection device acquires corresponding heart rate values;
step S500: the central information processing and control module compares and screens the acquired information to obtain the favorite commodity serial numbers of the users;
step S600: the central information processing and control module drives the execution terminal to display the recommended commodity information;
the step S100 includes: the image acquisition device acquires human face image information in a vending machine purchasing area in real time, if no person passes through the area or passes through the area but does not stay in the area, the image acquisition device outputs no identifiable human face front image in the information, the central information processing and control module judges that the image information analysis processing result is no potential user, and meanwhile, the commodity intelligent recommendation system is controlled to repeatedly execute the step; if someone is located in the purchasing area and views the commodity displayed in the commodity display area, the image acquisition device outputs a face front image which can be identified in the information, the central information processing and control module judges that the image information analysis processing result is a potential user, and meanwhile controls the intelligent commodity recommendation system to execute the next step;
the step S200 includes: the voice prompt component indicates the user to browse the commodity display area, if the user does not select the commodity within the system setting time, the step S300 is sequentially executed, and if the user browses and selects the commodity within the system setting time, the system displays specific commodity information according to the user selection;
the step S300 includes: the information calculation processing unit further processes and divides the face image to finish face recognition and marks pupil position coordinates as a reference, calculates the pupil position coordinates and analyzes the deviation amount of the user and the effective information acquisition area of the eye tracker, so that the position and posture adjustment mechanism of the eye tracker automatically and actively compensates and adjusts according to the current deviation amount until the pupil of the user is positioned in the effective information acquisition area of the eye tracker; meanwhile, the voice prompt component prompts a user to put the palm in an effective detection area of the palm touch terminal, when the electrocardio information acquisition component fails to detect heart rate information of the user, the voice prompt component prompts the user to correct the palm placement position, otherwise, the voice prompt component prompts the user to finish the appointed operation;
the step S400 includes: displaying commodities in the vending machine in the commodity display area, collecting user eye movement track data by the eye movement instrument when a user browses the commodities, collecting heart rate data of the user by the electrocardio detection device at a specified frequency, and transmitting the corresponding data to the central information processing and control module;
the step S500 includes: the central information processing and control module sequentially divides and numbers the regions of interest of the commodities according to the commodity display positions, obtains the position points of the pupils of the user staying in each region of interest by utilizing the eye movement track information acquired by the eye movement instrument, and counts the time length T corresponding to the position points of each region of interest 1 、T 2 、T 3 、…T n Simultaneously, the electrocardio detection device collects heart rate information of a user at the same sampling frequency, and information calculation processing is carried outThe unit compares the gazing duration of each commodity with a set threshold T, if the gazing duration T of the commodity serial number r r If T is less than or equal to T, the user favors the commodity, and the central information processing and control module does not count the heart rate of the time corresponding to the commodity number r watched; if the fixation time T of commodity number r r >T, the interest of the user on the commodity is high, and the central information processing and control module counts the heart rate value of the time corresponding to the commodity serial number r of fixation; the heart rate value of the commodity with the user watching time length being longer than the set threshold T collected by the electrocardio detection device is X respectively u 、X v 、…X z The information calculation processing unit calculates the heart rate value X acquired u 、X v 、…X z And (3) performing descending order arrangement, namely acquiring the recommended commodity corresponding to the first three orders of the heart rate value ordering, and transmitting a recommended commodity display instruction to the execution terminal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784731.5A CN111882365B (en) | 2020-08-06 | 2020-08-06 | Intelligent commodity recommendation system and method for efficient self-service vending machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010784731.5A CN111882365B (en) | 2020-08-06 | 2020-08-06 | Intelligent commodity recommendation system and method for efficient self-service vending machine |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111882365A CN111882365A (en) | 2020-11-03 |
| CN111882365B true CN111882365B (en) | 2024-01-26 |
Family
ID=73211466
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010784731.5A Active CN111882365B (en) | 2020-08-06 | 2020-08-06 | Intelligent commodity recommendation system and method for efficient self-service vending machine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111882365B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113111745B (en) * | 2021-03-30 | 2023-04-07 | 四川大学 | Eye movement identification method based on product attention of openposition |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102376061A (en) * | 2011-08-26 | 2012-03-14 | 浙江工业大学 | Omni-directional vision-based consumer purchase behavior analysis device |
| WO2016115895A1 (en) * | 2015-01-23 | 2016-07-28 | 北京工业大学 | On-line user type identification method and system based on visual behaviour |
| CN107247759A (en) * | 2017-05-31 | 2017-10-13 | 深圳正品创想科技有限公司 | A kind of Method of Commodity Recommendation and device |
| CN108446943A (en) * | 2018-01-30 | 2018-08-24 | 深圳市阿西莫夫科技有限公司 | Sales counter Method of Commodity Recommendation, device, computer equipment and storage medium |
| CN109166007A (en) * | 2018-08-23 | 2019-01-08 | 深圳码隆科技有限公司 | A kind of Method of Commodity Recommendation and its device based on automatic vending machine |
| CN109308762A (en) * | 2017-07-28 | 2019-02-05 | 橙果信息技术有限公司 | A kind of automatically vending system and method based on Identification of Images |
-
2020
- 2020-08-06 CN CN202010784731.5A patent/CN111882365B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102376061A (en) * | 2011-08-26 | 2012-03-14 | 浙江工业大学 | Omni-directional vision-based consumer purchase behavior analysis device |
| WO2016115895A1 (en) * | 2015-01-23 | 2016-07-28 | 北京工业大学 | On-line user type identification method and system based on visual behaviour |
| CN107247759A (en) * | 2017-05-31 | 2017-10-13 | 深圳正品创想科技有限公司 | A kind of Method of Commodity Recommendation and device |
| CN109308762A (en) * | 2017-07-28 | 2019-02-05 | 橙果信息技术有限公司 | A kind of automatically vending system and method based on Identification of Images |
| CN108446943A (en) * | 2018-01-30 | 2018-08-24 | 深圳市阿西莫夫科技有限公司 | Sales counter Method of Commodity Recommendation, device, computer equipment and storage medium |
| CN109166007A (en) * | 2018-08-23 | 2019-01-08 | 深圳码隆科技有限公司 | A kind of Method of Commodity Recommendation and its device based on automatic vending machine |
Non-Patent Citations (1)
| Title |
|---|
| 全自动无人售货与自动贩卖系统;马培立;张乃欣;陈哲;吴杰;杨雪;;兵工自动化(第03期);35-40 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111882365A (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7872635B2 (en) | Foveated display eye-tracking system and method | |
| EP2420305A1 (en) | Device capable of adjusting images according to body motion of user and performing method thereof | |
| US20190235624A1 (en) | Systems and methods for predictive visual rendering | |
| US20180342066A1 (en) | Apparatus and method for hybrid eye tracking | |
| CN103838378B (en) | A kind of wear-type eyes control system based on pupil identification positioning | |
| EP2470060B1 (en) | Visual perimeter measurement system and method | |
| US20180107274A1 (en) | Display control device, control method thereof, and display control system | |
| CN102830797A (en) | Man-machine interaction method and system based on sight judgment | |
| CN111882365B (en) | Intelligent commodity recommendation system and method for efficient self-service vending machine | |
| CN107694046A (en) | A kind of body building training method, device and computer-readable recording medium | |
| CN112223288A (en) | Visual fusion service robot control method | |
| AU2020102642A4 (en) | Machine Learning-based Smart Workout Mirror and Method Thereof | |
| CN111861589B (en) | Intelligent commodity recommendation system and intelligent recommendation method for self-service vending machine | |
| CN102467230A (en) | Method and system for operating and controlling vernier by human body | |
| Bardins et al. | Gaze-based interaction in various environments | |
| CN113100755B (en) | Limb rehabilitation training and evaluating system based on visual tracking control | |
| KR20040097477A (en) | Head mounted computer interfacing device and method using eye-gaze direction | |
| WO2022107651A1 (en) | Information processing device, system, information processing method, and information processing program | |
| CN108804246A (en) | The usability evaluation method of upper limb rehabilitation robot | |
| CN114052649A (en) | Alternate covering strabismus diagnosis method based on virtual reality and eye movement tracking technology | |
| CN109634431B (en) | Medium-free floating projection visual tracking interaction system | |
| CN111651043A (en) | Augmented reality system supporting customized multi-channel interaction | |
| CN116795144B (en) | Method for self-adaptively adjusting angle and height of screen of liquid crystal display | |
| CN116153510B (en) | Correction mirror control method, device, equipment, storage medium and intelligent correction mirror | |
| CN116458835B (en) | Detection and prevention system for myopia and amblyopia of infants |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |