WO2020007323A1 - 商超购物车定位方法、商超购物车定位系统及商超购物车 - Google Patents

商超购物车定位方法、商超购物车定位系统及商超购物车 Download PDF

Info

Publication number
WO2020007323A1
WO2020007323A1 PCT/CN2019/094574 CN2019094574W WO2020007323A1 WO 2020007323 A1 WO2020007323 A1 WO 2020007323A1 CN 2019094574 W CN2019094574 W CN 2019094574W WO 2020007323 A1 WO2020007323 A1 WO 2020007323A1
Authority
WO
WIPO (PCT)
Prior art keywords
shopping cart
positioning
partition
preset
distance
Prior art date
Application number
PCT/CN2019/094574
Other languages
English (en)
French (fr)
Inventor
许景涛
索健文
李月
徐博
张治国
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US16/633,393 priority Critical patent/US11288839B2/en
Publication of WO2020007323A1 publication Critical patent/WO2020007323A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • Embodiments disclosed in the present disclosure relate to a shopping mall shopping cart positioning method, a shopping mall shopping cart positioning system, and a shopping mall shopping cart.
  • smart shopping carts usually include positioning functions, and existing smart shopping carts usually use wireless positioning or visual positioning.
  • accuracy of wireless positioning methods such as Bluetooth positioning and WIFI positioning is poor, and it is obviously affected by environmental interference.
  • the size of the accessible partitions in supermarkets is often not too small or smaller than the positioning accuracy of Bluetooth or WIFI positioning, leading to smart shopping carts.
  • the positioning deviation is large, and the user experience is poor.
  • image positioning for visual positioning has high positioning accuracy, the positioning calculation is large, and the requirements for processing modules are high.
  • the electronics on the smart shopping cart cannot meet the visual positioning processing. Module requirements.
  • At least one embodiment of the present disclosure provides a shopping mall shopping cart positioning method, including: collecting a first reference image of a shopping mall, and processing the first reference image to obtain a partition in which a shopping cart is located, where the shopping cart is located.
  • the super includes a plurality of partitions; receives distance information sent by at least one signal transmitting module set in the positioning partition, wherein each partition is provided with at least one signal transmitting module; and sends according to at least one signal corresponding to the positioning partition.
  • the distance information sent by the module obtains the first positioning position of the shopping cart, and determines the actual position of the shopping cart based on the first positioning position of the shopping cart.
  • determining the actual position of the shopping cart based on the first positioning position of the shopping cart includes: setting the first positioning position of the first shopping cart As the actual location of the shopping cart.
  • the method for locating a shopping cart further includes: obtaining a plurality of first positioning positions of the shopping cart at a plurality of consecutive times, and determining a plurality of corresponding ones of the plurality of first positioning positions, respectively.
  • Partition determining the most numerous partition among the plurality of partitions corresponding to the plurality of first positioning positions as the current partition; if the current partition is different from the positioning partition, collecting a second reference image of the commercial supermarket again And processing the second reference image to obtain a second positioning position of the shopping cart, and using the second positioning position to replace the first positioning position corresponding to the last time among the plurality of consecutive times as the shopping cart If the current partition is the same as the positioning partition, the first positioning position corresponding to the last one of the plurality of consecutive times is used as the actual position of the shopping cart.
  • the shopping mall includes a shelf partition and a passing partition, and the passing partition includes a plurality of grids connected in sequence; the shopping cart positioning method The method further includes: obtaining a first positioning position of the shopping cart corresponding to the first time and a first positioning position of the shopping cart corresponding to the second time; and calculating the first positioning position corresponding to the first time and the quotient.
  • correcting the actual position of the shopping cart by a magnetometer sensor includes: obtaining a first angle of the magnetometer at the first moment The detection information and the second angle detection information of the magnetometer at the second time, and calculate the angle difference between the first time and the second time; if the angle difference is less than or equal to a preset angle value Taking the position of the center point of the grid corresponding to the minimum moment value of the first moment corresponding to the first moment as the actual position of the shopping cart; if the angle difference is greater than the preset angle value and the distance If the difference is less than or equal to the second preset distance value, the position of the center point of the grid corresponding to the second time minimum distance value corresponding to the second time is taken as the actual position of the shopping cart; if the angle difference If the value is greater than a preset angle value and the distance difference is greater than the second preset distance value, the position of the center point of the grid corresponding to the minimum distance value at the
  • the positioning partition corresponds to at least three signal sending modules
  • the shopping is obtained based on distance information sent by at least one signal sending module corresponding to the positioning partition.
  • the first positioning position of the vehicle includes: acquiring at least three distance information sent by the at least three signal transmitting modules corresponding to the positioning subdivision; determining the smallest three pieces of distance information among the at least three pieces of distance information; and The position coordinates of the three signal sending modules corresponding to the smallest three distance information; and the shopping cart is obtained based on the three point weighted centroid positioning algorithm based on the minimum three distance information and the position coordinates of the three signal sending modules The first positioning position.
  • the shopping cart further includes a plurality of preset identifiers, each partition corresponding to at least one preset identifier, and the plurality of preset identifiers include a first A preset identifier
  • the first reference image includes a first preset identifier in the supermarket
  • processing the first reference image to obtain a partition in which a shopping cart is located as a positioning partition includes processing the first reference image to Identifying a first preset identifier in the first reference image; and determining, according to the first preset identifier, a partition in which the shopping cart is located as a positioning partition.
  • the partition where the shopping cart is located corresponds to the first preset identifier in the first reference image.
  • the plurality of preset identifiers further include a second preset identifier
  • the second reference image includes the second preset identifier
  • At least one embodiment of the present disclosure also provides a shopping mall shopping cart positioning system, including: an image acquisition device configured to collect a first reference image of the shopping mall, wherein the shopping mall includes a plurality of partitions; a wireless communication device, Configured to receive distance information sent by a plurality of signal transmitting modules in the supermarket; a processing device configured to receive the first reference image, and process and obtain the first reference image to determine and determine a partition where the shopping cart is located A positioning partition, and obtaining a first positioning position of the shopping cart according to distance information sent by at least one signal sending module corresponding to the positioning partition, and determining an actual position of the shopping cart based on the first positioning position of the shopping cart.
  • the processing device is configured to use a first positioning position of the first shopping cart as an actual position of the shopping cart.
  • the processing device is further configured to: obtain multiple first positioning positions of the shopping cart at multiple consecutive times, and determine the multiple A plurality of partitions corresponding to the first positioning position, and determining that a partition with the largest number of the plurality of partitions corresponding to the plurality of first positioning positions is a current partition; if the current partition is different from the positioning partition, the The image acquisition device is further configured to acquire a second reference image of the commercial supermarket again, and the processing device is further configured to process the second reference image to obtain a second positioning position of the shopping cart, using the second The positioning position replaces the first positioning position corresponding to the last time of the plurality of consecutive times as the actual position of the shopping cart. If the current partition is the same as the positioning partition, the processing device is further configured to match the The first positioning position corresponding to the last time among the plurality of consecutive times is used as the actual position of the shopping cart.
  • the shopping cart positioning system further includes a magnetometer that can be fixed on the shopping cart; the shopping cart includes a shelf partition and a transit partition, and the transit partition includes a plurality of grids connected in sequence.
  • the processing device is further configured to: obtain a first positioning position of the shopping cart corresponding to the first time and a first positioning position of the shopping cart corresponding to the second time; calculate a first positioning position corresponding to the first time The distance between a positioning position and the center points of all the grids in the commercial supermarket to obtain a first set of distance values, and calculating the first positioning position corresponding to the second moment and all the grids in the commercial supermarket The distance from the center point to obtain the second set of distance values, the minimum distance value at the first moment in the first set of distance values and the minimum distance value at the second moment in the second set of distance values are selected, and The distance difference between the minimum distance value at the first moment and the minimum distance value at the second moment and the first preset distance value; when the distance difference is less than or equal to
  • the processing device is further configured to: acquire the first angle detection information of the magnetometer and the second angle at the first moment.
  • the second angle detection information of the magnetometer calculates the angle difference between the first time and the second time; if the angle difference is less than or equal to a preset angle value, the first time The position of the center point of the grid corresponding to the minimum distance value at the corresponding first moment is used as the actual position of the shopping cart; if the angle difference is greater than the preset angle value and the distance difference is less than or equal to the second preset value Set the distance value, and use the position of the center point of the grid corresponding to the second time minimum distance value corresponding to the second time as the actual position of the shopping cart; if the angle difference is greater than a preset angle value and The distance difference is greater than the second preset distance value, and the position of the center point of the grid corresponding to the minimum distance value at the first time corresponding to the first time is used as the actual position of the shopping cart; if the angle difference is greater than a preset angle value
  • the positioning partition corresponds to at least three signal sending modules
  • the processing device is configured to obtain the at least three corresponding to the positioning partition.
  • At least three distance information sent by the signal sending module; determining the smallest three distance information of the at least three distance information and the position coordinates of the three signal sending modules corresponding to the minimum three distance information; according to the The minimum three distance information and the position coordinates of the three signal sending modules are based on a three-point weighted centroid positioning algorithm to obtain a first positioning position of the shopping cart.
  • the supermarket also includes a plurality of preset identifiers, each partition corresponding to at least one preset identifier, and the plurality of preset identifiers include a first A preset identifier, the first reference image includes a first preset identifier in the supermarket, and the processing device is configured to process the first reference image to identify a first reference image in the first reference image A preset identifier; and determining the partition where the shopping cart is located as a positioning partition according to the first preset identifier.
  • the partition where the shopping cart is located corresponds to the first preset identifier in the first reference image.
  • a plurality of signal transmission modules corresponding to the plurality of partitions are respectively disposed at a plurality of preset positions in the supermarket.
  • the at least one signal sending module is a Bluetooth beacon.
  • the image acquisition device includes a camera, and an included angle between a shooting direction of the camera and a horizontal direction is 45 degrees to 90 degrees.
  • At least one embodiment of the present disclosure also provides a shopping mall shopping cart positioning system, including: an image acquisition device configured to collect a first reference image of the shopping mall, wherein the shopping mall includes a plurality of partitions; a memory is configured For storing computer-readable instructions; and a processor configured to run the computer-readable instructions, wherein the computer-readable instructions, when executed by the processor, execute the shopping mall shopping cart according to any one of the above embodiments Positioning method.
  • At least one embodiment of the present disclosure also provides a shopping mall shopping cart, including a car body and the shopping mall positioning system according to any one of the above embodiments.
  • FIG. 1 is a flowchart of an embodiment of a shopping mall positioning method provided by at least one embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a coordinate system of a camera calibration model
  • FIG. 3 illustrates a flowchart of positioning correction of a shopping mall by visual positioning according to an embodiment of at least one embodiment of the present disclosure
  • FIG. 4 shows a flowchart of positioning correction by a magnetometer in an embodiment of a shopping cart positioning method provided by at least one embodiment of the present disclosure
  • FIG. 5 illustrates a schematic diagram of an embodiment of a shopping mall positioning system provided by at least one embodiment of the present disclosure
  • FIG. 6 shows an application flowchart of an embodiment of a shopping mall shopping cart provided by at least one embodiment of the present disclosure.
  • the wireless positioning method may include wireless positioning methods such as Bluetooth positioning and WIFI positioning.
  • the current wireless positioning method has poor positioning accuracy and is obviously affected by environmental interference.
  • the usual positioning accuracy can only reach 3-5 meters, but the distance between two adjacent rows of shelves in the commercial supermarket is about 3 meters. Therefore, The wireless positioning method cannot meet the current positioning accuracy requirements of shopping carts in commercial supermarkets.
  • the positioning position of the shopping cart with high accuracy can be obtained through a large number of calculations.
  • the processing module on the shopping cart has a large amount of calculation during precise positioning, and requires higher performance of the processing module.
  • the cost is high, the current consumer electronics-grade processing module set on the shopping cart cannot meet the performance requirements of the visual positioning for the processing module.
  • An embodiment of the present disclosure provides a shopping mall shopping cart positioning method. By combining visual positioning and wireless positioning methods, the calculation amount of positioning is reduced, and the positioning accuracy of the shopping mall shopping cart is improved. In addition, visual recognition is used. And positioning technology can get a more accurate location information, and solve the problem that Bluetooth positioning cannot achieve accurate positioning. Embodiments of the present disclosure also provide a shopping mall positioning system and a shopping mall shopping cart.
  • FIG. 1 is a flowchart of an embodiment of a method for locating a shopping mall in accordance with at least one embodiment of the present disclosure.
  • a shopping cart positioning method 10 includes:
  • S100 Collect the first reference image of the supermarket, and process the first reference image to obtain the partition where the shopping cart is located as the positioning partition.
  • a first reference image including preset identifiers of different partitions in a commercial supermarket may be collected, and the first reference image is processed to obtain a positioning partition in which a car body of the shopping cart is located.
  • the business super plane can be divided into multiple partitions and multiple preset identifiers, and at least one preset identifier is set in each partition. For example, multiple (two, three, four, etc.) are set in each partition. ) Preset logo.
  • Multiple zones can be divided according to product categories. For example, multiple zones can include snacks, drinks, dairy, fresh, fruit, etc.
  • the preset logos set in each section can choose posters with different images.
  • the posters can include snack posters, drink posters, dairy posters, fruit posters, and so on. Each poster corresponds to one of the divided multiple partitions.
  • One positioning zone can be uniquely identified, and other markers can also be selected in practical applications. For example, different markers that can be identified through image processing can be used, and this disclosure is not limited thereto.
  • the plurality of preset identifiers include a first preset identifier
  • the first reference image includes a first preset identifier in a supermarket.
  • step S100 may include: collecting a first reference image of a supermarket; processing the first reference image to identify a first preset identifier in the first reference image; and determining, according to the first preset identifier, a partition where the shopping cart is located as Locate the partition.
  • the first reference image may be obtained by the image acquisition device 1 (shown in FIG. 5 which will be described in detail below).
  • the positions of all preset logos in the supermarket are recorded in advance, a first reference image including the first preset logo is collected, and then the first reference image is processed to identify a specific one in the first reference image.
  • a preset logo that is, the first preset logo
  • the position of the specific preset logo can be obtained according to the correspondence between different preset logos and logo positions, and according to the focal length and
  • the position of the image acquisition device 1 can be obtained in the shooting range, and the position of the shopping cart can be obtained by the relative position of the image acquisition device 1 and the shopping cart.
  • the position of the image acquisition device 1 can be used as At the initial position of the shopping cart, the partition in which the shopping cart is located is used as a positioning partition to further locate using a traditional wireless positioning method.
  • the first reference image including the first preset identifier may be captured by the image acquisition device 1 provided on the shopping cart, and then the first reference image may be acquired by the processing device 3 (shown in FIG. 5 which will be described in detail below).
  • the processing device 3 performs feature extraction on the first reference image and compares the preset reference identifiers with each other to obtain the identified first preset identifier and determine the identified first preset identifier.
  • the local partition is used as the positioning partition.
  • the position of the camera can also be obtained according to the preset position of the identified first preset identifier and the focal length and visible range of the camera in the image acquisition device 1.
  • the partition where the shopping cart is located corresponds to the first preset identifier in the first reference image.
  • the first preset identifier is an identifier corresponding to the snack area, it may be determined that the partition where the shopping cart is located is a snack area.
  • the image acquisition device 1 may select a camera for image acquisition.
  • the shooting angle of the camera may be between 45 degrees and 90 degrees, so as to ensure that a predetermined mark of a higher position can be captured. image.
  • the shooting angle of the camera may represent an included angle between the shooting direction of the camera and the horizontal direction, that is, the included angle between the shooting direction of the camera and the horizontal direction may be 45 degrees to 90 degrees.
  • S200 Receive distance information sent by at least one signal sending module set in the positioning partition.
  • At least one signal transmission module may be set in each of the plurality of partitions obtained by the division of the commercial supermarket, and the setting position of each signal transmission module may be recorded.
  • the signal sending module may be a Bluetooth beacon or the like. The distance information sent by each signal sending module can be received by the processing device 3 in the shopping cart.
  • multiple (three, four, etc.) signal transmission modules are provided in each partition.
  • step S200 distance information sent by a plurality of signal sending modules at a plurality of preset positions in the commercial supermarket may be received.
  • S300 Obtain a first positioning position of the shopping cart according to the distance information sent by at least one signal sending module corresponding to the positioning partition, and determine the actual position of the shopping cart based on the first positioning position of the shopping cart.
  • the positioning partition corresponds to multiple signal sending modules.
  • the first positioning position of the shopping cart is obtained according to the distance information of the plurality of signal sending modules corresponding to the positioning partition, and the actual position of the shopping cart is determined based on the first positioning position of the shopping cart.
  • determining the actual position of the shopping cart based on the first positioning position of the shopping cart includes: using the first positioning position of the first shopping cart as the actual position of the shopping cart.
  • the positioning subdivision of the shopping cart obtained in S100 may be obtained, and the distance information sent by multiple signal sending modules in the positioning subdivision may be obtained, and the first positioning position of the shopping cart may be calculated based on the multiple distance information as the shopping cart ’s
  • the actual position output eliminates the interference of the distance information sent by the signal sending module of the non-location partition, improves the accuracy of the shopping cart positioning, and reduces the amount of calculation.
  • the distance information sent by multiple signal sending modules in the positioning zone may be controlled to the processing device 3 according to the determined positioning zone; in other embodiments, all signal sending modules in the supermarket
  • the distance information sent to the processing device 3, and the processing device 3 may select the distance information sent by the sending module corresponding to the positioning partition from the obtained all distance information, and then process the distance information sent by the sending module corresponding to the positioning partition.
  • each partition corresponds to at least three signal sending modules, that is, the positioning partition corresponds to at least three signal sending modules.
  • obtaining the first positioning position of the shopping cart according to the distance information sent by at least one signal sending module corresponding to the positioning zone includes: obtaining at least three distance information sent by at least three signal sending modules corresponding to the positioning zone; determining at least three Among the three distance information, the minimum three distance information and the position coordinates of the three signal transmission modules corresponding to the minimum three distance information; according to the minimum three distance information and the position coordinates of the three signal transmission modules, based on three points
  • the weighted centroid positioning algorithm obtains the first positioning position of the shopping cart.
  • the distance information may include a distance value between a signal sending module and a shopping cart that sends the distance information.
  • the at least three distance information includes first distance information, second distance information, third distance information, fourth distance information, and fifth distance information, and the first distance information is greater than the second distance information, The second distance information is greater than the third distance information, the third distance information is greater than the fourth distance information, and the fourth distance information is greater than the fifth distance information.
  • the third distance information, the fourth distance information, and the fifth distance information are at least three distances. The smallest three distance messages in the message.
  • the first positioning position of the shopping cart calculated based on multiple distance information can be obtained by a three-point weighted centroid positioning algorithm.
  • the position and distance value of the signal sending module corresponding to the three distance information with the smallest distance value among the plurality of distance information in the received positioning partition may be selected, and the first positioning position of the shopping cart is obtained according to the three-point weighted centroid positioning algorithm. Specifically, it can be obtained by the following formula:
  • (x a , y a ), (x b , y b ), and (x c , y c ) are respectively the position coordinates of the three signal transmission modules corresponding to the three smallest distance information
  • d a , d b , d c is the distance value of the smallest three pieces of distance information sent by the three signal sending modules
  • x is the abscissa of the first positioning position
  • y is the ordinate of the first positioning position.
  • the distance between any two signal sending modules in multiple signal sending modules may be 2-4 meters.
  • the shopping cart positioning method further includes step 20 of unifying the coordinates of the first reference image and the position coordinates of the signal sending module into the same coordinate system before positioning the shopping cart.
  • step 20 may include: registering the positioning coordinate system of the camera with the positioning coordinate system in the commercial supermarket, so that the positioning coordinate system of the camera and the positioning coordinate system of the commercial supermarket are in the same world coordinate system and according to the commercial supermarket
  • the plane range of the commercial coordinate system sets the coordinate range of the Shangchao Super League in the world coordinate system and the coordinate ranges of each divided partition, and records the coordinates of each preset identifier to facilitate the calculation with a unified coordinate system.
  • FIG. 2 is a schematic diagram of a coordinate system of a camera calibration model.
  • camera calibration uses images taken by a camera to restore objects in space.
  • Camera calibration can use a pinhole model.
  • the positioning coordinate system of the camera ie, the camera coordinate system
  • the o c -x c y c z c coordinate system is a coordinate system established based on the camera.
  • the image coordinate system 472 (that is, the o p -x p y p coordinate system) is a coordinate system established based on an optical image (ie, a first reference image) of an object collected by a camera.
  • the positioning coordinate system of Shangchao is a coordinate system established on the basis of Shangchao.
  • the world coordinate system 482 (that is, the o w -x w y w z w coordinate system) is a coordinate system established on the basis of Mr. Objects.
  • the coordinate system 492 (that is, the o q -uv coordinate system) is a pixel coordinate system of the optical image (that is, the first reference image).
  • the world coordinate system 482 can be placed freely according to the computing needs.
  • the origin o c of the positioning coordinate system 462 may be located on the camera optical center (ie, the projection center), and the origin o p of the image coordinate system 472 may be located on the intersection (u 0 , v 0 ) of the optical axis of the camera and the imaging plane.
  • the z-axis of the positioning coordinate system 462 is the optical axis of the camera, and the x-axis and y-axis of the positioning coordinate system 462 are parallel to the x-axis and y-axis of the image coordinate system 472, respectively.
  • the x-axis and y-axis of the image coordinate system 472 are also parallel to the u-axis and v-axis of the pixel coordinate system 492, respectively.
  • the pixel coordinates (u, v) of each point in the pixel coordinate system 492 represent the number of columns and rows of pixels, and can be obtained from a camera.
  • the positioning coordinate system of the commercial supermarket may be the same as the world coordinate system 482.
  • the optical image in the image coordinate system 472 needs to be converted into the positioning coordinate system 462 first, and then into the world coordinate system 482.
  • each point in the optical image may correspond to a corresponding point in the world coordinate system 482.
  • the image coordinate system 472 and the positioning coordinate system 462 are mutually converted through perspective projection, and the positioning coordinate system 462 and the world coordinate system 482 are mutually converted through rigid body changes (rotation and translation).
  • the image coordinate system 472 and the pixel coordinate system 492 are two-dimensional coordinate systems
  • the positioning coordinate system 462 and the world coordinate system 482 are three-dimensional coordinate systems.
  • registering the positioning coordinate system of the camera with the positioning coordinate system in the commercial supermarket can be achieved by the following formula:
  • K is the internal parameter matrix of the camera
  • fx and fy are the normalized focal lengths of the camera on the x-axis and y-axis, respectively
  • cx is the x-axis of the center point of the image (for example, the first reference image) collected by the camera.
  • Coordinates, cy is the y-axis coordinate of the center point of the image (for example, the first reference image) collected by the camera
  • [R, t] is the pose transformation matrix of the camera position relative to the initial position
  • [u, v] T The coordinates of the spatial P point collected by the camera on the image coordinate system 472 of the image (for example, the first reference image)
  • P w is the coordinate of the spatial P point in the world coordinate system
  • u and v respectively represent the positioning coordinate system of the camera
  • the horizontal and vertical coordinate variables in, for example, can represent the horizontal and vertical coordinates of the space P point in the pixel coordinate system 492, and Z is the optical axis coordinate in the positioning coordinate system of the camera, that is, the distance from the space P point to the camera .
  • [R, t] is the external parameter matrix of the camera, which is used to determine the position and orientation of the camera in three-dimensional space.
  • R is the rotation matrix of the camera.
  • t is the translation matrix of the camera.
  • the commercial supermarket's flat partition can be divided into shelf partitions for setting up shelves and transit partitions for customers to pass between the shelves. Record the coordinate range of the shelf partition and the transit partition, and divide the transit partition into multiple successively connected grids, and record the center coordinates of each grid.
  • each segment of the supermarket may include at least a portion of the shelf partition and at least a portion of the transit partition.
  • each partition of the supermarket may include at least one of the shelf partitions. An area corresponding to one shelf and at least one grid in the pass zone.
  • FIG. 3 illustrates a flowchart of positioning correction by visual positioning according to an embodiment of a shopping cart positioning method according to at least one embodiment of the present disclosure.
  • the method for locating a supermarket shopping cart may further include:
  • S400 Obtain multiple first positioning positions of the shopping cart at multiple consecutive moments, determine multiple partitions corresponding to the multiple first positioning positions, and determine the largest number of the multiple partitions corresponding to the multiple first positioning positions to The current partition; if the current partition is different from the positioning partition, collect the second reference image of the commercial supermarket again, and process the second reference image to obtain the second positioning position of the shopping cart, and use the second positioning position to replace the last one among multiple consecutive moments
  • the first positioning position corresponding to the time is taken as the actual position of the shopping cart.
  • step S400 if the current partition is the same as the positioning partition, the first positioning position determined from the last time among a plurality of consecutive times is used as the actual position of the shopping cart.
  • the plurality of first positioning positions may be positioning positions obtained according to distance information sent by at least one signal sending module corresponding to the positioning partition at multiple consecutive times.
  • the plurality of preset identifiers further include a second preset identifier
  • the second reference image includes a second preset identifier
  • step S400 the first positioning positions of the shopping cart at multiple consecutive moments can be obtained, the partitions corresponding to the multiple first positioning positions are determined, and the partition with the largest number is the current partition.
  • the partition is different from the positioning partition, and a reference image (for example, a second reference image) including the preset identifier (for example, a second preset identifier) is acquired again by the image acquisition device 1 and the second reference image is processed.
  • the second positioning position of the shopping cart is obtained as the actual position of the shopping cart.
  • the method for determining the second positioning position of the shopping cart may be the same as the method for the first positioning position of the shopping cart, that is, it may be obtained through a three-point weighted centroid positioning algorithm.
  • the second positioning position may be used to replace the first positioning position at the previous moment (that is, the first positioning position determined at the last time among multiple consecutive times) as the actual position of the shopping cart. .
  • the time interval between any two adjacent moments in multiple consecutive moments may be the same; or, the multiple time intervals corresponding to the multiple consecutive moments may be at least partially different.
  • multiple consecutive moments may include a first moment, a second moment, a third moment, and a fourth moment in sequence, a first time interval between the first moment and the second moment, and a second moment and a third moment Is the second time interval and the third time interval is the third time interval, the first time interval, the second time interval, and the third time interval may all be the same, for example, all are 500 milliseconds; or, The first time interval, the second time interval, and the third time interval may be at least partially different.
  • the first time interval and the second time interval may be the same, and the first time interval and the third time interval are different. It should be noted that the last time among the multiple consecutive times is the fourth time.
  • the five partitions may be a drink area, a snack area, a drink area, a snack area, and a snack area. Since the five sub-regions include three snack regions and two drink regions, the most frequently occurring region among the five regions is the snack region, that is, the snack region is the current sub-region.
  • step S400 in some examples, if the acquired second reference image does not include a preset identifier, the actual position of the shopping cart is unchanged, that is, the actual position of the shopping cart is the first position of the shopping cart determined according to step S300. A positioning position.
  • step S400 if the acquired second reference image includes a preset identifier, the position of the image acquisition device 1 that acquires the second reference image is calculated, and shopping is obtained according to the position relationship between the image acquisition device 1 and the shopping cart.
  • the position of the cart is used as the actual position of the shopping cart. Real-time correction of the actual position of the shopping cart through visual positioning to prevent deviations in the positioning of the shopping cart and improve the accuracy of the positioning of the shopping cart.
  • FIG. 4 shows a flowchart of positioning correction by a magnetometer in an embodiment of a shopping cart positioning method provided by at least one embodiment of the present disclosure.
  • the method for locating a shopping mall may further include:
  • S500 Obtain the first positioning position of the shopping cart corresponding to the first time and the first positioning position of the shopping cart corresponding to the second time; calculate the first positioning position corresponding to the first time and the center points of all the grids in the supermarket To get the first set of distance values, calculate the distance between the first positioning position corresponding to the second moment and the center points of all the grids in the commercial supermarket to get the second set of distance values, and select the first set of distance values respectively The minimum distance value of the first moment in the second time and the minimum distance value of the second moment in the second set of distance values, and comparing the distance difference between the minimum distance value at the first moment and the minimum distance value at the second moment and the first preset distance value; When the distance difference is less than or equal to the first preset distance value, the position of the center point of the grid corresponding to the minimum distance value at the second moment corresponding to the second moment is used as the actual position of the shopping cart; when the distance difference is greater than the first preset Set the distance value to correct the actual position of the shopping cart with a magnet
  • the first group of distance values includes multiple distance values, and the minimum distance value at the first moment is the smallest distance value among the multiple distance values included in the first group of distance values.
  • the second set of distance values includes multiple distance values, and the minimum distance value at the second moment is the smallest distance value among the plurality of distance values included in the second set of distance values.
  • step S500 first, the actual position of the adjacent first time (that is, the first positioning position of the shopping cart corresponding to the first time) and the actual position of the shopping cart at the second time (that is, the second time) can be obtained. Corresponding second positioning position of the shopping cart).
  • the position of the center point of the grid corresponding to the second time minimum distance value corresponding to the second time is used as the actual position of the shopping cart;
  • the distance difference is greater than the first preset distance value, the actual position of the shopping cart is corrected by the magnetometer 4.
  • a first positioning position obtained at a first time and a second time in a certain time interval by using a conventional wireless positioning algorithm may be acquired.
  • the distances of the center points of all the grids so as to obtain multiple distances corresponding to the first time and multiple distances corresponding to the second time, respectively, to obtain two distance arrays, and further obtain the minimum values of the two distance arrays, respectively
  • the minimum distance value d0min at the first moment and the minimum distance value d1min at the second moment, and the coordinates of the center point of the passable grid corresponding to d0min (x (i), y (i)) and the passable grid corresponding to d1min are recorded.
  • the time interval between the first moment and the second moment can be selected as 500 milliseconds (ms), which is approximately equivalent to the time required for the customer to travel further. In practical applications, adaptive selection can also be made based on experience values.
  • d is less than or equal to the first preset distance value, it means that the distance between the coordinates of the first positioning position corresponding to the second moment and the actual point is within the error range. Then the coordinates (x (j), y (j)) of the center point of the passable grid corresponding to the minimum distance value d1min at the second moment is used as the actual position of the shopping cart, and the actual position of the shopping cart is selected to be positioned on the grid.
  • the center can avoid the situation where the positioning point of the actual position of the shopping cart floats on the shelf.
  • the first preset distance value may be selected according to a certain time interval, that is, 500 milliseconds in this embodiment as a reference, and 500ms is about a further distance, and the size of the grid is combined to obtain a first preset distance value.
  • the first preset distance value can be flexibly selected according to different actual conditions.
  • the first preset distance value can be 0.5-2 meters.
  • the actual position of the shopping cart may be further corrected by the magnetometer 4.
  • correcting the actual position of the shopping cart by the magnetometer 4 may include:
  • angle difference is less than or equal to a preset angle value, using the position of the center point of the grid corresponding to the minimum distance value at the first moment corresponding to the first moment as the actual position of the shopping cart;
  • the minimum distance value at the second moment corresponding to the second moment is The position of the center point of the corresponding grid is used as the actual position of the shopping cart.
  • the angle difference may represent a difference between an angle value in the first angle detection information and an angle value in the second angle detection information.
  • the preset angle value may be 90 degrees, that is, if the angle difference is less than or equal to 90 degrees, it means that the distance between the coordinates of the first positioning position corresponding to the second moment and the actual point exceeds the error range, the default value will be
  • the coordinates (x (i), y (i)) of the center point of the passable grid corresponding to the minimum distance value d0min at the first moment are output as the actual position of the final shopping cart.
  • the angle difference is greater than 90 degrees, it means that the user has turned, and it can be determined whether the distance difference d between the minimum distance value at the first moment and the minimum distance value at the second moment is within the second preset distance value, and the user has turned.
  • set a second preset distance value If the angle difference is less than or equal to the second preset distance value, set the center point of the passable grid corresponding to the minimum distance value d1min at the second moment.
  • the coordinates (x (j), y (j)) are output as the actual position of the shopping cart.
  • the settings of the first preset distance value and the second preset distance value may be obtained based on experience values obtained by performing multiple tests on an actual scene, and may be calculated by using a user's movement speed and step size.
  • the second preset distance value must be greater than the first preset distance value.
  • the second preset distance value can be a value between 1.5 and 3 meters.
  • the minimum distance at the first moment corresponding to the first moment is used as the actual location of the shopping cart.
  • the coordinates (x (i), y (i)) of the center point of the passable grid corresponding to the minimum distance value d0min at the first moment are output as the actual position of the shopping cart.
  • the commercial shopping cart positioning method disclosed in the present disclosure mainly uses a wireless positioning method to locate an indoor shopping cart in a supermarket, and simultaneously combines visual recognition technology and a magnetometer 4 sensor to implement path correction and fixed-point position correction, which can effectively improve the shopping cart positioning accuracy.
  • FIG. 5 is a schematic diagram of an embodiment of a shopping mall positioning system provided by at least one embodiment of the present disclosure.
  • the shopping mall positioning system includes a processing device 3, an image acquisition device 1, a wireless communication device 2, and a magnetometer 4.
  • the image acquisition device 1, the wireless communication device 2, and the magnetometer 4 can be disposed on the body of a shopping cart.
  • the components of the supermarket shopping cart positioning system shown in FIG. 5 are only exemplary and not restrictive. According to actual application requirements, the supermarket shopping cart positioning system may also have other components.
  • the processing device 3, the image acquisition device 1, and the magnetometer 4 may be interconnected through a bus system, a network, and / or other forms of connection mechanisms (not shown).
  • the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.
  • the wireless communication device 2 can communicate with other components in the shopping cart positioning system in a wireless manner.
  • the processing device 3, the image acquisition device 1, the wireless communication device 2 and the magnetometer 4 can communicate with each other directly or indirectly.
  • the image acquisition device 1 may acquire a first reference image, a second reference image, and the like.
  • the supermarket includes multiple partitions and multiple preset identifiers, and each partition includes at least one preset identifier.
  • multiple preset identifiers include a first preset identifier and a second preset identifier
  • the first reference image includes The first preset logo and the second reference image in the shopping mall include the second preset logo.
  • the first reference image and the second reference image may be an original image directly acquired by the image acquisition device 1, or may be an image obtained after preprocessing the original image.
  • the first reference image and the second reference image are the same size.
  • the wireless communication device 2 may receive distance information sent by a plurality of signal sending modules at a plurality of preset positions in a commercial supermarket.
  • a plurality of signal sending modules are respectively disposed at a plurality of preset positions in a commercial supermarket.
  • Each signal transmitting module may be a Bluetooth beacon.
  • the processing device 3 may be configured to receive a first reference image, process the first reference image, determine and determine a partition in which a vehicle body of a shopping cart is located, and send the positioning partition according to at least one signal corresponding to the positioning partition.
  • the distance information sent by the module obtains the first positioning position of the shopping cart, and determines the actual position of the shopping cart based on the first positioning position of the shopping cart.
  • the partition where the shopping cart is located corresponds to the first preset identifier in the first reference image.
  • the image acquisition device 1 may include a camera, and one or more cameras may be provided and configured to capture a first reference image.
  • the camera may be, for example, a camera of a smart phone, a camera of a tablet computer, or a web camera.
  • the included angle between the camera's shooting direction and the horizontal direction can be selected according to the actual set height of the identified preset identifier.
  • the included angle between the camera's shooting direction and the horizontal direction can be 45 degrees to 90 degrees, for example,
  • the angle between the camera's shooting direction and the horizontal direction is 60 degrees, so that the shooting range of the camera can cover the preset logo of the conventional settings.
  • the wireless communication device 2 may be a Bluetooth communication device, a Wi-Fi communication device, or the like.
  • the processing device 3 may be a processing device having a data processing capability and / or a program execution capability.
  • the processing device 3 includes, but is not limited to, one or more of a processor, a microcontroller, a digital signal processing (DSP), an application specific integrated circuit (ASIC), and other devices.
  • the processor may be, for example, a central processing unit (CPU), a field programmable gate array (FPGA), or a tensor processing unit (TPU).
  • the processing device 3 may include one or more chips among the above-mentioned devices.
  • the processing device 3 is configured to use the first positioning position of the first shopping cart as the actual position of the shopping cart.
  • the processing device 3 is further configured to obtain a plurality of first positioning positions of the shopping cart at a plurality of consecutive times, determine a plurality of partitions corresponding to the plurality of first positioning positions, and determine a plurality of first The partition with the largest number of the multiple partitions corresponding to the positioning position is the current partition. If the current subdivision is different from the positioning subdivision, the image acquisition device 1 is further configured to acquire the second reference image of the supermarket again, and the processing device 3 is further configured to process the second reference image to obtain the second positioning position of the shopping cart. The positioning position replaces the first positioning position corresponding to the last time among a plurality of consecutive times as the actual position of the shopping cart. If the current partition is the same as the positioning partition, the processing device 3 is further configured to use the first positioning position corresponding to the last time among a plurality of consecutive times as the actual position of the shopping cart.
  • the commercial supermarket includes shelf partitions and transit partitions, and transit partitions include multiple grids connected in sequence.
  • the processing device 3 is further configured to: obtain the first positioning position of the shopping cart corresponding to the first time and the first positioning position of the shopping cart corresponding to the second time; and calculate the first positioning position corresponding to the first time The distance from the center points of all the grids in the commercial supermarket to obtain the first set of distance values, and calculate the distances between the first positioning position corresponding to the second moment and the center points of all the grids in the commercial supermarket to obtain the second Set of distance values, select the minimum distance value at the first moment in the first group of distance values and the minimum distance value at the second moment in the second group of distance values, and compare the minimum distance value at the first moment and the minimum distance value at the second moment The distance difference and the first preset distance value; when the distance difference is less than or equal to the first preset distance value, the position of the center point of the grid corresponding to the second time minimum distance value corresponding to the second time is used as the Actual position
  • the processing device 3 is further configured to obtain the first angle detection information of the magnetometer at the first time and the second angle detection information of the magnetometer at the second time, and calculate the first time and the first time.
  • the angle difference between two times if the angle difference is less than or equal to the preset angle value, the position of the center point of the grid corresponding to the minimum distance value at the first time corresponding to the first time is used as the actual position of the shopping cart; if the angle difference is Greater than the preset angle value and the distance difference value is less than or equal to the second preset distance value, the position of the center point of the grid corresponding to the second time minimum distance value corresponding to the second time is taken as the actual position of the shopping cart; if the angle difference value Greater than the preset angle value and the distance difference greater than the second preset distance value, the position of the center point of the grid corresponding to the minimum distance value at the first moment corresponding to the first time is taken as the actual position of the shopping cart.
  • each partition corresponds to at least three signal sending modules, that is, the positioning partition corresponds to at least three signal sending modules.
  • the processing device 3 is configured to: obtain at least three distance information sent by at least three signal transmitting modules corresponding to the positioning partition; determine the smallest three of the at least three distance information and the smallest three The position coordinates of the three signal sending modules corresponding to the distance information; based on the minimum three distance information and the position coordinates of the three signal sending modules, a first positioning position of the shopping cart is obtained based on a three-point weighted centroid positioning algorithm.
  • the processing device 3 is further configured to: process the first reference image to identify a first preset identifier in the first reference image; and determine, according to the first preset identifier, a partition where the shopping cart is located as a positioning Partition.
  • the Supermarket shopping cart positioning system includes: an image acquisition device, a memory, and a processor.
  • the image acquisition device is configured to acquire a first reference image of a commercial supermarket.
  • the Supermarket includes multiple partitions.
  • the memory is configured to store computer-readable instructions;
  • the processor is configured to execute computer-readable instructions.
  • the method for positioning a shopping cart according to any one of the foregoing embodiments can be executed.
  • the image acquisition device, the memory, and the processor may be interconnected through a bus system, a network, and / or other forms of connection mechanisms (not shown).
  • the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.
  • components such as an image acquisition device, a memory, and a processor may communicate with each other directly or indirectly.
  • the processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or program execution capabilities, such as a field programmable gate array (FPGA) or tensor processing unit (TPU), etc.
  • the controller can control other components in the backlight brightness processing system to perform desired functions.
  • the central processing unit (CPU) may be an X86 or ARM architecture.
  • the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory.
  • the volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache).
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, and the like.
  • One or more computer instructions may be stored on the memory, and the processor may execute the computer instructions to implement various functions.
  • the computer-readable storage medium may also store various application programs and various data, and various data used and / or generated by the application programs.
  • an embodiment of the present disclosure further discloses a shopping mall shopping cart, which includes a vehicle body and the shopping mall positioning system according to any one of the above embodiments.
  • the processing device 3 may be disposed on a terminal of the vehicle body, or may be disposed on a mobile terminal outside the vehicle body.
  • data is transmitted through the wireless communication device 2 in the shopping cart positioning system and the processing device 3 in an external mobile terminal to transmit the received distance information.
  • the processing device 3 may further obtain the reference images (for example, the first reference image and the second reference image, etc.) collected by the image acquisition device 1 through the wireless communication device 2, and then Send to the processing device 3 to perform identification of the preset identifier.
  • the reference images for example, the first reference image and the second reference image, etc.
  • FIG. 6 shows an application flowchart of an embodiment of a shopping mall shopping cart provided by at least one embodiment of the present disclosure.
  • a tablet computer is provided on the shopping cart, and the tablet computer is equipped with a processing device and a wireless transmission device.
  • the shopping cart may further be provided with a camera, a camera and a processing device. Device communication connection.
  • the processing device of the tablet computer controls the camera to take a first reference image.
  • the first reference image includes a first poster (that is, a first preset identifier) in the shopping mall. ).
  • the processing device receives and processes the reference image to obtain a partition (that is, a positioning partition) where the initial position of the shopping cart is located, and then extracts a map of the partition where the initial position is located.
  • the processing device may also receive distance information sent by multiple Bluetooth beacons, and extract the distance information of Bluetooth beacons in the partition where the initial position of the shopping cart is located. Based on the received multiple distance information, select the three smallest distance information to pass the three The point weighted centroid positioning algorithm obtains the first positioning position of the shopping cart as the actual position of the shopping cart.
  • the processing device can obtain a first positioning position through a three-point weighted centroid positioning algorithm every certain time interval, and the processing device determines whether the most frequently occurring partition among the coordinates of the five first positioning positions obtained in total is the same as that passing through the camera.
  • the positioning subdivision of the shopping cart obtained by visual positioning is the same. If the subdivision maps are the same, the positioning of the shopping cart is continued through Bluetooth beacons. If they are not the same, the camera collects the second reference image and determines whether the camera recognizes the classification. Poster identification, for example, collecting a second reference image through the camera again.
  • the camera When the camera recognizes the classified poster identification, it extracts the map of the partition corresponding to the classified poster identification at the current time, calculates the position coordinates of the camera, and replaces the shopping cart at the previous time. Positioning coordinates (that is, the actual position of the shopping cart), that is, the second positioning position of the shopping cart can be obtained by visual positioning based on the second reference image, and the second positioning position is used to replace the first positioning position of the shopping cart as the shopping cart. Actual location.
  • the second reference image may include a second poster (that is, a second preset identifier).
  • the map of the partition is not changed, and the shopping cart positioning is continued through the Bluetooth beacon.
  • the processing device records the distance D0 between the positioning coordinates of the shopping cart at the previous time and the center point of the grid in the partition that can be passed in the partition, and the positioning coordinates of the shopping cart at the current time and the grid in which the partition can be passed.
  • Within a preset distance value T1 if d is within the first preset distance value T1, the coordinates (x (j), y (j)) of the center point of the passable grid corresponding to d1min are output as a shopping cart If d is not within the range of the first preset distance value T1, the actual position of the shopping cart is further corrected by the magnetometer 4.
  • the distance difference d is less than or equal to the second preset distance value T2
  • the position coordinates of the center point of the grid corresponding to d1min as the actual position of the shopping cart, that is, the position coordinates of the center point of the grid corresponding to the minimum distance value d1min corresponding to the second moment as the actual position of the shopping cart.
  • the distance difference d is greater than the second preset distance value T2, and the center point of the grid corresponding to d0min is output as the actual position of the shopping cart, that is, the center point of the grid corresponding to the minimum distance value d0min corresponding to the first moment is used as the shopping cart. Actual location.
  • the shopping mall positioning method, shopping mall positioning system, and shopping mall shopping cart reduce the amount of positioning calculation and improve the shopping mall shopping cart by combining visual positioning and wireless positioning methods. Positioning accuracy; in addition, a more accurate position information can be obtained through visual recognition and positioning technology, which solves the problem that Bluetooth positioning cannot achieve accurate positioning.
  • positioning a shopping cart it includes the The reference image of the preset identifier corresponding to the different partitions in the image and the image processing of the reference image can obtain the partition of the shopping mall where the shopping cart is located as the positioning partition of the shopping cart.
  • the wireless communication device on the shopping can receive the distance information sent by the signal transmitting modules at different preset positions in the supermarket, and the processing device can select the distance information sent by the multiple signal transmitting modules located in the positioning zone and calculate the shopping cart.
  • the first positioning position of is output as the actual position of the shopping cart.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Electromagnetism (AREA)
  • Handcart (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Cash Registers Or Receiving Machines (AREA)
  • Navigation (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Image Analysis (AREA)

Abstract

一种商超购物车定位方法、商超购物车定位系统和商超购物车，商超购物车定位方法(10)包括：采集商超的第一参考图像，处理第一参考图像得到购物车所在的分区为定位分区(S100)，其中，商超包括多个分区；接收在定位分区中设置的至少一个信号发送模块发送的距离信息(S200)，其中，每个分区中设有至少一个信号发送模块；根据定位分区对应的至少一个信号发送模块发送的距离信息得到购物车的第一定位位置，基于购物车的第一定位位置确定购物车的实际位置(S300)。

Description

商超购物车定位方法、商超购物车定位系统及商超购物车
本申请要求于2018年07月03日递交的中国专利申请第201810711492.3号的优先权，在此全文引用上述中国专利申请公开的内容以作为本申请的一部分。
技术领域
本公开公开的实施例涉及一种商超购物车定位方法、商超购物车定位系统及商超购物车。
背景技术
目前，智能购物车通常包括定位功能，现有的智能购物车通常采用无线定位或视觉定位。但是，蓝牙定位和WIFI定位等无线定位方法的精度较差，受环境干扰明显，超市中的可通行分区的尺寸往往与蓝牙或WIFI定位的定位精度差距不大或小于定位精度，导致智能购物车的定位偏差较大，用户体验差，而采用图像采集进行视觉定位虽然定位精度较高，但是定位计算量大，对于处理模块的要求较高，智能购物车上的电子产品无法满足视觉定位对处理模块的要求。
发明内容
本公开的至少一实施例提供一种商超购物车定位方法，包括：采集商超的第一参考图像，处理所述第一参考图像得到购物车所在的分区为定位分区，其中，所述商超包括多个分区；接收在所述定位分区中设置的至少一个信号发送模块发送的距离信息，其中，每个分区中设有至少一个信号发送模块；根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置，基于所述购物车的第一定位位置确定所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位方法中，基于所述购物车的第一定位位置确定所述购物车的实际位置包括：将所述第一购物车的第一定位位置作为所述购物车的实际位置。
例如，本公开一实施例提供的商超购物车定位方法还包括：得到多个连续时刻的所述购物车的多个第一定位位置，确定所述多个第一定位位置分别对应 的多个分区，确定所述多个第一定位位置对应的所述多个分区中数目最多的分区为当前分区；若所述当前分区与所述定位分区不同，再次采集所述商超的第二参考图像，并处理所述第二参考图像得到所述购物车的第二定位位置，利用所述第二定位位置替换与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置；若所述当前分区与所述定位分区相同，将与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位方法中，所述商超包括货架分区和通行分区，所述通行分区包括多个依次连接的栅格；所述商超购物车定位方法还包括：得到第一时刻对应的所述购物车的第一定位位置和第二时刻对应的所述购物车的第一定位位置；计算所述第一时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第一组距离值，计算所述第二时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取所述第一组距离值中的第一时刻最小距离值和所述第二组距离值中的第二时刻最小距离值，并比较所述第一时刻最小距离值和所述第二时刻最小距离值的距离差值与第一预设距离值；当所述距离差值小于等于所述第一预设距离值，则将所述第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；当所述距离差值大于所述第一预设距离值，则通过磁力计对所述购物车的实际位置进行校正。
例如，在本公开一实施例提供的商超购物车定位方法中，通过磁力计传感器对所述购物车的实际位置进行校正包括：获取在所述第一时刻时所述磁力计的第一角度检测信息和所述第二时刻时所述磁力计的第二角度检测信息，计算得到所述第一时刻和所述第二时刻的角度差值；若所述角度差值小于等于预设角度值，将所述第一时刻对应的第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；若所述角度差值大于所述预设角度值且所述距离差值小于等于第二预设距离值，将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；若所述角度差值大于预设角度值且所述距离差值大于所述第二预设距离值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位方法中，所述定位分区对 应至少三个信号发送模块，根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置包括：获取所述定位分区对应的所述至少三个信号发送模块发送的至少三个距离信息；确定所述至少三个距离信息中最小的三个距离信息和与所述最小的三个距离信息对应的三个信号发送模块的位置坐标；根据所述最小的三个距离信息和所述三个信号发送模块的位置坐标，基于三点加权质心定位算法得到所述购物车的第一定位位置。
例如，在本公开一实施例提供的商超购物车定位方法中，所述商超还包括多个预设标识，每个分区对应至少一个预设标识，所述多个预设标识包括第一预设标识，所述第一参考图像包含所述商超中的第一预设标识，处理所述第一参考图像得到购物车所在的分区为定位分区包括：处理所述第一参考图像，以识别所述第一参考图像中的第一预设标识；根据所述第一预设标识，确定所述购物车所在的分区为定位分区。
例如，在本公开一实施例提供的商超购物车定位方法中，所述购物车所在的分区与所述第一参考图像中的所述第一预设标识对应。
例如，在本公开一实施例提供的商超购物车定位方法中，所述多个预设标识还包括第二预设标识，所述第二参考图像包含所述第二预设标识。
本公开至少一实施例还提供一种商超购物车定位系统，包括：图像采集装置，被配置为采集商超的第一参考图像，其中，所述商超包括多个分区；无线通信装置，被配置为接收所述商超中的多个信号发送模块发送的距离信息；处理装置，被配置为接收所述第一参考图像，处理所述第一参考图像得到并确定购物车所在的分区为定位分区，并根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置，基于所述购物车的第一定位位置确定所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位系统中，所述处理装置被配置为：将所述第一购物车的第一定位位置作为所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位系统中，所述处理装置还被配置为：得到多个连续时刻的所述购物车的多个第一定位位置，确定所述多个第一定位位置分别对应的多个分区，确定所述多个第一定位位置对应的所述多个分区中数目最多的分区为当前分区；若所述当前分区与所述定位分区不同，所述图像采集装置还被配置为再次采集所述商超的第二参考图像，所述处理装置还被配置为处理所述第二参考图像得到所述购物车的第二定位位置，利用所 述第二定位位置替换与所述多个连续时刻中最后一个时刻对应的第一定位位置作为购物车的实际位置，若所述当前分区与所述定位分区相同，所述处理装置还被配置为将与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置。
例如，本公开一实施例提供的商超购物车定位系统还包括可固定在所述购物车上的磁力计；所述商超包括货架分区和通行分区，通行分区包括多个依次连接的栅格；所述处理装置还被配置为：得到第一时刻对应的所述购物车的第一定位位置和第二时刻对应的所述购物车的第一定位位置；计算所述第一时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第一组距离值，计算所述第二时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取所述第一组距离值中的第一时刻最小距离值和所述第二组距离值中的第二时刻最小距离值，并比较所述第一时刻最小距离值和所述第二时刻最小距离值的距离差值与第一预设距离值；当所述距离差值小于等于所述第一预设距离值，则将所述第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；当所述距离差值大于所述第一预设距离值，则通过所述磁力计对所述购物车的实际位置进行校正。
例如，在本公开一实施例提供的商超购物车定位系统中，所述处理装置还被配置为：获取在所述第一时刻时所述磁力计的第一角度检测信息和所述第二时刻时所述磁力计的第二角度检测信息，计算得到所述第一时刻和所述第二时刻的角度差值；若所述角度差值小于等于预设角度值，将所述第一时刻对应的第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；若所述角度差值大于所述预设角度值且所述距离差值小于等于第二预设距离值，将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；若所述角度差值大于预设角度值且所述距离差值大于所述第二预设距离值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置。
例如，在本公开一实施例提供的商超购物车定位系统中，所述定位分区对应至少三个信号发送模块，所述处理装置被配置为：获取所述定位分区对应的所述至少三个信号发送模块发送的至少三个距离信息；确定所述至少三个距离信息中最小的三个距离信息和与所述最小的三个距离信息对应的三个信号发 送模块的位置坐标；根据所述最小的三个距离信息和所述三个信号发送模块的位置坐标，基于三点加权质心定位算法得到所述购物车的第一定位位置。
例如，在本公开一实施例提供的商超购物车定位系统中，所述商超还包括多个预设标识，每个分区对应至少一个预设标识，所述多个预设标识包括第一预设标识，所述第一参考图像包含所述商超中的第一预设标识，所述处理装置被配置为：处理所述第一参考图像，以识别所述第一参考图像中的第一预设标识；根据所述第一预设标识，确定所述购物车所在的分区为定位分区。
例如，在本公开一实施例提供的商超购物车定位系统中，所述购物车所在的分区与所述第一参考图像中的所述第一预设标识对应。
例如，在本公开一实施例提供的商超购物车定位系统中，所述多个分区对应的多个信号发送模块分别设置在所述商超中的多个预设位置处。
例如，在本公开一实施例提供的商超购物车定位系统中，所述至少一个信号发送模块为蓝牙信标。
例如，在本公开一实施例提供的商超购物车定位系统中，所述图像采集装置包括摄像头，所述摄像头的拍摄方向与水平方向的夹角为45度～90度。
本公开至少一实施例还提供一种商超购物车定位系统，包括：图像采集装置，被配置为采集商超的第一参考图像，其中，所述商超包括多个分区；存储器，被配置为存储计算机可读指令；以及处理器，被配置为运行所述计算机可读指令，其中，所述计算机可读指令被所述处理器运行时执行上述任一实施例所述的商超购物车定位方法。
本公开至少一实施例还提供一种商超购物车，包括车体以及上述任一实施例所述的商超购物车定位系统。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例的附图作简单地介绍，显而易见地，下面描述中的附图仅仅涉及本公开的一些实施例，而非对本公开的限制。
图1示出本公开至少一实施例提供的一种商超购物车定位方法一个实施例的流程图；
图2为一种摄像机标定模型的坐标系示意图；
图3示出本公开至少一实施例提供的一种商超购物车定位方法一个实施例 通过视觉定位进行定位校正的流程图；
图4示出本公开至少一实施例提供的一种商超购物车定位方法一个实施例通过磁力计进行定位校正的流程图；
图5示出本公开至少一实施例提供的一种商超购物车定位系统一个实施例的示意图；
图6示出本公开至少一实施例提供的一种商超购物车一个实施例的应用流程图。
具体实施方式
为了使得本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例的附图，对本公开实施例的技术方案进行清楚、完整地描述。显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。基于所描述的本公开的实施例，本领域普通技术人员在无需创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
除非另外定义，本公开使用的技术术语或者科学术语应当为本公开所属领域内具有一般技能的人士所理解的通常意义。本公开中使用的“第一”、“第二”以及类似的词语并不表示任何顺序、数量或者重要性，而只是用来区分不同的组成部分。“包括”或者“包含”等类似的词语意指出现该词前面的元件或者物件涵盖出现在该词后面列举的元件或者物件及其等同，而不排除其他元件或者物件。“连接”或者“相连”等类似的词语并非限定于物理的或者机械的连接，而是可以包括电性的连接，不管是直接的还是间接的。“上”、“下”、“左”、“右”等仅用于表示相对位置关系，当被描述对象的绝对位置改变后，则该相对位置关系也可能相应地改变。
为了更清楚地说明本公开，下面结合优选实施例和附图对本公开做进一步的说明。附图中相似的部件以相同的附图标记进行表示。本领域技术人员应当理解，下面所具体描述的内容是说明性的而非限制性的，不应以此限制本公开的保护范围。
现有的商超智能购物车技术中，通常采用无线或视觉定位方法定位智能购物车的实时位置。其中，无线定位方法可包括蓝牙定位和WIFI定位等无线定位方法。目前的无线定位方法的定位精度较差，受环境的干扰明显，通常的定位精度只能达到3-5米，但是商超中相邻两排货架分区之间的距离在3米左右，因此，无线定位方法无法满足目前商超中购物车的定位精度要求，在购 物车定位过程中，容易造成购物车的定位位置落到货架或者间隔了货架的其他可通行分区内，导致购物车的定位偏差严重，易发生定位或导航错误，用户体验差。而采用精度较高的视觉定位方法，通过大量计算可得到精度较高的购物车的定位位置，但是在精确定位时购物车上的处理模块的计算量较大，对处理模块的性能要求较高，成本高，目前设于购物车上的消费电子级的处理模块无法满足视觉定位对于处理模块的性能需求。
本公开的实施例提供了一种商超购物车定位方法，通过将视觉定位与无线定位方法有机结合，减少了定位的计算量，提高了商超购物车的定位精度度；此外通过视觉的识别和定位技术能够得到一个较准确的位置信息，解决了蓝牙定位无法实现准确定位的问题。本公开的实施例还提供一种商超购物车定位系统和一种商超购物车。
本公开的一个方面提供了一种商超购物车定位方法。图1为本公开至少一实施例提供的一种商超购物车定位方法一个实施例的流程图。如图1所示，本实施例中，商超购物车定位方法10包括：
S100：采集商超的第一参考图像，处理第一参考图像得到购物车所在的分区为定位分区。
例如，在步骤S100中，可以采集包括商超中不同分区的预设标识的第一参考图像，处理第一参考图像得到购物车的车体所在的定位分区。
例如，商超平面可划分为多个分区和多个预设标识，并在每个分区中设置至少一个预设标识，例如，每个分区中设置多个(两个、三个、四个等)预设标识。多个分区可按照产品类别进行划分，例如，多个分区可以包括零食区、饮品区、奶制品区、生鲜区，水果区等。设置在每个分区中的预设标识可选用不同图像的海报，海报可以包括零食海报、饮品海报、奶制品海报、水果海报等。每张海报与划分得到的多个分区的其中一个分区对应，通过采集带有海报的第一参考图像，对第一参考图像进行特征提取等图像识别处理以提取第一参考图像上的海报，从而可唯一标定一个定位分区，在实际应用时也可选用其他标记物，例如，可通过图像处理识别的不同的标记物均可，本公开对此并不作限定。
例如，多个预设标识包括第一预设标识，第一参考图像包含商超中的第一预设标识。例如，步骤S100可以包括：采集商超的第一参考图像；处理第一参考图像，以识别第一参考图像中的第一预设标识；根据第一预设标识，确定购物车所在的分区为定位分区。
例如，第一参考图像可以通过图像采集装置1(如下面将会详细描述的图5所示)得到。
在设置预设标识时，预先记录商超中的所有预设标识的标识位置，采集包括第一预设标识的第一参考图像，然后处理第一参考图像以识别第一参考图像中的特定的预设标识(即第一预设标识)，进而根据不同预设标识与标识位置的对应关系可得到该特定的预设标识的位置，根据采集图像的图像采集装置1中的摄像头的焦距和可拍摄范围可得到图像采集装置1的位置，通过图像采集装置1与购物车的相对位置可得到购物车的位置，例如当图像采集装置1位于购物车上时，可将图像采集装置1的位置作为购物车的初始位置，得到购物车所在的分区作为定位分区，以进一步通过传统无线定位方法进行定位。
例如，可通过设置于购物车上的图像采集装置1拍摄包括第一预设标识的第一参考图像，然后可通过处理装置3(如下面将会详细描述的图5所示)获取所述第一参考图像，处理装置3对第一参考图像进行特征提取，并与预设的各个预设标识进行比对，以得到识别出的第一预设标识并确定该识别出的第一预设标识所在地分区作为定位分区。进一步地，还可根据识别出的第一预设标识的预设位置以及图像采集装置1中的摄像头的焦距和可视范围得到摄像头的位置。
例如，购物车所在的分区与第一参考图像中的第一预设标识对应。在一些示例中，当第一预设标识为与零食区对应的标识，则可以确定购物车所在的分区为零食区。
在可选地实施方式中，所述图像采集装置1可以选用摄像头进行图像采集，例如，摄像头的拍摄角度可在45度～90度之间，以保证可拍摄到较高位置的预设标识的图像。例如，摄像头的拍摄角度可以表示摄像头的拍摄方向与水平方向的夹角，也就是说，摄像头的拍摄方向与水平方向的夹角可以为45度～90度。
S200：接收在定位分区中设置的至少一个信号发送模块发送的距离信息。
例如，可在商超划分得到的多个分区的每一个分区中设置至少一个信号发送模块，并记录每个信号发送模块的设置位置。例如，该信号发送模块可以为蓝牙信标等。可通过购物车中的处理装置3接收各个信号发送模块发送的距离信息。
例如，每一个分区中设置有多个(三个、四个等)信号发送模块。
例如，多个分区对应的多个信号发送模块分别设置在商超中的多个预设位 置处。在步骤S200中，可以接收商超中多个预设位置处的多个信号发送模块发送的距离信息。
S300：根据定位分区对应的至少一个信号发送模块发送的距离信息得到购物车的第一定位位置，基于购物车的第一定位位置确定购物车的实际位置。
例如，定位分区对应多个信号发送模块。在步骤S300中，根据所述定位分区对应的多个信号发送模块的距离信息得到购物车的第一定位位置，并基于购物车的第一定位位置确定购物车的实际位置。
例如，在一些实施例中，基于购物车的第一定位位置确定购物车的实际位置包括：将第一购物车的第一定位位置作为购物车的实际位置。
例如，可获取S100中得到的购物车的定位分区，并获取该定位分区中的多个信号发送模块分别发送的距离信息，根据多个距离信息计算得到购物车的第一定位位置作为购物车的实际位置输出，排除非定位分区的信号发送模块发送的距离信息的干扰，提高购物车定位的准确性，且减少计算量。
例如，在一些实施例中，可以根据确定的定位分区，从而控制定位分区中的多个信号发送模块发送的距离信息至处理装置3；在另一些实施例中，商超中的所有信号发送模块均发送的距离信息至处理装置3，而处理装置3则可以从获取所有距离信息中选择与定位分区对应的发送模块发送的距离信息，然后处理与定位分区对应的发送模块发送的距离信息。
例如，在一些实施例中，每个分区对应至少三个信号发送模块，即定位分区对应至少三个信号发送模块。在步骤S300中，根据定位分区对应的至少一个信号发送模块发送的距离信息得到购物车的第一定位位置包括：获取定位分区对应的至少三个信号发送模块发送的至少三个距离信息；确定至少三个距离信息中最小的三个距离信息和与最小的三个距离信息对应的三个信号发送模块的位置坐标；根据最小的三个距离信息和三个信号发送模块的位置坐标，基于三点加权质心定位算法得到购物车的第一定位位置。
例如，距离信息可以包括发送该距离信息的信号发送模块与购物车之间的距离值。
例如，在一些实施例中，至少三个距离信息包括第一距离信息、第二距离信息、第三距离信息、第四距离信息和第五距离信息，且第一距离信息大于第二距离信息，第二距离信息大于第三距离信息，第三距离信息大于第四距离信息，第四距离信息大于第五距离信息，则第三距离信息、第四距离信息和第五距离信息为至少三个距离信息中最小的三个距离信息。
例如，根据多个距离信息计算得到购物车的第一定位位置可通过三点加 权质心定位算法得到。可选择接收到的定位分区内的多个距离信息中距离值最小的三个距离信息对应的信号发送模块的位置及距离值，根据三点加权质心定位算法得到购物车的第一定位位置。具体的，可通过以下公式得到：
Figure PCTCN2019094574-appb-000001
Figure PCTCN2019094574-appb-000002
其中，(x a,y a)，(x b,y b)，(x c,y c)分别为最小的三个距离信息对应的三个信号发送模块的位置坐标，d a、d b、d c分别为三个信号发送模块发送的最小的三个距离信息的距离值，x为第一定位位置的横坐标，y为第一定位位置的纵坐标。
例如，多个信号发送模块中任意两个信号发送模块之间的间距可以为2-4米。
在一些实施方式中，所述商超购物车定位方法进一步还包括在进行购物车定位之前将采集到的第一参考图像的坐标与信号发送模块的位置坐标统一至同一坐标系的步骤20。
例如，所述步骤20可以包括：将摄像头的定位坐标系与商超中的定位坐标系配准，使摄像头的定位坐标系和商超的定位坐标系在同一个世界坐标系下并根据商超的平面范围设定商超的在世界坐标系的坐标范围以及划分的各个分区的坐标范围，并记录各预设标识的坐标，以统一坐标系便于计算。
例如，图2为一种摄像机标定模型的坐标系示意图。例如，摄像机标定利用摄像头所拍摄到的图像来还原空间中的物体。摄像机标定可以采用针孔模型。如图2所示，在摄像机标定模型中，摄像头的定位坐标系(即摄像机坐标系)462(即o c-x cy cz c坐标系)是以摄像头为基准建立的坐标系。图像坐标系472(即o p-x py p坐标系)是以摄像机所采集的物体的光学图像(即第一参考图像)为基准建立的坐标系。商超的定位坐标系是以商超为基准建立的坐标系。世界坐标系482(即o w-x wy wz w坐标系)则是以先生物体为基准建立的坐标系。坐标系492(即o q-uv坐标系)为光学图像(即第一参考图像)的像素坐标系。世界坐标系482可以根据运算需求自由放置。定位坐标系462的原点o c可以位于摄像头光心(即投影中心)上，图像坐标系472的原点o p可以位于摄像头的光轴与成像平面的交点(u 0,v 0)上。定位坐标系462的z轴为摄像 头的光轴，定位坐标系462的x轴、y轴分别与图像坐标系472的x轴、y轴平行。图像坐标系472的x轴、y轴也分别与像素坐标系492的u轴、v轴平行。像素坐标系492中的每个点的像素坐标(u,v)表示像素的列数和行数，且能从摄像头中得到。
需要说明的是，在一些实施例中，商超的定位坐标系可与世界坐标系482相同。
例如，在坐标转换过程中，图像坐标系472下的光学图像需先转换到定位坐标系462中，然后转换到世界坐标系482中。由此，光学图像中的每个点可以与世界坐标系482中的相应点相对应。图像坐标系472和定位坐标系462通过透视投影实现相互转换，定位坐标系462和世界坐标系482则通过刚体变化(旋转和平移)实现相互转换。
例如，图像坐标系472和像素坐标系492为二维坐标系，定位坐标系462和世界坐标系482为三维坐标系。
例如，将摄像头的定位坐标系与商超中的定位坐标系配准可通过如下公式实现：
Figure PCTCN2019094574-appb-000003
Figure PCTCN2019094574-appb-000004
其中，K为摄像头的内参数矩阵，fx、fy分别为摄像头在x轴和y轴上的归一化焦距，cx为摄像头采集的图像(例如，第一参考图像)的中心点的x轴的坐标，cy为摄像头采集的图像(例如，第一参考图像)的中心点的y轴的坐标，[R,t]为摄像头的位置相对于初始位置的位姿变换矩阵，[u,v] T为摄像头采集的空间P点在图像(例如，第一参考图像)的图像坐标系472上的坐标，P w为空间P点在世界坐标系下的坐标，u和v分别表示摄像头的定位坐标系中的横、纵坐标变量，例如，可以表示空间P点在像素坐标系492下的横、纵坐标，Z是摄像头的定位坐标系中的光轴坐标，即可以表示空间P点到摄像头的距离。
例如，[R,t]是摄像头的外参矩阵，用来确定摄像头在三维空间中的位置和朝向，为了将世界坐标系转换为摄像头的定位坐标系使用的参数，R为摄像头的旋转矩阵，t为摄像头的平移矩阵。
例如，可将商超的平面分区划分为用于设置货架的货架分区以及位于货 架间的可供顾客通行的通行分区。记录货架分区和通行分区的坐标范围，并可将通行分区划分为多个依次连接的栅格，记录每个栅格的中心坐标。
例如，商超的每个分区(例如，奶制品区、零食区等)均可以包括货架分区的至少一部分和通行分区的至少一部分，例如，商超的每个分区中可以包括货架分区中的至少一个货架对应的区域和通行分区中的至少一个栅格。
图3示出本公开至少一实施例提供的一种商超购物车定位方法一个实施例通过视觉定位进行定位校正的流程图。
在一些实施方式中，如图3所示，所述商超购物车定位方法进一步还可包括：
S400：得到多个连续时刻的购物车的多个第一定位位置，确定多个第一定位位置分别对应的多个分区，确定多个第一定位位置对应的多个分区中数目最多的分区为当前分区；若当前分区与定位分区不同，再次采集商超的第二参考图像，并处理第二参考图像得到购物车的第二定位位置，利用第二定位位置替换与多个连续时刻中最后一个时刻对应的第一定位位置作为购物车的实际位置。
例如，在步骤S400中，若当前分区与定位分区相同，将与多个连续时刻中的最后一个时刻确定的第一定位位置作为购物车的实际位置。
例如，在步骤S400中，多个第一定位位置可以为在多个连续时刻根据定位分区对应的至少一个信号发送模块发送的距离信息得到的定位位置。
例如，在一些示例中，多个预设标识还包括第二预设标识，第二参考图像包含第二预设标识。
例如，在步骤S400中，可以得到多个连续时刻的所述购物车的第一定位位置，确定多个第一定位位置分别对应的分区，确定其中数目最多的分区为当前分区；若所述当前分区与所述定位分区不同，通过图像采集装置1再次采集包括所述预设标识(例如，第二预设标识)的参考图像(例如，第二参考图像)，并处理所述第二参考图像得到购物车的第二定位位置作为购物车的实际位置。
例如，确定购物车的第二定位位置的方法可以与上述购物车的第一定位位置的方法相同，即可以通过三点加权质心定位算法得到。
例如，在获得第二定位位置后，可以利用利用第二定位位置替换上一时刻的第一定位位置(即多个连续时刻中的最后一个时刻确定的第一定位位置)作为购物车的实际位置。
例如，在当前分区与定位分区不相同时，可以将当前分区替换原始的定位 分区作为当前的定位分区。
例如，多个连续时刻中任意相邻的两个时刻之间的时间间隔可以相同；或者，多个连续时刻对应的多个时间间隔也可以至少部分很不相同。在一些示例中，多个连续时刻可以依次包括第一时刻、第二时刻、第三时刻和第四时刻，第一时刻和第二时刻之间为第一时间间隔，第二时刻和第三时刻之间为第二时间间隔，第三时刻和第四时刻之间为第三时间间隔，则第一时间间隔、第二时间间隔和第三时间间隔可以均相同，例如均为500毫秒；或者，第一时间间隔、第二时间间隔和第三时间间隔可以至少部分不相同，例如，第一时间间隔和第二时间间隔可以相同，而第一时间间隔和第三时间间隔不相同。需要说明的是，多个连续时刻中最后一个时刻为第四时刻。
例如，可判断累计5个时刻对应的五个第一定位位置对应的五个分区中出现最多的分区是否与所述定位分区相同。例如，五个分区可以分别为饮品区、零食区、饮品区、零食区和零食区。由于在该五个分区包括三个零食区和两个饮品区，即五个分区中出现最多的分区为零食区，即零食区为当前分区。
例如，在步骤S400中，在一些示例中，若采集的第二参考图像中不包括预设标识，购物车的实际位置不变，即购物车的实际位置为根据步骤S300确定的购物车的第一定位位置。
例如，在步骤S400中，若采集的第二参考图像中包括预设标识，则计算得到采集第二参考图像的图像采集装置1的位置，并根据图像采集装置1与购物车的位置关系得到购物车的位置作为购物车的实际位置。通过视觉定位实时对购物车的实际位置进行校正，以防止购物车的定位出现偏差，提高购物车定位的准确度。
图4示出本公开至少一实施例提供的一种商超购物车定位方法一个实施例通过磁力计进行定位校正的流程图。
在一些实施方式中，如图4所示，所述商超购物车定位方法进一步还可包括：
S500：得到第一时刻对应的购物车的第一定位位置和第二时刻对应的购物车的第一定位位置；计算第一时刻对应的第一定位位置与商超中的所有栅格的中心点的距离，以得到第一组距离值，计算第二时刻对应的第一定位位置与商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取第一组距离值中的第一时刻最小距离值和第二组距离值中的第二时刻最小距离值，并比较第一时刻最小距离值和第二时刻最小距离值的距离差值与第一预设距离值；当距离差值小于等于第一预设距离值，则将第二时刻对应的第二时刻最小距离值 对应的栅格的中心点的位置作为购物车的实际位置；当距离差值大于第一预设距离值，则通过磁力计对购物车的实际位置进行校正。
例如，第一组距离值包括多个距离值，第一时刻最小距离值为第一组距离值包括的多个距离值中最小的距离值。类似地，第二组距离值包括多个距离值，第二时刻最小距离值为第二组距离值包括的多个距离值中最小的距离值。
例如，在步骤S500中，首先，可以得到相邻的第一时刻的实际位置(即第一时刻对应的购物车的第一定位位置)和第二时刻的购物车的实际位置(即第二时刻对应的购物车的第二定位位置)。
然后，分别计算第一时刻的实际位置(即第一时刻对应的购物车的第一定位位置)与所有所述栅格中心点的距离和第二时刻的实际位置(即第二时刻对应的购物车的第二定位位置)与所有所述栅格中心点的距离，得到两组距离值(即第一组距离值和第二组距离值)，分别选取两组距离值中的最小距离值并将两个最小距离值(即第一时刻最小距离值和第二时刻最小距离值)的距离差值与第一预设距离值对比。
最后，当所述距离差值小于等于所述第一预设距离值，则将所述第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置；当所述距离差值大于所述第一预设距离值，则通过磁力计4对购物车的实际位置进行校正。
例如，可获取一定时间间隔的第一时刻和第二时刻通过传统无线定位算法得到的第一定位位置。计算第一时刻的第一定位位置的坐标与对应的定位分区中的可通行的所有栅格的中心点的距离，计算第二时刻的第一定位位置的坐标与对应的定位分区中的可通行的所有栅格的中心点的距离，从而分别求出与第一时刻对应的多个距离和第二时刻对应的多个距离，以得到两个距离数组，进一步获取两个距离数组的最小值分别为第一时刻最小距离值d0min和第二时刻最小距离值d1min，并记录d0min对应的可通行栅格的中心点的坐标(x(i)，y(i))和d1min对应的可通行栅格的中心点的坐标(x1(j)，y1(j))。其中，第一时刻和第二时刻的时间间隔可选为500毫秒(ms)，大约相当于顾客行进一步所需的时间，在实际应用时，也可根据经验值作适应性选择。
例如，将两个最小距离值d0min和d1min的距离差值与第一预设距离值对比，计算距离差值d＝|d0min-d1min|，判断d是否在第一预设距离值范围内。
例如，若d小于等于第一预设距离值，说明第二时刻对应的第一定位位置的坐标与实际点之间的距离在误差范围内。则将第二时刻最小距离值d1min对应的可通行栅格的中心点的坐标(x(j)，y(j))作为购物车的实际位置，选择将 购物车的实际位置定位到栅格的中心，可以避免出现购物车的实际位置的定位点飘到货架上的情况。
例如，第一预设距离值可以根据一定时间时隔即本实施例中的500毫秒作为参考来选取，500ms大约行进一步的距离，再结合的栅格大小，得到一个第一预设距离值，第一预设距离值可根据实际情况的不同灵活选取，例如，第一预设距离值可以为0.5-2米。
例如，若d大于第一预设距离值，则可进一步通过磁力计4对购物车的实际位置进行校正。
在一些实施方式中，通过磁力计4对购物车的实际位置进行校正可包括：
获取在所述第一时刻时所述磁力计的第一角度检测信息和所述第二时刻时磁力计4的第二角度检测信息，计算得到第一时刻和第二时刻的角度差值；
若所述角度差值小于等于预设角度值，将所述第一时刻对应的第一时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置；
若所述角度差值大于预设角度值且第一时刻最小距离值和第二时刻最小距离值的距离差值小于等于第二预设距离值，将第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置。
例如，角度差值可以表示第一角度检测信息中的角度值和第二角度检测信息中的角度值之间的差值。
例如，本实施例中，预设角度值可以为90度，即若角度差值小于等于90度，说明第二时刻对应的第一定位位置的坐标与实际点的距离超出误差范围，则默认将第一时刻最小距离值d0min对应的可通行栅格的中心点的坐标(x(i),y(i))作为最终购物车的实际位置而被输出。
例如，若角度差值大于90度，则说明用户已经转弯，可以判断第一时刻最小距离值和第二时刻最小距离值的距离差值d是否在第二预设距离值范围内，用户已经转弯，要考虑货架的宽度等因素，设定一个第二预设距离值，若角度差值小于等于第二预设距离值，将第二时刻最小距离值d1min对应的可通行栅格的中心点的坐标(x(j),y(j))作为购物车的实际位置输出。
例如，第一预设距离值和第二预设距离值的设置可根据实际场景进行多次测试得到的经验值，可利用用户运动速度和步长计算求得。第二预设距离值需大于第一预设距离值，例如，第二预设距离值可选用1.5-3米间的数值。
例如，若所述角度差值大于预设角度值且第一时刻最小距离值和第二时刻最小距离值的距离差值大于第二预设距离值，将第一时刻对应的第一时刻最小距离值对应的栅格的中心点作为购物车的实际位置。若角度差值大于预 设角度值且第一时刻最小距离值和第二时刻最小距离值的距离差值大于第二预设距离值，则说明第二时刻对应的第一定位位置存在不可靠性，则将第一时刻最小距离值d0min对应的可通行栅格的中心点的坐标(x(i),y(i))作为购物车的实际位置输出。
本公开的商超购物车定位方法主要通过无线定位方法对超市的室内购物车进行定位，同时结合视觉识别技术和磁力计4传感器实现路径纠偏和定点的位置纠正，能够有效的提高购物车定位精度，能够避免定位得到的坐标出现在不可通行分区(货架等分区)，并且纠偏后的定位点坐标与真实坐标之间的定位误差在可控范围内，且纠偏后的定位点坐标和无线定位算法求得的定位坐标之间的定位误差也在可控范围内；利用视觉定位得到的准确定位坐标进行纠偏，克服利用历史时刻位置纠偏时存在的累计误差的问题；结合磁力计4传感器进行方向角度判断，能够预估用户转弯或者直行的状态，提高阈值判断的准确性。
基于相同原理，根据本公开的另一方面，本公开的至少一个实施例还公开了一种商超购物车定位系统。图5示出本公开至少一实施例提供的一种商超购物车定位系统一个实施例的示意图。
例如，如图5所示，本实施例中，该商超购物车定位系统包括处理装置3、图像采集装置1、无线通信装置2和磁力计4。所述图像采集装置1、无线通信装置2和磁力计4可以设于购物车的车体上。应当注意，图5所示的商超购物车定位系统的组件只是示例性的，而非限制性的，根据实际应用需要，该商超购物车定位系统还可以具有其他组件。
例如，处理装置3、图像采集装置1和磁力计4可以通过总线系统、网络和/或其它形式的连接机构(未示出)互连。网络可以包括无线网络、有线网络、和/或无线网络和有线网络的任意组合。无线通信装置2可以通过无线方式与商超购物车定位系统中的其他组件通信。
例如，处理装置3、图像采集装置1、无线通信装置2和磁力计4之间可以直接或间接地互相通信。
例如，所述图像采集装置1可采集第一参考图像和第二参考图像等。例如，商超包括多个分区和多个预设标识，每个分区包括至少一个预设标识，例如，多个预设标识包括第一预设标识和第二预设标识，第一参考图像包括商超中的第一预设标识，第二参考图像包括第二预设标识。
例如，第一参考图像和第二参考图像可以是图像采集装置1直接采集得到的原始图像，也可以是对原始图像进行预处理之后获得的图像。
例如，第一参考图像和第二参考图像的尺寸相同。
例如，所述无线通信装置2可接收商超中多个预设位置处的多个信号发送模块发送的距离信息。
例如，多个信号发送模块分别设置在商超中的多个预设位置处。每个信号发送模块可以为蓝牙信标。
例如，所述处理装置3可被配置为接收第一参考图像，处理所述第一参考图像得到并确定购物车的车体所在的分区为定位分区，根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到购物车的第一定位位置，基于所述购物车的第一定位位置确定购物车的实际位置。
例如，购物车所在的分区与第一参考图像中的第一预设标识对应。
例如，图像采集装置1可以包括摄像头，摄像头可设置为一个或多个，被配置为拍摄第一参考图像。摄像头例如可以为智能手机的摄像头、平板电脑的摄像头、或者网络摄像头等。
例如，摄像头的拍摄方向与水平方向的夹角可根据识别的预设标识的实际设置高度选取，在一些示例中，摄像头的拍摄方向与水平方向的夹角可为45度～90度，例如，摄像头的拍摄方向与水平方向的夹角为60度，使摄像头的拍摄范围可覆盖到常规设置的预设标识。
例如，无线通信装置2可以为蓝牙通信装置、Wi-Fi通信装置等。
例如，处理装置3可以是具有数据处理能力和/或程序执行能力的处理装置。处理装置3包括但不限于处理器、单片机、数字信号处理(Digital Signal Process,DSP)、专用集成电路(Application Specific Integrated Circuits,ASIC)等器件中的一种或多种。处理器例如可以为中央处理单元(CPU)、现场可编程门阵列(FPGA)或张量处理单元(TPU)等。处理装置3可以包括上述器件中的一个或多个芯片。
例如，在一些示例中，处理装置3被配置为：将第一购物车的第一定位位置作为购物车的实际位置。
例如，在一些示例中，处理装置3还被配置为：得到多个连续时刻的购物车的多个第一定位位置，确定多个第一定位位置分别对应的多个分区，确定多个第一定位位置对应的多个分区中数目最多的分区为当前分区。若当前分区与定位分区不同，图像采集装置1还被配置为再次采集商超的第二参考图像，处理装置3还被配置为处理第二参考图像得到购物车的第二定位位置，利用第二定位位置替换与多个连续时刻中最后一个时刻对应的第一定位位置作为购物 车的实际位置。若当前分区与定位分区相同，处理装置3还被配置为将与多个连续时刻中最后一个时刻对应的第一定位位置作为购物车的实际位置。
例如，商超包括货架分区和通行分区，通行分区包括多个依次连接的栅格。在一些示例中，处理装置3还被配置为：得到第一时刻对应的购物车的第一定位位置和第二时刻对应的购物车的第一定位位置；计算第一时刻对应的第一定位位置与商超中的所有栅格的中心点的距离，以得到第一组距离值，计算第二时刻对应的第一定位位置与商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取第一组距离值中的第一时刻最小距离值和第二组距离值中的第二时刻最小距离值，并比较第一时刻最小距离值和第二时刻最小距离值的距离差值与第一预设距离值；当距离差值小于等于第一预设距离值，则将第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置；当距离差值大于第一预设距离值，则通过磁力计对购物车的实际位置进行校正。
例如，在一些示例中，处理装置3还被配置为：获取在第一时刻时磁力计的第一角度检测信息和第二时刻时磁力计的第二角度检测信息，计算得到第一时刻和第二时刻的角度差值；若角度差值小于等于预设角度值，将第一时刻对应的第一时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置；若角度差值大于预设角度值且距离差值小于等于第二预设距离值，将第二时刻对应的第二时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置；若角度差值大于预设角度值且距离差值大于第二预设距离值，将第一时刻对应的第一时刻最小距离值对应的栅格的中心点的位置作为购物车的实际位置。
例如，每个分区对应至少三个信号发送模块，即定位分区对应至少三个信号发送模块。在一些示例中，处理装置3被配置为：获取定位分区对应的至少三个信号发送模块发送的至少三个距离信息；确定至少三个距离信息中最小的三个距离信息和与最小的三个距离信息对应的三个信号发送模块的位置坐标；根据最小的三个距离信息和三个信号发送模块的位置坐标，基于三点加权质心定位算法得到购物车的第一定位位置。
例如，在一些示例中，处理装置3还被配置为：处理第一参考图像，以识别第一参考图像中的第一预设标识；根据第一预设标识，确定购物车所在的分区为定位分区。
需要说明的是，关于通过图5所示的商超购物车定位系统实现商超购物车定位方法的详细说明可以参考商超购物车定位方法的实施例中的相关描述，重复之处不再赘述。
本公开的至少一个实施例还公开了一种商超购物车定位系统。商超购物车定位系统包括：图像采集装置、存储器和处理器。
例如，图像采集装置被配置为采集商超的第一参考图像。例如，商超包括多个分区。
例如，存储器被配置为存储计算机可读指令；处理器被配置为运行计算机可读指令。计算机可读指令被处理器运行时能够执行根据上述任一实施例所述的商超购物车定位方法。
例如，图像采集装置、存储器和处理器之间可以通过总线系统、网络和/或其它形式的连接机构(未示出)互连。网络可以包括无线网络、有线网络、和/或无线网络和有线网络的任意组合。
例如，图像采集装置、存储器和处理器等组件之间可以直接或间接地互相通信。
例如，处理器可以是中央处理单元(CPU)或者具有数据处理能力和/或程序执行能力的其它形式的处理单元，例如现场可编程门阵列(FPGA)或张量处理单元(TPU)等，处理器可以控制背光亮度处理系统中的其它组件以执行期望的功能。又例如，中央处理器(CPU)可以为X86或ARM架构等。
例如，存储器可以包括一个或多个计算机程序产品的任意组合，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器、闪存等。在存储器上可以存储一个或多个计算机指令，处理器可以运行所述计算机指令，以实现各种功能。在计算机可读存储介质中还可以存储各种应用程序和各种数据以及应用程序使用和/或产生的各种数据等。
需要说明的是，关于商超购物车定位方法的定位过程的详细说明可以参考商超购物车定位方法的实施例中的相关描述，重复之处不再赘述。
根据本公开的还一方面，本公开的实施例还公开了一种商超购物车，该商超购物车包括车体以及如上述任一实施例所述的商超购物车定位系统。
例如，对于图5所示的商超购物车定位系统，所述处理装置3可设置于车体的终端上，也可设置在车体外部的移动终端中，例如可选择设置在车体外部的手机或平板电脑中，通过商超购物车定位系统中的无线通信装置2与外部的移动终端中的处理装置3进行数据传输，以传输接收的距离信息。当处理装置3位于外部的移动终端中时，处理装置3可进一步通过无线通信装置2获取图像采集装置1采集的参考图像(例如，第一参考图像和第二参考图像等)，然后将参考想发送至处理装置3以进行预设标识的识别。
图6示出本公开至少一实施例提供的一种商超购物车一个实施例的应用流程图。
下面通过一个具体实施例对本公开作进一步地说明，如图6所示，购物车上设置有平板电脑，平板电脑搭载有处理装置和无线传输装置，购物车上还可以设有摄像头，摄像头与处理装置通信连接。首先，启动购物车的平板电脑以确认购物车的初始位置，并提取该初始位置所在的分区的地图。例如，用户在使用商超购物车时，保持平板电脑的开启状态，平板电脑的处理装置控制摄像头拍摄第一参考图像，第一参考图像包括商超内的第一海报(即第一预设标识)。处理装置接收并处理参考图像得到购物车的初始位置所在的分区(即定位分区)，进而提取该初始位置所在的分区的地图。处理装置还可以接收多个蓝牙信标发出的距离信息，并提取购物车的初始位置所在的分区内的蓝牙信标的距离信息，根据接收的多个距离信息，选取三个最小的距离信息通过三点加权质心定位算法得到购物车的第一定位位置作为购物车的实际位置。
然后，处理装置每经过一定时间间隔就可以通过三点加权质心定位算法得到一个第一定位位置，处理装置判断累计求得的5个第一定位位置的坐标中出现最多的分区是否与通过摄像头的视觉定位得到的购物车的定位分区相同，如果相同，则不改变分区的地图，继续通过蓝牙信标进行购物车的定位，如果不相同，则摄像头采集第二参考图像并判断摄像头是否识别到分类海报标识，例如，再次通过摄像头采集第二参考图像，当摄像头识别到分类海报标识，则提取当前时刻的分类海报标识对应的分区的地图，计算摄像头的位置坐标，并代替上一时刻的购物车的定位坐标(即购物车的实际位置)，也就是说，可以基于第二参考图像进行视觉定位得到购物车的第二定位位置，利用第二定位位置替换购物车的第一定位位置作为购物车的实际位置。当摄像头识别到分类海报标识时，例如，第二参考图像可以包括第二海报(即第二预设标识)。当摄像头没有识别到分类海报标识，则不改变分区的地图，继续通过蓝牙信标进行购物车的定位。然后，处理装置计录上一时刻的购物车的定位坐标 与该分区中可通行分区的栅格中心点的距离值D0与当前时刻的购物车的定位坐标与该分区中可通行分区的栅格中心点的距离值D1，以得到两组距离值；然后排序两组距离值，求两组距离值中的最小的距离值d0min和d1min，判断距离差值d＝|d0min-d1min|是否在第一预设距离值T1范围内，如果d在第一预设距离值T1范围内，则输出d1min对应的可通行栅格的中心点的坐标(x(j)，y(j))作为购物车的实际位置，如果d不在第一预设距离值T1范围内，则进一步通过磁力计4对购物车的实际位置进行校正。例如，判断磁力计4在每相隔500ms的时间间隔内的角度差|角度是否大于90度，如果角度差|度差小于等于90度，则将第二时刻对应的最小距离值d1min对应的栅格的中心点作为购物车的实际位置，如果角度差|度差大于90度，则判断距离差值d是否在第二预设距离值T2内，如果距离差值d小于等于第二预设距离值T2，则输出d1min对应的栅格的中心点的位置坐标作为购物车的实际位置，即将第二时刻对应的最小距离值d1min对应的栅格的中心点的位置坐标作为购物车的实际位置，如果距离差值d大于第二预设距离值T2，则输出d0min对应的栅格的中心点作为购物车的实际位置，即将第一时刻对应的最小距离值d0min对应的栅格的中心点作为购物车的实际位置。
本公开的有益效果包括：
本公开实施例提供的商超购物车定位方法、商超购物车定位系统和商超购物车，通过将视觉定位与无线定位方法有机结合，减少了定位的计算量，提高了商超购物车的定位精度度；此外通过视觉的识别和定位技术能够得到一个较准确的位置信息，解决了蓝牙定位无法实现准确定位的问题，具体地，在对商超购物车定位时，通过采集包括与商超中不同分区对应的预设标识的参考图像，并对参考图像进行图像处理可得到购物车所在的商超的分区作为购物车的定位分区。可通过购物上的无线通信装置接收商超中位于不同预设位置处的信号发送模块发送的距离信息，通过处理装置选取位于定位分区中的多个信号发送模块发送的距离信息并计算得到购物车的第一定位位置作为购物车的实际位置输出。通过采集参考图像进行视觉定位以确定购物车的定位分区，并选取定位分区内的信号发送模块的距离信息以计算购物车的位置，能够排除易造成定位误差的距离信息，从而提高购物车定位的准确度。
对于本公开，还有以下几点需要说明：
(1)本公开实施例附图只涉及到与本公开实施例涉及到的结构，其他结构可参考通常设计。
(2)在不冲突的情况下，本公开的实施例及实施例中的特征可以相互组合以得到新的实施例。
显然，本公开的上述实施例仅仅是为清楚地说明本公开所作的举例，而并非是对本公开的实施方式的限定，对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动，这里无法对所有的实施方式予以穷举，凡是属于本公开的技术方案所引伸出的显而易见的变化或变动仍处于本公开的保护范围之列。本公开的保护范围应以所述权利要求的保护范围为准。

Claims (22)

  1. 一种商超购物车定位方法，包括：
    采集商超的第一参考图像，处理所述第一参考图像得到购物车所在的分区为定位分区，其中，所述商超包括多个分区；
    接收在所述定位分区中设置的至少一个信号发送模块发送的距离信息，其中，每个分区中设有至少一个信号发送模块；
    根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置，基于所述购物车的第一定位位置确定所述购物车的实际位置。
  2. 根据权利要求1所述的商超购物车定位方法，其中，基于所述购物车的第一定位位置确定所述购物车的实际位置包括：
    将所述第一购物车的第一定位位置作为所述购物车的实际位置。
  3. 根据权利要求2所述的商超购物车定位方法，还包括：
    得到多个连续时刻的所述购物车的多个第一定位位置，确定所述多个第一定位位置分别对应的多个分区，确定所述多个第一定位位置对应的所述多个分区中数目最多的分区为当前分区；
    若所述当前分区与所述定位分区不同，再次采集所述商超的第二参考图像，并处理所述第二参考图像得到所述购物车的第二定位位置，利用所述第二定位位置替换与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置；
    若所述当前分区与所述定位分区相同，将与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置。
  4. 根据权利要求1-3任一项所述的商超购物车定位方法，其中，所述商超包括货架分区和通行分区，所述通行分区包括多个依次连接的栅格；
    所述商超购物车定位方法还包括：
    得到第一时刻对应的所述购物车的第一定位位置和第二时刻对应的所述购物车的第一定位位置；
    计算所述第一时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第一组距离值，计算所述第二时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取所述第 一组距离值中的第一时刻最小距离值和所述第二组距离值中的第二时刻最小距离值，并比较所述第一时刻最小距离值和所述第二时刻最小距离值的距离差值与第一预设距离值；
    当所述距离差值小于等于所述第一预设距离值，则将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    当所述距离差值大于所述第一预设距离值，则通过磁力计对所述购物车的实际位置进行校正。
  5. 根据权利要求4所述的商超购物车定位方法，其中，通过磁力计传感器对所述购物车的实际位置进行校正包括：
    获取在所述第一时刻时所述磁力计的第一角度检测信息和所述第二时刻时所述磁力计的第二角度检测信息，计算得到所述第一时刻和所述第二时刻的角度差值；
    若所述角度差值小于等于预设角度值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    若所述角度差值大于所述预设角度值且所述距离差值小于等于第二预设距离值，将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    若所述角度差值大于所述预设角度值且所述距离差值大于所述第二预设距离值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置。
  6. 根据权利要求1-5任一项所述的商超购物车定位方法，其中，所述定位分区对应至少三个信号发送模块，
    根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置包括：
    获取所述定位分区对应的所述至少三个信号发送模块发送的至少三个距离信息；
    确定所述至少三个距离信息中最小的三个距离信息和与所述最小的三个距离信息对应的三个信号发送模块的位置坐标；
    根据所述最小的三个距离信息和所述三个信号发送模块的位置坐标，基于三点加权质心定位算法得到所述购物车的第一定位位置。
  7. 根据权利要求2所述的商超购物车定位方法，其中，所述商超还包括多个预设标识，每个分区对应至少一个预设标识，所述多个预设标识包括第一预设标识，所述第一参考图像包含所述商超中的第一预设标识，
    处理所述第一参考图像得到购物车所在的分区为定位分区包括：
    处理所述第一参考图像，以识别所述第一参考图像中的第一预设标识；
    根据所述第一预设标识，确定所述购物车所在的分区为定位分区。
  8. 根据权利要求7所述的商超购物车定位方法，其中，所述购物车所在的分区与所述第一参考图像中的所述第一预设标识对应。
  9. 根据权利要求7或8所述的商超购物车定位方法，其中，所述多个预设标识还包括第二预设标识，所述第二参考图像包含所述第二预设标识。
  10. 一种商超购物车定位系统，包括：
    图像采集装置，被配置为采集商超的第一参考图像，其中，所述商超包括多个分区；
    无线通信装置，被配置为接收所述商超中的多个信号发送模块发送的距离信息；
    处理装置，被配置为接收所述第一参考图像，处理所述第一参考图像得到并确定购物车所在的分区为定位分区，并根据所述定位分区对应的至少一个信号发送模块发送的距离信息得到所述购物车的第一定位位置，基于所述购物车的第一定位位置确定所述购物车的实际位置。
  11. 根据权利要求10所述的商超购物车定位系统，其中，所述处理装置被配置为：将所述第一购物车的第一定位位置作为所述购物车的实际位置。
  12. 根据权利要求11所述的商超购物车定位系统，其中，所述处理装置还被配置为：得到多个连续时刻的所述购物车的多个第一定位位置，确定所述多个第一定位位置分别对应的多个分区，确定所述多个第一定位位置对应的所述多个分区中数目最多的分区为当前分区；
    若所述当前分区与所述定位分区不同，所述图像采集装置还被配置为再次采集所述商超的第二参考图像，所述处理装置还被配置为处理所述第二参考图像得到所述购物车的第二定位位置，利用所述第二定位位置替换与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位置，
    若所述当前分区与所述定位分区相同，所述处理装置还被配置为将与所述多个连续时刻中最后一个时刻对应的第一定位位置作为所述购物车的实际位 置。
  13. 根据权利要求10-12任一项所述的商超购物车定位系统，还包括可固定在所述购物车上的磁力计；
    所述商超包括货架分区和通行分区，所述通行分区包括多个依次连接的栅格；
    所述处理装置还被配置为：
    得到第一时刻对应的所述购物车的第一定位位置和第二时刻对应的所述购物车的第一定位位置；
    计算所述第一时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第一组距离值，计算所述第二时刻对应的第一定位位置与所述商超中的所有栅格的中心点的距离，以得到第二组距离值，分别选取所述第一组距离值中的第一时刻最小距离值和所述第二组距离值中的第二时刻最小距离值，并比较所述第一时刻最小距离值和所述第二时刻最小距离值的距离差值与第一预设距离值；
    当所述距离差值小于等于所述第一预设距离值，则将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    当所述距离差值大于所述第一预设距离值，则通过所述磁力计对所述购物车的实际位置进行校正。
  14. 根据权利要求13所述的商超购物车定位系统，其中，所述处理装置还被配置为：
    获取在所述第一时刻时所述磁力计的第一角度检测信息和所述第二时刻时所述磁力计的第二角度检测信息，计算得到所述第一时刻和所述第二时刻的角度差值；
    若所述角度差值小于等于预设角度值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    若所述角度差值大于所述预设角度值且所述距离差值小于等于第二预设距离值，将所述第二时刻对应的所述第二时刻最小距离值对应的栅格的中心点的位置作为所述购物车的实际位置；
    若所述角度差值大于预设角度值且所述距离差值大于所述第二预设距离值，将所述第一时刻对应的所述第一时刻最小距离值对应的栅格的中心点的位 置作为所述购物车的实际位置。
  15. 根据权利要求10-14任一项所述的商超购物车定位系统，其中，所述定位分区对应至少三个信号发送模块，
    所述处理装置被配置为：获取所述定位分区对应的所述至少三个信号发送模块发送的至少三个距离信息；确定所述至少三个距离信息中最小的三个距离信息和与所述最小的三个距离信息对应的三个信号发送模块的位置坐标；根据所述最小的三个距离信息和所述三个信号发送模块的位置坐标，基于三点加权质心定位算法得到所述购物车的第一定位位置。
  16. 根据权利要求10-15任一项所述的商超购物车定位系统，其中，所述商超还包括多个预设标识，每个分区对应至少一个预设标识，所述多个预设标识包括第一预设标识，所述第一参考图像包含所述商超中的第一预设标识，
    所述处理装置被配置为：处理所述第一参考图像，以识别所述第一参考图像中的第一预设标识；根据所述第一预设标识，确定所述购物车所在的分区为定位分区。
  17. 根据权利要求16所述的商超购物车定位系统，其中，所述购物车所在的分区与所述第一参考图像中的所述第一预设标识对应。
  18. 根据权利要求10-17任一项所述的商超购物车定位系统，其中，所述多个分区对应的多个信号发送模块分别设置在所述商超中的多个预设位置处。
  19. 根据权利要求10-18任一项所述的商超购物车定位系统，其中，所述至少一个信号发送模块为蓝牙信标。
  20. 根据权利要求10-19任一项所述的商超购物车定位系统，其中，所述图像采集装置包括摄像头，所述摄像头的拍摄方向与水平方向的夹角为45度～90度。
  21. 一种商超购物车定位系统，包括：
    图像采集装置，被配置为采集商超的第一参考图像，其中，所述商超包括多个分区；
    存储器，被配置为存储计算机可读指令；以及
    处理器，被配置为运行所述计算机可读指令，其中，所述计算机可读指令被所述处理器运行时执行根据权利要求1-9任一项所述的商超购物车定位方法。
  22. 一种商超购物车，包括车体以及如权利要求10-21任一项所述的商超购物车定位系统。
PCT/CN2019/094574 2018-07-03 2019-07-03 商超购物车定位方法、商超购物车定位系统及商超购物车 WO2020007323A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/633,393 US11288839B2 (en) 2018-07-03 2019-07-03 Supermarket shopping cart positioning method, supermarket shopping cart positioning system, and supermarket shopping cart

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810711492.3A CN108845289B (zh) 2018-07-03 2018-07-03 一种商超购物车定位方法、系统及商超购物车
CN201810711492.3 2018-07-03

Publications (1)

Publication Number Publication Date
WO2020007323A1 true WO2020007323A1 (zh) 2020-01-09

Family

ID=64201160

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/094574 WO2020007323A1 (zh) 2018-07-03 2019-07-03 商超购物车定位方法、商超购物车定位系统及商超购物车

Country Status (3)

Country Link
US (1) US11288839B2 (zh)
CN (1) CN108845289B (zh)
WO (1) WO2020007323A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114485395A (zh) * 2022-01-06 2022-05-13 深圳中科飞测科技股份有限公司 承载方法及相关装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108845289B (zh) 2018-07-03 2021-08-03 京东方科技集团股份有限公司 一种商超购物车定位方法、系统及商超购物车
CN112689234B (zh) * 2020-12-28 2023-10-17 北京爱笔科技有限公司 室内车辆定位方法、装置、计算机设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6470267B1 (en) * 1999-09-20 2002-10-22 Pioneer Corporation, Increment P Corporation Man navigation system
CN103995250A (zh) * 2014-05-29 2014-08-20 南京泰系信息技术有限公司 射频标签轨迹追踪方法
CN105438346A (zh) * 2015-11-26 2016-03-30 小米科技有限责任公司 平衡车的操纵杆高度调节方法及装置
CN106372556A (zh) * 2016-08-30 2017-02-01 西安小光子网络科技有限公司 一种光标签的识别方法
CN107766114A (zh) * 2017-10-19 2018-03-06 广东欧珀移动通信有限公司 显示商品信息的方法和装置
CN108845289A (zh) * 2018-07-03 2018-11-20 京东方科技集团股份有限公司 一种商超购物车定位方法、系统及商超购物车

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8229158B2 (en) * 2008-04-29 2012-07-24 International Business Machines Corporation Method, system, and program product for determining a state of a shopping receptacle
US7448542B1 (en) * 2008-05-05 2008-11-11 International Business Machines Corporation Method for detecting a non-scan at a retail checkout station
US20100053329A1 (en) * 2008-08-27 2010-03-04 Flickner Myron D Exit security
US8317086B2 (en) * 2011-02-16 2012-11-27 International Business Machines Corporation Communication of transaction data within a self-checkout environment
US8235285B1 (en) * 2011-06-24 2012-08-07 American Express Travel Related Services Company, Inc. Systems and methods for gesture-based interaction with computer systems
US9202105B1 (en) * 2012-01-13 2015-12-01 Amazon Technologies, Inc. Image analysis for user authentication
CN103575293B (zh) * 2012-07-25 2016-08-10 华为终端有限公司 一种磁力计方向角校正方法及磁力计
CN103281778B (zh) 2013-06-03 2016-04-13 上海北大方正科技电脑系统有限公司 基于无线传感网络的物联网智能手机室内定位方法及系统
US20160110791A1 (en) * 2014-10-15 2016-04-21 Toshiba Global Commerce Solutions Holdings Corporation Method, computer program product, and system for providing a sensor-based environment
US20180225647A1 (en) * 2015-04-08 2018-08-09 Heb Grocery Company Lp Systems and methods for detecting retail items stored in the bottom of the basket (bob)
US20160300212A1 (en) * 2015-04-08 2016-10-13 Heb Grocery Company Lp Systems and methods for detecting retail items stored in the bottom of the basket (bob)
KR20160144665A (ko) * 2015-06-09 2016-12-19 에스케이플래닛 주식회사 객체 인식 및 데이터베이스 매칭 결과를 표시하는 사용자 장치, 그의 제어 방법 및 컴퓨터 프로그램이 기록된 기록매체
US10088549B2 (en) * 2015-06-25 2018-10-02 Appropolis Inc. System and a method for tracking mobile objects using cameras and tag devices
CN105093172A (zh) * 2015-08-10 2015-11-25 联想(北京)有限公司 一种定位方法及电子设备
CN105548961A (zh) * 2016-01-22 2016-05-04 刘思超 一种定位方法、购物车、服务器和购物车定位系统
US20180025500A1 (en) * 2016-07-22 2018-01-25 Appropolis Inc. Method of tracking one or more mobile objects in a site and a system employing same
CN107770858A (zh) * 2016-08-17 2018-03-06 上海新飞凡电子商务有限公司 基于rssi和区域划分的快速三点定位法及其系统
CN106793078B (zh) 2017-01-05 2019-12-24 西安电子科技大学 基于rssi修正值双重定位的蓝牙室内定位方法
US20180211300A1 (en) * 2017-01-26 2018-07-26 Wal-Mart Stores, Inc. System and method for assessing wait times in a facility
EP3646242A4 (en) * 2017-06-29 2021-02-17 Hewlett-Packard Development Company, L.P. DETECTION OF THE DIRECTION OF A MOBILE COMPUTER DEVICE TOWARDS A VISUAL CODE
CN107883950B (zh) * 2017-11-03 2020-04-28 深圳市沃特沃德股份有限公司 停车场导航方法、装置和系统
US10657691B2 (en) * 2018-03-27 2020-05-19 Faro Technologies, Inc. System and method of automatic room segmentation for two-dimensional floorplan annotation
CN108692720B (zh) * 2018-04-09 2021-01-22 京东方科技集团股份有限公司 定位方法、定位服务器及定位系统
US10572737B2 (en) * 2018-05-16 2020-02-25 360Ai Solutions Llc Methods and system for detecting a threat or other suspicious activity in the vicinity of a person
US10572738B2 (en) * 2018-05-16 2020-02-25 360Ai Solutions Llc Method and system for detecting a threat or other suspicious activity in the vicinity of a person or vehicle
US10572740B2 (en) * 2018-05-16 2020-02-25 360Ai Solutions Llc Method and system for detecting a threat or other suspicious activity in the vicinity of a motor vehicle
US10572739B2 (en) * 2018-05-16 2020-02-25 360Ai Solutions Llc Method and system for detecting a threat or other suspicious activity in the vicinity of a stopped emergency vehicle
US20190356885A1 (en) * 2018-05-16 2019-11-21 360Ai Solutions Llc Camera System Securable Within a Motor Vehicle
EP3654305B1 (de) * 2018-11-14 2024-02-14 Bizerba SE & Co. KG Verkaufsvorrichtung zum selbst-checkout
US11966900B2 (en) * 2019-07-19 2024-04-23 Walmart Apollo, Llc System and method for detecting unpaid items in retail store transactions
US11265518B2 (en) * 2019-09-03 2022-03-01 BOB Profit Partners LLC Camera system monitor for shopping cart bottom shelf

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6470267B1 (en) * 1999-09-20 2002-10-22 Pioneer Corporation, Increment P Corporation Man navigation system
CN103995250A (zh) * 2014-05-29 2014-08-20 南京泰系信息技术有限公司 射频标签轨迹追踪方法
CN105438346A (zh) * 2015-11-26 2016-03-30 小米科技有限责任公司 平衡车的操纵杆高度调节方法及装置
CN106372556A (zh) * 2016-08-30 2017-02-01 西安小光子网络科技有限公司 一种光标签的识别方法
CN107766114A (zh) * 2017-10-19 2018-03-06 广东欧珀移动通信有限公司 显示商品信息的方法和装置
CN108845289A (zh) * 2018-07-03 2018-11-20 京东方科技集团股份有限公司 一种商超购物车定位方法、系统及商超购物车

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114485395A (zh) * 2022-01-06 2022-05-13 深圳中科飞测科技股份有限公司 承载方法及相关装置

Also Published As

Publication number Publication date
CN108845289A (zh) 2018-11-20
CN108845289B (zh) 2021-08-03
US11288839B2 (en) 2022-03-29
US20200234465A1 (en) 2020-07-23

Similar Documents

Publication Publication Date Title
CN110411441B (zh) 用于多模态映射和定位的系统和方法
WO2021233029A1 (en) Simultaneous localization and mapping method, device, system and storage medium
CN111199564B (zh) 智能移动终端的室内定位方法、装置与电子设备
US9727580B2 (en) Information processing device, map update method, program, and information processing system
CN107230225B (zh) 三维重建的方法和装置
WO2020223974A1 (zh) 更新地图的方法及移动机器人
WO2020007323A1 (zh) 商超购物车定位方法、商超购物车定位系统及商超购物车
US11610373B2 (en) Method of generating three-dimensional model data of object
CN107610176A (zh) 一种基于Kinect的栈板动态识别与定位方法、系统及介质
CN112667837A (zh) 图像数据自动标注方法及装置
CN112419374B (zh) 一种基于图像配准的无人机定位方法
CN110073362A (zh) 用于车道标记检测的系统及方法
Yu et al. Robust robot pose estimation for challenging scenes with an RGB-D camera
CN105354841B (zh) 一种快速遥感影像匹配方法及系统
KR20210116507A (ko) 교정 방법, 위치 확정 방법, 장치, 전자 디바이스 및 저장매체
CN107368790B (zh) 行人检测方法、系统、计算机可读存储介质及电子设备
CN107220996B (zh) 一种基于三角结构一致的无人机线阵与面阵影像匹配方法
KR102303779B1 (ko) 복수 영역 검출을 이용한 객체 탐지 방법 및 그 장치
CN109559347A (zh) 对象识别方法、装置、系统及存储介质
CN112652020A (zh) 一种基于AdaLAM算法的视觉SLAM方法
Chen et al. A study of sensor-fusion mechanism for mobile robot global localization
CN116129037A (zh) 视触觉传感器及其三维重建方法、系统、设备及存储介质
WO2022062853A1 (zh) 遥感图像的配准方法、装置、设备、存储介质及系统
JP2023503750A (ja) ロボットの位置決め方法及び装置、機器、記憶媒体
CN109785388B (zh) 一种基于双目摄像头的短距离精确相对定位方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19831396

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19831396

Country of ref document: EP

Kind code of ref document: A1