WO2021092883A1 - 物品管理方法、终端装置和物品管理设备及存储介质 - Google Patents

物品管理方法、终端装置和物品管理设备及存储介质 Download PDF

Info

Publication number
WO2021092883A1
WO2021092883A1 PCT/CN2019/118726 CN2019118726W WO2021092883A1 WO 2021092883 A1 WO2021092883 A1 WO 2021092883A1 CN 2019118726 W CN2019118726 W CN 2019118726W WO 2021092883 A1 WO2021092883 A1 WO 2021092883A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
article
storage unit
item
article management
Prior art date
Application number
PCT/CN2019/118726
Other languages
English (en)
French (fr)
Inventor
任翔
薛冰
徐升
黄捷汶
黄瑞浩
Original Assignee
深圳市微蓝智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市微蓝智能科技有限公司 filed Critical 深圳市微蓝智能科技有限公司
Priority to PCT/CN2019/118726 priority Critical patent/WO2021092883A1/zh
Priority to CN201980022416.4A priority patent/CN112005244A/zh
Publication of WO2021092883A1 publication Critical patent/WO2021092883A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes

Definitions

  • the present disclosure relates to the technical field of article management, and in particular, to an article management method, as well as non-temporary computer storage media, terminal devices, and article management equipment.
  • the management of items includes identifying whether the type of the item is correct, and identifying the status of the item, such as whether it is returned or lent.
  • the current article management technologies mainly include RFID (Radio Frequency Identification) technology, lining sensors, line scan camera technology, etc.
  • RFID technology uses RFID radio frequency technology to scan the electronic tags to identify the status of the items by attaching electronic tags to them.
  • this technology has some shortcomings, for example, the electronic tag is not easy to stick to the article, the use of the article is affected after the electronic label is attached, and the misidentification due to the signal delay of the radio wave is caused.
  • a lining sensor such as a weight sensor
  • a sensor is placed on the inner lining of the article to identify the current state of the article.
  • this method also has some disadvantages, such as high recognition error rate and high manufacturing cost.
  • the line scan camera technology For the line scan camera technology, at least one set of line scan cameras is set above the storage device, and a periscope mirror set on the front eaves of the storage device is used to project the light 90 degrees to the camera, and the illuminated area in the drawer Is the scanned area.
  • the line scan camera needs to cooperate with the motion feedback sensor to detect the motion speed and motion position of the drawer, in order to assemble a complete image, and the cost of this solution is extremely high.
  • the present disclosure aims to solve at least one of the technical problems existing in the prior art.
  • an embodiment of the present disclosure proposes an item management method, which has a high accuracy rate and low cost for identifying items.
  • the embodiment of the present disclosure also proposes a non-transitory computer storage medium.
  • the embodiment of the present disclosure also provides a terminal device.
  • the embodiment of the present disclosure also proposes an item management device.
  • the article management method includes: acquiring the original image of the article in the storage unit; The image is processed to obtain the target image of the article; the state of the article and the category of the article are identified according to the target image of the article.
  • the article management method of the embodiment of the present disclosure by acquiring the image of the article in the storage unit and performing image processing, the state and category of the article are recognized, that is, whether the article is stored and the type of the stored article, so as to determine whether the article is returned or Whether it is misplaced, so as to achieve the purpose of managing items, the plan is simple to deploy and low in cost.
  • the acquiring the original image of the article in the storage unit includes: detecting that the storage unit has reached a preset state, and collecting the image of the article in the storage unit as the original image.
  • processing the original image of the article to obtain the target image of the article includes: performing matting processing on the original image of the article to obtain the target image of the article.
  • processing the original image of the article before performing the matting process, further includes: obtaining key frame images in the original image of the article; and filtering the key frame images, thereby Can improve the accuracy of image recognition.
  • processing the original image of the article before performing the matting process, further includes: correcting the original image or the filtered image of the article to obtain a corrected image; Alignment processing is performed with the template image of the article stored in the storage unit, so that the correction image is aligned with the pixels of the template image.
  • Alignment processing is performed with the template image of the article stored in the storage unit, so that the correction image is aligned with the pixels of the template image.
  • the acquiring the key frame image in the original image of the article includes: determining the side length information of the storage unit according to the original image of the article; and according to the side length of the storage unit The length information determines the image with the largest side length when the storage unit changes from static to dynamic during the closing process, as the key frame image.
  • the screening of the key frame images includes: using an image screening algorithm to select images with complete and unobstructed images in the key frame images as the screened images to improve the efficiency of image recognition And accuracy.
  • the correcting the original image or the filtered image of the article to obtain the corrected image includes: using at least one of a perspective transformation algorithm and an affine transformation algorithm to perform a correction on the article.
  • the original image or the filtered image is corrected to obtain the corrected image.
  • aligning the corrected image with the template image of the article stored in the storage unit to align the pixels of the corrected image with the template image includes: identifying that the corrected image Align the marked pixels of the corrected image with the marked pixels of the template image, so that the corrected image is aligned with the pixels of the template image, so as to improve the accuracy of image recognition.
  • performing the matting process includes: acquiring the coordinate values of the contour of the article in the template image; and performing matting processing on the corrected image according to the coordinate value of the contour of the article in the template image.
  • the performing matting processing further includes: identifying the marked pixels in the corrected image and the marked pixels in the template image; calculating the marked pixels in the corrected image and the template The coordinate difference of the marked pixel points of the image; the coordinate value of the contour of the article in the template image is corrected according to the coordinate difference, so as to perform matting processing and improve the accuracy of image recognition.
  • the recognizing the state of the article and the category of the article according to the target image of the article includes: using an artificial intelligence algorithm to recognize the target image of the article to determine whether the stored article exists and to identify the The category of the item.
  • the article management method further includes: prompting according to the state of the article and the category of the article.
  • the item management personnel can know or make inquiries in time.
  • prompting according to the state of the item and the category of the item includes at least one of the following:
  • the non-transitory computer storage medium proposed by the embodiment of the present disclosure has a computer program stored thereon, and when the computer program is executed, the article management method is realized.
  • the terminal device proposed by the embodiment of the present disclosure includes: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and When the instruction is executed by the at least one processor, the at least one processor is caused to execute the item management method.
  • the terminal device of the embodiment of the present disclosure by executing the item management method through the processor, the intelligent management of items can be realized, the solution is easy to implement, and the cost is low.
  • the article management equipment of the embodiment of the present disclosure includes: a cabinet and a storage unit for storing articles; an image acquisition device for collecting images of articles stored in the storage unit; and the terminal device.
  • the image of the article in the storage unit is collected by the image acquisition device, and the terminal device performs image processing and recognizes the state and category of the article, that is, whether the article is stored and the type of the stored article can be determined Whether the items are returned or misplaced, so as to achieve the purpose of managing items, the plan is simple to deploy and the cost is low.
  • the item management equipment further includes a communication device for transmitting image data to the terminal device, which can realize remote item management, reduce the amount of local data processing, and improve data processing efficiency .
  • the storage unit can be opened and closed on the cabinet; the image acquisition device is used to collect images of the articles in the storage unit when the storage unit reaches a preset state.
  • the image capture device is arranged on the cabinet and above the storage unit.
  • the image capture device is provided in the cabinet, and the storage unit can move away from the cabinet and return to the cabinet after items are stored.
  • the article management equipment further includes: a detection device, which is provided in the cabinet, and is used to detect whether the storage unit reaches a preset state.
  • the item management equipment further includes a prompting device, which is used for prompting according to the feedback information of the terminal device, so that the item management personnel can know it in time or facilitate inquiries.
  • the image acquisition device includes: at least one sensor arranged in the cabinet and located above the storage unit when the storage unit reaches a preset state; and at least one light source for Light is emitted to form a surface light on the surface of the storage unit.
  • the item management equipment further includes a power supply device for supplying power to the image acquisition device.
  • the power supply device includes a wire power supply or a battery.
  • the battery is provided in the image capture device.
  • the battery is provided in the cabinet or the storage unit.
  • the power supply device is also used to supply power to the terminal device or the communication device.
  • Fig. 1 is a block diagram of an item management device according to an embodiment of the present disclosure
  • Figure 2 is a block diagram of an item management device according to another embodiment of the present disclosure.
  • Fig. 3 is a flowchart of an item management method according to an embodiment of the present disclosure.
  • Figure 4 is a schematic diagram of an item management device according to an embodiment of the present disclosure.
  • Fig. 5 is a schematic diagram of an item management device according to another embodiment of the present disclosure.
  • Fig. 6 is a schematic diagram of a cabinet and a storage unit according to an embodiment of the present disclosure
  • FIG. 7 is a schematic diagram of the configuration of an image capture device according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 9 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 11 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 12 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 13 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 14 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 15 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 16 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 17 is a schematic diagram of the configuration of an image capture device according to another embodiment of the present disclosure.
  • FIG. 18 is a schematic diagram of the side length observed by the storage unit and its selected key frame image according to an embodiment of the present disclosure
  • Fig. 19 is a variation curve of the side length of the storage unit with time according to an embodiment of the present disclosure.
  • Fig. 20 is a schematic diagram of correcting and transforming an image according to an embodiment of the present disclosure.
  • FIG. 21 is a schematic diagram of the position of the marking point on the storage unit according to an embodiment of the present disclosure.
  • FIG. 22 is a flowchart of a process of identifying items according to an embodiment of the present disclosure.
  • FIG. 23 is a block diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 24 is a block diagram of a terminal device according to another embodiment of the present disclosure.
  • first and second are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” may explicitly or implicitly include one or more of these features. Further, in the description of the present disclosure, unless otherwise specified, “plurality” means two or more than two.
  • items used by personnel usually need to be stored in specified lockers, storage boxes or warehouses, such as inspection tools used by medical personnel Or surgical tools, tools used by aircraft maintenance personnel, firearms or supplies used by armed forces personnel, tools used by factory workers, etc., need to be returned to the corresponding location after use to avoid unknown whereabouts or storage errors. Therefore, right Item management is necessary.
  • the present disclosure proposes an item management method and an item management device, in which, as shown in FIG. 1 is a functional block diagram of an item management device according to an embodiment of the present disclosure.
  • the item management device 1 includes a cabinet 11 and a storage unit for storing items 12.
  • the image acquisition device 13 is used to acquire an image of the article stored in the storage unit 12 as the original image of the article.
  • the terminal device 14 realizes the management of the items according to the original images of the items in the storage unit.
  • the terminal device 14 may be a local device such as a computer device, an image processing device, etc., set in the place where the storage unit 12 is located, or a mobile smart terminal such as a mobile phone, a tablet computer, etc., or a remote computer or a cloud server. Wait.
  • the item management device 1 may also include a communication device 15 that transmits image data to a terminal device 14 such as a remote computer or cloud server, and the remote computer or cloud server implements item management based on the image data.
  • Fig. 3 is a flowchart of an item management method according to an embodiment of the present disclosure. As shown in Fig. 3, the item management method of an embodiment of the present disclosure at least includes steps S1-S3, and each step is described as follows.
  • Step S1 Obtain the original image of the item in the storage unit.
  • the image of the article in the storage unit can be collected by the image collection device, and the image data can be transmitted to a terminal device, such as a local computer device, or the image data can be transmitted to the terminal device through a communication device, such as a mobile smart terminal or the cloud.
  • the server and the terminal device receive the image data and use the image as the original image of the item in the storage unit.
  • step S2 the original image of the article is processed to obtain the target image of the article.
  • the terminal device After the terminal device receives the original image of the item in the storage unit, it processes the original image, for example, selects at least one action such as selecting a key frame image, correcting the image, aligning the image, etc., and performing matting in the image , To obtain an image that can be used to identify the status and category of the item, and use this as the target image.
  • Step S3 Recognizing the state of the article and the category of the article according to the target image of the article.
  • the item status can be understood as whether the item is returned, whether it is complete, etc.
  • the category of the item includes what the item is, which category it belongs to, and so on.
  • the terminal device After obtaining the target image of the item, the terminal device performs image recognition on the target image to identify whether there is an item that should be stored in the image, what the item is actually stored, what is stored in a specific location, etc., so as to determine whether the item is returned or returned Whether the items are correct, whether the storage location is correct, etc., so as to realize the management of the items.
  • the article management method of the embodiment of the present disclosure by acquiring the image of the article in the storage unit and performing image processing, the state and category of the article are recognized, that is, whether the article is stored and the type of the stored article, so as to determine whether the article is returned or Whether it is misplaced, so as to achieve the effect of managing the items, the plan is simple to deploy and the cost is low.
  • the storage unit 12 can be arranged in cooperation with the cabinet 11, or can be arranged independently from the cabinet 11.
  • the image acquisition device 13 may adopt an external mode or a built-in mode.
  • the image acquisition device 13 may be arranged on the cabinet 11 and above the storage unit 12 to facilitate the acquisition of images of the items in the storage unit 12, or the image acquisition device 13 may be arranged on a support frame outside the cabinet 11
  • the image acquisition device 13 is set at a certain position of the place where the storage unit 12 is located, or the image acquisition device 13 is set at another location where the image of the article in the storage unit 12 can be collected.
  • the storage unit 12 can be opened and closed on the cabinet 11, that is, the storage unit 12 and the cabinet 11 are integrated.
  • FIG. 4 the item management according to an embodiment of the present disclosure is shown.
  • the multiple drawers are respectively arranged in the cabinet body 11 to be opened and closed.
  • the drawers can be opened when storing items, and the drawers can be closed after the storage is completed.
  • the image acquisition device 13 is used to acquire an image of the article in the storage unit 12 when the storage unit 12 reaches a preset state, as an original image for image processing.
  • the preset state may be a state during the closing process of the storage unit 12, for example, the storage unit 12 is completely closed or about to be completely closed, or there is a pause during the closing process, or a signal triggered by a sensor.
  • the image acquisition device 13 can be integrated with the storage unit 12 and the cabinet 11 and adopt a built-in manner.
  • the image acquisition device 13 is provided on the cabinet 11 and located above the storage unit 12.
  • the image capture device 13 captures an image of the article in the storage unit 12 as the original image; or, the image capture device 13 is integrated with the storage unit 12 and the cabinet 11 and adopts Externally, for example, the image capture device 13 is arranged on a support frame outside the cabinet 11.
  • the image acquisition device 13 collects images of the items in the storage unit 12; or, the image acquisition device 13 can also be set independently, for example, set at a certain position near the cabinet 11, for example, an image
  • the collection device 13 is installed on the wall near the cabinet 11. After storing the items, move the storage unit 12 and the cabinet 11 to the image capture device 13.
  • the image capture device 13 can capture images of the items in the storage unit 12 to As the original image.
  • the image capture device 13 is installed in the cabinet 11, and the storage unit 12 can move away from the cabinet 11 and return to the cabinet 11 after the items are stored, that is, the image capture device 13 and the cabinet 11 are integrated.
  • the storage unit 12 is set independently.
  • FIG. 5 shows a schematic diagram of an item management device according to an embodiment of the present disclosure, where the storage unit 12 is a freely movable storage cart, and the storage cart includes at least one tray to store items.
  • the terminal device 14 is arranged on the cabinet body 11; for multiple storage units 12, the cabinet body 11 can be provided with multiple cabinet doors, which can reduce the interference of external environmental factors on image collection.
  • the image acquisition device 13 can be built-in, for example, set inside the cabinet 11.
  • the image acquisition device 13 is located above the storage cart to collect the items in the storage cart.
  • Image specifically, when you need to store items, open the cabinet door, pull out the storage cart, and after storing the items, return the storage cart to the corresponding cabinet 11, the image acquisition device 13 can collect the storage cart
  • the image of the upper item is used as the original image of the item in the drawer. This method is less disturbed by the external environment, and higher quality images can be obtained.
  • the image acquisition device 13 can also be externally installed, for example, set on a support frame outside the cabinet 11. After the storage cart stores items, the image capture device 13 is in the process of returning the storage cart to the cabinet 11. It can record the video of the items in the storage cart or continuously collect multiple continuous shooting images.
  • the image acquisition device 13 may include at least one sensor 131 and at least one light source 132, and the at least one sensor 131 is arranged in the cabinet 11 and The storage unit 12 is located above the storage unit 12 when it reaches a preset state, and at least one light source 132 is used to emit light to form a surface light on the surface of the storage unit 12.
  • FIG. 6 is a schematic diagram of a cabinet 11 and a storage unit 12 according to an embodiment of the present disclosure.
  • the storage unit 12 includes a plurality of drawers, and an image capture device 13 may be provided above each drawer, such as an image capture device. 13 is arranged on the drawer bottom plate 133 above the drawer and faces the drawer.
  • the image acquisition device 13 corresponding to the uppermost drawer can be arranged on the inner top of the cabinet 11 to acquire images of the corresponding items in the drawer through the image acquisition device 13.
  • Figures 7-17 respectively show schematic diagrams of sensors and light sources with different numbers and positions.
  • the image acquisition device 13 may use one sensor 131 and one light source 132, and both light source 132 and sensor 131 It is arranged on the edge of the longest side of the drawer bottom plate 133, so that the sensor 131 can obtain a large enough field of view, and the light source 132 can cover a large enough area.
  • the image acquisition device 13 uses one sensor 131 and two light sources 132, the sensor 131 is arranged on the edge of the longest side of the drawer bottom plate 133, and the two light sources 132 are arranged on two diagonal corners of the drawer bottom plate 133. The location ensures the intensity and coverage of the light source 132.
  • the image acquisition device 13 may use one sensor 131 and four light sources 132.
  • the sensor 131 is arranged at the center of the drawer bottom plate 133, and the light sources 132 are arranged at four diagonal positions of the drawer bottom plate 133 to ensure Ample light.
  • the image acquisition device 13 may use one sensor 131 and six light sources 132, wherein the sensor 131 is arranged at the center of the drawer bottom plate 133, and the four light sources 132 are arranged at four diagonal corners of the drawer bottom plate 133.
  • the other two light sources 132 are arranged in the middle of the long sides of the drawer bottom plate 133 to ensure sufficient light and obtain clearer images.
  • FIG. 10 the image acquisition device 13 may use one sensor 131 and four light sources 132.
  • the sensor 131 is arranged at the center of the drawer bottom plate 133, and the light sources 132 are arranged at four diagonal corners of the drawer bottom plate 133.
  • the other two light sources 132 are arranged in the middle of the long sides of the drawer bottom plate 133 to ensure
  • the image acquisition device 13 may adopt a sensor 131 and eight light sources 132, the sensor 131 is arranged at the center of the drawer bottom plate 133, and the four light sources 132 are arranged at four diagonal corners of the drawer bottom plate 133. The other four light sources 132 are arranged in the middle of each side of the drawer bottom plate 133 to ensure sufficient light to obtain a clearer image.
  • the image acquisition device 13 may adopt two sensors 131 and two light sources 132, and the two sensors 131 are separated by a preset distance along the central axis of the long side of the drawer bottom plate 133.
  • the two light sources 132 are arranged at diagonal positions of the drawer bottom plate 133, which can take images from more angles and ensure sufficient light to obtain better image information.
  • the image acquisition device 13 may adopt two sensors 131 and four light sources 132, the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, and the four light sources 132 are arranged at The four diagonal positions of the drawer bottom 133.
  • FIG. 13 the image acquisition device 13 may adopt two sensors 131 and four light sources 132, the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, and the four light sources 132 are arranged at The four diagonal positions of the drawer bottom 133.
  • FIG. 13 the image acquisition device 13 may adopt two sensors 131 and four light sources 132, the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, and the four light sources 132 are arranged at
  • the image acquisition device 13 may adopt two sensors 131 and six light sources 132, and the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, of which four light sources 132 They are arranged at the four diagonal positions of the drawer bottom plate 133, and the other two light sources 132 are arranged at the middle positions of the two long sides of the drawer bottom plate 133. Or, as shown in FIG.
  • the image acquisition device 13 may use two sensors 131 and six light sources 132, the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, and the four light sources 131 are arranged at The four diagonal positions of the drawer bottom plate 133, and the other four light sources 132 are arranged in the middle positions of each side of the drawer bottom plate 133.
  • the image acquisition device 13 adopts a sensor 131 and a strip light source 132, the sensor 131 is arranged at the center of the drawer bottom plate 133, and the strip light source 132 is attached to the four peripheral edges of the drawer bottom plate 133.
  • FIG. 16 the image acquisition device 13 adopts a sensor 131 and a strip light source 132, the sensor 131 is arranged at the center of the drawer bottom plate 133, and the strip light source 132 is attached to the four peripheral edges of the drawer bottom plate 133.
  • the image acquisition device 13 may adopt two sensors 131 and a strip light source 132, and the two sensors 131 are arranged at a predetermined distance along the central axis of the long side of the drawer bottom plate 133, and the strip light source 132 is attached to it. Close the four peripheral edges of the drawer bottom plate 133.
  • the stability of the visible/non-visible light source 132 can be ensured without being affected by external conditions; it can reduce the impact on the sensor 131 during operation such as physical impact and vibration. This greatly improves the accuracy of image information acquired by the sensor 131.
  • the image acquisition device 13 needs power supply to work.
  • the article management equipment 1 further includes a power supply device 16 for supplying power to the image acquisition device 13.
  • the power supply device 16 may be a wire power supply or a battery.
  • the power supply device 16 is a wire power supply, which is connected to the image acquisition device 13 through a wire harness, and the item management device 1 is started.
  • the wire power supply can supply power to the image acquisition device 13.
  • the image acquisition device 13 can collect image information .
  • a wire power supply can be used to power the image acquisition device 13; for the cabinet 11 and the storage unit 12, the image acquisition device 13 is integrated and the image acquisition device 13 is independent.
  • the provided item management device 1 can also use a wire power supply to supply power to the image capture device 13; or, for the item management device 1 in which the cabinet 11 and the image capture device 13 are integrated and the storage unit 12 can be separated from the cabinet 11, it can also be A wire power supply is used to power the image acquisition device 13.
  • the power supply device 16 is a battery, and the image acquisition device 13 is powered by the battery.
  • the battery is provided in the image acquisition device 13 and directly supplies power to the image acquisition device 13. Specifically, it can be controlled by software.
  • the image acquisition device 13 is turned off and the battery does not output electric energy.
  • the battery supplies power to the image acquisition device 13 to wake up the image acquisition device 13 , Which saves more power, in which the item management device 1 can be activated or deactivated by pressing a button or other means; in other embodiments, the battery is provided in the cabinet 11 or the storage unit 12.
  • the battery Connect with the image acquisition device 13 to supply power to the image acquisition device 13.
  • the battery is placed in the cabinet 11 or the storage unit 12.
  • the item management device 1 is activated, and the battery Connect to the image acquisition device 13 to supply power to the image acquisition device 13.
  • the image acquisition device 13 can collect image information; for another example, the cabinet 11 and the storage unit 12 are integrated and the image acquisition device 13 is independent
  • the battery can be set in the storage unit 12 or the cabinet 11.
  • the battery is connected to the image acquisition device 13 to supply power to the image acquisition device 13.
  • the battery can be installed in the storage unit 12.
  • the cabinet 11 when the item management device 1 is started, the battery is connected to the image acquisition device 13 to supply power to the image acquisition device 13.
  • the power supply device 16 can also supply power to the terminal device 14.
  • the power supply device 16 supplies power to the image acquisition device 13 and the terminal device 14 respectively.
  • the device 16 may be one or more than one.
  • the power supply device 16 can also supply power to the communication device 15.
  • the power supply device 16 is the communication device 15 respectively. Power is supplied to the image acquisition device 13, and the power supply device 16 may be one or more than one.
  • an image capture device when capturing images, an image capture device such as a camera can be called to capture the entire process of storing items, obtain a video of the item storage process, or take multiple continuous shooting images, and The video data or continuous shooting image data is transmitted to the terminal device as the original image of the items in the storage unit.
  • the image acquisition device can be called to take at least one image according to the trigger signal of the sensor. For example, as shown in FIG. To detect whether the storage unit 12 reaches a preset state.
  • a laser transceiver can be arranged on the cabinet to match the opening and closing position of the storage unit.
  • the storage unit such as a drawer
  • the laser receiver will not receive the laser light.
  • the storage unit reaches the preset state at this time, and the image acquisition device is triggered to collect the image in the storage unit as the original image.
  • the sensor can also be a position sensor, a motion sensor, a distance sensor, a contact sensor, etc., which can be judged according to the position information, operating status, distance change, or action information of the storage unit. Whether to reach the preset state.
  • electromechanical control can be used to trigger the image acquisition device to collect image information, similar to the trigger setting of triggering the lighting when the refrigerator door is opened, or the lighting when the kitchen door of a hotel closet is opened.
  • the original image of the article in the storage unit is obtained, the original image is processed, for example, the original image of the article is matted to obtain the target image of the article.
  • the terminal device prestores an image collected when the correct item is stored in the storage unit as a template image.
  • the coordinate value of the item outline in the template image is obtained, and the original image is matted according to the coordinate value of the item outline in the template image, so as to obtain the image corresponding to the correct item position in the original image, that is, obtain the target image.
  • the items in the target image are identified, and it is judged whether there is a correct corresponding item in the storage location of the corresponding item and whether the category of the item is correct, so as to realize the management of the item.
  • the original image Since some image acquisition devices, such as fisheye cameras, are used to obtain the original image, the original image is usually distorted. Therefore, in the embodiments of the present disclosure, the original image can be corrected, aligned, and cut out; or, for external The image acquisition device obtains the original image by collecting a video stream or multiple continuous shooting images. If each frame of image is used as the original image, it will cause a very large amount of data processing and low efficiency. And in actual operation, there will also be situations where the image acquisition field of view is blocked, which results in defects in the original image. In the embodiment of the present disclosure, the original image can be filtered, corrected, aligned, and cut out to improve the accuracy of image recognition. Each processing step will be described separately below.
  • the key frame image Before matting, the key frame image can be screened first, specifically, the key frame image in the original image of the article can be obtained, and the key frame image can be screened.
  • the image acquisition device when the detection device detects that the storage unit reaches the preset state, the image acquisition device is triggered to collect an image of the article in the storage unit, and the image can be used as the original image, and the original image is It is a key frame image, and in this case, the shooting field of view will not be affected by the outside world. Therefore, this key frame image is the filtered image.
  • the image capture device can record the video of the items in the storage unit during the storage of the items or collect multiple continuous shooting images to obtain the original image, and the side length of the storage unit can be determined according to the original image of the item Information, according to the side length information of the storage unit, determine the image with the largest side length when the storage unit changes from static to dynamic during the closing process, as the key frame image.
  • the observation side length of a storage unit such as a drawer is marked.
  • the length of the observation side length changes when the drawer is opened and closed.
  • the length information of the observation side length can be used to measure the extent of the drawer being opened. .
  • the variation curve of the side length of the storage unit with time as shown in FIG. 19 is generated, wherein,
  • the storage unit is in a static state, as shown in Figure 19, when the length of the side length does not change with time, it corresponds to the pause state when the drawer is closed. At this time, there is often an action of taking out or putting in items. Therefore, the storage unit can be changed from The image with the largest side length when turning from static to dynamic, for example, the image collected at the time of curve jump shown in FIG. 19, is used as the key frame image.
  • the image capturing field of view is easily blocked, so the key frame images can be filtered.
  • the image filtering algorithm is used to select the key frame images with complete and unobstructed images as The filtered images are the most complete images that are free of foreign objects, such as hand occlusion, and have normal illumination, which are most conducive to recognition.
  • the images captured by the image capture devices usually have distortions, so the images can be further corrected and aligned.
  • the original image of the article or the filtered image is corrected to obtain a corrected image; the corrected image is aligned with the template image of the article stored in the storage unit to align the corrected image with the pixel points of the template image.
  • At least one of the perspective transformation algorithm and the affine transformation algorithm is used to correct the original image or the filtered image of the article to obtain a corrected image.
  • affine transformation and perspective transformation can be more intuitively called “plane transformation” and “space transformation” or “two-dimensional coordinate transformation” and “three-dimensional coordinate transformation”.
  • the affine transformation is two-dimensional coordinates (x, y)
  • the perspective transformation is three-dimensional coordinates (x, y, z).
  • the transformation can be achieved through a matrix. In the embodiment, as shown in FIG.
  • the original image is a fisheye image collected by a fisheye camera, and the original image or a filtered image such as a fisheye image is converted into a trapezoidal image through a perspective transformation algorithm or a radiation transformation algorithm, and then The trapezoid image is converted into a rectangular image to obtain a corrected image, which is convenient for subsequent alignment processing.
  • identifying the marked pixels in the corrected image aligning the marked pixels in the corrected image with the marked pixels of the template image, so that the corrected image is aligned with the pixels of the template image.
  • physical marking points are set on the storage unit, for example, stickers or easily recognizable color points are marked on the storage unit as physical marking points.
  • the marking position can be, for example, as shown in FIG. 21, selecting the marking at the intersection of the two sides of the storage unit or a certain position on the long side, and when the objects are correctly stored, an image of the object in the storage unit is collected as a template image.
  • the collected template image carries marking pixels corresponding to the physical marking points, and similarly, there are marked pixels in the corrected image.
  • the alignment process identify the marked pixels in the corrected image, and shift the marked pixels in the corrected image and the marked pixels of the template image until the marked pixels of the corrected image are aligned with the marked pixels of the template image.
  • it provides a basis for matting.
  • the coordinate value of the contour of the article in the template image is obtained; and the corrected image is matted according to the coordinate value of the contour of the article in the template image.
  • a cropping algorithm can be used to locate the target image by using pixel information, align the coordinate value of the item outline in the template image with the coordinate in the corrected image, and perform matting, so as to obtain the correct storage location of the item. Image, and use the image obtained by matting as the target image.
  • the coordinates of the article in the corrected image and the coordinate value of the article in the template image often have errors.
  • the errors can be corrected to obtain a more accurate target image. Specifically, identify the marked pixels in the corrected image and the marked pixels in the template image; calculate the coordinate difference between the marked pixels in the corrected image and the marked pixels of the template image; compare the contours of the items in the template image according to the coordinate differences Correct the coordinate value of, and align and cut out the image according to the corrected coordinate value, so that the target image corresponding to the correct storage location of the article can be obtained, and the accuracy of the identification of the article can be improved.
  • an artificial intelligence algorithm is used to identify the target image of the item to determine whether there is a correct corresponding item in the storage location of the corresponding item and to identify the category of the item.
  • a neural network algorithm or a deep learning algorithm or other artificial intelligence algorithms can be used to identify the target image, and the target image is input to the artificial intelligence model, and the artificial intelligence model outputs the calculation result based on the artificial intelligence algorithm, as shown in FIG. 22, according to the present disclosure.
  • the flowchart of the process of identifying an item in an embodiment specifically includes the following steps.
  • S10 Recognize the target image and classify the items based on the artificial intelligence algorithm.
  • the input target image is divided into two situations by using the dichotomy method, one of which is that the item is occluded and the category cannot be determined, then step S20 is executed, and the other is that the item is not occluded or is rarely occluded and the item category can be determined. Then step S30 is executed.
  • S30 Directly judge whether the category of the article is correct, and the output result is that the article is correct or incorrect.
  • the management of the items in the storage unit can be realized.
  • the article management device 1 of the embodiment of the present disclosure further includes a prompting device 18, and the prompting device 18 is used for prompting according to the feedback information of the terminal device 14.
  • the image acquisition device 13 collects the original image of the article in the storage unit 12, and transmits the original image to the terminal device 14, such as a computer device or an image processing device, or transmits it to a cloud server or a mobile smart terminal through the communication device 15.
  • the original image is processed and recognized, the state and category of the article are recognized, and the state and category information of the article is sent to the prompt device 18, and the prompt device 18 prompts to make the article management
  • the personnel can know the storage status of the items in time, such as whether the items are returned or the type and location of the items are correct.
  • the prompting device 18 can be implemented in various forms. For example, at least one of the following can be used for prompting: displaying the item status and item category through the display device; determining when the item is missing according to the item status and item category , For notifications, such as pop-up warning boxes or prompt boxes or voice prompts; record the item status and item type in the system log, and the item management personnel can find out the storage status of the item by querying the system log.
  • the item management method and item management device of the embodiments of the present disclosure determine the status and category of items in the storage unit based on image recognition, which is easy to implement, does not need to cooperate with other more sensor devices, and has low cost.
  • the acquisition device can be built-in or external to acquire images in a static manner. Compared with the dynamic acquisition of line scan camera technology, the cost is lower.
  • the use of a built-in image acquisition device can avoid the influence of external environmental factors on image acquisition, and the image acquisition device can select different sensor and light source position settings according to needs to achieve better image acquisition effects.
  • the collected original image can be processed, such as matting, correction, alignment, etc., wherein the alignment is performed based on the marked pixels of the image, and the image alignment is corrected according to the error of the marked pixels, so as to obtain
  • the target image corresponding to the accurate location of the item improves the accuracy of item recognition.
  • the recognition result is more accurate.
  • the item identification result can also be reminded through the reminding device, so that the item management personnel can know and supervise in time.
  • the method and equipment of the embodiments of the present disclosure reduce system misjudgments, minimize the impact of environmental factors and human factors, save costs, and realize intelligent management of items.
  • the present disclosure also proposes a non-temporary computer storage medium on which a computer program is stored, and the computer program can realize the item management method of the above embodiment when the computer program is executed.
  • the present disclosure also proposes a terminal device.
  • the terminal device 14 in an embodiment of the present disclosure includes at least one processor 141 and a memory 142 communicatively connected to the at least one processor 141.
  • the instruction executed by the at least one processor 141 when the instruction is executed by the at least one processor 141, causes the at least one processor 141 to execute the item management method of the above embodiment.
  • the terminal device 14 may be a fixed computer device, image processing device, etc., or a mobile smart terminal such as a mobile phone, a tablet computer, etc., in some embodiments, the terminal device 14 may also be a remote computer. Or cloud server, there is no specific restriction here.
  • FIG. 24 is a block diagram of a terminal device according to an embodiment of the present disclosure, which may further include a communication interface (Communication Interface) 143 and a bus 144 .
  • the processor 141, the memory 142, and the communication interface 143 can communicate with each other through the bus 144.
  • the communication interface 143 can be used for information transmission.
  • the processor 141 may call the logic instructions in the memory 142 to execute the item management method of the foregoing embodiment.
  • the aforementioned logic instructions in the memory 142 can be implemented in the form of a software functional unit and when sold or used as an independent product, they can be stored in a computer readable storage medium.
  • the memory 142 can be used to store software programs and computer-executable programs, such as program instructions/modules corresponding to the methods in the embodiments of the present disclosure.
  • the processor 141 executes functional applications and data processing by running software programs, instructions, and modules stored in the memory 142, that is, implements the deep-belief network-based medical image classification method in the foregoing method embodiment.
  • the memory 142 may include a program storage area and a data storage area.
  • the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal device, and the like.
  • the memory 142 may include a high-speed random access memory, and may also include a non-volatile memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)

Abstract

一种物品管理方法、终端装置和物品管理设备（1）及存储介质，其中，物品存放在储物单元（12）中，物品管理方法包括：获取所述储物单元中物品的原始图像（S1）；对物品的原始图像进行处理，以获得物品的目标图像（S2）；根据物品的目标图像识别物品状态以及物品的类别（S3）。

Description

物品管理方法、终端装置和物品管理设备及存储介质 技术领域
本公开涉及物品管理技术领域，具体而言，涉及一种物品管理方法，以及非临时性计算机存储介质、终端装置和物品管理设备。
背景技术
对于物品的管理，包括识别物品的种类是否正确，以及识别物品的状态，例如是否归还或借出等。目前采用的物品管理技术主要包括RFID(Radio Frequency Identification，射频识别)技术、设置内衬传感器、线扫相机技术等。
其中，RFID技术通过给物品贴上电子标签，并使用RFID射频技术扫描电子标签来识别物品的状态。但该技术存在一些缺陷，例如电子标签不易黏贴到物品上、贴上电子标签后物品的使用受到影响、由于无线电波的信号延迟造成误识别等。对于设置内衬传感器例如重量传感器的方法，通过给物品内衬放置传感器来识别当前物品的状态。但该方法也存在一些缺陷，例如识别错误率高、制造成本高等。对于线扫相机技术，在储物装置的上方设置至少一组线扫相机，通过设置在储物装置前檐的一个潜望式反光镜将光线转折90度投射到相机，抽屉中被照亮区域就是扫描的区域。线扫相机需要配合运动反馈传感器探测抽屉的运动速度和运动位置，才能拼合成一幅完整的图像，该方案成本极高。
发明内容
本公开旨在至少解决现有技术中存在的技术问题之一。
为此，本公开实施例提出一种物品管理方法，该方法识别物品的正确率高、成本低。
本公开实施例还提出一种非临时性计算机存储介质。
本公开实施例还提出一种终端装置。
本公开实施例还提出一种物品管理设备。
为了解决上述问题，本公开实施例提出的物品管理方法，其中，物品存放在储物单元中，所述物品管理方法包括：获取所述储物单元中物品的原始图像；对所述物品的原始图像进行处理，以获得所述物品的目标图像；根据所述物品的目标图像识别物品状态以及所述物品的类别。
根据本公开实施例的物品管理方法，通过获取储物单元中物品的图像，并进行图像处理，识别物品状态和类别，即识别物品是否存放以及存放的物品的类型，进而可以确定物品是否归还或是否放错，从而达到管理物品的目的，方案部署简单，成本低。
在一些实施例中，所述获取所述储物单元中物品的原始图像包括：检测到所述储物单元达到预设状态，采集所述储物单元中物品的图像以作为所述原始图像。
在一些实施例中，对所述物品的原始图像进行处理，以获得所述物品的目标图像，包括：对所述物品的原始图像进行抠图处理，以获得所述物品的目标图像。
在一些实施例中，在进行抠图处理前，对所述物品的原始图像进行处理还包括：获取所述物品的原始图像中的关键帧图像；以及，对所述关键帧图像进行筛选，从而可以提高图像识别的准确性。
在一些实施例中，在进行抠图处理之前，对所述物品的原始图像进行处理还包括：对所述物品的原始图像或筛选出的图像进行矫正，以获得矫正图像；将所述矫正图像与所述储物单元存放物品的模板图像进行对齐处理，以使所述矫正图像与所述模板图像的像素点对齐。从而，可以提高图像识别的准确性。
在一些实施例中，所述获取所述物品的原始图像中的关键帧图像包括：根据所述物品的原始图像确定所述储物单元的边长长度信息；根据所述储物单元的边长长度信息，确定在关闭过程中所述储物单元由静态转为动态时边长长度最大的图像，以作为所述关键帧图像。
在一些实施例中，所述对所述关键帧图像进行筛选，包括：采用图像筛选算法选取所述关键帧图像中图像完整、无遮挡的图像，以作为筛选出的图像，提高图像识别的效率和准确性。
在一些实施例中，所述对所述物品的原始图像或筛选出的图像进行矫正，以获得矫正图像，包括：采用透视变换算法和仿射变换算法中的至少一种方法对所述物品的原始图像或筛选出的图像进行矫正，以获得所述矫正图像。
在一些实施例中，将所述矫正图像与所述储物单元存放物品的模板图像进行对齐处理，以使所述矫正图像与所述模板图像的像素点对齐，包括：识别所述矫正图像中的标记像素点；将所述矫正图像中的标记像素点与模板图像的标记像素点进行对齐，以使得所述矫正图像与所述模板图像的像素点对齐，以提高图像识别准确性。
在一些实施例中，进行抠图处理包括：获取所述模板图像中物品轮廓的坐标值；根据所述模板图像中物品轮廓的坐标值对所述矫正图像进行抠图处理。
在一些实施例中，所述进行抠图处理还包括：识别所述矫正图像中的标记像素点和所述 模板图像中的标记像素点；计算所述矫正图像中的标记像素点与所述模板图像的标记像素点的坐标差值；根据所述坐标差值对所述模板图像中物品轮廓的坐标值进行修正，以进行抠图处理，提高图像识别准确性。
在一些实施例中，所述根据所述物品的目标图像识别物品状态以及所述物品的类别包括：采用人工智能算法对所述物品的目标图像进行识别，以判断存放物品是否存在以及识别所述物品的类别。
在一些实施例中，所述物品管理方法还包括：根据所述物品状态和所述物品的类别进行提示。从而，物品管理人员可以及时知晓或者进行查询。
在一些实施例中，根据所述物品状态和所述物品的类别进行提示包括以下至少一项：
通过显示装置对所述物品状态和所述物品的类别进行显示；
根据所述物品状态和所述物品的类别确定物品缺失时，进行通知；
将所述物品状态和所述物品的类型记录在系统日志中。
本公开实施例提出的非临时性计算机存储介质，其上存储有计算机程序，所述计算机程序被执行时实现所述的物品管理方法。
本公开实施例提出的终端装置包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行时，使所述至少一个处理器执行所述的物品管理方法。
根据本公开实施例的终端装置，通过处理器执行所述的物品管理方法，可以实现对物品的智能化管理，方案容易实现，成本低。
本公开实施例的物品管理设备包括：柜体和存放物品的储物单元；图像采集装置，用于采集存放在所述储物单元的物品的图像；所述的终端装置。
根据本公开实施例的物品管理设备，通过图像采集装置采集储物单元中物品的图像，终端装置进行图像处理并识别物品状态和类别，即识别物品是否存放以及存放的物品的类型，进而可以确定物品是否归还或是否放错，从而达到管理物品的目的，方案部署简单，成本低。
在一些实施例中，所述物品管理设备还包括通信装置，所述通信装置用于将图像数据传输给所述终端装置，可以实现远程物品管理，以及可以减少本地数据处理量，提高数据处理效率。
在一些实施例中，所述储物单元可开合地设置在所述柜体；所述图像采集装置用于采集所述储物单元达到预设状态时所述储物单元中物品的图像。
在一些实施例中，所述图像采集装置设置在所述柜体且位于所述储物单元的上方。
在一些实施例中，所述图像采集装置设置在所述柜体，所述储物单元可脱离所述柜体移动以及在物品存放后归位至所述柜体。
在一些实施例中，所述物品管理设备还包括：检测装置，设置在所述柜体，用于检测所述储物单元是否到达预设状态。
在一些实施例中，所述物品管理设备还包括提示装置，提示装置用于根据所述终端装置的反馈信息进行提示，从而物品管理人员可以及时知晓或者便于进行查询。
在一些实施例中，所述图像采集装置包括：至少一个传感器，设置在所述柜体且在所述储物单元达到预设状态时位于所述储物单元的上方；至少一个光源，用于发射光线以在所述储物单元表面形成面光。从而，可以提高采集图像的清晰度和图像质量。
在一些实施例中，所述物品管理设备还包括用于为所述图像采集装置供电的供电装置。
在一些实施例中，所述供电装置包括电线电源或电池。
在一些实施例中，所述电池设置在所述图像采集装置。
在一些实施例中，所述电池设置在所述柜体或所述储物单元。
在一些实施例中，所述供电装置还用于为所述终端装置或通信装置供电。
本公开的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本公开的实践了解到。
附图说明
本公开的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1是根据本公开的一个实施例的物品管理设备的框图；
图2是根据本公开的另一个实施例的物品管理设备的框图；
图3是根据本公开的一个实施例的物品管理方法的流程图；
图4是根据本公开的一个实施例的物品管理设备的示意图；
图5是根据本公开的另一个实施例的物品管理设备的示意图；
图6是根据本公开的一个实施例的柜体和储物单元的示意图；
图7是根据本公开的一个实施例的图像采集装置设置的示意图；
图8是根据本公开的另一个实施例的图像采集装置设置的示意图；
图9是根据本公开的另一个实施例的图像采集装置设置的示意图；
图10是根据本公开的另一个实施例的图像采集装置设置的示意图；
图11是根据本公开的另一个实施例的图像采集装置设置的示意图；
图12是根据本公开的另一个实施例的图像采集装置设置的示意图；
图13是根据本公开的另一个实施例的图像采集装置设置的示意图；
图14是根据本公开的另一个实施例的图像采集装置设置的示意图；
图15是根据本公开的另一个实施例的图像采集装置设置的示意图；
图16是根据本公开的另一个实施例的图像采集装置设置的示意图；
图17是根据本公开的另一个实施例的图像采集装置设置的示意图；
图18是根据本公开的一个实施例的储物单元及其选取关键帧图像观测的边长的示意图；
图19是根据本公开的一个实施例的储物单元的边长长度随时间变化的变化曲线；
图20是根据本公开的一个实施例的对图像进行矫正变换的示意图；
图21是根据本公开的一个实施例的储物单元上标记点位置的示意图；
图22是根据本公开的一个实施例的识别物品过程的流程图；
图23是根据本公开的一个实施例的终端装置的框图；
图24是根据本公开的另一个实施例的终端装置的框图。
具体实施方式
下面详细描述本公开的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本公开，而不能理解为对本公开的限制。
在本公开的描述中，需要理解的是，术语“中心”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开的限制。
需要说明的是，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。进一步地，在本公开的描述中，除非另有说明，“多个”的含义是两个或两个以上。
下面对本公开实施例的物品管理方法以及物品管理设备、存储介质和终端装置分别进行说明。
在一些场所例如医院、工厂、机场、铁路、船坞、警署、武装部队驻地等，人员使用的物品通常需要存放到规定的储物柜、储物箱或仓库中，例如医务人员使用的检查工具或手术工具、飞机维修人员使用的工具、武装部队人员使用的枪械或用品、工厂工人使用的工具等都需要在使用完毕后归还到对应的位置，以避免物品去向不明或存放错误，因此，对物品进行管理是必要的。
本公开提出了物品管理方法和物品管理设备，其中，如图1所示为根据本公开的一个实施例的物品管理设备的功能框图，物品管理设备1包括柜体11、存放物品的储物单元12、图像采集装置13和终端装置14。其中，图像采集装置13用于采集存放在储物单元12的物品的图像以作为物品的原始图像。终端装置14根据储物单元中物品的原始图像实现对物品的管理。在实施例中，终端装置14可以是设置在储物单元12所在场所的计算机设备、图像处理设备等本地设备，也可以是移动智能终端例如手机、平板电脑等，也可以是远程计算机或者云端服务器等。如图2所示，物品管理设备1还可以包括通信装置15，通信装置15将图像数据传输给终端装置14例如远程计算机或者云端服务器，进而由远程计算机或云端服务器来基于图像数据实现物品管理。
图3是根据本公开的一个实施例的物品管理方法的流程图，如图3所示，本公开实施例的物品管理方法至少包括步骤S1-S3，每个步骤的说明如下。
步骤S1，获取储物单元中物品的原始图像。
具体地，可以通过图像采集装置采集储物单元中物品的图像，并将图像数据传输给终端装置，例如本地计算机设备，或者，将图像数据通过通信装置传送给终端装置，例如移动智能终端或者云端服务器，终端装置接收到该图像数据，并将该图像作为储物单元中物品的原始图像。
步骤S2，对物品的原始图像进行处理，以获得物品的目标图像。
具体地，终端装置接收到储物单元中物品的原始图像后，对原始图像进行处理，例如，选取关键帧图像、对图像进行矫正、对齐图像等至少一项动作，以及在图像中进行抠图，以获得可用于识别物品状态和类别的图像，并以此作为目标图像。
步骤S3，根据物品的目标图像识别物品状态以及物品的类别。
具体地，物品状态可以理解为物品是否归还、是否齐全等，物品的类别包括物品是什么、属于用户自定义的哪个种类等。终端装置在获得物品的目标图像后，对目标图像进行图像 识别，以识别图像中是否存在应存放的物品、实际存放的物品是什么、特定位置存放的是什么等，从而判定物品是否归还、归还的物品是否正确、存放位置是否正确等，从而实现对物品的管理。
根据本公开实施例的物品管理方法，通过获取储物单元中物品的图像，并进行图像处理，识别物品状态和类别，即识别物品是否存放以及存放的物品的类型，进而可以确定物品是否归还或是否放错，从而达到管理物品的效果，方案部署简单，成本低。
进一步地，在实施例中，储物单元12可以与柜体11配合设置，也可以脱离柜体11独立设置。图像采集装置13可以采用外置的方式，也可以采用内置的方式。例如，图像采集装置13可以设置在柜体11并位于储物单元12的上方，以便于采集储物单元12中的物品的图像，或者，图像采集装置13设置在柜体11外侧的支撑架上，或者，图像采集装置13设置在储物单元12所在场所的某个位置，或者，图像采集装置13设置在可采集到存储单元12中物品图像的其它位置。
在一些实施例中，储物单元12可开合地设置在柜体11，即储物单元12和柜体11一体设置，例如，如图4所示为根据本公开的一个实施例的物品管理设备的示意图，其中，柜体11设置多层，储物单元12包括多个抽屉，多个抽屉分别可开合地设置在柜体11，存放物品时打开抽屉，在存放完成后可以合上抽屉，图像采集装置13用于采集储物单元12达到预设状态时储物单元12中物品的图像，以作为图像处理的原始图像。其中，预设状态可以是储物单元12关闭过程中的某个状态，例如储物单元12完全关闭或者即将完全关闭或者关闭过程中存在停顿时的状态，也可以是由传感器触发的信号。其中，图像采集装置13可以与储物单元12和柜体11一体设置，并采用内置方式，例如，图像采集装置13设置在柜体11并位于储物单元12的上方。在储物单元12达到预设状态时，图像采集装置13采集储物单元12中物品的图像，以作为原始图像；或者，图像采集装置13与储物单元12和柜体11一体设置，并采用外置方式，例如，图像采集装置13设置在柜体11外侧的支撑架上。储物单元12达到预设状态时，图像采集装置13采集储物单元12中物品的图像；或者，图像采集装置13也可以独立设置，例如设置在柜体11附近的某个位置，具体例如图像采集装置13设置在柜体11附近的墙壁上。在存放物品后，将储物单元12和柜体11移动至图像采集装置13处，在储物单元12到达预设状态时，图像采集装置13即可采集储物单元12中物品的图像，以作为原始图像。
在另一些实施例中，图像采集装置13设置在柜体11，储物单元12可脱离柜体11移动以及在物品存放后归位至柜体11，即图像采集装置13与柜体11一体设置，储物单元12独 立设置。例如，图5所示为根据本公开的一个实施例的物品管理设备的示意图，其中，储物单元12为可自由移动的储物车，储物车包括至少一个托盘，以存放物品，终端装置14设置在柜体11上；对于多个储物单元12，柜体11可以设置多个柜门，设置柜门可以减少外界环境因素对图像采集的干扰。在本实施例中，图像采集装置13可以采用内置，例如设置在柜体11内部，在储物车归位后，图像采集装置13位于储物车的上方，即可采集到储物车中物品图像，具体地，需要存放物品时，打开柜门，将储物车拉出，以及在存放物品后，将储物车归位至对应的柜体11后，图像采集装置13可以采集储物车上物品的图像，以作为抽屉中物品的原始图像。该方式受外界环境干扰少，可以获得更加高质量的图像。或者，图像采集装置13也可以采用外置，例如设置在柜体11外部的支撑架上，在储物车存放物品后，将储物车归位至柜体11的过程中，图像采集装置13可以记录储物车中物品的视频或者连续采集多张连拍图像。
进一步地，在实施例中，为了获得更好的原始图像信息，如图2所示，图像采集装置13可以包括至少一个传感器131和至少一个光源132，至少一个传感器131设置在柜体11且在储物单元12达到预设状态时位于储物单元12的上方，至少一个光源132用于发射光线以在储物单元12表面形成面光。
其中，传感器131和光源132的数量和设置位置可选，需保证在储物单元12表面形成面光。例如，图6为根据本公开的一个实施例的柜体11和储物单元12的示意图，储物单元12包括多个抽屉，在每个抽屉的上方可以设置图像采集装置13，例如图像采集装置13设置在抽屉的上方的抽屉底板133上并朝向该抽屉，最上方的抽屉对应的图像采集装置13可以设置在柜体11的内顶部，以通过图像采集装置13采集对应抽屉中物品的图像。
图7-图17分别示出了不同数量和设置位置的传感器和光源的示意图，具体地，如图7所示，图像采集装置13可以采用一个传感器131和一个光源132，光源132和传感器131均设置在抽屉底板133的最长边的边缘，使得传感器131可以获取足够大的视野，光源132可以覆盖足够大的面积。或者，如图8所示，图像采集装置13采用一个传感器131和两个光源132，传感器131设置在抽屉底板133最长边的边缘，两个光源132设置在抽屉底板133的两个对角的位置，保证光源132的强度与覆盖面。或者，如图9所示，图像采集装置13可以采用一个传感器131和四个光源132，传感器131设置于抽屉底板133的中心位置，光源132设置在抽屉底板133的四个对角位置，以保证充足的光线。或者，如图10所示，图像采集装置13可以采用一个传感器131和6个光源132，其中，传感器131设置于抽屉底板133的中心位置，4个光源132设置在抽屉底板133的四个对角位置，另外两个光源 132设置在抽屉底板133的长边的中间位置，以保证充足的光线，获得更清晰的图像。或者，如图11所示，图像采集装置13可以采用一个传感器131和8个光源132，传感器131设置于抽屉底板133的中心位置，其中，4个光源132设置于抽屉底板133的四个对角位置，另外四个光源132设置在抽屉底板133的各边的中间位置，以保证充足的光线，以获得更加清晰的图像。或者，如图12中(a)和(b)所示，图像采集装置13可以采用两个传感器131和两个光源132，两个传感器131沿抽屉底板133的长边的中轴线相隔预设距离设置，两个光源132设置在抽屉底板133的对角位置，既可以拍摄更多角度的图像又可以保证充足光线，获得更好的图像信息。或者，如图13所示，图像采集装置13可以采用两个传感器131和四个光源132，两个传感器131沿抽屉底板133的长边的中轴线相隔预设距离设置，四个光源132设置在抽屉底板133的四个对角位置。或者，如图14所示，图像采集装置13可以采用两个传感器131和六个光源132，两个传感器131沿抽屉底板133的长边的中轴线相隔预设距离设置，其中，4个光源132设置在抽屉底板133的四个对角位置，另外2个光源132设置在抽屉底板133的两个长边的中间位置。或者，如图15所示，图像采集装置13可以采用两个传感器131和六个光源132，两个传感器131沿抽屉底板133的长边的中轴线相隔预设距离设置，4个光源131设置在抽屉底板133的四个对角位置，另外4个光源132设置在抽屉底板133的各边的中间位置。或者，如图16所示，图像采集装置13采用一个传感器131和一个带状光源132，传感器131设置在抽屉底板133的中心位置，带状光源132贴合抽屉底板133的四周边沿。或者，如图17所示，图像采集装置13可以采用两个传感器131和一个带状光源132，两个传感器131沿抽屉底板133的长边的中轴线相隔预设距离设置，带状光源132贴合抽屉底板133的四周边沿。
通过采用上面实施例的传感器131和光源132的设置方式，可以保证可见/非可见光源132的稳定性，不受外界条件的影响；可以降低在操作过程中如物理撞击、震动等对传感器131的影响，极大提高了传感器131获取图像信息的精确度。
图像采集装置13工作需要供电，如图2所示，物品管理设备1还包括用于为图像采集装置13供电的供电装置16。在实施例中，供电装置16可以为电线电源或者电池。
例如，供电装置16为电线电源，通过线束连接至图像采集装置13，启动物品管理设备1，电线电源即可为图像采集装置13供电，在启动图像采集时，图像采集装置13即可采集图像信息。对于柜体11、储物单元12和图像采集装置13一体设置的物品管理设备1，可以采用电线电源为图像采集装置13供电；对于柜体11和储物单元12一体设置而图像采集装置13独立设置的物品管理设备1，也可以采用电线电源为图像采集装置13供电；或者， 对于柜体11和图像采集装置13一体设置而储物单元12可脱离柜体11的物品管理设备1，也可以采用电线电源来为图像采集装置13供电。
再例如，供电装置16为电池，由电池为图像采集装置13供电。在一些实施例中，电池设置在图像采集装置13，直接为图像采集装置13供电。具体地，可以通过软件控制，不启动物品管理设备1时，图像采集装置13处于关闭状态，电池不输出电能，在启动物品管理设备1时，电池为图像采集装置13供电以唤醒图像采集装置13，更加节省电能，其中，可以通过按键或其他方式来启动或关闭物品管理设备1；在另一些实施例中，电池设置在柜体11或储物单元12，在启动物品管理设备1时，电池与图像采集装置13连接，为图像采集装置13供电。例如，对于柜体11、储物单元12和图像采集装置13一体设置的物品管理设备1，将电池放置在柜体11或储物单元12，在进行物品管理时，启动物品管理设备1，电池与图像采集装置13连接，来为图像采集装置13供电，启动图像采集时，图像采集装置13即可采集图像信息；再例如，对于柜体11和储物单元12一体设置而图像采集装置13独立设置的物品管理设备1，电池可以设置在储物单元12或柜体11，在启动物品管理设备1时，电池与图像采集装置13连接，来为图像采集装置13供电。再例如，对于柜体11和图像采集装置13一体设置而储物单元12可脱离柜体11的物品管理设备1，例如储物单元12为储物车的场景，电池可以设置在储物单元12或柜体11，在启动物品管理设备1时，电池与图像采集装置13连接，来为图像采集装置13供电。
在实施例中，对于终端装置14采用本地设备的场景，供电装置16还可以为终端装置14供电，在启动物品管理设备1时，供电装置16分别为图像采集装置13和终端装置14供电，供电装置16可以为一个也可以包括多个。在另一些实施例中，对于终端装置14采用移动终端或远程设备或云端设备的场景，供电装置16还可以为通信装置15供电，在启动物品管理设备1时，供电装置16分别为通信装置15和图像采集装置13供电，供电装置16可以为一个也可以包括多个。
下面对采集原始图像以及对原始图像进行处理以获得目标图像、根据目标图像实现物品管理的过程进一步说明。
具体地，对于外置的图像采集装置，在采集图像时，调用图像采集装置例如摄像头，可以对存放物品的全过程进行拍摄，获得物品存放过程的视频，或者，拍摄多张连拍图像，并将视频数据或连拍图像数据传送给终端装置，来作为储物单元中物品的原始图像。
或者，对于内置的图像采集装置，可以根据传感器的触发信号调用图像采集装置拍摄至少一次图像，例如图2所示，物品管理设备1还包括检测装置17，检测装置17设置在柜体 11，用于检测储物单元12是否到达预设状态。
举例说明，可以在柜体上配合储物单元开合的位置相对设置激光收发器，当储物单元例如抽屉关闭时，如果抽屉挡住激光发射器发射的激光，激光接收器将接收不到激光，则认为此时储物单元到达预设状态，触发图像采集装置采集储物单元中的图像以作为原始图像。其中，传感器除了可以采用上面实施例的激光收发器外，也可以采用例如位置传感器、运动传感器、距离传感器、接触传感器等，根据储物单元的位置信息、运行状态、距离变化或者动作信息判断其是否到达预设状态。或者，可以通过机电控制，来实现触发图像采集装置采集图像信息，类似于冰箱开门即触发照明灯照明或宾馆衣橱厨门打开即亮灯的触发设置。
进一步地，在获得储物单元中物品的原始图像后，对原始图像进行处理，例如对物品的原始图像进行抠图，以获得物品的目标图像。具体地，终端装置预存有储物单元中存放正确物品时采集的图像，以作为模板图像。在进行抠图时，获取模板图像中物品轮廓的坐标值，根据模板图像中物品轮廓的坐标值对原始图像进行抠图，从而获得原始图像中对应正确物品位置的图像，即获得目标图像。进而对目标图像的物品进行识别，判断对应物品存放位置是否存在正确的对应物品以及物品的类别是否正确，从而实现对物品的管理。
由于采用某些图像采集装置，如鱼眼相机，获得的原始图像通常存在畸变，因而，在本公开实施例中，可以对原始图像进行矫正、对齐以及抠图等处理；或者，对于外置的图像采集装置，通过采集视频流或多张连拍图像来获得原始图像，如果将每帧图像都作为原始图像，将造成数据处理量非常大，效率低。以及在实际操作中，也会存在图像采集视野被遮挡的情况，从而造成原始图像存在缺陷。在本公开实施例中，可以对原始图像进行筛选、矫正、对齐以及抠图等处理，以提高图像识别的准确性，下面对每个处理步骤分别进行说明。
在进行抠图之前，可以先进行关键帧图像的筛选，具体地，获取物品的原始图像中的关键帧图像，以及对关键帧图像进行筛选。
其中，对于内置的图像采集装置，通过检测装置检测到储物单元到达预设状态时，触发图像采集装置采集储物单元中物品的图像，该图像即可作为原始图像，以及，该原始图像即为关键帧图像，并且在此情况下，拍摄视野不会受到外界的影响，因此，此关键帧图像即为筛选出的图像。
对于外置的图像采集装置，图像采集装置可以录制物品存放过程中储物单元中物品的视频或采集多张连拍图像以获得原始图像，可以根据物品的原始图像确定储物单元的边长长 度信息，根据储物单元的边长长度信息，确定在关闭过程中储物单元由静态转为动态时边长长度最大的图像，以作为关键帧图像。其中，如图18所示，标示出了储物单元例如抽屉的观测边长，在抽屉开合时此观测边长的长度变化，可以通过该观测边长的长度信息衡量抽屉被拉开的程度。具体地，根据采集的每帧图像的时间戳或者视频的时间信息以及储物单元的边长的长度信息，生成如图19所示的储物单元边长长度随时间变化的变化曲线，其中，储物单元处于静态时如图19中观测边长的长度不随时间变化的阶段，对应关闭抽屉时的停顿状态，此时往往会存在物品取出或放入的动作，所以，可以将储物单元由静态转为动态时边长长度最大的图像，例如图19所示的曲线跳变时刻采集的图像，作为关键帧图像。
进一步地，对于图像采集装置外置的方案，图像拍摄视野容易被遮挡，所以可以对关键帧图像进行筛选，具体地，采用图像筛选算法选取关键帧图像中图像完整、无遮挡的图像，以作为筛选出的图像，即筛选出最完整、无异物例如无手遮挡且光照正常的最利于识别的图像。
进一步，对于图像采集装置内置或外置的方案，图像采集装置采集的图像通常都会存在畸变，所以可以进一步对图像进行矫正和对齐。具体地，对物品的原始图像或筛选出的图像进行矫正，以获得矫正图像；将矫正图像与储物单元存放物品的模板图像进行对齐处理，以使矫正图像与模板图像的像素点对齐。
在实施例中，采用透视变换算法和仿射变换算法中的至少一种方法对物品的原始图像或筛选出的图像进行矫正，以获得矫正图像。其中，仿射变换和透视变换更直观的叫法可以为“平面变换”和“空间变换”或者“二维坐标变换”和“三维坐标变换”。仿射变换是二维坐标(x,y)，透视变换是三维坐标(x,y,z)，具体可以通过矩阵方式实现变换。在实施例中，如图20所示，原始图像为鱼眼相机采集到的鱼眼图像，通过透视变换算法或放射变换算法将原始图像或筛选出的图像例如鱼眼图像转换为梯形图像，进而将梯形图像转换为矩形图像，以获得矫正图像，便于后续的对齐处理。
进一步地，识别矫正图像中的标记像素点；将矫正图像中的标记像素点与模板图像的标记像素点进行对齐，以使得矫正图像与模板图像的像素点对齐。具体地，在储物单元上设置物理标记点，例如，将贴纸或者易识别的颜色点，标记在储物单元上以作为物理标记点。标记位置可以例如图21所示，选择标记在储物单元的两条边的交叉点或者长边的某个位置，并且在物品正确存放时，采集储物单元中物品的图像以作为模板图像。所采集的模板图像中携带有对应物理标记点的标记像素点，同样地，矫正图像中也会存在标记像素点。在进行对齐处理时，识别矫正图像中的标记像素点，并将矫正图像中的标记像素点与模板图像 的标记像素点进行位移，直至矫正图像的标记像素点与模板图像的标记像素点对齐，以使得矫正图像与模板图像的像素点对齐，为进行抠图提供基础。
进而，在矫正图像与模板图像的像素点对齐后，获取模板图像中物品轮廓的坐标值；根据模板图像中物品轮廓的坐标值对矫正图像进行抠图。具体地，可以使用裁剪算法，利用像素信息对目标图像进行坐标定位，将模板图像中物品轮廓的坐标值与矫正图像中的坐标进行对齐，并进行抠图，从而可以获得对应正确存放物品位置的图像，并将抠图获得的图像作为目标图像。
进一步地，矫正图像中物品的坐标与模板图像中物品的坐标值往往会存在误差，在本公开实施例中，可以对误差进行修正，以获得更加准确位置的目标图像。具体地，识别矫正图像中的标记像素点和模板图像中的标记像素点；计算矫正图像中的标记像素点与模板图像的标记像素点的坐标差值；根据坐标差值对模板图像中物品轮廓的坐标值进行修正，并根据修正后的坐标值进行对齐以及抠图，从而可以获得对应物品正确存放位置的目标图像，进而可以提高物品识别的准确性。
在获得目标图像后，对目标图像中的物品进行识别和分类，以识别物品状态和类别。具体地，采用人工智能算法对物品的目标图像进行识别，以判断对应物品存放位置是否存在正确的对应物品以及识别物品的类别。例如可以采用神经网络算法或者深度学习算法或者其它人工智能算法对目标图像进行识别，将目标图像输入人工智能模型，人工智能模型基于人工智能算法输出计算结果，如图22所示，为根据本公开的一个实施例的识别物品过程的流程图，具体包括以下步骤。
S10，基于人工智能算法对目标图像进行识别并将物品分类。
例如采用二分法将输入目标图像分为两种情况，其中一种情况为物品被遮挡且无法判断类别，则执行步骤S20，另一种情况是物品未遮挡或者遮挡很少且可以判断物品类别，则执行步骤S30。
S20，通过二分法判断该物品的状态，输出结果为物品存在或不在。
S30，直接判断物品的类别是否正确，输出结果为物品正确或不正确。
简言之，通过上面实施例进行图像处理和识别，并基于人工智能算法识别储物单元中物品的状态和类别，可以实现对储物单元中物品的管理。
进一步地，如图2所示，本公开实施例的物品管理设备1还包括提示装置18，提示装置18用于根据终端装置14的反馈信息进行提示。具体地，图像采集装置13采集储物单元12中物品的原始图像，并将原始图像传输给终端装置14例如计算机设备或者图像处理设备 或者通过通信装置15传送给云端服务器或者移动智能终端，终端装置14按照上面实施例的物品管理方法对原始图像进行处理和识别，识别出物品的状态和类别，并将物品的状态和类别信息下发给提示装置18，提示装置18进行提示，以使物品管理人员可以及时知晓物品存放情况例如物品是否归还或者存放物品类别、位置是否正确等。
在实施例中，提示装置18可以通过多种形式实现，例如可以采用以下的至少一项进行提示：通过显示装置对物品状态和物品的类别进行显示；根据物品状态和物品的类别确定物品缺失时，进行通知，例如弹出警示框或提示框或者通过声音提示；将物品状态和物品的类型记录在系统日志中，物品管理人员可以通过查询系统日志知晓物品存放情况。
概括来说，本公开实施例的物品管理方法和物品管理设备，基于图像识别来判断储物单元中物品的状态和类别，容易实现，无需配合其它更多的传感器设备，成本低，其中，图像采集装置可以采用内置和外置，以静态方式采集图像，相较于线扫相机技术的动态采集，成本更低。以及，采用内置的图像采集装置可以避免外界环境因素对图像采集的影响，并且图像采集装置可以根据需要选择不同的传感器和光源位置设置，以达到更好的图像采集效果。以及，通过终端装置可以对采集的原始图像进行处理例如抠图、矫正、对齐等处理，其中，基于图像的标注像素点进行对齐处理，并根据标注像素点的误差对图像对齐进行修正，可以获得对应物品准确位置的目标图像，提高物品识别的准确性。以及，基于人工智能算法来对目标图像中的物品状态和类别进行识别，识别结果更加准确。以及，还可以通过提示装置对物品识别结果进行提示，物品管理人员可以及时知晓和监督。本公开实施例的方法和设备，降低了系统误判，最大程度地降低了环境因素以及人为因素的影响，节约成本，实现了对物品的智能化管理。
基于上面实施例的物品管理方法，本公开还提出一种非临时性计算机存储介质，其上存储有计算机程序，该计算机程序被执行时可实现上面实施例的物品管理方法。
本公开还提出了一种终端装置，如图23所示，本公开实施例的终端装置14包括至少一个处理器141和与该至少一个处理器141通信连接的存储器142，存储器142存储有可被至少一个处理器141执行的指令，该指令被至少一个处理器141执行时，使至少一个处理器141执行上面实施例的物品管理方法。在实施例中，终端装置14可以是固定设置的计算机设备、图像处理设备等，也可以是可移动的智能终端例如手机、平板电脑等，在一些实施例中，终端装置14也可以是远程计算机或者云端服务器，在此不作具体限制。
例如，以一个处理器(processor)141和存储器(memory)142为例，图24为根据本公开的一个实施例的终端装置的框图，其中，还可以包括通信接口(Communication Interface) 143和总线144。其中，处理器141、存储器142、通信接口143可以通过总线144完成相互间的通信。通信接口143可以用于信息传输。处理器141可以调用存储器142中的逻辑指令，以执行上述实施例的物品管理方法。
此外，上述的存储器142中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。
存储器142作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序，如本公开实施例中的方法对应的程序指令/模块。处理器141通过运行存储在存储器142中的软件程序、指令以及模块，从而执行功能应用以及数据处理，即实现上述方法实施例中的基于深度置信网络的医学图像分类方法。
存储器142可包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端设备的使用所创建的数据等。此外，存储器142可以包括高速随机存取存储器，还可以包括非易失性存储器。
需要说明的是，在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示意性实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本公开的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
尽管已经示出和描述了本公开的实施例，本领域的普通技术人员可以理解：在不脱离本公开的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本公开的范围由权利要求及其等同物限定。

Claims (29)

  1. 一种物品管理方法，其特征在于，物品存放在储物单元，所述物品管理方法包括：
    获取所述储物单元中物品的原始图像；
    对所述物品的原始图像进行处理，以获得所述物品的目标图像；
    根据所述物品的目标图像识别物品状态以及所述物品的类别。
  2. 根据权利要求1所述的物品管理方法，其特征在于，所述获取所述储物单元中物品的原始图像包括：
    检测到所述储物单元达到预设状态，采集所述储物单元中物品的图像以作为所述原始图像。
  3. 根据权利要求1所述的物品管理方法，其特征在于，对所述物品的原始图像进行处理，以获得所述物品的目标图像，包括：
    对所述物品的原始图像进行抠图处理，以获得所述物品的目标图像。
  4. 根据权利要求3所述的物品管理方法，其特征在于，在进行抠图处理前，对所述物品的原始图像进行处理还包括：
    获取所述物品的原始图像中的关键帧图像；以及，
    对所述关键帧图像进行筛选。
  5. 根据权利要求3或4所述的物品管理方法，其特征在于，在进行抠图处理之前，对所述物品的原始图像进行处理还包括：
    对所述物品的原始图像或筛选出的图像进行矫正，以获得矫正图像；
    将所述矫正图像与所述储物单元存放物品的模板图像进行对齐处理，以使所述矫正图像与所述模板图像的像素点对齐。
  6. 根据权利要求4所述的物品管理方法，其特征在于，所述获取所述物品的原始图像中的关键帧图像包括：
    根据所述物品的原始图像确定所述储物单元的边长长度信息；
    根据所述储物单元的边长长度信息，确定在关闭过程中所述储物单元由静态转为动态时边长长度最大的图像，以作为所述关键帧图像。
  7. 根据权利要求4所述的物品管理方法，其特征在于，所述对所述关键帧图像进行筛选，包括：
    采用图像筛选算法选取所述关键帧图像中图像完整、无遮挡的图像，以作为筛选出的 图像。
  8. 根据权利要求5-7任一项所述的物品管理方法，其特征在于，所述对所述物品的原始图像或筛选出的图像进行矫正，以获得矫正图像，包括：
    采用透视变换算法和仿射变换算法中的至少一种方法对所述物品的原始图像或筛选出的图像进行矫正，以获得所述矫正图像。
  9. 根据权利要求5-8任一项所述的物品管理方法，其特征在于，将所述矫正图像与所述储物单元存放物品的模板图像进行对齐处理，以使所述矫正图像与所述模板图像的像素点对齐，包括：
    识别所述矫正图像中的标记像素点；
    将所述矫正图像中的标记像素点与模板图像的标记像素点进行对齐，以使得所述矫正图像与所述模板图像的像素点对齐。
  10. 根据权利要求3-9任一项所述的物品管理方法，其特征在于，进行抠图处理包括：
    获取所述模板图像中物品轮廓的坐标值；
    根据所述模板图像中物品轮廓的坐标值对所述矫正图像进行抠图处理。
  11. 根据权利要求10所述的物品管理方法，其特征在于，所述进行抠图处理还包括：
    识别所述矫正图像中的标记像素点和所述模板图像中的标记像素点；
    计算所述矫正图像中的标记像素点与所述模板图像的标记像素点的坐标差值；
    根据所述坐标差值对所述模板图像中物品轮廓的坐标值进行修正，以进行抠图处理。
  12. 根据权利要求1-11任一项所述的物品管理方法，其特征在于，所述根据所述物品的目标图像识别物品状态以及所述物品的类别包括：
    采用人工智能算法对所述物品的目标图像进行识别，以判断存放物品是否存在以及识别所述物品的类别。
  13. 根据权利要求1-12任一项所述的物品管理方法，其特征在于，所述物品管理方法还包括：根据所述物品状态和所述物品的类别进行提示。
  14. 根据权利要求13所述的物品管理方法，其特征在于，根据所述物品状态和所述物品的类别进行提示包括以下至少一项：
    通过显示装置对所述物品状态和所述物品的类别进行显示；
    根据所述物品状态和所述物品的类别确定物品缺失时，进行通知；
    将所述物品状态和所述物品的类型记录在系统日志中。
  15. 一种非临时性计算机存储介质，其上存储有计算机程序，其特征在于，所述计算 机程序被执行时实现如权利要求1-14任一项所述的物品管理方法。
  16. 一种终端装置，其特征在于，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行时，使所述至少一个处理器执行权利要求1-14任一项所述的物品管理方法。
  17. 一种物品管理设备，其特征在于，包括：
    柜体和存放物品的储物单元；
    图像采集装置，用于采集存放在所述储物单元的物品的图像；
    如权利要求16所述的终端装置。
  18. 根据权利要求17所述的物品管理设备，其特征在于，所述物品管理设备还包括通信装置，所述通信装置用于将图像数据传输给所述终端装置。
  19. 根据权利要求17所述的物品管理设备，其特征在于，
    所述储物单元可开合地设置在所述柜体；
    所述图像采集装置用于采集所述储物单元达到预设状态时所述储物单元中物品的图像。
  20. 根据权利要求19所述的物品管理设备，其特征在于，所述图像采集装置设置在所述柜体且位于所述储物单元的上方。
  21. 根据权利要求17所述的物品管理设备，其特征在于，所述图像采集装置设置在所述柜体，所述储物单元可脱离所述柜体移动以及在物品存放后归位至所述柜体。
  22. 根据权利要求19所述的物品管理设备，其特征在于，所述物品管理设备还包括：
    检测装置，设置在所述柜体，用于检测所述储物单元是否到达预设状态。
  23. 根据权利要求17-22任一项所述的物品管理设备，其特征在于，所述物品管理设备还包括：
    提示装置，用于根据所述终端装置的反馈信息进行提示。
  24. 根据权利要求17所述的物品管理设备，其特征在于，所述图像采集装置包括：
    至少一个传感器，设置在所述柜体且在所述储物单元达到预设状态时位于所述储物单元的上方；
    至少一个光源，用于发射光线以在所述储物单元表面形成面光。
  25. 根据权利要求19-21任一项所述的物品管理设备，其特征在于，所述物品管理设备 还包括用于为所述图像采集装置供电的供电装置。
  26. 根据权利要求25所述的物品管理设备，其特征在于，所述供电装置包括电线电源或电池。
  27. 根据权利要求26所述的物品管理设备，其特征在于，所述电池设置在所述图像采集装置。
  28. 根据权利要求26所述的物品管理设备，其特征在于，所述电池设置在所述柜体或所述储物单元。
  29. 根据权利要求25所述的物品管理设备，其特征在于，所述供电装置还用于为所述终端装置或通信装置供电。
PCT/CN2019/118726 2019-11-15 2019-11-15 物品管理方法、终端装置和物品管理设备及存储介质 WO2021092883A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/118726 WO2021092883A1 (zh) 2019-11-15 2019-11-15 物品管理方法、终端装置和物品管理设备及存储介质
CN201980022416.4A CN112005244A (zh) 2019-11-15 2019-11-15 物品管理方法、终端装置和物品管理设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/118726 WO2021092883A1 (zh) 2019-11-15 2019-11-15 物品管理方法、终端装置和物品管理设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021092883A1 true WO2021092883A1 (zh) 2021-05-20

Family

ID=73461567

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/118726 WO2021092883A1 (zh) 2019-11-15 2019-11-15 物品管理方法、终端装置和物品管理设备及存储介质

Country Status (2)

Country Link
CN (1) CN112005244A (zh)
WO (1) WO2021092883A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113392801A (zh) * 2021-06-30 2021-09-14 深圳市斯博科技有限公司 图像处理方法、系统、设备及存储介质
CN115471769A (zh) * 2022-08-16 2022-12-13 上海航翼高新技术发展研究院有限公司 一种工具柜中工具存在状态的视觉识别方法
CN116612224A (zh) * 2023-07-18 2023-08-18 山东新天地智能工程有限公司 一种数字映射可视化管理系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116129337B (zh) * 2023-04-04 2023-10-20 广东车海洋环保科技有限公司 一种自助洗车机用品智能识别方法、系统、设备及介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012193873A (ja) * 2011-03-15 2012-10-11 Nikon Corp 収納装置
CN104889959A (zh) * 2008-08-08 2015-09-09 实耐宝公司 基于图像的库存控制系统和方法
CN105046280A (zh) * 2015-08-10 2015-11-11 北京小豹科技有限公司 一种衣柜智能管理装置及方法
CN105868772A (zh) * 2016-03-23 2016-08-17 百度在线网络技术（北京）有限公司 图像识别方法及装置
CN108932496A (zh) * 2018-07-03 2018-12-04 北京佳格天地科技有限公司 区域内目标物的数量统计方法及装置
CN109145901A (zh) * 2018-08-14 2019-01-04 腾讯科技（深圳）有限公司 物品识别方法、装置、计算机可读存储介质和计算机设备
CN109271920A (zh) * 2018-09-12 2019-01-25 上海了物网络科技有限公司 支持物品识别的视觉系统及方法与介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0617604D0 (en) * 2006-09-07 2006-10-18 Zeroshift Ltd Inventory control system
CN107463946B (zh) * 2017-07-12 2020-06-23 浙江大学 一种结合模板匹配与深度学习的商品种类检测方法
CN109176435A (zh) * 2018-10-29 2019-01-11 龙铁纵横（北京）轨道交通科技股份有限公司 一种带盘点功能的工具柜

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104889959A (zh) * 2008-08-08 2015-09-09 实耐宝公司 基于图像的库存控制系统和方法
JP2012193873A (ja) * 2011-03-15 2012-10-11 Nikon Corp 収納装置
CN105046280A (zh) * 2015-08-10 2015-11-11 北京小豹科技有限公司 一种衣柜智能管理装置及方法
CN105868772A (zh) * 2016-03-23 2016-08-17 百度在线网络技术（北京）有限公司 图像识别方法及装置
CN108932496A (zh) * 2018-07-03 2018-12-04 北京佳格天地科技有限公司 区域内目标物的数量统计方法及装置
CN109145901A (zh) * 2018-08-14 2019-01-04 腾讯科技（深圳）有限公司 物品识别方法、装置、计算机可读存储介质和计算机设备
CN109271920A (zh) * 2018-09-12 2019-01-25 上海了物网络科技有限公司 支持物品识别的视觉系统及方法与介质

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113392801A (zh) * 2021-06-30 2021-09-14 深圳市斯博科技有限公司 图像处理方法、系统、设备及存储介质
CN115471769A (zh) * 2022-08-16 2022-12-13 上海航翼高新技术发展研究院有限公司 一种工具柜中工具存在状态的视觉识别方法
CN115471769B (zh) * 2022-08-16 2023-04-07 上海航翼高新技术发展研究院有限公司 一种工具柜中工具存在状态的视觉识别方法
CN116612224A (zh) * 2023-07-18 2023-08-18 山东新天地智能工程有限公司 一种数字映射可视化管理系统
CN116612224B (zh) * 2023-07-18 2023-10-13 山东新天地智能工程有限公司 一种数字映射可视化管理系统

Also Published As

Publication number Publication date
CN112005244A (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
WO2021092883A1 (zh) 物品管理方法、终端装置和物品管理设备及存储介质
CN106020227B (zh) 无人机的控制方法、装置
JP6108159B2 (ja) 情報処理システム、情報処理装置およびその制御方法と制御プログラム
US20160021342A1 (en) Device and method for determination
CN102541059B (zh) 可自动移动的设备
US7710455B2 (en) Node management system and node managing program using sensing system
CN103049734A (zh) 一种在公共场所找人的方法及系统
CN109255568A (zh) 一种基于图像识别的智能仓储系统
US20120105630A1 (en) Electronic device and method for recognizing and tracking suspects
US20200041200A1 (en) Artificial intelligence refrigerator
CN210436111U (zh) 一种工具管理柜
US20130075201A1 (en) Elevator control apparatus and method
CN103856762A (zh) 多摄像头智能选择和视频优先级判断系统及选择方法
CN204808523U (zh) 区域监控告警系统
US10713536B1 (en) Camera method with animal recognition function and monitoring camera
US10943155B2 (en) System of recognizing identity of object and method of automatically recognizing identity of object
CN104981820A (zh) 即时识别和定位对象的方法,系统与处理器
US20150300652A1 (en) Systems and methods for cookware detection
CN110929658A (zh) 图像识别方法、图像识别装置及冰箱
EP4207068A1 (en) Target object detection method and apparatus, and electronic device, storage medium and program
JP2006103857A (ja) 物品管理システム，方法及びプログラム並びに記録媒体
CN205523945U (zh) 一种车厢内遗留物检测装置
CN107610260B (zh) 一种基于机器视觉的智能考勤系统及考勤方法
CN108652161A (zh) 一种智能识别旅行箱及其控制方法
WO2023155521A1 (zh) 分拣系统、分拣方法、控制方法及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19952683

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19952683

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.01.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 19952683

Country of ref document: EP

Kind code of ref document: A1