WO2024130584A1 - 储物柜的管理方法及装置 - Google Patents

储物柜的管理方法及装置 Download PDF

Info

Publication number
WO2024130584A1
WO2024130584A1 PCT/CN2022/140637 CN2022140637W WO2024130584A1 WO 2024130584 A1 WO2024130584 A1 WO 2024130584A1 CN 2022140637 W CN2022140637 W CN 2022140637W WO 2024130584 A1 WO2024130584 A1 WO 2024130584A1
Authority
WO
WIPO (PCT)
Prior art keywords
locker
image data
foreign object
feature information
time
Prior art date
Application number
PCT/CN2022/140637
Other languages
English (en)
French (fr)
Inventor
陈明轩
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to PCT/CN2022/140637 priority Critical patent/WO2024130584A1/zh
Publication of WO2024130584A1 publication Critical patent/WO2024130584A1/zh

Links

Images

Definitions

  • the present disclosure relates to the field of image processing technology, and in particular to a locker management method and device.
  • the smart container will sound an alarm.
  • the smart container will sound frequent alarms, false alarms, or missed alarms. For example, if the smart container misjudges an item that belongs in the container as a foreign object, a false alarm will occur, or if the smart container misjudges a foreign object as an item that belongs in the smart container, a missed alarm will occur. These situations will cause inconvenience to the staff.
  • a locker management method includes: in response to opening the locker, obtaining multiple frames of image data inside the locker at the current time.
  • the locker is used to store items. If the time interval between the current time and the first time is greater than a preset time, and the number of times the locker is opened between the current time and the first time is greater than a first preset number of times, it is determined whether there is a foreign object in the locker based on the multiple frames of image data, and the first time is the time when an alarm message is sent before the current time, and the alarm message is used to prompt the presence of a foreign object in the locker. If there is a foreign object in the locker, and the number of times the presence of a foreign object in the locker is determined between the current time and the first time is greater than a second preset number of times, an alarm message is sent.
  • the above-mentioned "obtaining multiple frames of image data inside the locker at the current time” may specifically include: when the locker door opening angle is within a preset angle range and the angular acceleration of the door is within a preset angular acceleration range, capturing multiple frames of image data inside the locker.
  • the preset angle range is any angle range between 45 degrees and 50 degrees
  • the preset angular acceleration range is any angular acceleration range between 0 and 10 degrees/second squared.
  • the above-mentioned "determining whether there is a foreign object in the locker based on multiple frames of image data” may specifically include: obtaining feature information of an object in each frame of image data in the multiple frames of image data, and determining whether there is a foreign object in the locker based on the feature information of the object in the multiple frames of image data and a preset feature information library; the preset feature information library includes feature information of multiple objects in the locker, and foreign objects refer to objects that do not belong to the locker.
  • the above-mentioned "determining whether there is a foreign object in the locker based on the feature information of the items in the multi-frame image data and the preset feature information library” may specifically include: if there is an item among all the items included in the multi-frame image data whose similarity between the feature information of any item in the preset feature information library is less than a first preset value, and the total number of times the item is identified is greater than or equal to the first preset value, then it is determined that there is a foreign object in the locker; if the similarity between the feature information of all the items included in the multi-frame image data and the feature information of the items in the preset feature information library is greater than or equal to the first preset value, or the total number of times the item is identified is less than the first preset value, then it is determined that there is no foreign object in the locker.
  • the method further includes: when the foreign body feature library includes feature information of at least one foreign body, if it is determined that there is a foreign body in the first image data, and the similarity between the feature information of the foreign body in the first image data and the feature information of the foreign body in the foreign body feature information library is greater than a second preset value, then the number of times the foreign body is identified is increased by a first value to obtain the total number of times the foreign body is identified.
  • the foreign body feature information library includes feature information of foreign bodies identified in image data whose shooting time is before the first image data in multiple frames of image data.
  • the method further includes: if the foreign object feature information library is empty and there is a foreign object in the first image data, writing the feature information of the foreign object in the first image data into the foreign object feature information library.
  • the method further includes: if the time interval between the current time and the first time is less than or equal to a preset time length, or the number of times the locker is opened between the current time and the first time is less than or equal to a first preset number of times, then deleting multiple frames of image data.
  • the number of multiple frames of image data is greater than a first preset number, at least one frame of image data with the earliest shooting time among the multiple frames of image data is deleted, and the number of image data after deletion is less than or equal to the first preset number.
  • the above-mentioned “determining whether there is a foreign object in the locker based on multiple frames of image data” may specifically include: in response to closing the locker, determining whether there is a foreign object in the locker based on multiple frames of image data.
  • the above-mentioned "determining the presence of a foreign object in the locker" may specifically include: if the total number of times the first foreign object is identified is greater than a third preset number of times, then it is determined that there is a foreign object in the locker, and the first foreign object is the foreign object that is identified the most times among the multiple foreign objects.
  • any two frames of image data in the multiple frames of image data correspond to different shooting angles.
  • a locker management device which includes an acquisition unit, a processing unit, and a sending unit.
  • the acquisition unit is configured to: in response to opening the locker, acquire multiple frames of image data inside the locker at the current time, where the locker is used to store items.
  • the processing unit is configured to: if the time interval between the current time and the first time is greater than a preset number of times, and the number of times the locker is opened between the current time and the first time is greater than the first preset number of times, determine whether there is a foreign object in the locker based on multiple frames of image data, the first time is the time when the alarm information is sent before the current time, and the alarm information is used to prompt the presence of foreign objects in the locker.
  • the sending unit is configured to send an alarm message if there is a foreign object in the locker and the number of times the foreign object is determined to be in the locker between the current time and the first time is greater than a second preset number of times.
  • the acquisition unit is specifically configured to capture multiple frames of image data inside the locker when the door opening angle of the locker is within a preset angle range and the angular acceleration of the door is within a preset angular acceleration range.
  • the preset angle range is any angle range between 45 degrees and 50 degrees
  • the preset angular acceleration range is any angular acceleration range between 0 and 10 degrees/second.
  • the processing unit is specifically configured to: obtain feature information of an object in each frame of image data in the multiple frames of image data, and determine whether there is a foreign object in the locker based on the feature information of the object in the multiple frames of image data and a preset feature information library.
  • the preset feature information library includes feature information of multiple objects in the locker, and the foreign object refers to an object that does not belong to the locker.
  • the processing unit is specifically configured as follows: if there is an item among all items included in the multi-frame image data, the similarity between the feature information of which is less than a first preset value with respect to any item in the preset feature information library, and the total number of times the item is identified is greater than or equal to the first preset value, then it is determined that there is a foreign object in the locker, or that a foreign object has invaded the locker; if the similarity between the feature information of all items included in the multi-frame image data and the feature information of the items in the preset feature information library is greater than or equal to the first preset value, or the total number of times the item is identified is less than the first preset value, then it is determined that there is no foreign object in the locker.
  • the processing unit is further configured as follows: when the foreign body feature information library includes feature information of at least one foreign body, if it is determined that there is a foreign body in the first image data, and the similarity between the feature information of the foreign body in the first image data and the feature information of the foreign body in the foreign body feature information library is greater than a second preset value, then the number of times the foreign body is identified is increased by a first value to obtain the total number of times the foreign body is identified.
  • the foreign body feature information library includes feature information of foreign bodies in the image data of the multiple frames of image data whose shooting time is before the first image data.
  • the processing unit is further configured to: if the foreign object feature information library is empty and there is a foreign object in the first image data, write the feature information of the foreign object in the first image data into the foreign object feature information library.
  • the processing unit is further configured to delete multiple frames of image data if the time interval between the current time and the first time is less than or equal to a preset time length, or the number of times the locker is opened between the current time and the first time is less than or equal to a first preset number of times.
  • the processing unit is further configured to: after determining whether there is a foreign object in the locker based on the multiple frames of image data, delete the multiple frames of image data.
  • the processing unit is further configured to: if the number of multiple frames of image data is greater than a first preset number, delete at least one frame of image data with the earliest shooting time among the multiple frames of image data, and the number of image data after deletion is less than or equal to the first preset number.
  • the processing unit is specifically configured to: in response to closing the locker, determine whether there is a foreign object in the locker based on the multiple frames of image data.
  • the processing unit is specifically configured to: if the total number of times the first foreign object is identified is greater than a third preset number of times, it is determined that there is a foreign object in the locker, and the first foreign object is the foreign object that is identified the most times among the multiple foreign objects.
  • any two frames of image data in the multiple frames of image data correspond to different shooting angles.
  • a locker management device comprising a processor and a communication interface.
  • the communication interface is coupled to the processor.
  • the processor is used to run a computer program or instruction to implement the locker management method of the first aspect or any embodiment of the first aspect.
  • a locker comprising the above-mentioned management device and a camera.
  • the management device is connected to the camera.
  • the management device is used to execute the locker management method of the first aspect or any embodiment of the first aspect.
  • the camera is used to capture multiple frames of image data in the locker in response to opening the locker.
  • the locker further comprises an angle sensor, which is arranged at the connection between the locker door and the cabinet body of the locker.
  • the angle sensor is used to detect the opening angle of the locker door.
  • the locker further comprises one or more light strips, which are disposed on a door frame side of the locker.
  • the camera is a fisheye camera.
  • a non-transitory computer-readable storage medium stores computer program instructions, which, when executed on a computer (eg, a locker), cause the computer to execute the locker management method as described in any of the above embodiments.
  • a computer program is provided.
  • the computer program When the computer program is executed on a computer (eg, a locker), the computer program enables the computer to execute the locker management method as described in any one of the above embodiments.
  • FIG1 is a structural diagram of a locker according to some embodiments.
  • FIG2 is a schematic diagram of image data according to some embodiments.
  • FIG3 is a structural diagram of a locker according to some embodiments.
  • FIG4 is a flow chart of a method for managing a locker according to some embodiments.
  • FIG5 is a schematic diagram of updating an image data list according to some embodiments.
  • FIG6 is a flow chart of a method for managing a locker according to some embodiments.
  • FIG7 is a flow chart of a method for managing a locker according to some embodiments.
  • FIG8 is a schematic diagram of rotation information of an object according to some embodiments.
  • FIG9 is a flow chart of a method for managing a locker according to some embodiments.
  • FIG10 is a block diagram of a management device according to some embodiments.
  • FIG. 11 is a block diagram of a management device according to some embodiments.
  • first and second are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.
  • a feature defined as “first” or “second” may explicitly or implicitly include one or more of the features.
  • plural means two or more.
  • the expressions “coupled” and “connected” and their derivatives may be used.
  • the term “connected” may be used to indicate that two or more components are in direct physical or electrical contact with each other.
  • the term “coupled” may be used to indicate that two or more components are in direct physical or electrical contact.
  • the term “coupled” or “communicatively coupled” may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the contents of this document.
  • At least one of A, B, and C has the same meaning as “at least one of A, B, or C” and both include the following combinations of A, B, and C: A only, B only, C only, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B, and C.
  • a and/or B includes the following three combinations: A only, B only, and a combination of A and B.
  • the term “if” is optionally interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context.
  • the phrases “if it is determined that” or “if [a stated condition or event] is detected” are optionally interpreted to mean “upon determining that” or “in response to determining that” or “upon detecting [a stated condition or event]” or “in response to detecting [a stated condition or event],” depending on the context.
  • lockers With the widespread use of smart containers (hereinafter referred to as lockers for unified description), while bringing convenience to people, it also brings some troubles to the staff. For example, items that do not belong to the lockers appear in the lockers, which leads to the problem of abnormal number of items when the staff counts the number of remaining items in the lockers. For example, the sum of the items actually taken out and the items remaining in the locker is inconsistent with the total number of items originally in the locker. For example, there were 10 items in the locker, and 6 items were taken out. Once there is a foreign object in the locker, the number of items remaining in the locker will exceed 4. In this way, the staff needs to look for items that do not belong to the locker (that is, foreign objects), which wastes time and energy.
  • the locker In order to remind the staff that there is a foreign object in the locker, usually, when the locker detects a foreign object, it will output an alarm message to remind the staff that there is a foreign object in the locker.
  • the locker can play the alarm message through voice broadcast, or the locker can also output the alarm message through a text message, or the locker can also display the alarm message on a display screen.
  • lockers may have problems with frequent alarms, false alarms, and missed alarms, causing inconvenience to staff.
  • the locker when the locker is opened frequently, the locker will output an alarm message every time it detects an abnormality. For example, in some time periods, the staff does not need to count the number of remaining items in the locker, and the locker will still output an alarm message, increasing the workload of the staff.
  • the locker when a locker identifies an item that originally belonged to the locker as a foreign object, the locker will also output an alarm message, causing the staff to spend time and energy searching for the foreign object.
  • the locker when the locker identifies a foreign object as belonging to an item in a container, the locker will not output an alarm message, which may cause the number of items remaining in the locker to be inconsistent with the number of items counted, resulting in an error in the counted number.
  • the embodiment of the present application provides a locker management method, when the number of times the locker is opened exceeds a first preset number of times, and the time interval between the current time and the time when the alarm information was last output exceeds a preset time length, it is determined whether there is a foreign object in the locker according to the multi-frame image data in the locker obtained at the current time; if there is a foreign object in the locker, and the number of times the locker is determined to have a foreign object in the locker from the last time the alarm information was output to the current time exceeds a second preset number of times, then the alarm information is output.
  • the locker will only output the alarm information when the number of times the locker is opened is relatively large, the time from the alarm is long, and the number of times the locker is determined to have a foreign object is relatively large. In this way, while reminding the management personnel to clean up the foreign objects in the locker, the locker will not have the problem of frequent alarms.
  • the locker has an abnormal recognition (such as misidentifying the items in the locker as foreign objects), so frequent alarms may occur in the locker, increasing the workload of the management personnel.
  • the locker can perform a comprehensive analysis based on the recognition results of the image data taken when the door is opened multiple times, and determine whether to issue an alarm based on the analysis results. On the one hand, the phenomenon of abnormal recognition is reduced, and the problem of frequent alarms is avoided; on the other hand, the workload of the management personnel is reduced due to the reduction in the number of alarms.
  • the locker when determining whether there is a foreign object in the locker, the locker can detect whether there is a foreign object in the locker based on multi-frame image data of the locker, thereby increasing the accuracy of foreign object identification and reducing the probability of false alarms and missed alarms.
  • Figure 1 is a storage cabinet provided in an embodiment of the present application.
  • the storage cabinet may include a cabinet body and a cabinet door.
  • the cabinet body is connected to the cabinet door.
  • the cabinet is used to place items.
  • a camera A is arranged at the cabinet door. The camera A faces the cabinet. The camera A is used to capture image data inside the cabinet.
  • the camera A can be set at the edge of the cabinet door farthest from the connection between the cabinet door and the cabinet body. In this way, it can be ensured that the camera A can capture the information of the items in the cabinet body.
  • the camera A can also be set at other positions, for example, it can also be set at the edge of the cabinet body.
  • camera A in order to ensure that camera A can capture all items in the locker, camera A can be a fisheye camera. Since the shooting range of a fisheye camera is large, the shooting range of the camera can cover the entire interior of the locker.
  • this is image data captured by a camera A provided in an embodiment of the present application. From the image data in Figure 2, it can be seen that the camera A can capture image data of all items in the locker.
  • light strips can be provided on both sides of the cabinet body of the locker.
  • the locker In response to the opening of the cabinet door, the locker can turn on the light strip, so that the brightness of the locker can be increased, making the image data captured by the camera clearer.
  • the locker can be provided with more cameras, for example, the locker can be provided with camera B and camera C.
  • Camera B and camera C can be used to assist camera A. For example, when camera A fails or the lens of camera A is blocked, image data inside the locker cannot be obtained. At this time, the locker can use camera B and/or camera C to capture image data inside the locker.
  • the locker may also be provided with a camera D.
  • the camera D is provided on the outside of the door and is oriented in the same direction as the opening direction of the locker.
  • the camera D may be used to capture image data of a person in front of the locker.
  • the locker may detect whether the person has the authority to open the locker based on the image data captured by the camera D. If it is determined that the person has the authority to open the locker, the locker may respond to the person's operation of opening the locker and may not input a warning message. If it is determined that the person does not have the authority to open the locker, the locker may output a warning message in response to the person's operation of opening the locker.
  • the locker may also be provided with a management device, which is connected to the camera, for example, via a system bus.
  • the management device may be used to perform image recognition on the image data captured by the camera to determine whether there is a foreign object in the image data.
  • the specific recognition process may refer to the following embodiment.
  • the management device can be a board, a processor (such as a central processing unit (CPU)), etc.
  • a processor such as a central processing unit (CPU)
  • the locker may also be provided with a display screen.
  • the display screen may be provided on the outside of the door. In this way, the locker may output warning information through the display screen.
  • the locker provided in the embodiment of the present application may also be provided with a voice playback device (such as a sound system, a speaker, etc.).
  • a voice playback device such as a sound system, a speaker, etc.
  • the locker can input an alarm message through the voice playback device.
  • the locker can also display other voice messages through the voice playback device, for example, it can also play "Welcome”, "Please close the door”, "Insufficient items, please replenish items in time”, etc.
  • the alarm message and other voice messages can be pre-configured for the locker.
  • the locker provided in the embodiment of the present application may also be provided with one or more sensors.
  • the one or more sensors may be used to detect the state of the door (such as open or closed).
  • the sensor may be provided at the connection side between the door and the cabinet.
  • the one or more sensors may include a distance sensor.
  • the distance sensor detects that the distance between the cabinet door and the cabinet body is greater than a preset distance, it indicates that the locker is in an open state; when the distance sensor detects that the distance between the cabinet door and the cabinet body is less than or equal to the preset distance, it indicates that the locker is in a closed state.
  • the locker may be provided with a counter, which may be used to count the number of times the locker is opened. For example, if the state information of the door detected by the one or more sensors includes: being in an open state in a first time period, and being in a closed state in a second time period adjacent to and after the first time period, the locker may control the counter to increase a first value (such as 1). In this way, subsequent lockers may continue to determine the number of times the locker is opened based on the state information of the door detected by the one or more sensors, and count the number of times the locker is opened through the counter.
  • a counter such as 1
  • the locker can also initialize the counter (such as initializing the value to 0).
  • the locker may also be provided with a timer.
  • the timer may be used for timing.
  • the timer may be used to record each alarm time, each time the door is opened, and the like.
  • the one or more sensors may further include an angle sensor, which may be used to detect an opening angle of the cabinet door.
  • the locker may determine an angular acceleration of the cabinet door opening process according to the opening angles of the cabinet door at multiple times.
  • the opening angle of the door at time T1 is ⁇ 1
  • the opening angle of the door at time T2 after time T1 is ⁇ 2
  • the opening angle of the door at time T3 after time T2 is ⁇ 3
  • the angular velocity ⁇ 2 of the locker door from time T2 to time T3 ( ⁇ 3 - ⁇ 2 )/(T3-T2).
  • the angular acceleration of the locker ( ⁇ 2- ⁇ 1)/(T3-T1).
  • the angular acceleration of the locker door when the angular acceleration of the locker door is too large, it means that the locker door is rotating too fast.
  • the image data captured by the camera may not be clear; when the angular acceleration of the locker door is too small (such as less than 0), it means that the locker door rotates too slowly.
  • the image data that the camera may capture may be the same or similar. Therefore, when the locker detects that the angular acceleration of the door is within the preset angular acceleration range, the camera can be controlled to start shooting, thereby improving the shooting quality of the image data and reducing the energy consumption caused by shooting too many images.
  • the senor can continuously detect the state of the locker. That is, when the locker is in a closed state, the sensor can detect the state of the locker. When the locker is in an open state, the sensor can detect the state of the locker.
  • the lockers may be unmanned vending machines, temperature control equipment (such as refrigerators, freezers, etc.), and of course may also be other containers without limitation.
  • the locker may also be provided with a communication module, which may be connected to the management device, or may be used to send alarm information to a staff member's terminal.
  • the management device may be used to manage the locker. For example, the number of remaining items may be obtained from the locker.
  • the management device may be a server or a computer.
  • the terminal may include a mobile phone, a tablet computer, a personal computer, etc.
  • the following describes the locker management method provided in the embodiment of the present application in conjunction with the locker shown in FIG. 1 .
  • the execution subject of the embodiment of the present application may be a locker, or a device in the locker, such as a chip or a system on a chip of the locker.
  • the following takes the execution subject as an example of a locker to illustrate the locker management method provided in the embodiment of the present application.
  • a locker management method is provided in an embodiment of the present application.
  • the method may include S401 to S403 .
  • S401 In response to opening a locker, obtaining multiple frames of image data inside the locker at the current time.
  • the current time may refer to the time from opening the locker to closing the locker.
  • the locker may determine the state of the locker based on state information detected by a sensor.
  • the locker can capture multiple frames of images inside the locker through the camera.
  • the locker can perform image recognition on the multiple frames of image data captured by the camera to determine whether there is a foreign object in the locker.
  • the specific process can be referred to the subsequent description and will not be repeated here.
  • the locker in order to ensure the quality of image data captured by the camera, when the opening angle of the cabinet door is within a preset angle range and the angular acceleration of the cabinet door is within a preset angular acceleration range, the locker starts capturing image data inside the locker through the camera.
  • the preset angle range and the preset angular acceleration range can be set as needed.
  • the preset angle range can be any range between 45 degrees and 50 degrees.
  • the preset angle range can be 45 degrees to 50 degrees, or 46 degrees to 59 degrees, etc., without limitation.
  • the preset angular acceleration range can be any range between 0 and 10 degrees/second squared.
  • the preset angular acceleration range can be 0 to 10 degrees/second squared, 1 to 5 degrees/second squared, etc., without limitation.
  • the problem that the camera cannot capture all items in the locker when the locker is opened at a small angle is avoided.
  • the camera can start capturing image data. Since the camera can capture all items in the locker when the locker is opened at an angle between 45 and 50 degrees, the availability of image data can be improved. At the same time, the number of times the camera captures can be reduced, thereby reducing the power consumption of the locker.
  • the image data captured by the camera may be blurred, which is not conducive to subsequent image recognition. Therefore, when the angular acceleration of the locker door is within the preset angular acceleration range, the camera captures the image data to ensure the clarity of the image data. For example, when the angular acceleration of the door is between 0 and 10 degrees per second squared, the rotation speed of the locker door will not be too fast or too slow, avoiding blurring of the captured image data and capturing too much repeated image data.
  • S402 If the time interval between the current time and the first time is greater than a preset time, and the number of times the locker is opened between the current time and the first time is greater than a preset number, determine whether there is a foreign object in the locker based on the multiple frames of image data.
  • the first time refers to the time before the current time when the locker sends the alarm information.
  • Foreign objects refer to objects that do not belong in the locker.
  • the preset time and the preset number of times can be set as needed.
  • the preset time can be 1 hour, 2 hours, etc.
  • the preset number of times can be 10 times, 20 times, 30 times, etc., without limitation.
  • the locker can determine the time interval between the current time and the time when the alarm information was last sent according to the time recorded by the timer, and determine the number of times the locker was opened between the current time and the time when the alarm information was last sent according to the number of times the locker was opened counted by the counter.
  • the locker can perform a foreign object identification process.
  • the foreign object identification process may include a process of determining whether a foreign object exists and a process of determining the number of foreign objects. The specific process can be referred to the subsequent description and will not be repeated here.
  • the locker may also be provided with a counting algorithm and a timing algorithm, so that the locker may count the number of times it is opened by the counting algorithm and count the duration by the timing algorithm.
  • the locker may determine whether there is a foreign object in the locker based on multiple frames of image data in response to closing the locker.
  • the locker in order to process the image data in time, can perform image recognition on a frame of image data after acquiring the frame of image data to determine whether there is a foreign object in the frame of image data. In this way, the locker can dynamically recognize the image data to avoid the problem that the locker is opened again before the image data acquired at the current time is recognized, resulting in inaccurate foreign object recognition results.
  • the number of times that a foreign object is determined to exist in the locker can also be described as the number of times that a foreign object invades the locker.
  • the number of times that a foreign object is determined to exist in the locker between the current time and the first time refers to the number of times that a foreign object is determined to exist in the locker during multiple door opening processes between the current time and the first time.
  • the preset number of times can be set as needed, for example, it can be 1/2 of the number of times the door is opened, and of course, it can also be other values without limitation.
  • the preset number of times is 3, and the number of times the locker is opened in the time period between the current time and the first time is 5 times. Among them, there are 3 times that foreign objects are determined to be present in the locker, and an alarm message can be output.
  • the locker when the number of times the locker is opened exceeds a preset number of times and the time interval between the current time and the time when the alarm information was last output exceeds a preset time length, the locker will judge whether there is a foreign object in the locker based on the acquired multi-frame image data. If the number of times the existence of a foreign object in the locker is determined between the current time and the time when the alarm information was last output exceeds a preset number of times, the locker will output the alarm information, thereby reducing the alarm frequency. At the same time, in the embodiment of the present application, the locker detects whether there is a foreign object in the locker based on multi-frame images, thereby improving the accuracy of foreign object detection and reducing the probability of misjudgment.
  • the locker in combination with the preset angle range and the preset angular acceleration range in S401, when the door opening angle rotates from 0 degrees to 45 degrees, the locker starts to capture image data inside the locker through camera A until the door opening angle of the locker reaches 50 degrees. When the locker opening angle exceeds 50 degrees, the locker can control camera A to stop capturing image data inside the locker.
  • the locker can continue to capture the image data inside the locker through camera A.
  • the opening process of the door can refer to the process in which the opening angle of the door is continuously increased.
  • the closing process of the door refers to the process in which the opening angle of the door of the locker is continuously reduced.
  • the locker can determine the state of the door according to the change of the opening angle of the door detected by the angle sensor.
  • the opening angle of the cabinet door at time T3 is ⁇ 3
  • the opening angle at time T4 is ⁇ 4
  • time T4 is after time T3. If ⁇ 3 is greater than ⁇ 4 , it means that the cabinet door is in the process of closing.
  • the locker when the door opening angle decreases from 90 degrees to 50 degrees, the locker can continue to capture image data inside the locker through the camera until the opening angle decreases to 45 degrees. When the door opening angle is less than 45 degrees, the locker controls the camera A to stop capturing image data inside the locker.
  • the locker in order to reduce the time of subsequent image data recognition, in the embodiments of the present application, can continuously update the captured image data during the process of capturing the image data of the locker, so that the number of image data used for foreign object recognition does not exceed a second preset number.
  • the second preset number can be set as needed, for example, it can be any value between 5 and 15.
  • the second preset number can also be related to the performance of the camera, the performance of the processor, and the performance of the image recognition algorithm. For example, the higher the performance of the camera, the larger the value of the second preset number can be. For another example, the better the performance of the processor, the larger the value of the second preset number can be. For another example, the higher the recognition efficiency of the image recognition algorithm, the larger the value of the second preset number can be.
  • the locker may delete part of the image data in the multiple frames of image data so that the amount of image data is less than or equal to the second preset amount.
  • the locker can control the camera to stop capturing image data, or the locker can continue to capture image data through the camera and update the multiple frames of image data.
  • the process of updating the multiple frames of image data may refer to the locker replacing the image data with the earliest shooting time in the multiple frames of image data with the image data with the latest shooting time of the camera.
  • the second preset data is 5.
  • the camera captures 5 frames of image data (respectively, image data 1 to image data 5) from the start of capturing to the first time, and the 5 frames of image data are sorted in the order of the capturing time to obtain an image data list.
  • the arrangement order in the image data list is determined according to the capturing time. For example, in the image data list, image data 1 is the image data with the earliest capturing time among the 5 frames of image data, and image data 5 is the image data with the latest capturing time among the 5 frames of image data.
  • the locker continues to capture image data in the locker through the camera. For example, as shown in FIG5 , after the first time, another frame of image data (such as can be marked as image data 6) is captured, and the locker can delete image data 1 and write image data 6 into the image data list to obtain an updated image data list. Similarly, later, if the locker continues to capture image data through the camera, the locker can continue to update the image data list until the locker controls the camera to stop shooting.
  • another frame of image data (such as can be marked as image data 6) is captured, and the locker can delete image data 1 and write image data 6 into the image data list to obtain an updated image data list.
  • the locker can continue to update the image data list until the locker controls the camera to stop shooting.
  • the angles corresponding to any two frames of image data in the multiple frames of image data are different.
  • the angle difference corresponding to the image data captured adjacently in time may be a preset angle.
  • the preset angle may be set as needed, for example, 1 to 2 degrees.
  • the locker can obtain the shooting angle corresponding to the image data according to the angle sensor. In this way, the locker can obtain the shooting angle corresponding to each frame of the multiple frames of image data.
  • the locker When the locker continues to shoot with the camera, it can compare whether the shooting angle corresponding to the captured image data is the same as the shooting angle of the multiple frames of image data. If there is image data with the same shooting angle, the locker deletes the image data or replaces the corresponding image data with the frame of image data.
  • the locker when the cabinet door rotates, the locker captures image data inside the locker through the camera.
  • the locker controls the camera to stop capturing image data inside the locker.
  • the locker can determine whether the door is rotating according to the change in the door opening angle detected by the angle sensor. For example, when the angle sensor detects that the door is opened for multiple consecutive angles and remains unchanged, it means that the door has stopped rotating. When the angle sensor detects that the door is opened for multiple consecutive angles and gradually increases or decreases, it means that the door is rotating. In this way, the locker can accurately determine whether the door is rotating.
  • the locker detects that the door opening angle is increasing, and the locker can capture image data inside the locker through the camera.
  • the locker detects that the door opening angle remains unchanged, and the locker can control the camera to continue capturing image data inside the locker.
  • the locker detects that the door opening angle is decreasing, and the locker can continue capturing image data inside the locker through the camera.
  • the locker in combination with the above angle range, if the locker detects that the opening angle of the door exceeds the preset angle range during the time period T7 to T8, the locker can control the camera to stop capturing image data inside the locker.
  • the locker in combination with the above angular acceleration range, if the locker detects that the angular acceleration of the door exceeds the preset angular acceleration range during the time period T7 to T8, the locker can control the camera to stop capturing image data inside the locker.
  • the locker in combination with the fact that the number of the above-mentioned multiple frames of image data does not exceed the second preset number, if before the time period T7 to T8, the number of image data captured by the locker through the camera reaches the second preset number, the locker can control the camera to stop capturing image data inside the locker, or the locker can control the camera to continue capturing image data inside the locker and update the multiple frames of image data captured before the time period T7 to T8.
  • the locker may determine whether to update the multiple frames of image data according to the shooting angle of the image data. For example, the locker captures the image data through a camera. The locker may compare the shooting angle of the image data with the shooting angle of each frame of image data in the multiple frames of image data. If there is image data with the same shooting angle as the image data in the multiple frames of image data, the locker may not update the multiple frames of image data. If there is no image data with the same shooting angle as the image data in the multiple frames of image data, the locker may update the multiple frames of image data.
  • the foreign body identification process may include S601 to S602 .
  • the feature information of the object may be the feature information of all objects in the image data.
  • the locker may be provided with an object recognition model, and the object recognition model may be used to recognize feature information of an object.
  • the locker can input multiple frames of image data into the object recognition model respectively to obtain feature information of the object in each frame of image data.
  • the item recognition model can be obtained by training the feature information of multiple samples according to a preset algorithm.
  • the multiple samples may include items belonging to the locker, and may also include items not belonging to the locker.
  • the preset algorithm may be a neural network algorithm or a deep learning algorithm, without limitation.
  • the above-mentioned multiple samples may also include the characteristic information of part (or part) of the object.
  • the locker can also obtain the characteristic information of the object when recognizing the image data through the object recognition model.
  • the locker can also obtain feature information of the object in the image data by selecting a detection method.
  • a detection method please refer to the description of the embodiment of FIG. 7 below.
  • S602 Determine whether there is a foreign object in the locker based on the feature information of the object in the multiple frames of image data and a preset feature information library.
  • the preset feature information library includes multiple feature information of items in the locker.
  • the preset feature information library can be preset for the locker. For example, it can be input into the locker by a staff member, or it can be obtained by the locker from other devices. For example, the locker can obtain it from the management device without restriction.
  • the locker can calculate the similarity between the object and the objects in the preset feature information library. For example, the similarity between the feature information of the object and each feature information in the preset feature information library can be calculated.
  • the similarity can be cosine similarity.
  • the calculation method of cosine similarity can refer to the prior art and will not be described in detail.
  • the locker can calculate the similarity between the feature information of the item and each feature information in the preset feature information library. If there is feature information in the preset feature information library whose similarity with the feature information of the item is greater than or equal to a preset value, the locker can determine that the item belongs to the locker. If the similarity between any feature information in the preset feature information library and the feature information of the item is less than a preset value, the locker can determine that the item is a foreign object.
  • the locker can perform image recognition on multiple frames of image data acquired during the door opening process to determine the foreign objects recognized in each frame of image data and the number of foreign objects.
  • the total number of times the foreign objects are recognized in the multiple frames of image data is greater than a first preset value, it is determined that there are foreign objects in the locker or the locker is invaded by foreign objects.
  • the first preset value can be set as needed, for example, it can be 1/2 of the number of multiple frames of image data, of course, it can also be other values, without limitation.
  • the total number of times a foreign object is identified can refer to the total number of times the first foreign object is identified among multiple different foreign objects included in the multiple frames of image data, the first foreign object refers to the foreign object with the largest total number of identifications among multiple foreign objects, or it can refer to the total number of times all foreign objects are identified in the multiple frames of image data, or it can refer to the number of image data in which foreign objects are identified in the multiple frames of image data, or it can refer to the total number of foreign objects identified in multiple locker openings between the current time and the first time.
  • the feature information of an object can be used to characterize an object.
  • the feature information of an object may include the shape, color, pattern, etc. of the object.
  • the feature information of the same object located at different positions in the image data may also be different.
  • the feature information of an object is the shape of the object, and the shape displayed in the image data of the same object located at different positions may also be different.
  • the total number of times that foreign objects are identified refers to the total number of times that the first foreign object is identified among multiple different foreign objects included in multiple frames of image data.
  • the locker can count the total number of times each foreign object is recognized, and determine whether to output an alarm message according to the total number of times the first foreign object is recognized. For example, when the total number of times a foreign object is recognized is greater than a third preset number, an alarm message is output.
  • the third preset number can be set as needed and is not limited.
  • the number of the multiple frames of image data is 5 (respectively image data 1 to image data 5).
  • the number of times that foreign objects are identified in image data 1 to image data 5 and the number of times that foreign objects are identified can be shown in Table 1.
  • Table 1 is merely exemplary and may include more or less image data and the number of times that foreign objects are identified.
  • the locker can output an alarm message because the total number of times the foreign object 1 is recognized is greater than 5. If the first preset value is 10, the locker may not output an alarm message because the total number of times the foreign object 1 is recognized is less than 10. In this case, the locker can delete the above multiple frames of image data.
  • the locker can save the value of the counter and continue to control the counter to count the number of foreign objects included in the multi-frame image data in the next opening of the locker until the number of foreign objects exceeds the first preset number.
  • the locker can output an alarm message and initialize the counter. In other words, the locker can restart the above S401 to S403.
  • the total number of times foreign objects are identified refers to the total number of times all foreign objects are identified in multiple frames of image data.
  • the locker can count the number of foreign objects identified in each frame of image data, and determine the total number of times all foreign objects in the multiple frames of image data are identified based on the statistical results.
  • the number of times foreign objects are recognized refers to the number of image data in which foreign objects are recognized as existing in multiple frames of image data.
  • the locker can determine whether there is a foreign object in each frame of image data, thereby obtaining the number of image data containing foreign objects in the multiple frames of image data.
  • the first preset value is 1/2 of the number of multiple frames of image data, the number of multiple frames of image data is 5, and the number of image data in which foreign objects are identified in the 5 frames of image data is 3, that is, the number of image data in which foreign objects are detected in the 5 frames of image data is 3.
  • the locker can output an alarm message.
  • the multiple frames of image data may be deleted, so that when the locker is opened again later, the locker may obtain the multiple frames of image data in the locker again.
  • the total number of times foreign objects are identified refers to the total number of foreign objects identified during multiple locker openings between the current time and the first time.
  • the locker can determine the foreign objects and the number of foreign objects included in the multiple frames of image data according to the multiple frames of image data obtained during the opening operation.
  • the opening operation refers to the operation of opening the locker and closing the locker. In this way, the locker can determine the total number of foreign objects according to the number of foreign objects identified each time in the continuous multiple opening operations. In the continuous multiple first operations, the locker does not output alarm information.
  • the locker outputs an alarm message at the first time, and between the second time and the third time after the first time, the locker is opened three times, and the locker does not output an alarm message from the second time to the third time.
  • the locker is opened three times from the second time to the third time.
  • the locker recognizes that there are foreign objects 1 and 2 in the multi-frame image data, and the number of times foreign objects 1 and 2 are recognized is 3;
  • the locker recognizes that there are foreign objects 1, 2, and 3 in the multi-frame image data, and the number of times foreign objects 1 and 2 are recognized is 3, and the number of times foreign object 3 is recognized is 1;
  • the locker recognizes that there are foreign objects 1, 2, and 3 in the multi-frame image data, and the number of times foreign objects 1 and 2 are recognized is 3, and the number of times foreign object 3 is recognized is 1.
  • the locker can determine whether to output an alarm message based on the total number of times foreign object 1 or foreign object 3 is identified.
  • the locker can calculate the similarity between the feature information of the foreign object and the feature information in the foreign object feature information library.
  • the foreign object feature information library can be used to store the feature information of the foreign object in the multi-frame image data.
  • the foreign object feature information library when the locker obtains multiple frames of image data and does not identify the multiple frames of image data, the foreign object feature information library is empty. Subsequently, the foreign object feature information library can store feature information of foreign objects identified in the multiple frames of image data.
  • the locker detects that the first image data includes a foreign object (referred to as foreign object 1).
  • the foreign object feature information library includes at least one foreign object
  • the locker can calculate the similarity between the feature information of foreign object 1 and the feature information of each foreign object in the at least one foreign object.
  • the at least one foreign object refers to a foreign object identified in the image data in the multiple frames of image data whose shooting time is before the first image data.
  • the foreign object feature information library is empty, the locker can write the feature information of foreign object 1 into the foreign object feature information library.
  • the locker may increase the first number of foreign objects 1 by a first value (such as 1).
  • the first number refers to the number of foreign objects 1 in the image data whose shooting time is before the first image data in the multiple frames of image data.
  • the locker can write the characteristic information of foreign object 1 into the foreign object characteristic information library, or the locker can update the foreign object characteristic information library, and the updated foreign object characteristic information library includes the characteristic information of foreign object 1.
  • the locker when the locker recognizes that a frame of image data includes multiple different foreign objects, if there is a foreign object among the multiple foreign objects whose similarity with the feature information of at least one foreign object in the foreign object feature information library is less than a preset value, the locker can update the foreign object feature information library, and the updated foreign object feature information library includes the feature information of the foreign object.
  • the locker detects that the first image data includes foreign matter 1 and foreign matter 2, and the locker can respectively calculate the similarity between the feature information of foreign matter 1 and the feature information of foreign matter 2 and the feature information in the foreign matter feature information library.
  • the locker can update the foreign matter feature information library, and the updated foreign matter feature information library includes the feature information of foreign matter 1.
  • the locker may increase the first number of foreign object 1 by a first value (such as 1).
  • a first value such as 1
  • the locker may not calculate the similarity between the characteristic information of foreign object 2 and the characteristic information of foreign object a. In this way, the amount of calculation can be reduced.
  • the similarity between the characteristic information of foreign object 1 and the characteristic information of foreign object a is greater than the preset value, it means that foreign object 1 and foreign object a are the same object. However, foreign object 1 and foreign object 2 are different, so the characteristic information of foreign object 2 and the characteristic information of foreign object a are also different. Therefore, in order to reduce the amount of calculation, the similarity between the characteristic information of foreign object 2 and the characteristic information of foreign object a may not be calculated.
  • the locker can update the foreign object feature information library, and the updated foreign object feature information library includes the feature information of foreign object 1.
  • the locker when the locker calculates the similarity between the feature information of foreign object 2 and the feature information of each foreign object in the updated foreign object feature information library, the locker may not calculate the feature information of foreign object 2 and the feature information of foreign object 1.
  • the locker in order to more accurately determine whether there is a foreign object in each door opening process, the locker can initialize the foreign object feature information library after completing the recognition of multiple frames of image data. That is, delete all feature information in the foreign object feature information library. In this way, in the next door opening process, the locker can restart adding feature information in the foreign object feature information library based on the acquired multiple frames of image data.
  • the locker may write the feature information of the foreign object into the foreign object feature information library.
  • the locker when the locker acquires 5 frames of image data during a door opening process, the locker detects that the first frame of image data among the 5 frames of image data includes foreign object 1.
  • the foreign object feature information library is empty.
  • the locker can write the feature information of foreign object 1 into the foreign object feature information library, and record the number of foreign objects 1 as the first value (such as 1).
  • the locker recognizes that there is a foreign object in the second frame of image data, and can calculate the similarity between the feature information of the foreign object and the feature information of foreign object 1 in the foreign object feature information library. If the similarity is greater than the preset value, it means that the foreign object in the second frame of image data is the same as foreign object 1.
  • the locker can increase the number of foreign objects 1 by the first value to the second value (i.e., 2).
  • the locker can calculate the similarity between the feature information of foreign object 2 and the feature information of foreign object 1. If the similarity is less than or equal to the preset value, the locker can write the feature information of foreign object 2 into the foreign object feature information library, and record the number of foreign objects 2 as the first value. Alternatively, the locker can also not calculate the similarity between foreign objects 1 and foreign objects 2, directly write the feature information of foreign object 2 into the foreign object feature information library, and record the number of foreign objects 2 as the first value.
  • the foreign object feature information library includes the feature information of foreign object 1 and the feature information of foreign object 2. If foreign object 3 is also identified in the second frame of image data, the locker can calculate the similarity between the feature information of foreign object 3 and the feature information of foreign object 1. If the similarity between the feature information of foreign object 3 and the feature information of foreign object 1 is less than or equal to the preset value, the locker can directly write the feature information of foreign object 3 into the foreign object feature information library. Similarly, each time the locker identifies the presence of foreign objects in the image data, the locker can calculate the similarity between the foreign objects and the foreign objects in the foreign object feature information library, and count the total number of times each foreign object is identified. In this way, the locker can obtain the foreign objects identified in the multiple frames of image data and the total number of times the foreign objects are identified.
  • the locker after acquiring the feature information of the object in each frame of image data, the locker can detect whether the feature information of the object in the image data belongs to the locker according to the preset feature information library. Since multiple objects in the preset feature information library belong to the locker, the locker can accurately detect whether there is a foreign object in each frame of image data.
  • the locker may also obtain feature information of the object in the image data by means of rotation detection, and the process may specifically include S701 to S702 .
  • S701 Input each frame of image data into an object detection model to obtain a detection frame of the object in the frame of image data.
  • the object detection module can be used to detect the detection frame of the object in the image data.
  • the detection frame can be a rectangular detection frame.
  • the input of the object detection model is the image data, and the object has a detection frame in the output image data.
  • the detection library of an object may refer to a rectangular frame with the smallest area that can include the object.
  • the object detection model may be pre-configured by the locker or obtained by the locker from other devices, without limitation.
  • the object detection model may be obtained by training a plurality of samples with detection frames according to a preset algorithm.
  • S702 Determine the rotation information of the object according to the detection frame of the object, and determine the feature information of the object according to the rotation information of the object.
  • the rotation information of the item refers to the position information of the item in the locker.
  • the position information includes coordinate data, the length and width of the detection frame, and the rotation angle.
  • the coordinate information may refer to the coordinate data of the center point of the item, and the rotation angle may refer to the angle between the bottom line of the detection frame of the item and the horizontal plane.
  • the locker can use the position information of the camera as the origin O2, the horizontal line as the X-axis, and the vertical line as the Y-axis to establish a plane coordinate system xoy.
  • the rotation information of item 1 is (x, y, k, s, ⁇ ).
  • x is the horizontal coordinate of the center point O1 of item 1 in the plane coordinate system xoy
  • y is the vertical coordinate of the center point of item 1 in the plane coordinate system xoy
  • l is the length of the detection frame of item 1
  • s is the width of the detection frame of item 1
  • ⁇ is the angle between the bottom line of the detection frame of item 1 and the x-axis (i.e., the rotation angle).
  • the locker can determine the vertex coordinates of the detection frame of the object (such as the coordinates of the four vertices of the detection frame of object 1) according to the rotation information of the object.
  • the specific process can refer to the prior art and will not be described in detail.
  • the locker can determine the sub-image data of the object included in the image data (that is, the minimum area of the object included in the image data) based on the vertex coordinates of the detection frame of the object, and input the sub-image data into the object recognition model to obtain the feature information of the object.
  • sub-image data including only one object is first determined from the image data, and then each sub-image data is detected by the object recognition model to obtain the characteristic information of the object. Since the sub-image data only includes one object, the object recognition model can quickly obtain the characteristic information of the object. At the same time, the interference of other objects in the image data is reduced, and the accuracy of recognition is improved.
  • FIG. 9 another locker management method is provided in an embodiment of the present application, and the method includes S901 to S909.
  • the foreign object intrusion recognition function means that the locker starts to obtain image data inside the locker, and identifies whether the image data includes foreign objects and counts the number of foreign object intrusions.
  • S904 For the image data including foreign matter, obtain feature information of the foreign matter in the image data, and perform a secondary comparison based on the feature information of the foreign matter.
  • the secondary comparison may refer to comparing the feature information of the foreign matter with the feature information in the foreign matter feature information library.
  • S906 Count the number of foreign objects identified between the current time and the time when the alarm information was last output, and the total number of times the foreign objects were identified.
  • S905 can also describe the statistics of foreign objects identified and the total number of times foreign objects are identified during multiple door opening processes between the current time and the time when the alarm information was last output.
  • S907 Determine whether the total number of times the first foreign object is identified is greater than a preset value.
  • S908 is executed; if the total number of times the first foreign object is identified is less than or equal to the preset value, S901 is executed.
  • S908 Detect whether the time interval between the current time and the last time the alarm information was output exceeds a preset time length.
  • S909 Output alarm information, clear foreign body intrusion information, and restart timing.
  • clearing foreign object records means deleting the counted number of foreign object intrusions.
  • the locker when the number of times the locker is opened exceeds the preset number of times and the time interval between the current time and the time when the last alarm information was output exceeds the preset time length, the locker will judge whether there is a foreign object in the locker based on the acquired multi-frame image data, and only when the cumulative number of foreign objects counted exceeds the preset number, the locker will output the alarm information, thereby reducing the alarm frequency. Since the locker detects whether there is a foreign object in the locker based on multi-frame images, the accuracy of foreign object detection can be improved, and the probability of misjudgment can be reduced.
  • an embodiment of the present application provides a locker management method, the method comprising: in response to an operation of opening the locker, obtaining multiple frames of image data in the locker at the current time; increasing the number of opening times by a first value; performing image recognition on the multiple frames of image data; if the number of opening times is greater than a preset threshold, and the time interval between the current time and the last time an alarm message was sent is greater than a preset duration, outputting an alarm message and deleting the recorded number of opening times.
  • the locker can clear the counted opening times each time the alarm message is output. In this way, when the door is opened again later, the locker can start counting the opening times again.
  • the embodiment of the present application can divide the locker management device into functional modules or functional units according to the above method example.
  • each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module.
  • the above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units.
  • the division of modules or units in the embodiment of the present application is schematic, which is only a logical functional division. There may be other division methods in actual implementation.
  • FIG10 it is a schematic diagram of the structure of a locker management device provided in an embodiment of the present application.
  • the device includes: an acquisition unit 101, a processing unit 102, and a sending unit 103.
  • the acquisition unit 101 is configured to: in response to opening the locker, acquire multiple frames of image data inside the locker at the current time.
  • the processing unit 102 is configured to: determine whether there is a foreign object in the locker based on multiple frames of image data when the time interval between the current time and the first time is greater than a preset time length and the number of times the locker is opened between the current time and the first time is greater than a preset number of times, and the first time is the time when the alarm information is sent before the current time, and the alarm information is used to prompt the presence of foreign objects in the locker.
  • the sending unit 103 is configured to send an alarm message if there is a foreign object in the locker and the number of times the foreign object is determined to be present in the locker between the current time and the first time is greater than a preset number.
  • the acquisition unit 101 is specifically configured to capture multiple frames of image data inside the locker when the door opening angle of the locker is within a preset angle range and the angular acceleration of the door is within a preset angular acceleration range.
  • the preset angle range is any angle range between 45 degrees and 50 degrees
  • the preset angular acceleration range is any angular acceleration range between 0 and 10 degrees/second squared.
  • the processing unit 102 is specifically configured to: obtain feature information of an object in each frame of image data in multiple frames of image data, and determine whether there is a foreign object in the locker based on the feature information of the object in the multiple frames of image data and a preset feature information library, wherein the preset feature information library includes feature information of multiple objects in the locker, and a foreign object refers to an object that does not belong to the locker.
  • the processing unit 102 is specifically configured as follows: if there is an item among all items included in the multi-frame image data, the similarity between the feature information of which is less than a first preset value with respect to any item in the preset feature information library, and the total number of times the item is identified is greater than or equal to the first preset value, then it is determined that there is a foreign object in the locker; if the similarity between the feature information of all items included in the multi-frame image data and the feature information of the items in the preset feature information library is greater than or equal to the first preset value, or the total number of times the item is identified is less than the first preset value, then it is determined that there is no foreign object in the locker.
  • the processing unit 102 is further configured to: when the foreign body feature information library includes feature information of at least one foreign body, if it is determined that there is a foreign body in the image data, and the similarity between the feature information of the foreign body in the image data and the feature information of the foreign body in the foreign body feature information library is greater than a second preset value, then the number of times the foreign body is identified is increased by a first value to obtain the total number of times the foreign body is identified; if the similarity between the feature information of the foreign body in the image data and the feature information of the foreign body in the foreign body feature information library is less than or equal to the second preset value, then the feature information of the foreign body in the image data is written into the foreign body feature information library.
  • the foreign body feature information library includes feature information of foreign bodies in image data whose shooting time is before the image data in the multiple frames of image data.
  • the foreign matter feature information library is empty and there is a foreign matter in the image data, the foreign matter feature information in the image data is written into the foreign matter feature information library.
  • the processing unit 102 is further configured to delete multiple frames of image data if the time interval between the current time and the first time is less than or equal to a preset time length, or the number of times the locker is opened between the current time and the first time is less than or equal to a first preset number of times.
  • the processing unit 102 is further configured to: if the number of multiple frames of image data is greater than a first preset number, delete at least one frame of image data with the earliest shooting time among the multiple frames of image data, and the number of image data after deletion is less than or equal to the first preset number.
  • the processing unit 102 is specifically configured to: in response to closing the locker, determine whether there is a foreign object in the locker based on the multiple frames of image data.
  • the processing unit 102 is specifically configured as follows: if the total number of times the first foreign object is identified is greater than a third preset number of times, it is determined that there is a foreign object in the locker, and the first foreign object is the foreign object that is identified the most times among the multiple foreign objects.
  • any two frames of image data in the multiple frames of image data correspond to different shooting angles.
  • the acquisition unit 101 and the sending unit 103 in the embodiment of the present application can be integrated on the communication interface, and the processing unit 102 can be integrated on the processor.
  • the specific implementation is shown in FIG11 .
  • FIG11 shows another possible structural diagram of the locker management device involved in the above embodiment.
  • the communication device includes: a processor 1102 and a communication interface 1103.
  • the processor 1102 is used to control and manage the actions of the device, for example, to execute the steps performed by the above processing unit 102, and/or to execute other processes of the technology described herein.
  • the communication interface 1103 is used to support the communication between the device and other network entities, for example, to execute the steps performed by the above acquisition unit 101.
  • the device may also include a memory 1101 and a bus 1104, and the memory 1101 is used to store program codes and data of the device.
  • the memory 1101 can be a memory in the device, etc., and the memory can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory can also include a combination of the above types of memory.
  • a volatile memory such as a random access memory
  • the memory can also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive
  • the memory can also include a combination of the above types of memory.
  • the processor 1102 may be a processor that implements or executes various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present application.
  • the processor may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.
  • the processor may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of the present application.
  • the processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
  • the bus 1104 may be an Extended Industry Standard Architecture (EISA) bus, etc.
  • EISA Extended Industry Standard Architecture
  • the bus 1104 may be divided into an address bus, a data bus, a control bus, etc.
  • FIG11 only uses one thick line, but does not mean that there is only one bus or one type of bus.
  • the device in FIG11 may also be a chip, which includes one or more than two (including two) processors 1102 and a communication interface 1103 .
  • the chip further includes a memory 1105, which may include a read-only memory and a random access memory, and provides operation instructions and data to the processor 1102.
  • a portion of the memory 1105 may also include a non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1105 stores the following elements, execution modules or data structures, or a subset thereof, or an extended set thereof.
  • the corresponding operation is performed by calling the operation instruction stored in the memory 1105 (the operation instruction may be stored in the operating system).
  • Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) having computer program instructions stored therein.
  • a computer e.g., a locker
  • the computer executes the locker management method as described in any of the above embodiments.
  • the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or magnetic tapes, etc.), optical disks (e.g., CD (Compact Disk), DVD (Digital Versatile Disk), etc.), smart cards and flash memory devices (e.g., EPROM (Erasable Programmable Read-Only Memory), cards, sticks or key drives, etc.).
  • the various computer-readable storage media described in the present disclosure may represent one or more devices and/or other machine-readable storage media for storing information.
  • the term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and/or carrying instructions and/or data.
  • Some embodiments of the present disclosure also provide a computer program product, for example, the computer program product is stored on a non-transitory computer-readable storage medium.
  • the computer program product includes computer program instructions, and when the computer program instructions are executed on a computer (e.g., a locker), the computer program instructions cause the computer to perform the synchronization method described in the above embodiments.
  • Some embodiments of the present disclosure further provide a computer program.
  • the computer program When the computer program is executed on a computer (eg, a locker), the computer program enables the computer to execute the locker management method as described in the above embodiments.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Landscapes

  • Alarm Systems (AREA)

Abstract

一种储物柜的管理方法及装置。方法包括：响应于开启储物柜，获取当前时间储物柜内的多帧图像数据（S401）。储物柜用于存储物品。若当前时间与第一时间之间的时间间隔大于预设时长，且当前时间与第一时间之间储物柜的开启次数大于第一预设次数，则根据多帧图像数据，确定储物柜内是否存在异物（S402），第一时间为当前时间之前发送告警信息的时间，告警信息用于提示储物柜内存在异物。若储物柜内存在异物，且当前时间与第一时间之间确定储物柜内存在异物的次数大于第二预设次数，则发送告警信息（S403）。

Description

储物柜的管理方法及装置 技术领域
本公开涉及图像处理技术领域，尤其涉及一种储物柜的管理方法及装置。
背景技术
随着技术的不断发展，智能货柜(如自动售货机、无人货柜等)越来越多。智能货柜的广泛应用，给人们带来便利的同时，也大大减少了工作人员的工作时间。
通常情况下，一旦智能货柜检测到智能货柜内出现将不属于货柜的物品/商品(也即异物)放置于货柜，智能货柜就会告警。但是，在一些情况下，智能货柜会出现频繁告警、误告警、漏告警的情况。例如，智能货柜将属于货柜内的物品误判为异物，就会出现误告警的情况，或者将异物误判为属于智能货柜内的物品，就会出现漏告警的情况。这些情况都会给工作人员的使用带来不便。
发明内容
一方面，提供一种储物柜的管理方法。该方法包括：响应于开启储物柜，获取当前时间储物柜内部的多帧图像数据。该储物柜用于储存物品。若当前时间与第一时间之间的时间间隔大于预设时长，且当前时间与第一时间之间储物柜的开启次数大于第一预设次数，根据多帧图像数据，确定储物柜内是否存在异物，第一时间为当前时间之前发送告警信息的时间，告警信息用于提示储物柜内存在异物。若储物柜内存在异物，且当前时间与第一时间之间确定储物柜内存在异物的次数大于第二预设次数，则发送告警信息。
在一些实施例中，上述“获取当前时间储物柜内部的多帧图像数据”具体可以包括：当储物柜的柜门开启角度位于预设角度范围，且柜门的角加速度位于预设角加速度范围时，拍摄储物柜内的多帧图像数据。
在一些实施例中，预设角度范围为45度～50度之间的任一角度范围，预设角加速度范围为0～10度/秒平方之间的任一角加速度范围。
在一些实施例中，上述“根据多帧图像数据，确定储物柜内是否存在异物”具体可以包括：获取多帧图像数据中每帧图像数据中的物品的特征信息，根据多帧图像数据的物品的特征信息以及预设特征信息库，确定储物柜内是否存在异物；预设特征信息库包括多个属于储物柜内的物品的特征信息，异 物是指不属于储物柜内的物品。
在一些实施例中，上述“根据多帧图像数据的物品的特征信息以及预设特征信息库，确定储物柜内是否存在异物”具体可以包括：若多帧图像数据包括的全部物品中存在与预设特征信息库中任意物品的特征信息之间的相似度均小于第一预设值的物品，且该物品被识别到的总次数大于或等于第一预设数值，则确定储物柜内存在异物；若多帧图像数据包括的全部物品的特征信息与预设特征信息库中的物品的特征信息之间的相似度均大于或等于第一预设值，或者该物品被识别到的总次数小于第一预设数值，则确定储物柜内不存在异物。
在一些实施例中，针对多帧图像数据中的任一帧图像数据，如第一图像数据，该方法还包括：当异物特征库包括至少一个异物的特征信息时，若确定第一图像数据中存在异物，且第一图像数据中异物的特征信息与异物特征信息库中异物的特征信息之间的相似度大于第二预设值，则将该异物被识别到的次数增加第一数值，得到异物被识别到的总次数。若第一图像数据中异物的特征信息与异物特征信息库中的异物的特征信息之间的相似度小于或等于第二预设值，则将第一图像数据中的异物的特征信息写入异物特征信息库。其中，该异物特征信息库包括多帧图像数据中拍摄时间位于第一图像数据之前的图像数据中被识别到的异物的特征信息。
在一些实施例中，该方法还包括：若异物特征信息库为空，且第一图像数据存在异物，则将第一图像数据中的异物的特征信息写入异物特征信息库。
在一些实施例中，该方法还包括：若当前时间与第一时间之前的时间间隔小于或等于预设时长，或当前时间与第一时间之间储物柜的开启次数小于或等于第一预设次数，则删除多帧图像数据。
在一些实施例中，若多帧图像数据的数量大于第一预设数量，则删除多帧图像数据中拍摄时间最早的至少一帧图像数据，删除后的图像数据的数量小于或等于第一预设数量。
在一些实施例中，上述“根据多帧图像数据，确定储物柜内是否存在异物”具体可以包括：响应于关闭储物柜，根据多帧图像数据，确定储物柜内是否存在异物。
在一些实施例中，若多帧图像数据包括多个异物且所述多个异物不同，上述“确定储物柜内存在异物”具体可以包括：若第一异物被识别到的总次数大于第三预设次数，则确定储物柜内存在异物，第一异物为多个异物中被识别的次数最多的异物。
在一些实施例中，多帧图像数据中任意两帧图像数据对应的拍摄角度不同。
另一方面，提供一种储物柜的管理装置。该管理装置包括：获取单元及处理单元、发送单元。
获取单元，被配置为：响应于开启储物柜，获取当前时间储物柜内部的多帧图像数据，储物柜用于存储物品。
处理单元，被配置为：当前时间与第一时间之间的时间间隔大于预设次数，且当前时间与第一时间之间储物柜的开启次数大于第一预设次数，则根据多帧图像数据，确定储物柜内是否存在异物，第一时间为当前时间之前发送告警信息的时间，告警信息用于提示储物柜内存在异物。
发送单元，被配置为：若储物柜内存在异物，且当前时间与第一时间之间确定储物柜内异物的次数大于第二预设次数，则发送告警信息。
在一些实施例中，获取单元，具体被配置为：当储物柜的柜门开启角度位于预设角度范围，且柜门的角加速度位于预设角加速度范围时，拍摄储物柜内的多帧图像数据。
在一些实施例中，预设角度范围为45度～50度之间的任一角度范围，预设角加速度范围为0～10度/秒之间的任一角加速度范围。
在一些实施例中，处理单元，具体被配置为：获取多帧图像数据中每帧图像数据中的物品的特征信息，根据多帧图像数据的物品的特征信息以及预设特征信息库，确定储物柜内是否存在异物。预设特征信息库包括多个属于储物柜内的物品的特征信息，异物是指不属于储物柜内的物品。
在一些实施例中，处理单元，具体被配置为：若多帧图像数据包括的全部物品中存在与预设特征信息库中任一物品的特征信息之间的相似度均小于第一预设值的物品，且该物品被识别到的总次数大于或等于第一预设数值，则确定储物柜内存在异物，或者确定储物柜存在异物入侵；若多帧图像数据包括的全部物品的特征信息与预设特征信息库中的物品的特征信息之间的相似度均大于或等于第一预设值，或该物品被识别到的总次数小于第一预设数值，则确定储物柜内不存在异物。
在一些实施例中，针对多帧图像数据中的任一帧图像数据，如第一图像数据，处理单元，还被配置为：当异物特征信息库包括至少一个异物的特征信息时，若确定第一图像数据中存在异物，且第一图像数据中异物的特征信息与异物特征信息库中的异物的特征信息之间的相似度大于第二预设值，则将该异物被识别到的次数增加第一数值，得到异物被识别到的总次数。若第 一图像数据中异物的特征信息与异物特征信息库中的异物的特征信息之间的相似度小于或等于第二预设值，则将第一图像数据中异物的特征信息写入异物特征信息库。其中，异物特征信息库包括多帧图像数据中拍摄时间位于第一图像数据之前的图像数据的异物的特征信息。
在一些实施例中，处理单元，还被配置为：若异物特征信息库为空，且第一图像数据存在异物，则将第一图像数据中的异物的特征信息写入异物特征信息库。
在一些实施例中，处理单元，还被配置为：若当前时间与第一时间之前的时间间隔小于或等于预设时长，或当前时间与第一时间之间储物柜的开启次数小于或等于第一预设次数，则删除多帧图像数据。
在一些实施例中，处理单元，还被配置为：在根据多帧图像数据，确定储物柜是否存在异物之后，删除多帧图像数据。
在一些实施例中，处理单元，还被配置为：若多帧图像数据的数量大于第一预设数量，则删除多帧图像数据中拍摄时间最早的至少一帧图像数据，删除后的图像数据的数量小于或等于第一预设数量。
在一些实施例中，处理单元，具体被配置为：响应于关闭储物柜，根据多帧图像数据，确定储物柜内是否存在异物。
在一些实施例中，若多帧图像数据包括多个异物且所述多个异物不同，处理单元，具体被被配置为：若第一异物被识别到的总次数大于第三预设次数，则确定储物柜内存在异物，第一异物为多个异物中被识别到的次数最多的异物。
在一些实施例中，多帧图像数据中任意两帧图像数据对应的拍摄角度不同。
又一方面，提供一种储物柜的管理装置，包括处理器和通信接口。通信接口和处理器耦合。处理器用于运行计算机程序或指令，以实现第一方面或第一方面任一实施例的储物柜的管理方法。
又一方面，提供一种储物柜，包括上述管理装置、摄像头。管理装置与摄像头连接。管理装置用于执行第一方面或第一方面任一实施例的储物柜的管理方法。摄像头用于响应于开启储物柜，拍摄储物柜内的多帧图像数据。
在一些实施例中，储物柜还包括角度传感器，角度传感器设置在储物柜的柜门与柜体的连接处。该角度传感器用于检测储物柜的柜门的开启角度。
在一些实施例中，储物柜还包括一个或多个灯带，该一个或多个设置在储物柜的门框侧。
在一些实施例中，摄像头为鱼眼摄像头。
再一方面，提供一种非瞬态计算机可读存储介质。所述计算机可读存储介质存储有计算机程序指令，所述计算机程序指令在计算机(例如，储物柜)上运行时，使得所述计算机执行如上述任一实施例所述的储物柜的管理方法。
又一方面，提供一种计算机程序产品。所述计算机程序产品包括计算机程序指令，在计算机(例如，储物柜)上执行所述计算机程序指令时，所述计算机程序指令使计算机执行如上述任一实施例所述的储物柜的管理方法。
又一方面，提供一种计算机程序。当所述计算机程序在计算机(例如，储物柜)上执行时，所述计算机程序使计算机执行如上述任一实施例所述的储物柜的管理方法。
附图说明
为了更清楚地说明本公开中的技术方案，下面将对本公开一些实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例的附图，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。此外，以下描述中的附图可以视作示意图，并非对本公开实施例所涉及的产品的实际尺寸、方法的实际流程、信号的实际时序等的限制。
图1为根据一些实施例的储物柜的结构图；
图2为根据一些实施例的图像数据的示意图；
图3为根据一些实施例的储物柜的结构图；
图4为根据一些实施例的储物柜的管理方法的流程图；
图5为根据一些实施例的更新图像数据列表的示意图；
图6为根据一些实施例的储物柜的管理方法的流程图；
图7为根据一些实施例的储物柜的管理方法的流程图；
图8为根据一些实施例的物品的旋转信息的示意图；
图9为根据一些实施例的储物柜的管理方法的流程图；
图10为根据一些实施例的管理装置的结构图；
图11为根据一些实施例的管理装置的结构图。
具体实施方式
下面将结合附图，对本公开一些实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开所提供的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本公开保护的范围。
除非上下文另有要求，否则，在整个说明书和权利要求书中，术语“包括(comprise)”及其其他形式例如第三人称单数形式“包括(comprises)”和现在分词形式“包括(comprising)”被解释为开放、包含的意思，即为“包含，但不限于”。在说明书的描述中，术语“一个实施例(one embodiment)”、“一些实施例(some embodiments)”、“示例性实施例(exemplary embodiments)”、“示例(example)”、“特定示例(specific example)”或“一些示例(some examples)”等旨在表明与该实施例或示例相关的特定特征或特性包括在本公开的至少一个实施例或示例中。上述术语的示意性表示不一定是指同一实施例或示例。此外，所述的特定特征或特点可以以任何适当方式包括在任何一个或多个实施例或示例中。
以下，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本公开实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
在描述一些实施例时，可能使用了“耦接”和“连接”及其衍伸的表达。例如，描述一些实施例时可能使用了术语“连接”以表明两个或两个以上部件彼此间有直接物理接触或电接触。又如，描述一些实施例时可能使用了术语“耦接”以表明两个或两个以上部件有直接物理接触或电接触。然而，术语“耦接”或“通信耦合(communicatively coupled)”也可能指两个或两个以上部件彼此间并无直接接触，但仍彼此协作或相互作用。这里所公开的实施例并不必然限制于本文内容。
“A、B和C中的至少一个”与“A、B或C中的至少一个”具有相同含义，均包括以下A、B和C的组合：仅A，仅B，仅C，A和B的组合，A和C的组合，B和C的组合，及A、B和C的组合。
“A和/或B”，包括以下三种组合：仅A，仅B，及A和B的组合。
如本文中所使用，根据上下文，术语“如果”任选地被解释为意思是“当……时”或“在……时”或“响应于确定”或“响应于检测到”。类似地，根据上下文，短语“如果确定……”或“如果检测到[所陈述的条件或事件]”任选地被解释为是指“在确定……时”或“响应于确定……”或“在检测到[所陈述的条件或事件]时”或“响应于检测到[所陈述的条件或事件]”。
本文中“适用于”或“被配置为”的使用意味着开放和包容性的语言，其不排除适用于或被配置为执行额外任务或步骤的设备。
另外，“基于”的使用意味着开放和包容性，因为“基于”一个或多个 所述条件或值的过程、步骤、计算或其他动作在实践中可以基于额外条件或超出所述的值。
随着智能货柜(后续为了统一描述，称为储物柜)的广泛应用，在给人们带来便利的同时，也给工作人员带来了一些麻烦，比如，储物柜内出现不属于储物柜的物品，从而导致工作人员在统计储物柜内剩余物品的数量时，就会出现物品的数量异常的问题。例如，实际取出的物品与储物柜内剩余的物品的总和与储物柜内原有的物品的总数量不一致。比如说，储物柜内原有10件物品，取出了6件。一旦储物柜内存在异物时，储物柜内剩余的物品的数量就会超过4件。这样一来，就需要工作人员再去查找不属于储物柜内的物品(也即异物)，浪费了时间和精力。
为了提醒工作人员储物柜内存在异物，通常情况下，储物柜检测到异物时，会输出告警信息，用于提示储物柜内存在异物。例如，储物柜可以通过语音广播播放告警信息，或者，储物柜也可以通过短信的方式输出告警信息，或者，储物柜也可以通过显示屏显示告警信息。
但是，在一些情况下，储物柜可能会出现频繁告警、误告警、漏告警的问题，给工作人员带来了不便。
例如，当储物柜开启比较频繁时，储物柜每检测到存在异常就会输出告警信息，比如，在一些时间段，工作人员不需要统计储物柜内剩余物品的数量，储物柜还会输出告警信息，增大了工作人员的工作量。
又例如，当储物柜将原本属于储物柜内的物品识别为异物，储物柜也会输出告警信息，导致工作人员需要耗费时间和精力查找异物。
又例如，当储物柜将异物识别为属于货柜内的物品，储物柜不会输出告警信息，这样，会导致储物柜剩余物品的数量与统计的物品的数量不一致，导致统计的数量错误。
鉴于此，本申请实施例提供了一种储物柜的管理方法，当储物柜开启的次数超过第一预设次数、且当前时间与上一次输出告警信息的时间之间的时间间隔超过预设时长时，根据当前时间获取的储物柜内的多帧图像数据判断储物柜内是否存在异物；若储物柜存在异物，且从上一次输出告警信息到当前时间这一个时间段内确定储物柜存在异物的次数超过第二预设次数，则输出告警信息。这样一来，只有当储物柜的开启次数比较多、距离告警时间较长且确定储物柜内存在异物的次数较多时，储物柜才输出告警信息，如此，在提醒管理人员清理储物柜内的异物的同时，储物柜不会出现频繁告警的问题。例如，在一些情况下，储物柜出现识别异常(如将属于储物柜内的物品 错误的识别为异物)，这样就可能出现储物柜频繁告警，增大了管理人员的工作量。而本申请实施例中，储物柜可以根据多次开门拍摄的图像数据的识别结果，进行综合分析，并根据分析结果，确定是否告警。一方面减少了识别异常的现象，避免出现频繁告警的问题；另一方面，由于告警次数的减少，减少了管理人员的工作量。
另外，本申请实施例中，储物柜在判断储物柜是否存在异物时，可以根据储物柜的多帧图像数据检测储物柜内是否存在异物，增大了异物识别的准确性，减少了误告警和漏告警的概率。
下面将结合说明书附图，对本申请实施例的实施方式进行详细描述。
如图1所示，图1为本申请实施例提供的一种储物柜。该储物柜可以包括柜体和柜门。柜体与柜门连接。
其中，柜体用于放置物品。柜门处设置有摄像头A。该摄像头A的朝向为柜体方向。该摄像头A用于拍摄柜体内的图像数据。
一种可能的实现方式中，该摄像头A可以设置在柜门的距离柜门与柜体的连接处最远的边缘位置。如此，可以保证该摄像头A能够拍摄到柜体内的物品信息。当然，摄像头A也可以设置在其他位置，例如，还可以设置在柜体的边缘。
一种示例中，为了保证摄像头A能够拍摄到储物柜内的全部物品，该摄像头A可以鱼眼摄像头。由于鱼眼摄像头的拍摄范围较大，因此，摄像头的拍摄范围能够覆盖整个储物柜的内部。
例如，如图2所示，为本申请实施例提供的一种摄像头A拍摄的图像数据。由图2的图像数据可知，摄像头A能够拍摄到位于储物柜内的所有物品的图像数据。
又一种示例中，为了使得摄像头拍摄的图像数据更加清楚，如图1所示，该储物柜的柜体两侧还可以设置有灯带。响应于柜门的开启，储物柜可以开启灯带，如此，可以增加储物柜的亮度，使得摄像头拍摄的图像数据更加清楚。
当然，储物柜还可以设置更多的摄像头，例如，储物柜还可以设置有摄像头B和摄像头C。摄像头B和摄像头C可以用于辅助摄像A。例如，当摄像头A出现故障或摄像头A的镜头被遮挡时，无法获取到储物柜内的图像数据，此时，储物柜可以通过摄像B和/或摄像头C拍摄储物柜内的图像数据。
又一种示例中，如图3所示，该储物柜还可以设置有摄像头D。摄像头D设置在柜门的外侧，且朝向为与储物柜的开口方向一致。摄像头D可以用于 拍摄位于储物柜前的人员的图像数据。储物柜可以根据摄像头D拍摄的图像数据，检测该人员是否具有开启储物柜的权限。若确定该人员有开启储物柜的权限，则储物柜可以响应于该人员开启储物柜的操作，可以不输入警告信息。若确定该人员不具有开启储物柜的权限，则储物柜可以响应于该人员开启储物柜的操作，输出警告信息。
又一种可能的实现方式中，该储物柜还可以设置有管理装置，该管理装置与摄像头连接，比如，通过系统总线连接。该管理装置可以用于对摄像头拍摄的图像数据进行图像识别，确定图像数据中是否存在异物。具体的识别过程可以参照下述的实施例。
一些场景中，该管理装置可以为板卡、处理器(如中央处理器(central processing unit，CPU))等。
又一种可能的实现方式中，如图3所示，储物柜还可以设置有显示屏。该显示屏可以设置在柜门的外侧。如此，储物柜可以通过该显示屏输出告警信息。
又一种可能的实现方式中，本申请实施例提供的储物柜还可以设置有语音播放装置(如音响、扬声器等)。如此，储物柜可以通过该语音播放装置输入告警信息。当然，储物柜还可以通过该语音播放装置显示其他语音信息，比如，还可以播放“欢迎使用”、“请关好柜门”、“物品数量不足，请及时补充物品”等。本申请实施例中，告警信息以及其他语音信息可以为储物柜预先配置的。
又一种可能的实现方式中，本申请实施例提供的储物柜还可以设置有一个或多个传感器。该一个或多个传感器可以用于检测柜门的状态(如开启或关闭)。例如，传感器可以设置在柜门与柜体的连接侧。
一种示例中，该一个或多个传感器可以包括距离传感器。当距离传感器检测到柜门与柜体之间的距离大于预设距离时，说明储物柜处于开启状态；当距离传感器检测到柜门与柜体之间的距离小于或等于预设距离时，说明储物柜处于关闭状态。
进一步，储物柜还可以设置有计数器，该计数器可以用于统计储物柜的开启次数。例如，该一个或多个传感器检测到的柜门的状态信息包括：在第一时间段内处于开启状态，在与第一时间段相邻且位于第一时间段之后的第二时间段内处于关闭状态，则储物柜可以控制计数器增加第一数值(如1)。如此，后续储物柜可以继续根据该一个或多个传感器检测到的柜门的状态信息，确定储物柜的开启次数，并通过计数器统计储物柜的开启次数。
需要说明的是，储物柜还可以将计数器进行初始化(如将数值初始化为0)。
又一种示例中，储物柜还可以设置有计时器。该计时器可以用于计时。例如，计时器可以用于记录每次的告警时间、每次柜门被打开的时间等。
又一种示例中，该一个或多个传感器还可以包括角度传感器，该角度传感器可以用于检测柜门的开启角度。储物柜可以根据该柜门的多个时间的开启角度，确定柜门开启过程的角加速度。
例如，在储物柜的开启过程中，在T1时间柜门的开启角度为θ 1，在T1时间之后的T2时间柜门的开启角度为θ 2，在T2时间之后的T3时间柜门的开启角度为θ 3，则在T1时间～T2时间储物柜的角速度α1＝(θ 2-θ 1)/(T2-T1)，在T2时间～T3时间储物柜的柜门的角速度α2＝(θ 3-θ 2)/(T3-T2)。基于该多个角速度，储物柜的角加速度＝(α2-α1)/(T3-T1)。
基于该角加速度，当储物柜的柜门的角加速度过大时，则说明储物柜的柜门转动速度太快。当储物柜的柜门转动速度太快时，可能会导致摄像头拍摄的图像数据不清晰；当储物柜的柜门的角加速度过小(如小于0)时，则说明储物柜的柜门转动速度太慢。当储物柜的柜门转动过慢时，摄像头可能会拍摄的图像数据可能会相同或相似。因此，当储物柜检测到柜门的角加速度位于预设角加速度范围时，可以控制摄像头开始拍摄，提高图像数据的拍摄质量以及减少拍摄图像过多带来的能耗。
需要说明的是，本申请实施例中，传感器可以不间断的检测储物柜的状态。也即，当储物柜处于关闭状态时，传感器可以检测储物柜的状态。当储物柜处于开启状态时，传感器可以检测储物柜的状态。
需要说明的是，本申请实施例中，储物柜可以为无人售货机、温控设备(如冰箱、冰柜等)，当然还可以为其他货柜，不予限制。
又一种可能的实现方式中，该储物柜还可以设置有通信模块，该通信模块可以连接管理设备，或者可以用于向工作人员的终端发送告警信息。
其中，管理设备可以用于管理储物柜。例如，可以从储物柜获取剩余的物品的数量等。管理设备可以为服务器或者计算机。终端可以包括手机、平板电脑、个人计算机等。
下面结合图1所示的储物柜，对本申请实施例提供的储物柜的管理方法进行说明。
需要说明的是，本申请实施例的执行主体可以为储物柜，也可以为储物柜中器件，如储物柜的芯片或片上系统。下面以执行主体为储物柜为例，对 本申请实施例提供的储物柜的管理方法进行说明。
如图4所示，为本申请实施例提供的一种储物柜的管理方法，该方法可以包括S401～S403。
S401、响应于开启储物柜，获取当前时间储物柜内部的多帧图像数据。
其中，当前时间可以是指从开启储物柜到关闭储物柜之间的时间。
一种示例中，储物柜可以根据传感器检测到的状态信息，确定储物柜的状态。
例如，当传感器检测到的状态信息中，储物柜在当前时间的状态为开启状态，在当前时间之前的第一时间，储物柜的状态为关闭状态，则说明储物柜在当前时间被打开。在该情况下，储物柜可以通过摄像头拍摄储物柜内部的多帧图像。
又例如，当传感器检测到的状态信息中，储物柜在当前时间的状态为关闭状态，在当前时间之前的第一时间，储物柜的状态为开启状态，则说明储物柜在当前时间被关闭。在这种情况下，储物柜可以对摄像头拍摄的多帧图像数据进行图像识别，确定储物柜内是否具有异物。具体过程可以参照后续的描述，此处不予赘述。
一种可能的实现方式中，为了保证摄像头拍摄的图像数据的质量，当柜门的开启角度位于预设角度范围，且柜门的角加速度位于预设角加速度范围内时，储物柜通过摄像头开始拍摄储物柜内的图像数据。
其中，预设角度范围、预设角加速度范围可以根据需要设置，例如，预设角度范围可以为45度～50度之间的任一范围，比如，预设角度范围可以45度～50度，也可以为46度～59度等，不予限制。预设角加速度范围可以为0～10度/秒平方之间的任一范围。比如，预设角加速度范围可以为0～10度/秒平方、1～5度/秒平方等，不予限制。
基于该实现方式，避免当储物柜的开启角度较小(如储物柜刚被开启的时刻)时，出现摄像头不能拍摄到位于储物柜内全部问题，当储物柜的开启角度位于45度～50度之间时，摄像头可以开始拍摄图像数据。由于当储物柜的开启角度位于45度～50度之间时，摄像头能够拍摄到储物柜内的全部物品，因此，可以提高了图像数据的可用性。同时还可以减少拍摄头拍摄次数，从而降低了储物柜的耗电量。
另外，当储物柜的柜门旋转速度较快时，可能会造成摄像头拍摄的图像数据出现模糊的问题，不利于后续的图像识别，因此，当储物柜的柜门的角加速度位于预设角加速度范围时，摄像头才拍摄的图像数据，保证了图像数 据的清晰度。例如，当柜门的角加速度位于0～10度/秒平方之间，此时，储物柜的柜门的转动速度不会太快，也不会太慢，避免出现拍摄的图像数据模糊以及拍摄过多重复的图像数据。
S402、若当前时间与第一时间之间的时间间隔大于预设时长，且当前时间与第一时间之间储物柜的开启次数大于预设次数，则根据多帧图像数据，确定储物柜内是否存在异物。
其中，第一时间是指当前时间之前储物柜发送告警信息的时间。异物是指不属于储物柜内的物品。预设时长、预设次数可以根据需要设置，例如，预设时长可以为1小时、2小时等，预设次数可以为10次、20次、30次等，不予限制。
一种示例中，储物柜可以根据计时器记录的时间确定当前时间与上一次发送告警信息的时间之间的时间间隔，并根据计数器统计的开启次数，确定当前时间与上一次发送告警信息的时间之间的开启次数。当时间间隔超过预设时长，且开启次数超过预设次数时，储物柜可以进行异物识别流程。异物识别流程可以包括判断是否存在异物的过程以及确定异物的数量的过程。具体过程可以参照后续的描述，此处不予赘述。
又一种示例中，储物柜也可以设置有计数算法和计时算法，如此，储物柜可以通过计数算法统计开启次数，并通过计时算法统计时长。
一种可能的实现方式中，为了降低储物柜的管理装置的负荷，储物柜可以响应于关闭储物柜，根据多帧图像数据，确定储物柜内是否存在异物。
又一种可能的实现方式中，为了及时处理图像数据，储物柜可以在获取到一帧图像数据之后，便对该帧图像数据进行图像识别，确定该帧图像数据中是否存在异物。如此，储物柜可以动态的对图像数据进行识别，避免出现当前时间获取的图像数据还未识别完成，储物柜就被再次打开，导致异物识别结果不准确的问题。
S403、若确定储物柜内存在异物，且当前时间与第一时间之间确定储物柜内存在异物的次数大于预设次数，则输出告警信息。
其中，确定储物柜内存在异物的次数也可以描述为储物柜的异物入侵次数。当前时间与第一时间之间确定储物柜内存在异物的次数是指在当前时间与第一时间之间的多次开门过程中，确定储物柜内存在异物的次数。预设次数可以根据需要设置，例如，可以为开启次数的1/2，当然，也可以为其他数值，不予限制。
一种示例中，预设次数为3，当前时间与第一时间之间的时间段内储物柜 的开启次数为5次，其中，有3次确定储物柜内存在异物，则可以输出告警信息。
基于图4的技术方案，当储物柜开启的次数超过预设次数、且当前时间与上一次输出告警信息的时间之间的时间间隔超过预设时长时，储物柜才根据获取到的多帧图像数据，判断储物柜是否存在异物，若当前时间与上一次输出告警信息的时间之间确定储物柜内存在异物的次数超过预设次数，储物柜才输出告警信息，减少了告警频次。同时，本申请实施例中，储物柜是根据多帧图像检测储物柜内是否存在异物，因此可以提高异物检测的准确率，减低了误判的概率。
一些实施例中，结合上述S401中的预设角度范围以及预设角加速度范围，当柜门的开启角度从0度旋转至45度时，储物柜通过摄像头A开始拍摄储物柜内的图像数据，直至储物柜的柜门的开启角度达到50度。当储物柜的开启角度超过50度时，储物柜可以控制摄像头A停止拍摄储物柜内的图像数据。
进一步的，为了避免出现当储物柜被关闭的过程中储物柜出现异物，而柜门的开启过程中储物柜没有异物，导致后续异物检测结果错误的情况。本申请实施例中，在储物柜的柜门的关闭过程中，储物柜可以通过摄像头A继续拍摄储物柜内的图像数据。柜门的开启过程可以是指柜门的开启角度不断变大的过程。柜门的关闭过程是指储物柜的柜门的开启角度不断变小的过程。
其中，储物柜可以根据角度传感器检测到的柜门的开启角度的变化情况，确定柜门的状态。
结合上述例子，在T3时间柜门的开启角度为θ 3，在T4时间的开启角度为θ 4，且在T4时间位于T3时间之后。如果θ 3大于θ 4，则说明柜门处于关闭过程。
一种示例中，当柜门的开启角度从90度减小到50度时，储物柜可以通过摄像头继续拍摄储物柜内的图像数据，直至开启角度减小至45度。当柜门的开启角度小于45度时，储物柜控制摄像头A停止拍摄储物柜内的图像数据。
需要说明的是，当柜门的角加速度超过预设角加速度范围时，说明柜门的运动速度过快，可能会导致拍摄的图像数据的清晰度较低，导致识别后续的图像识别的准确率较低。
基于该实施例，可以避免出现柜门被关闭过程中出现异物，导致漏告警的问题。
又一些实施例中，为了减少后续图像数据识别的时间，本申请实施例中，储物柜在通过拍摄储物柜的图像数据过程中，可以不断更新拍摄的图像数据， 以使得用于异物识别的图像数据的数量不超过第二预设数量。
其中，第二预设数量可以根据需要设置，例如，可以为5～15中的任一数值。第二预设数量还可以根据摄像头的性能、处理器的性能以及图像识别算法的性能相关。比如，摄像头的性能越高，第二预设数量的数值可以越大。又比如，处理器的性能越好，第二预设数量的数值可以越大。又比如，图像识别算法的识别效率越高，第二预设数量的数值可以越大。
一种示例中，当拍摄的图像数据的数量大于第二预设数量时，储物柜可以删除多帧图像数据中的部分图像数据，以使得图像数据的数量小于或等于第二预设数量。
一种场景中，当拍摄头拍摄到的图像数据的数量超过第二预设数量时，储物柜可以控制摄像头停止拍摄图像数据，或者，储物柜也可以通过摄像头继续拍摄图像数据，并更新该多帧图像数据。
其中，更新该多帧图像数据过程可以是指储物柜将摄像头拍摄的时间最晚的图像数据替换掉该多帧图像数据中拍摄时间最早的图像数据。
例如，第二预设数据为5，如图5所示，摄像头从开始拍摄到第一时间之间拍摄了5帧图像数据(分别为图像数据1～图像数据5)，并将5帧图像数据按照拍摄的时间顺序排序，得到一个图像数据列表。该图像数据列表中排列顺序是根据拍摄的时间确定的。例如，图像数据列表中，图像数据1为该5帧图像数据中拍摄时间最早的图像数据，图像数据5为该5帧图像数据中拍摄时间最晚的图像数据。
之后，储物柜通过拍摄头继续拍摄储物柜内的图像数据。例如，如图5所示，在第一时间之后，又拍摄了1帧图像数据(如可以标记为图像数据6)，储物柜可以删除图像数据1，并将图像数据6写入该图像数据列表中，得到一个更新后的图像数据列表。类似的，后续，若储物柜通过摄像头继续拍摄到图像数据，储物柜可以继续更新图像数据列表，直至储物柜控制摄像头停止拍摄。
基于该实施例，不仅能保证获取到的多帧图像数据为最新的图像数据，还能避免出现在后续图像识别过程中，由于识别速度过慢，导致下一次储物柜的柜门被打开时，没有识别完图像数据，从而导致异物检测不正确，给工作人员的使用造成不便。
一些实施例中，为了保证尽可能的拍摄到位于储物柜的物品信息，该多帧图像数据中任意两帧图像数据对应的角度不同。多帧图像数据中，拍摄时间相邻的图像数据对应的角度差值可以为预设角度。预设角度可以根据需要 设置，例如，可以为1度～2度。
例如，储物柜在控制摄像头拍摄储物柜内的图像数据之后，可以根据角度传感器获取该图像数据对应的拍摄角度。如此，储物柜可以获取到多帧图像数据中每一帧图像数据对应的拍摄角度。
储物柜后续在通过摄像头继续拍摄的过程中，可以比较拍摄的图像数据对应的拍摄角度是否与多帧图像数据的拍摄角度是否相同。若存在相同拍摄角度的图像数据，储物柜删除该图像数据，或者将该帧图像数据替换掉对应的图像数据。
一种示例中，当柜门的旋转时，储物柜通过摄像头拍摄储物柜内的图像数据。当柜门停止旋转时，储物柜控制摄像头停止拍摄储物柜内的图像数据。
其中，储物柜可以根据角度传感器检测到的柜门开启的角度的变化情况，确定柜门是否旋转状态。例如，当角度传感器检测到柜门开启的连续多个角度保持不变，则说明柜门停止旋转。当角度传感器检测到柜门开启的连续多个角度逐渐增多或逐渐减小时，则说明柜门正在旋转。如此，储物柜可以准确的判断柜门是否旋转。
一种可能的场景中，在T5～T6时间段内，储物柜检测到柜门的开启角度不断增大，储物柜可以通过拍摄头拍摄储物柜内的图像数据。在T6～T7时间段内，储物柜检测到柜门的开启角度保持不变，储物柜可以控制摄像头继续拍摄储物柜内的图像数据。在T7～T8时间段内，储物柜检测到柜门的开启角度不断减小，储物柜可以通过摄像头继续拍摄储物柜内的图像数据。
一种示例中，结合上述角度范围，若在T7～T8时间段内，储物柜检测到柜门的开启角度超过预设角度范围，则储物柜可以控制摄像头停止拍摄储物柜内的图像数据。
又一种示例中，结合上述角加速度范围，若在T7～T8时间段内，储物柜检测到柜门的角加速度超过预设角加速度范围，则储物柜可以控制摄像头停止拍摄储物柜内的图像数据。
又一种示例中，结合上述多帧图像数据的数量不超过第二预设数量，若在T7～T8时间段之前，储物柜通过摄像头拍摄的图像数据的数量达到第二预设数量，储物柜可以控制摄像头停止拍摄储物柜内的图像数据，或者，储物柜可以控制摄像头继续拍摄储物柜内的图像数据，并更新T7～T8时间段之前拍摄的多帧图像数据。
进一步的，储物柜在更新拍摄的多帧图像数据的过程中，储物柜可以根据图像数据的拍摄角度，确定是否更新该多帧图像数据。例如，储物柜又通 过摄像头拍摄到图像数据。储物柜可以对比该图像数据的拍摄角度与该多帧图像数据中每帧图像数据的拍摄角度。若该多帧图像数据中存在与该图像数据拍摄角度相同的图像数据，则储物柜可以不更新该多帧图像数据。若该多帧图像数据中不存在与该图像数据拍摄角度的图像数据，则储物柜可以更新该多帧图像数据。
基于该实施例，由于储物柜获取到图像数据不会过多，因此，可以提供异物识别的速度。
又一些实施例中，如图6所示，上述S402中，异物识别流程可以包括S601～S602。
S601、获取每帧图像数据中的物品的特征信息。
其中，物品的特征信息可以图像数据中全部物品的特征信息。
一种可能的实现方式中，储物柜可以设置有物品识别模型，该物品识别模型可以用于识别物品的特征信息。
例如，储物柜可以将多帧图像数据分别输入该物品识别模型，得到每帧图像数据中的物品的特征信息。
需要说明的是，该物品识别模型可以根据预设算法，对多个样品的特征信息进行训练得到。该多个样品可以包括属于储物柜的物品，也可以包括不属于储物柜的物品。预设算法可以为神经网络算法、深度学习算法，不予限制。
进一步的，为了避免出现由于物品被遮挡，导致无法识别到物品的特征信息，上述多个样本还可以包括物品的部分(或局部)的特征信息。如此，当图像数据中的物品的部分被遮挡时，储物柜在通过该物品识别模型对图像数据进行识别时，也可以获取到该物品的特征信息。
又一种示例中，储物柜还可以通过选择检测的方式，获取图像数据中物品的特征信息。具体的，可以参照下述图7的实施例的描述。
S602、根据多帧图像数据的物品的特征信息以及预设特征信息库，确定储物柜内是否存在异物。
其中，预设特征信息库包括多个属于储物柜内的物品的特征信息。该预设特征信息库可以为储物柜预设配置的。例如，可以为工作人员输入到储物柜的，也可以为储物柜从其他设备处获取的。比如，储物柜可以从管理设备处获取，不予限制。
一种可能的实现方式中，针对每一帧图像数据，储物柜在识别到该帧图像数据中的物品之后，可以计算该物品与预设特征信息库中的物品的相似度。 例如，可以计算该物品的特征信息与预设特征信息库中每个特征信息之间的相似度。该相似度可以为余弦相似度。余弦相似度的计算方法可以参照现有技术，不予赘述。
具体的，针对多帧图像数据中包括的每一个物品，储物柜可以计算该物品的特征信息与预设特征信息库中的每个特征信息之间的相似度。若预设特征信息库中存在与该物品的特征信息的相似度大于或等于预设值的特征信息，则储物柜可以确定该物品属于储物柜。若预设特征信息库中任意特征信息与该物品的特征信息的相似度均小于预设值，则储物柜可以确定该物品为异物。
进一步的，每次开门过程中，储物柜都可以对该次开门过程中获取到的多帧图像数据进行图像识别，确定每帧图像数据被识别到的异物以及异物的数量。当多帧图像数据中异物被识别到的总次数大于第一预设数值时，则确定储物柜存在异物或者储物柜被异物入侵。
其中，第一预设数值可以根据需要设置，例如，可以为多帧图像数据的数量的1/2，当然，也可以为其他数值，不予限制。异物被识别到的总次数可以是指多帧图像数据包括的多个不同的异物中第一异物被识别到的总次数，第一异物是指多个异物中被识别到的总次数最多的异物，也可以是指多帧图像数据中全部异物被识别到的总次数、也可以是指多帧图像数据中被识别到存在异物的图像数据的数量，也可以是指当前时间与第一时间之间的多次开启储物柜中识别到异物的总数量。
需要说明的是，不同的异物是指异物的特征信息不同。一个物品的特征信息可以用于表征一个物品，例如，物品的特征信息可以包括该物品的形状、颜色、图案等。本申请实施例中，由于不同物品与摄像头之间的角度/距离不同，则位于不同位置的相同物品，在图像数据中的特征信息也有可能不同。例如，物品的特征信息为物品的形状，位于不同位置的同一物品在图像数据中的显示的形状也可能不同。
下面分别对上述四种异物被识别到的总次数进行说明。
一、异物被识别到的总次数是指多帧图像数据包括的多个不同的异物中第一异物被识别到总次数。
其中，储物柜在上述异物识别流程中可以分别统计每个异物被识别到总次数，并根据第一异物被识别到的总次数确定是否输出告警信息。例如，当一异物被识别到总次数大于第三预设次数，则输出告警信息。第三预设次数可以根据需要设置，不予限制。
例如，该多帧图像数据的数量为5(分别为图像数据1～图像数据5)。其中，图像数据1～图像数据5中被识别到异物以及异物被识别到的次数可以如表1所示。
表1
Figure PCTCN2022140637-appb-000001
需要说的是，表1中的数值仅为示例性的，还可以包括更多或更少的图像数据以及异物被识别到次数。
结合上述表1，若第一预设数值为5，由于异物1被识别到的总次数大于5，因此储物柜可以输出告警信息。若第一预设数值为10，由于异物1被识别到的总次数小于10，则储物柜可以不输出告警信息。在这种情况下，储物柜可以删除上述多帧图像数据。
一种示例中，储物柜可以保存计数器的数值，继续控制计数器统计下一次开启储物柜中的多帧图像数据中包括的异物的数量，直至异物的数量超过第一预设数量。当异物的数量超过第一预设数量时，储物柜可以输出告警信息，并对计数器进行初始化。也即是说，储物柜可以重新开始执行上述S401～S403。
二、异物被识别到的总次数是指多帧图像数据中全部异物被识别到的总次数。
其中，储物柜在获取到多帧图像数据中，储物柜可以统计每帧图像数据中识别到的异物的数量，并根据统计结果，确定多帧图像数据中全部异物被识别到的总次数。
例如，多帧图像数据的数量为5，该5帧图像数据每帧图像数据中异物被识别的总次数分别为1、1、1、1、1，则异物被识别到总次数为1+1+1+1+1＝5。
三、异物被识别到次数是指多帧图像数据被识别到存在异物的图像数据的数量。
其中，储物柜可以分别确定每帧图像数据是否存在异物，从而可以得到该多帧图像数据中存在异物的图像数据的数量。
例如，第一预设数值为多帧图像数据的数量的1/2，多帧图像数据的数量为5，该5帧图像数据中被识别到存在异物的图像数据的数量为3，也即是说，5帧图像数据中检测到存在异物的图像数据的数量为3。在这种情况下，储物柜可以输出告警信息。
进一步，当储物柜输出告警信息之后，可以删除多帧图像数据。如此，后续当储物柜再次被打开时，储物柜可以再次获取储物柜内的多帧图像数据。
四、异物被识别到的总次数是指当前时间与第一时间之间的多次开启储物柜的过程中识别到异物的总数量。
其中，储物柜每执行一次开启操作后，储物柜可以根据该开启操作过程中获取的多帧图像数据，确定该多帧图像数据中包括的异物以及异物的数量。开启操作是指打开储物柜并关闭储物柜的操作。如此，储物柜可以根据连续的多次开启操作中每次识别到的异物的数量，确定异物的总数量。在该连续的多次第一操作中，储物柜未输出告警信息。
例如，储物柜在第一时间输出了告警信息，在第一时间之后的第二时间到第三时间之间内，储物柜被执行了3次开启操作，且第二时间到第三时间内，储物柜未输出告警信息。其中，第1次储物柜被执行开启操作时，储物柜获取到多帧图像数据，并对该多帧图像数据进行识别，异物被识别到的总次数为3；第2次储物柜被执行开启操作时，储物柜再次获取到多帧图像数据，并对该多帧图像数据进行识别，异物被识别到的总次数为4；第3次储物柜被执行开启操作时，储物柜又获取到多帧图像数据，并对该多帧图像数据进行识别，异物被识别到的总次数为3。则第二时间与第三时间，储物柜识别到的异物的总数量为3+4+3＝10次。
又例如，在第二时间到第三时间内储物柜被执行了3次开启操作。其中，第1次储物柜被执行开启操作时，储物柜识别到多帧图像数据中存在异物1和异物2，且异物1和异物2被识别到的次数均为3；第2次储物柜被执行开启操作时，储物柜识别到多帧图像数据中存在异物1、异物2和异物3，异物1和异物2被识别到的次数均为3，异物3被识别到的次数为1；第3次储物柜被执行开启操作时，储物柜识别到多帧图像数据中存在异物1、异物2、异物3，异物1、异物2被识别到的次数均为3，异物3被识别到的次数为1。则第二时间与第三时间之间，异物1被识别到总次数为3+3+3＝9、异物2被识别到的总次数为3+3+3＝9，异物3被识别到总次数为0+1+1＝2。储物柜可以 根据该异物1或异物3被识别到的总次数，确定是否输出告警信息。
进一步的，储物柜在识别到多帧图像数据中的异物后，可以计算该异物的特征信息与异物特征信息库中的特征信息之间的相似度。异物特征信息库可以用于存储该多帧图像数据中的异物的特征信息。
需要说明的是，在储物柜获取到多帧图像数据，且未对该多帧图像数据进行识别时，异物特征信息库为空。后续，异物特征信息库可以存储该多帧图像数据中被识别到的异物的特征信息。
一种示例中，对于该多帧图像数据中的任一帧图像数据(记做第一图像数据)，储物柜检测到该第一图像数据包括一个异物(记做异物1)。当异物特征信息库包括至少一个异物时，储物柜可以计算异物1的特征信息与该至少一个异物中每一个异物的特征信息之间的相似度。该至少一个异物是指该多帧图像数据中拍摄时间位于第一图像数据之前的图像数据中识别到的异物。当异物特征信息库为空时，储物柜可以将异物1的特征信息写入异物特征信息库。
例如，当异物特征信息库包括至少一个异物时，若异物1的特征信息与该至少一个异物中某个异物的特征信息之间的相似度大于预设值，则储物柜可以将异物1的第一数量增加第一数值(如1)。第一数量是指多帧图像数据中拍摄时间位于第一图像数据之前的图像数据中异物1的数量。
若异物1的特征信息与该至少一个异物的特征信息之间的相似度均小于或等于预设值，则说明该异物特征信息库中不包括该异物的特征信息。储物柜可以将异物1的特征信息写入异物特征信息库，或者储物柜可以更新异物特征信息库，更新后的异物特征信息库包括异物1的特征信息。
又一种示例中，当储物柜识别到一帧图像数据中包括多个不同的异物时，若该多个异物中存在与异物特征信息库中至少一个异物的特征信息之间的相似度均小于预设值的异物，储物柜可以更新异物特征信息库，更新后的异物特征信息库包括该异物的特征信息。
例如，储物柜检测到第一图像数据包括异物1和异物2，储物柜可以分别计算异物1的特征信息和异物2的特征信息与异物特征信息库中的特征信息之间的相似度。其中，异物1的特征信息1与异物特征信息库中至少一个异物的特征信息之间的相似度均小于预设值，储物柜可以更新异物特征信息库，更新后的异物特征信息库包括异物1的特征信息。
若异物1的特征信息与异物特征信息库中的异物a的特征信息之间的相似度大于预设值，则储物柜可以将异物1的第一数量增加第一数值(如1)。 当计算异物2的特征信息与异物特征信息中异物的特征信息之间的相似度时，储物柜可以不计算异物2的特征信息与异物a的特征信息之间的相似度。如此，可以减少计算量。
需要说明的是，由于异物1的特征信息与异物a的特征信息的相似度大于预设值，说明异物1与异物a为同一物品。而异物1和异物2不同，因此，异物2的特征信息和异物a的特征信息也不同。因此，为了减少计算量，可以不计算异物2的特征信息与异物a的特征信息之间的相似度。
例如，若异物1的特征信息与异物特征信息库中每个异物的特征信息之间的相似度均小于或等于预设值，则储物柜可以更新异物特征信息库，更新后的异物特征信息库包括异物1的特征信息。
接下来，储物柜在计算异物2的特征信息与更新后的异物特征信息库中每个异物的特征信息的相似度时，储物柜可以不计算异物2的特征信息与异物1的特征信息。
需要说明的是，本申请实施例中，为了更加准确的确定每次开门过程中是否存在异物，储物柜在对多帧图像数据识别完成之后，可以将异物特征信息库初始化。也即，删除异物特征信息库中的所有特征信息。如此，在下一次开门过程中，储物柜可以根据获取到的多帧图像数据，重新开始增加异物特征信息库中的特征信息。
进一步的，在储物柜检测到图像数据中包括异物，若异物特征信息库为空或者该异物特征信息库中不包括该异物的特征信息，储物柜可以将该异物的特征信息写入异物特征信息库中。
例如，在储物柜在一次开门过程中获取到5帧图像数据，储物柜检测到该5帧图像数据中的第一帧图像数据包括异物1，此时异物特征信息库为空，储物柜可以将该异物1的特征信息写入异物特征信息库，并将异物1的数量记做第一数值(如1)。储物柜识别到第二帧图像数据存在异物，可以计算该异物的特征信息与异物特征信息库中的异物1的特征信息之间的相似度。若相似度大于预设值，则说明该第二帧图像数据中的异物与异物1相同。储物柜可以将异物1的数量增加第一数值，变为第二数值(也即2)。
又例如，若第二帧图像数据还被识别到其他异物(如异物2)，储物柜可以计算异物2的特征新与异物1的特征信息的相似度，若相似度小于或等于预设值，储物柜可以将该异物2的特征信息写入异物特征信息库中，并将异物2的数量记做第一数值。或者，储物柜也可以不计算异物1和异物2的相似度，直接将异物2的特征信息写入异物特征信息库中，并将异物2的数量 记做第一数值。
此时，异物特征信息库包括异物1的特征信息、异物2的特征信息。若第二帧图像数据中还被识别到异物3，则储物柜可以计算异物3的特征信息与异物1的特征信息的相似度。若异物3的特征信息与异物1的特征信息之间的相似度小于或等于预设值，则储物柜可以直接将异物3的特征信息写入异物特征信息库中。类似的，储物柜每次识别到图像数据中存在异物，储物柜柜可以分别计算异物与异物特征信息库中的异物的相似度，并统计每个异物被识别到的总次数。如此，储物柜可以得到该多帧图像数据中被识别的异物以及异物被识别到的总次数。
基于图6的技术方案，储物柜在获取到每帧图像数据的物品的特征信息之后，可以根据包括预设特征信息库检测图像数据中的物品的特征信息是否属于储物柜。由于预设特征信息库中的多个物品为属于储物柜的物品，因此，储物柜可以准确的检测每帧图像数据中是否存在异物。
一些实施例中，如图7所示，上述S601中，储物柜还可以通过旋转检测的方式，获取图像数据中物品的特征信息的过程具体可以包括S701～S702。
S701、将每帧图像数据输入物品检测模型，得到该帧图像数据中的物品的检测框。
其中，物品检测模块可以用于检测图像数据中的物品的检测框。该检测框可以为矩形检测框。该物品检测模型的输入为图像数据，输出的图像数据中物品具有检测框。
例如，结合图2的图像数据，如图8所示，物品的检测库可以是指能够包括该物品的面积最小的矩形框。
其中，物品检测模型可以为储物柜预先配置的，也可以为储物柜从其他设备处获取的，不予限制。该物品检测模型可以根据预设算法对多个具有检测框的样本训练得到的。
S702、根据物品的检测框，确定物品的旋转信息，并根据物品的旋转信息，确定物品的特征信息。
其中，物品的旋转信息是指物品在储物柜的位置信息。该位置信息包括坐标数据、检测框的长度和宽度以及旋转角度。其中，坐标信息可以是指物品的中心点的坐标数据，旋转角度可以是指物品的检测框的底线与水平面之间的角度。
例如，如图8所示，储物柜可以将摄像机的位置信息作为原点O2，水平线为X轴，垂直线为Y轴，建立平面坐标系xoy。物品1的旋转信息为(x， y，k，s，α)。其中，x为物品1的中心点O1在该平面坐标系xoy中的横坐标，y为物品1的中心点在该平面坐标系xoy中的纵坐标，l为物品1的检测框的长度，s为物品1的检测框的宽度，α为物品1的检测框的底线与x轴之间的夹角(也即旋转角度)。
一种可能的实现方式中，在得到物品的旋转信息之后，储物柜可以根据物品的旋转信息，确定物品的检测框的顶点坐标(如物品1的检测框的四个顶点的坐标)。具体过程可以参照现有技术，不予赘述。
进一步的，在得到图像数据中物品的检测框的顶点坐标之后，储物柜可以根据物品的检测框的顶点坐标确定图像数据中包括的物品的子图像数据(也即，图像数据中包括该物品的最小区域)，并将子图像数据输入物品识别模型，得到该物品的特征信息。
基于图7的技术方案，由于旋转检测的方式中，先从图像数据中确定只包括一个物品的子图像数据，然后通过物品识别模型检测每个子图像数据，得到该物品的特征信息。由于子图像数据只包括一个物品，因此，物品识别模型可以快速的获取到该物品的特征信息。同时，减少了图像数据中其他物品的干扰，提高了识别的准确度。
一些实施例中，如图9所示，为本申请实施例提供的另一种储物柜的管理方法，该方法包括S901～S909。
S901、响应于开启储物柜，启动异物入侵识别功能。
其中，异物入侵识别功能是指储物柜开始获取储物柜内的图像数据，并识别该图像数据中是否包括异物以及统计异物入侵的次数。
S902、当柜门的开启角度位于预设角度范围内，通过摄像头A拍摄获取当前时间储物柜内部的多帧图像数据。
S903、检测图像数据中是否存在异物。
若存在异物，则执行S904；若不存在异物，则重新执行S901。
S904、对于包括异物的图像数据，获取该图像数据中异物的特征信息，并根据该异物的特征信息进行二次对比。
其中，二次对比可以是指将异物的特征信息与异物特征信息库中的特征信息进行对比。
S905、统计该多帧图像数据中识别的异物以及异物被识别到的次数。
S906、统计当前时间与上次输出告警信息的时间之间识别到的异物以及异物被识别到总次数。
其中，S905也可以描述统计当前时间与上次输出告警信息的时间之间的 多次开门过程中，识别到的异物以及异物被识别到的总次数。
S907、确定第一异物被识别到的总次数是否大于预设值。
其中，若第一异物被识别到的总次数大于预设值，则执行S908；若第一异物被识别到的总次数小于或等于预设值，则执行S901。
S908、检测当前时间与上一次输出告警信息之间的时间间隔是否超过预设时长。
若超过，则执行S909；若未超过，则执行S901。
S909、输出告警信息，并清空异物入侵情况以及重新开始计时。
其中，清空异物记录是指删除统计的异物入侵次数。
基于图9的技术方案，当储物柜开启的次数超过预设次数、且当前时间与上一次输出告警信息的时间之间的时间间隔超过预设时长时，储物柜才根据获取到的多帧图像数据，判断储物柜是否存在异物，仅当统计的异物的数量累计超过预设数量时，储物柜才输出告警信息，减少了告警频次。由于储物柜是根据多帧图像检测储物柜内是否存在异物，因此可以提高异物检测的准确率，减低了误判的概率。
一些实施例中，本申请实施例提供了一种储物柜的管理方法，该方法包括：响应于开启储物柜的操作，获取当前时间储物柜内的多帧图像数据；将开启次数增加第一数值；对该多帧图像数据进行图像识别；若开启次数大于预设阈值，且当前时间与上一次发送告警信息的时间之间的时间间隔大于预设时长，输出告警信息，并删除记录的开启次数。
基于该实施例，当储物柜输出告警信息之后，为了避免出现频繁告警的问题，每当输出告警信息之后，储物柜可以清空统计的开启次数，如此，后续当柜门被再次打开时，储物柜可以重新开始统计开启次数。
需要指出的是，本申请各实施例之间可以相互借鉴或参考，例如，相同或相似的步骤，方法实施例、系统实施例和装置实施例之间，均可以相互参考，不予限制。
本申请实施例可以根据上述方法示例对储物柜的管理装置进行功能模块或者功能单元的划分，例如，可以对应各个功能划分各个功能模块或者功能单元，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块或者功能单元的形式实现。其中，本申请实施例中对模块或者单元的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图10所示，为本申请实施例提供的一种储物柜的管理装置的结构示意 图，该装置包括：获取单元101及处理单元102、发送单元103。
获取单元101，被配置为：响应于开启储物柜，获取当前时间储物柜内部的多帧图像数据。
处理单元102，被配置为：当前时间与第一时间之间的时间间隔大于预设时长，且当前时间与第一时间之间储物柜的开启次数大于预设次数，根据多帧图像数据，确定储物柜内是否存在异物，第一时间为当前时间之前发送告警信息的时间，告警信息用于提示储物柜内存在异物。
发送单元103，被配置为：若储物柜内存在异物，且当前时间与第一时间之间确定储物柜内存在异物的次数大于预设次数，发送告警信息。
在一些实施例中，获取单元101，具体被配置为：当储物柜的柜门开启角度位于预设角度范围，且柜门的角加速度位于预设角加速度范围时，拍摄储物柜内的多帧图像数据。
在一些实施例中，预设角度范围为45度～50度之间的任一角度范围，预设角加速度范围为0～10度/秒平方之间的任一角加速度范围。
在一些实施例中，处理单元102，具体被配置为：获取多帧图像数据中每帧图像数据中的物品的特征信息，根据多帧图像数据的物品的特征信息以及预设特征信息库，确定储物柜内是否存在异物，预设特征信息库包括多个属于储物柜内的物品的特征信息，异物是指不属于储物柜内的物品。
在一些实施例中，处理单元102，具体被配置为：若多帧图像数据包括的全部物品中存在与预设特征信息库中任意物品的特征信息之间的相似度均小于第一预设值的物品，且该物品被识别到的总次数大于或等于第一预设数值，则确定储物柜内存在异物；若多帧图像数据包括的全部物品的特征信息与预设特征信息库中的物品的特征信息之间的相似度均大于或等于第一预设值，或者该物品被识别到的总次数小于第一预设数值，则确定储物柜内不存在异物。
在一些实施例中，针对多帧图像数据中的任一帧图像数据，处理单元102，还被配置为：当异物特征信息库包括至少一个异物的特征信息时，若确定该图像数据中存在异物，且该图像数据中异物的特征信息与异物特征信息库中的异物的特征信息之间的相似度大于第二预设值，则将该异物被识别到的次数增加第一数值，得到异物被识别到的总次数；若该图像数据中异物的特征信息与异物特征信息库中的异物的特征信息之间的相似度小于或等于第二预设值，则将该图像数据中的异物的特征信息写入异物特征信息库。异物特征信息库包括多帧图像数据中拍摄时间位于该图像数据之前的图像数据的异物 的特征信息。
在一些实施例中，若异物特征信息库为空，且图像数据存在异物，则将该图像数据中的异物特征信息写入异物特征信息库。
在一些实施例中，处理单元102，还被配置为：若当前时间与第一时间之前的时间间隔小于或等于预设时长，或当前时间与第一时间之间储物柜的开启次数小于或等于第一预设次数，则删除多帧图像数据。
在一些实施例中，处理单元102，还被配置为：若多帧图像数据的数量大于第一预设数量，则删除多帧图像数据中拍摄时间最早的至少一帧图像数据，删除后的图像数据的数量小于或等于第一预设数量。
在一些实施例中，处理单元102，具体被配置为：响应于关闭储物柜，根据多帧图像数据，确定储物柜内是否存在异物。
在一些实施例中，若多帧图像数据包括多个异物且所述多个异物不同，处理单元102，具体被配置为：若第一异物被识别到的总次数大于第三预设次数，则确定储物柜内存在异物，第一异物为多个异物中被识别的次数最多的异物。
在一些实施例中，多帧图像数据中任意两帧图像数据对应的拍摄角度不同。
在通过硬件实现时，本申请实施例中的获取单元101和发送单元103可以集成在通信接口上，处理单元102可以集成在处理器上。具体实现方式如图11所示。
图11示出了上述实施例中所涉及的储物柜的管理装置的又一种可能的结构示意图。该通信装置包括：处理器1102和通信接口1103。处理器1102用于对装置的动作进行控制管理，例如，执行上述处理单元102执行的步骤，和/或用于执行本文所描述的技术的其它过程。通信接口1103用于支持装置与其他网络实体的通信，例如，执行上述获取单元101执行的步骤。该装置还可以包括存储器1101和总线1104，存储器1101用于存储装置的程序代码和数据。
其中，存储器1101可以是该装置中的存储器等，该存储器可以包括易失性存储器，例如随机存取存储器；该存储器也可以包括非易失性存储器，例如只读存储器，快闪存储器，硬盘或固态硬盘；该存储器还可以包括上述种类的存储器的组合。
上述处理器1102可以是实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。该处理器可以是中央处理器，通用处理器， 数字信号处理器，专用集成电路，现场可编程门阵列或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。该处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等。
总线1104可以是扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。总线1104可以分为地址总线、数据总线、控制总线等。为便于表示，图11中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
图11中的装置还可以为芯片。该芯片包括一个或两个以上(包括两个)处理器1102和通信接口1103。
可选的，该芯片还包括存储器1105，存储器1105可以包括只读存储器和随机存取存储器，并向处理器1102提供操作指令和数据。存储器1105的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。
在一些实施方式中，存储器1105存储了如下的元素，执行模块或者数据结构，或者他们的子集，或者他们的扩展集。
在本申请实施例中，通过调用存储器1105存储的操作指令(该操作指令可存储在操作系统中)，执行相应的操作。
本公开的一些实施例提供了一种计算机可读存储介质(例如，非暂态计算机可读存储介质)，该计算机可读存储介质中存储有计算机程序指令，计算机程序指令在计算机(例如，储物柜)上运行时，使得计算机执行如上述实施例中任一实施例所述的储物柜的管理方法。
示例性的，上述计算机可读存储介质可以包括，但不限于：磁存储器件(例如，硬盘、软盘或磁带等)，光盘(例如，CD(Compact Disk，压缩盘)、DVD(Digital Versatile Disk，数字通用盘)等)，智能卡和闪存器件(例如，EPROM(Erasable Programmable Read-Only Memory，可擦写可编程只读存储器)、卡、棒或钥匙驱动器等)。本公开描述的各种计算机可读存储介质可代表用于存储信息的一个或多个设备和/或其它机器可读存储介质。术语“机器可读存储介质”可包括但不限于，无线信道和能够存储、包含和/或承载指令和/或数据的各种其它介质。
本公开的一些实施例还提供了一种计算机程序产品，例如，该计算机程序产品存储在非瞬时性的计算机可读存储介质上。该计算机程序产品包括计 算机程序指令，在计算机(例如，储物柜)上执行该计算机程序指令时，该计算机程序指令使计算机执行如上述实施例所述的同步方法。
本公开的一些实施例还提供了一种计算机程序。当该计算机程序在计算机(例如，储物柜)上执行时，该计算机程序使计算机执行如上述实施例所述的储物柜的管理方法。
上述计算机可读存储介质、计算机程序产品及计算机程序的有益效果和上述一些实施例所述的储物柜的管理方法的有益效果相同，此处不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、设备和方法，可以通过其它的方式实现。例如，以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (19)

  1. 一种储物柜的管理方法，其中，该方法包括：
    响应于开启所述储物柜，获取当前时间所述储物柜内的多帧图像数据；所述储物柜用于储存物品；
    若所述当前时间与第一时间之间的时间间隔大于预设时长，且所述当前时间与所述第一时间之间所述储物柜的开启次数大于第一预设次数，则根据所述多帧图像数据，确定所述储物柜内是否存在异物；所述第一时间为所述当前时间之前发送告警信息的时间，所述告警信息用于提示所述储物柜内存在异物；
    若所述储物柜内存在异物，且所述当前时间与所述第一时间之间确定所述储物柜内存在所述异物的次数大于第二预设次数，则发送所述告警信息。
  2. 根据权利要求1所述的方法，其中，所述获取当前时间所述储物柜内的多帧图像数据，包括：
    当所述储物柜的柜门的开启角度位于预设角度范围，且所述柜门的角加速度位于预设角加速度范围时，多次拍摄所述储物柜内的图像，得到所述多帧图像数据。
  3. 根据权利要求2所述的方法，其中，所述预设角度范围为45度～50度之间的任一角度范围，所述预设角加速度范围为0～10度/秒平方之间的任一角加速度范围。
  4. 根据权利要求1-3任一项所述的方法，其中，所述根据所述多帧图像数据，确定所述储物柜内是否存在异物，包括：
    获取所述多帧图像数据中每帧图像数据的物品的特征信息；
    根据所述多帧图像数据的物品的特征信息以及预设特征信息库，确定所述储物柜内是否存在异物；所述预设特征信息库包括多个属于所述储物柜内的物品的特征信息，所述异物是指不属于所述储物柜内的物品。
  5. 根据权利要求4所述的方法，其中，所述根据所述多帧图像数据的物品的特征信息以及预设特征信息库，确定所述储物柜内是否存在异物，包括：
    若所述多帧图像数据包括的全部物品中存在与所述预设特征信息库中任一物品的特征信息之间的相似度均小于第一预设值的物品，且所述物品被识别到的总次数大于或等于第一预设数值，则确定所述储物柜内存在异物；
    若所述多帧图像数据包括的全部物品的特征信息与所述预设特征信息库中的物品的特征信息之间的相似度均大于或等于所述第一预设值，或者所述物品被识别到的总次数小于所述第一预设数值，则确定所述储物柜内不存在异物。
  6. 根据权利要求5所述的方法，其中，针对第一图像数据，所述第一图像数据为所述多帧图像数据中的任一图像数据，所述方法还包括：
    当异物特征信息库包括至少一个异物的特征信息时，若确定所述第一图像数据中存在异物，且所述第一图像数据中异物的特征信息与所述异物特征信息库中的异物的特征信息之间的相似度大于第二预设值，则将所述异物被识别到的次数增加第一数值，得到所述异物被识别到的总次数；若所述第一图像数据中异物的特征信息与所述异物特征信息库中的异物的特征信息之间的相似度小于或等于所述第二预设值，则将所述第一图像数据中的异物的特征信息写入所述异物特征信息库；
    其中，所述异物特征信息库包括所述多帧图像数据中拍摄时间位于所述第一图像数据之前的图像数据中被识别到的异物的特征信息。
  7. 根据权利要求6所述的方法，其中，所述方法还包括：
    若所述异物特征信息库为空，且所述第一图像数据存在异物，则将所述第一图像数据中异物的特征信息写入所述异物特征信息库。
  8. 根据权利要求1-7任一项所述的方法，其中，所述方法还包括：
    若所述当前时间与所述第一时间之间的时间间隔小于或等于所述预设时长，或所述当前时间与所述第一时间之间所述储物柜的开启次数小于或等于所述第一预设次数，则删除所述多帧图像数据。
  9. 根据权利要求1-8任一项所述的方法，其中，所述方法还包括：
    若所述多帧图像数据的数量大于第一预设数量，则删除所述多帧图像数据中拍摄时间最早的至少一帧图像数据，删除后的图像数据的数量小于或等于所述第一预设数量。
  10. 根据权利要求1-9任一项所述的方法，其中，所述根据所述多帧图像数据，确定所述储物柜内是否存在异物，包括：
    响应于关闭所述储物柜，根据所述多帧图像数据，确定所述储物柜内是否存在异物。
  11. 根据权利要求5所述的方法，其中，若所述多帧图像数据包括多个异物且所述多个异物不同，所述确定所述储物柜内存在异物，包括：
    若第一异物被识别到的总次数大于第三预设次数，则确定所述储物柜内存在异物；所述第一异物为所述多个异物中被识别到的次数最多的异物。
  12. 根据权利要求1-11任一项所述的方法，其中，所述多帧图像数据中任意两帧图像数据对应的拍摄角度不同。
  13. 一种储物柜的管理装置，其中，所述装置包括获取单元和处理单元、 发送单元；
    所述获取单元，被配置为：响应于开启所述储物柜，获取当前时间所述储物柜内的多帧图像数据；所述储物柜用于存储物品；
    所述处理单元，被配置为：若所述当前时间与第一时间之间的时间间隔大于预设时长，且所述当前时间与所述第一时间之间所述储物柜的开启次数大于第一预设次数，则根据所述多帧图像数据，确定所述储物柜内是否存在异物；所述第一时间为所述当前时间之前发送告警信息的时间，所述告警信息用于提示所述储物柜内存在异物；
    所述发送单元，被配置为：若确定所述储物柜内存在异物，且所述当前时间与所述第一时间之间确定所述储物柜内存在异物的次数大于第二预设次数，发送所述告警信息。
  14. 一种储物柜的管理装置，其中，包括：处理器和通信接口；所述通信接口和所述处理器耦合，所述处理器用于运行计算机程序或指令，以实现如权利要求1-12任一项所述的储物柜的管理方法。
  15. 一种储物柜，其中，包括如权利要求14所述的管理装置以及摄像头，所述管理装置与所述摄像头连接，所述摄像头设置在所述储物柜的柜门中间位置，且朝向与所述储物柜的柜体方向；
    所述摄像头，用于响应于开启所述储物柜，拍摄所述储物柜内的多帧图像数据。
  16. 根据权利要求15所述的储物柜，其中，所述储物柜还包括角度传感器，所述角度传感器设置在所述储物柜的柜门与柜体的连接处；
    所述角度传感用于检测所述储物柜的柜门的开启角度。
  17. 根据权利要求15或16所述的储物柜，其中，所述储物柜还包括一个或多个灯带，所述一个或多个灯带设置在所述储物柜的门框侧。
  18. 根据权利要求15-17任一项所述的储物柜，其中，所述摄像头为鱼眼摄像头。
  19. 一种非瞬态计算机可读存储介质，其中，所述计算机可读存储介质中存储有指令，当计算机执行所述指令时，所述计算机执行上述权利要求1-12任一项所述的储物柜的管理方法。
PCT/CN2022/140637 2022-12-21 2022-12-21 储物柜的管理方法及装置 WO2024130584A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/140637 WO2024130584A1 (zh) 2022-12-21 2022-12-21 储物柜的管理方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/140637 WO2024130584A1 (zh) 2022-12-21 2022-12-21 储物柜的管理方法及装置

Publications (1)

Publication Number Publication Date
WO2024130584A1 true WO2024130584A1 (zh) 2024-06-27

Family

ID=91587311

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/140637 WO2024130584A1 (zh) 2022-12-21 2022-12-21 储物柜的管理方法及装置

Country Status (1)

Country Link
WO (1) WO2024130584A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018033850A (ja) * 2016-09-02 2018-03-08 グローリー株式会社 ロッカー管理システム、ロッカー管理装置及びロッカー管理方法
CN109872448A (zh) * 2019-01-19 2019-06-11 创新奇智(广州)科技有限公司 一种异物检测方法、计算机可读存储介质及检测系统
CN109959974A (zh) * 2019-01-19 2019-07-02 创新奇智(重庆)科技有限公司 一种异物检测方法、计算机可读存储介质及检测系统
CN113763629A (zh) * 2021-03-25 2021-12-07 北京京东乾石科技有限公司 一种智能售货柜以及检测异物的方法
CN114500832A (zh) * 2021-12-31 2022-05-13 合肥美的智能科技有限公司 储物柜、图像采集方法、可读存储介质和计算机程序产品
CN114724025A (zh) * 2022-03-18 2022-07-08 上海商汤智能科技有限公司 异物检测方法、装置、电子设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018033850A (ja) * 2016-09-02 2018-03-08 グローリー株式会社 ロッカー管理システム、ロッカー管理装置及びロッカー管理方法
CN109872448A (zh) * 2019-01-19 2019-06-11 创新奇智(广州)科技有限公司 一种异物检测方法、计算机可读存储介质及检测系统
CN109959974A (zh) * 2019-01-19 2019-07-02 创新奇智(重庆)科技有限公司 一种异物检测方法、计算机可读存储介质及检测系统
CN113763629A (zh) * 2021-03-25 2021-12-07 北京京东乾石科技有限公司 一种智能售货柜以及检测异物的方法
CN114500832A (zh) * 2021-12-31 2022-05-13 合肥美的智能科技有限公司 储物柜、图像采集方法、可读存储介质和计算机程序产品
CN114724025A (zh) * 2022-03-18 2022-07-08 上海商汤智能科技有限公司 异物检测方法、装置、电子设备及存储介质

Similar Documents

Publication Publication Date Title
WO2019134346A1 (zh) 人脸识别方法、应用服务器及计算机可读存储介质
US20180115749A1 (en) Surveillance system and surveillance method
CN110443833B (zh) 对象跟踪方法和设备
CN104601918B (zh) 视频录制方法和装置
WO2021238194A1 (zh) 室内监控方法、装置、系统、存储介质及摄像装置
EP2998960B1 (en) Method and device for video browsing
CN107122743B (zh) 安防监控方法、装置和电子设备
US10657783B2 (en) Video surveillance method based on object detection and system thereof
Xiang et al. Activity based surveillance video content modelling
CN111259868A (zh) 基于卷积神经网络的逆行车辆检测方法、系统及介质
CN105608462A (zh) 字符相似性的判断方法及装置
CN110569770A (zh) 人体入侵行为识别方法、装置、存储介质及电子设备
CN111666915A (zh) 监控方法、装置、设备以及存储介质
JP2011070629A (ja) 広告効果測定システム及び広告効果測定装置
WO2024130584A1 (zh) 储物柜的管理方法及装置
JP5202419B2 (ja) 警備システムおよび警備方法
JP2011070629A5 (zh)
CN111986229A (zh) 视频目标检测方法、装置及计算机系统
JP5758165B2 (ja) 物品検出装置および静止人物検出装置
US8126212B2 (en) Method of detecting moving object
KR101513180B1 (ko) 감시카메라 영상을 이용한 실시간 결제 이벤트 요약 시스템 및 그 방법
JP2007233495A (ja) 分散型画像処理装置
CN114051624A (zh) 检测游戏区域上游戏道具的方法及装置、设备、存储介质
JPWO2022181575A5 (zh)
KR20120120780A (ko) 통계에 기반한 적응적 감시카메라 시스템 구동 방법 및 시스템