JP2016196981A - Storage shed management system and storage shed management method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a storage shed management system capable of intuitively grasping a stock period of an accommodated article visually through an image.SOLUTION: A storage shed management system is composed of: a storage 100 including an image pick-up portion 101, an image storage portion 103 for storing information on an image, a region extracting portion 102 for extracting a prescribed region by comparing information on a newly picked-up image and the information on stored image, a hysteresis information management portion 105 for managing information relating to change hysteresis of articles accommodated in the storage 100 on the basis of the region extracted by the region extracting portion 102, and a transmitting portion for transmitting the managed information or stored information; and a display terminal 200 including a receiving portion, a visualization processing portion 203 for determining a stock period of an article accommodated in the storage 100 from the received information, and implementing the visualization by image processing according to the stock period, and a display portion 205 for displaying the image processed by the visualization processing portion 203.SELECTED DRAWING: Figure 1

Description

  The present invention relates to management in a storage such as a cold storage or a heat storage.

  When storing items in the cold storage or the storage of the heat storage (hereinafter sometimes referred to only as “storage”), the user remembers all the items that have been in the storage from when. It is difficult to keep. So, for example, in the case of a refrigerator, you forget that the food is left in the warehouse and the expiration date has passed, or you have purchased the same food as the food already in the warehouse. Can happen.

  In view of this, several methods have been devised in which the interior is imaged with an imaging device such as a camera and the interior management is performed based on information obtained from the captured image.

  For example, each time the refrigerator door is closed, the interior is photographed, and the storage position at the time of the previous photographing is compared with the storage position at the time of the current photographing to determine the entry / exit of the article, and the inventory period exceeds the warning period. And an inventory management device that issues a warning by illuminating the shelf or bottom surface in which articles are stored (see Patent Document 1).

  In addition, a refrigerator system is described in which the image inside the store is stored together with the shooting date and time so that the search for the image inside the store can be performed by specifying the date and time (see Patent Document 2).

JP 2010-121784 A JP 2014-196851 A

However, in Patent Document 1, when the inventory period exceeds the warning period, a warning is issued by illuminating the shelf or the bottom where the articles are stored. As the number increases, the number of light-emitting elements must be increased. In addition, even if the number of light-emitting elements is increased, changes in the depth direction of the shelf cannot be detected, so if multiple items are lined up before and after the same shelf, which item's inventory period has exceeded the warning period I can't judge.
In addition, when entering and leaving the goods for each article is determined in comparison with the image taken at the previous time, when a plurality of articles are put in and out and the goods in the warehouse are moved by one opening and closing of the door. It can happen that the state of the inside of the cabinet is completely different between immediately after opening the door and before closing. In such a case, as in Patent Document 1, in the method of comparing the images taken every time the door is opened and closed to determine entry / exit, the change in the storage position is too large to correctly determine entry / exit. . Further, since the inventory period is confirmed by opening the refrigerator door, it is not possible to confirm the storage period of stock items (whether the food is still eatable, for example) from the outside.

  In Patent Document 2, the image in the warehouse can be searched by specifying the date and time. However, in order to utilize this system, the user needs to roughly grasp the date and time of warehousing for each article. It is difficult for the user to memorize the warehousing date and time of the goods for each warehouse, and there is a problem in operability.

  The storage management system of the present invention is a storage management system that includes a storage for storing articles, and a display terminal that receives information transmitted from the storage and displays an image based on the received information. The storage includes an imaging unit that captures an inside, an image storage unit that stores information on an image captured by the imaging unit, information on an image newly captured by the imaging unit, and the image storage unit. Comparing with information of stored images, an area extraction unit that extracts a predetermined area, and information on a change history of articles stored in the storage, based on the area extracted by the area extraction unit An information management unit for managing, and a transmission unit for transmitting information managed by the information management unit or information stored in the image storage unit, wherein the display terminal displays the information transmitted from the transmission unit. A receiving unit for receiving, and the receiving unit; Obtaining an inventory period of the article stored in the storage from the received information, a visualization processing unit that performs visualization by image processing according to the inventory period, and a display unit that displays an image processed by the visualization processing unit; , Are provided.

  According to the present invention, the user can intuitively grasp the inventory period of the articles stored in each region in the warehouse through an image.

It is a block diagram which shows the structure of the storage management system in Embodiment 1 of this invention. It is a flowchart which shows the process of the storage in Embodiment 1 of this invention. It is the figure which showed typically the image which image | photographed the inside of a store | warehouse | chamber. It is a schematic diagram which shows a mode that the image which image | photographed the inside is divided | segmented vertically and horizontally. It is the schematic diagram which showed concretely management of taking in and out of articles | goods. It is a flowchart which shows the flow of a process of the display terminal in Embodiment 1 of this invention. It is the schematic which shows the screen visualized so that a user can grasp | ascertain intuitively visually. It is the schematic which shows the screen visualized so that a user can grasp | ascertain intuitively visually. It is a block diagram which shows the structure of the storage management system in Embodiment 2 of this invention. An example of the process of the article | item area | region extraction part of the warehouse management system in Embodiment 2 of this invention is shown. It is a block diagram which shows the structure of the warehouse management system in Embodiment 3 of this invention.

Embodiment 1 FIG.
Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic and ratios of dimensions and the like are different from actual ones. Therefore, specific dimensions and the like should be determined in consideration of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings.

  FIG. 1 is a block diagram showing the configuration of the warehouse management system according to Embodiment 1 of the present invention. In the figure, the storage management system includes a storage 100, a display terminal 200, and a server 300. Hereinafter, as a storage, a refrigerator will be described as an example, but it is assumed that an object is put in and out in a state of being provided in advance, such as a heat storage, a cold storage, or a cold storage, etc. It is applicable if the conditions described below are satisfied. In addition, storage such as warehouses where people and heavy machinery come and go and each item is covered and managed, or storage that can be carried and the contents are supposed to move to some extent In some cases, the effects of the present invention may not be achieved.

  In addition, although the structure which communicates between the storage 100 and the display terminal 200 via the server 300 is demonstrated here, it is good also as a structure which communicates directly not via the server 300. FIG. Each configuration will be described below.

  The storage 100 is stored in an image capturing unit 101 that captures an image of the interior, an image storage unit 103 that stores and manages images captured by the image capturing unit 101, and an image captured by the image capturing unit 101 and an image storage unit 103. A region extraction unit 102 that compares the image and extracts a different (changed) region (hereinafter may be referred to as a change region), and history information that has changed based on the region extracted by the region extraction unit 102 A history information management unit 105 that manages the information, and a storage communication unit 106 that transmits information of a newly captured image obtained from the image storage unit 103 and history information obtained from the history information management unit 105 to the server 300. Consists of.

  The display terminal 200 includes a terminal communication unit 201 that receives the image information and the history information transmitted from the storage communication unit 106 from the server 300, a storage unit 202 that stores the image information and the history information, and the history information. A visualization processing unit 203 that performs processing for displaying information relating to an inventory period of an article obtained from image information in a visible form, and a change-time information presentation unit 204 that presents details of history information according to a user operation. And a display unit 205 for displaying an image.

  Next, the operation will be described. FIG. 2 is a flowchart showing the processing of the storage in Embodiment 1 of the present invention.

  Step S101 is a step of detecting that the door of the storage 100 is closed. Step S102 is a step of acquiring an image. When the imaging unit 101 detects that the door of the storage 100 is closed, the imaging unit 101 captures an image of the storage 100. While the door is closed, the goods in the storage 100 do not move, so the image taken when the door is closed last becomes an image showing the latest situation in the storage.

  The inside of the cabinet may be photographed with a camera or the like provided inside the door. Although there is no restriction | limiting in particular in the position provided, it is necessary to image | photograph the whole inside of a store | warehouse | chamber. It is better to use a wide-angle lens to photograph the entire interior from the vicinity of the article, but this is not particularly necessary when the entire interior is small. Since the main purpose is not to obtain an image to know the detailed state of the article but to detect a change in the position of the article, the number of pixels required for the image sensor of the camera is not large. A high-definition camera may be used.

  In addition, when shooting after the door is closed, it is assumed that the interior is dark, so a camera equipped with some lighting equipment is preferable, but in particular, in a refrigerator in advance, such as a refrigerator. In the case where an illumination device is provided, it is also possible to perform settings such that shooting is performed in conjunction with the provided illumination device.

  Further, here, the setting is made such that a picture is taken every time the door is closed, but the moment when the door is opened may be taken. Further, a continuous image may be taken while the door is opened or closed, and a three-dimensional image may be constructed based on a plurality of images. In such a case, a certain shutter speed is required.

  If the refrigerator is already equipped with a sensor or the like, it is sufficient to use a function provided in advance. However, if the storage does not have lighting etc. in the storage, there may be products that do not have such a function. In such a case, a separate contact sensor or light sensor is provided on the door. Then, the opening / closing of the door may be detected.

  FIG. 3 is a diagram schematically illustrating an image obtained by photographing the interior obtained in steps S101 and S102. The figure shows a case where articles F01 to F05 are placed on the respective shelves T01 to T04.

  Step S103 is a step in which the region extraction unit 102 compares a newly captured image with the previously captured image and extracts a difference pixel that is a pixel whose difference is detected. The previously captured image is stored in the image storage unit 103.

  For example, when the difference area is extracted by comparing the newly captured image and the previously captured image for each corresponding pixel, for example, each pixel is composed of RGB and is indicated by a gradation of 8 bits. This can be done by calculating the absolute value of the gradation value difference for each RGB and extracting pixels whose sum is equal to or greater than a certain threshold Thd. In this case, in order to reduce the influence of noise, it is desirable to perform expansion processing and / or contraction processing of the difference pixels. A process of changing all non-candidate pixels (pixels that are not difference pixels in the present invention) within a predetermined expansion range to candidate pixels with reference to a certain candidate pixel (difference pixels in the present invention) is performed on all difference pixels. It is processing. Further, the contraction process is a process in which a process for changing all candidate pixels within a predetermined contraction range to a non-candidate pixel with respect to a certain non-candidate pixel is performed on all the non-candidate pixels. As the expansion range and the contraction range, for example, a 3 × 3 range is used.

  Step S <b> 104 is a step of extracting an area where a certain number or more of pixels are recognized by the area extraction unit 102. FIG. 4 is a schematic diagram showing a state in which an image taken inside the warehouse is divided vertically and horizontally. Whether or not the pixel is a region where a difference is recognized is determined based on the region obtained by dividing the captured image (hereinafter, only referred to as a divided region), and the area of the pixel that has changed more than a certain value within the region. Judgment by occupying Here, as the divided area, an area in which the vertical is divided into four and the horizontal is divided into five is illustrated. Here, although the divided areas are represented by the same area, for example, the middle part of the shelf on which articles are concentrated and placed is subdivided to have a smaller area, and divided into larger areas as it goes up and down or around. The entire screen may be composed of divided areas having different areas.

  Step S105 is a step of updating the image that has been shot but is stored in the image storage unit 103 using the image after extracting the change area. The updated image is transmitted to the server as a new photographed image, and is used as a comparative image at the time of difference extraction at the next photographing. Here, the image data recording / updating work is performed by using the newly captured image data to update the previously captured image data. However, when the data is updated in this way, the image storage is performed. There is an advantage that the unit 103 can be configured with a small amount of memory. If there is a margin in memory, it is possible to know a long-term change history based on stored image data by storing multiple images with different shooting times instead of replacing images. .

  In step S106, the history information management unit 105 updates the change history information based on the changed region extracted by the region extracting unit 102. The history information about the change area of the image (hereinafter sometimes referred to only as change history information) includes a last change date and time table that is a table showing the date and time of the last change for each divided area and a change time that is an image at the time of change. Consists of images.

  The last change date and time table is a table representing the last change date and time for each divided area. The last change date and time is the date and time when the region extraction unit 102 determined that the last change occurred in the divided region. The inventory period represents a period in which there is no change in the divided area, and is managed in the format of the last change date.

  The image at the time of change represents an image obtained by cutting out the image of the divided area from each of the photographed image before the change when the change occurs in the divided area and the photographed image after the change. Hereinafter, they are referred to as a pre-change divided region image and a post-change divided region image, respectively.

  FIG. 5 is a schematic diagram specifically showing how to divide an image area and manage the loading and unloading of articles. FIG. 5A is an image diagram showing an image taken last time. FIG. 5B is an image diagram showing an image taken this time. FIG. 5C is an image diagram showing a difference area. Here, the portions shown in white are areas recognized as differences. In this example, since the article F02 has been received, that portion is displayed as a white area as a difference area. FIG. 5D is an image diagram showing the positional relationship between the difference area and the divided area.

  If the criterion for determining that there is a change in the divided region is that the ratio of the “area of the difference region” to the “area of the divided region” is 20% or more, the divided region R2-4 satisfies the criterion. Therefore, it can be regarded as an area where there has been a change. Therefore, the history information management unit 105 updates the last change date and time of the divided area R2-4 to the date and time when the current image was taken, and corresponds to the divided area R2-4 from FIG. The pre-change divided region image (FIG. 5E) obtained by extracting the image of the region, and the post-change divided image that is an image obtained by extracting the region corresponding to the divided region R2-4 from FIG. The area image (FIG. 5F) is stored in correspondence with the divided area R2-4.

  Step S <b> 107 is a step in which the storage communication unit 106 transmits information of a newly captured image obtained from the image storage unit 103 and history information obtained from the history information management unit 105 to the server 300.

  FIG. 6 is a flowchart showing a process flow of the display terminal according to Embodiment 1 of the present invention. Hereinafter, the processing of the display terminal 200 will be described with reference to the drawings.

  Step S201 is a step in which the terminal communication unit 201 of the display terminal 200 receives new photographed image information and change history information from the server 300, for example, triggered by a user operation such as a button press. At this time, from the viewpoint of performing efficient processing, it is preferable to inquire of the server whether there is unacquired information, and to proceed only when there is unacquired information.

  Further, the reception processing may be performed periodically by the terminal communication unit 201 even when there is no user operation. Moreover, you may perform when the application for the management in a store | warehouse | chamber which the display terminal 200 preserve | saves starts.

  Step S202 is a step of editing the image so that the user can intuitively grasp the inventory period based on the last change date and time table in the change history information received by the visualization processing unit 203. As an example of editing, red is superimposed on the entire captured image in the divided area, and the user does not put in or out the article by performing image processing so that the red color becomes darker as the period without change is longer A region with a long period can be recognized at a glance.

  Here, the darkness of the color is set to represent the length of the inventory period, but regardless of the color density, the length of the inventory period can be increased by changing the hue, changing the hue, or changing the brightness. You may make it express.

  In addition, here, the shade of color was changed only in the inventory period regardless of the shelf or the item, but for example, an article having a relatively long storage period, a food stored in the refrigerator, If foods with a long shelf life are placed on the top and bottom shelves, and foods with a short shelf life are placed on the center shelf, the shelf is placed above and below the center. You may make it change the period which a color changes with the arrange | positioned shelf. By setting in this way, it becomes easy to manage the shelf life of articles for each shelf, and a finer response is possible.

  In addition, when storing food on a shelf, it may be possible to make a setting that changes the color in consideration of the period corresponding to the expiration date of the food for the divided area where the food is placed. . By adopting a configuration that can be set in this way, finer settings corresponding to the storage deadline for each article to be stored are possible.

  Step S203 is a step of displaying a screen after the visualization processing unit 203 performs image processing of the captured image.

  Step S204 is a step of selecting a divided area from the displayed screen. For example, when the display unit 205 of the display terminal 200 is a liquid crystal panel having a contact sensor on the surface, the user can select a certain coordinate on the screen by touching the surface of the display unit 205.

  Step S205 is a step of displaying change history information related to the divided area including the coordinates selected in step S204. As an example, the last change date and time, the pre-change divided region image, and the post-change divided region image related to the divided region are displayed on the display unit 205. Further, when no divided area is selected, the change history information can be displayed in detail in each divided area with a touch on the screen as a trigger. Alternatively, one group may be formed from a plurality of divided areas, and change history information regarding the divided areas included in the group may be displayed by being enlarged so that the entire group is displayed on the entire display unit 205. Further, all the change history information related to the divided areas may be sequentially displayed while being shifted in time.

  FIG. 7 is a schematic diagram showing a screen visualized so that the user can intuitively grasp it visually. FIG. 7A shows an example in which change history information is displayed in all divided areas. Here, in the change history information, the storage period, that is, the case of the difference between the current date and time and the last change date and time of each divided area in the last change date and time table is shown. FIG. 7B is a diagram showing an image when the color to be superimposed on the photographed image is darkened on a daily basis. The image output by the visualization processing unit 203 is as follows. Here, it is assumed that all the colors to be overlaid on the divided areas having no change for five days or more are the same. From the image shown in FIG. 7B, it can be seen at a glance that the object F02 has recently been received, but the object F05 has been received at least five days ago.

  FIG. 8 is a schematic diagram showing a screen visualized so that the user can intuitively grasp it visually. A display example for displaying the change history information is shown in FIG. For example, when the user selects the region R2-4 shown in FIG. 4 by a touch operation or the like, the date and time when the change time information presentation unit 204 has changed, here, 2014/1/1 12:00, The pre-change divided area image and the post-change divided area image of the date and time are displayed as a pop-up screen P01. Thereby, the user can easily confirm when and how the area designated by the user has changed. When the user selects a different divided area during the pop-up screen display, the current pop-up screen P01 is deleted and a new pop-up screen is displayed. Note that the current pop-up screen P01 may be displayed in an overlapping manner without being erased.

  As described above, in the warehouse management system according to the present embodiment, the user can intuitively grasp the inventory period according to the last change date for each divided area, so the user views the edited warehouse image. The inventory period of each area in the warehouse can be grasped immediately.

  In addition, the user can grasp not only articles that do not move but also shelf boards that are not being used. For example, when the user is storing unevenly in the refrigerator, the shelf indicated by the divided area is used by displaying the divided area where the shelf is present in dark red indicating that there is no change for a long time. Since you can see the period when it is not, the cooling efficiency will drop if you store it unevenly, which will improve it. Thereby, the user can give the information for cooling the inside efficiently by using the inside equally.

  Moreover, since the information of the captured image and the change history information in the storage are transmitted from the storage to the display terminal, it is possible to grasp the state in the storage without opening the storage door. Therefore, it is not necessary to open and close the door for confirmation, and an energy saving effect can be obtained.

  Further, since inventory management is performed through the display screen of the display terminal, it is not necessary to provide a light emitting element or the like on the shelf of the storage, it is not necessary to provide a complicated structure in the storage, and it is easy to manufacture. Furthermore, when the number or size of the divided areas is changed, it is not necessary to change the storage itself, and it can be dealt with by changing the control software.

  Moreover, depending on the position where the camera is installed, for example, a position where a predetermined shelf is photographed obliquely from above or below, or a position where the predetermined shelf is photographed from the left and right directions, the photographing is performed not only from the front but also in the depth direction of the shelf. If a three-dimensional divided area is set, each inventory period can be presented even when articles are arranged before and after the shelf. Note that the number of cameras is not limited to one, and two, three, or more cameras may be installed, or may be installed in association with each shelf board. In this case, the image storage unit may manage images captured for each camera in association with each camera, or may combine them and manage them as one image.

  In addition, it does not detect the change in dust such as the contour of each item, but detects the rate of change in the area where the item is placed, so a control device such as a microcomputer is required. The amount of calculation can be reduced, and the capacity of a storage device such as a memory can be reduced. Furthermore, a camera with a small number of pixels can be used depending on the processing method or the setting method.

  Further, in the present embodiment, since the change history information is displayed for the divided areas in the in-store shot image, the user can grasp not only intuitive grasp but also specific detailed information. Since the user can easily confirm whether the visualized inventory period is really correct, the user can use the system with peace of mind.

  Furthermore, in the warehouse management system in the present embodiment, only the latest photographed image information and change history information are uploaded to the server, so all image information photographed in the past or photographed within a certain period in the past. Compared with the case where a plurality of pieces of image information are stored in the server, the capacity of the storage area of the server can be reduced. For example, when a 4 × 5 divided area is provided, the data stored in the server includes the latest captured image, the last change date / time table, 20 pre-change divided area images, and 20 post-change divided area images. Can be limited. In addition, since the amount of communication between the display terminal and the server can be suppressed, the time required to display the display terminal can also be shortened. If there is a sufficient storage capacity in the server, all image information taken in the past or a plurality of pieces of image information taken within a certain period in the past may be stored in the server. Needless to say.

  In addition, although the example which transmits a picked-up image whenever a storage image | photographs the inside of a store | warehouse | chamber was shown in the storage management system in this Embodiment, you may make it transmit an image according to the request | requirement from a display terminal. Good. By doing so, the time required to display an image on the display terminal may increase, but the amount of communication with the server can be reduced. In some cases, this leads to cost reduction.

  Further, instead of transmitting to the server every time shooting is performed, it may be transmitted after a certain time has elapsed after shooting. In this way, when the door is opened and closed a plurality of times in a short time, transmission to the server can be repeated to prevent an increase in the processing load on the server.

  In the present embodiment, the presentation to the user is performed in units of divided areas and pre-change divided area images and post-change divided area images, but the entire internal image may be presented. As a display method, for example, only a change date and time is displayed as a pop-up when a divided area is selected, and when the user selects a pop-up, a transition is made to display the entire image before and after the change. By doing so, the amount of communication between the storage and the display terminal is increased, but the user can grasp the entire state of the storage at the time of change.

  In the present embodiment, the display terminal displays a captured image in which the inventory period is visualized, but an image in which the inventory period is visualized and an image in which the inventory period is not visible (an image as it is) can be switched by a key operation or the like. Good. In this way, the display terminal needs to store both the image with the inventory period visualized and the image with the inventory period not visible, but the user can more easily check the internal situation.

  Of course, it is possible to make the step of darkening the overlapping color finer or increase the number of divided areas. Moreover, you may make it visualize using a difference (a several color) of a color according to a preservation | save period irrespective of the lightness or darkness of a color. By doing so, the amount of calculation and the amount of communication increase, but the user can more easily recognize changes for each object. However, if it seems that it becomes difficult to identify an article if the divided area is too fine, it is preferable that the divided area has a certain size.

  Further, the color may be changed stepwise near the boundary between adjacent divided regions. By doing so, it is possible to reduce a situation where the color suddenly changes between adjacent divided regions and the user feels uncomfortable.

  In the present embodiment, the inventory period is visualized when a new photographed image and change history information are received. However, the inventory period may be visualized. For example, the inventory period may be recalculated by the user pressing the update button. In this way, information reflecting the current date and time can be reflected on the display at an arbitrary timing without receiving new information.

  In this embodiment, only one image is handled as an image in the storage. However, when the image is divided into a plurality of rooms such as a refrigerator room, a vegetable room, and a freezer room, for example, A camera may be installed in the room and processed in the same way. However, since it is assumed that the freezer compartment and the like are considerably cold, it goes without saying that a camera that operates at a low temperature is necessary. Further, in a place where food materials are stacked and stored like a freezer, detailed information during change (while food is being taken out) may be required.

Embodiment 2. FIG.
In the first embodiment, the inventory period is visualized regardless of whether or not the article in the divided area exists. However, in the second embodiment, the inventory period only in the area where the article exists is visualized. Is different from the first embodiment.

  FIG. 9 is a block diagram showing the configuration of the warehouse management system according to Embodiment 2 of the present invention. The display device 209 has a point having an article region extraction unit 206 that extracts a region where an article is present from the photographed image, and a point that the visualization processing unit 207 visualizes using the article region extracted by the article region extraction unit 206. Different from the display device 200 of the first embodiment.

  FIG. 10 shows an example of processing of the article region extraction unit 206 in the present embodiment. 10A is a photographed image, FIG. 10B is an initial image in a storage room where only a shelf before an article is placed, and FIG. 10C is a difference between the photographed image and the initial image. FIG. 10D shows an example of an output image generated by the visualization processing unit 203. In the first embodiment, the visualization processing unit 203 generates an image as shown in FIG. Here, how the image output by the visualization processing unit 207 changes in this embodiment will be described. FIG. 10B shows an initial image in the storage room in which only the shelf before the article enters is captured with respect to the photographed image of FIG. The initial image is stored in a storage area provided in the image storage unit 103 at the time of factory shipment, for example.

  The article region extraction unit 206 generates FIG. 10C, which is an image obtained by extracting the difference between the newly captured image of FIG. 10A and the initial image of FIG. The extraction of the difference can be performed, for example, by the same processing as the region extraction unit 102 in the first embodiment. A region displayed in white in FIG. 10C indicates a region having a difference from the initial image, that is, a region where an article exists.

  Subsequently, the visualization processing unit 207 edits the new photographed image so that the color to be overlaid becomes dark according to the inventory period as in the in-house management system in the first embodiment. However, a region where colors are superimposed is limited to a region displayed in white in FIG. 10C, that is, a region where an article exists. That is, the image generated by the visualization processing unit 203 is as shown in FIG.

  As described above, in the present embodiment, since the inventory period is visualized only in the region where the article exists, it becomes easier for the user to grasp an article that has not changed for a long time, that is, an article that has been stored for a long time.

  Further, in this embodiment, since the inventory period is visualized using only the change for each divided area and the difference from the initial image, compared to a system that performs management for each article at the time of entering and leaving. The inventory period can be visualized with a small amount of calculation. Also in the present embodiment, the inventory period is managed based on the divided areas as in the first embodiment, but this is used to limit the area for visualizing the inventory period to the article area.

  In FIG. 10A and the like, the number of articles is very small for simplicity, but there may be a case where articles such as food are contained without gaps in the actual warehouse. In such a case, if the inventory period is managed in the difference area, if the article is large and overlaps with another small article, the area may be divided despite the same article, and the inventory period may be displayed differently. possible. In such a case, such a situation can be prevented by using a large divided region in advance. In some cases, such as extracting the difference between the contours of the image, it may not be possible to accurately grasp the shape of the object. However, if the idea of the divided area is used, for example, a criterion for determining that there is a change in the divided area is used. When the ratio of the “area of the difference area” to the “area of the divided area” is 20% or more, it is possible to cope with the case where the shape of the difference area is unclear.

  In the present embodiment, the color is overlapped only in the region where the article is present. However, the region where there is no change from the initial image, that is, the region where the article does not exist may be overlaid with a different color depending on the inventory period. By doing so, it is possible to easily grasp not only the articles stored for a long period of time but also the areas that have not been used for a long period of time.

Embodiment 3 FIG.
In the above embodiment, the period after the change has occurred in the divided area is visualized. However, in this third embodiment, the method for visualizing the inventory period is changed depending on whether the change is inbound or outbound. This is different from the above embodiment.

  FIG. 11 is a block diagram showing the configuration of the warehouse management system according to Embodiment 3 of the present invention. The storage 100 has a point having an entry / exit determination unit 107 that determines whether an article has been added or removed for each change area, and whether the visualization processing unit 208 of the display terminal 210 has been added or not is removed. It is different from the display device 200 of the first embodiment in that it is visualized so that it can be distinguished.

  The entry / exit determination unit 107 determines whether the change for each region extracted by the region extraction unit 102 is due to the addition of an article or the removal of an article. Whether or not the change in the change region is due to the addition of an article is determined based on the post-change divided image and the contour image of the difference region extracted by the region extraction unit 102. The determination as to whether or not the change in the change region is due to the removal of the article is performed based on the pre-change divided image and the contour image of the difference region.

  For example, when the contour extracted from the difference area is similar to the contour extracted from the pre-change divided image, the article shown in the position of the difference area disappears from the position in the storage box corresponding to the position of the difference area. It can be determined that it has been (removed). Further, when the contour extracted from the difference area is similar to the contour extracted from the post-change divided image, an article that appears in the position of the difference area appears at the position in the storage box corresponding to the position of the difference area. (Added) can be determined.

  Here, “similar” to the outline means that the degree of similarity is greater than or equal to a predetermined threshold, and includes the same case. The contour similarity referred to here is determined based on, for example, the ratio of the contour extracted from the difference area to the contour extracted from the pre-change divided image.

  Information indicating whether an article has been added or removed for each change area is added as change history information and transmitted from the storage 100 to the display terminal 210.

  The visualization processing unit 208 edits a new photographed image so that the color to be overlaid becomes dark according to the inventory period. However, the divided area where the article is added and the divided area where the article is removed are edited so as to have different colors. For example, the color is overlaid so that the added divided area becomes darker as the period after the addition is longer, and the red becomes darker as the period after the removal is removed becomes longer. .

  As described above, in the present embodiment, the inventory period is visualized so that it can be distinguished from the period after the article is added or the period after the article is removed. , The unused period of the shelf, etc.).

  In addition, the user distinguishes when an article in front is removed from a state in which a plurality of articles are overlapped in the same divided area, and when another article is added in front of one article. Can do.

  In this embodiment, the colors are superimposed on each of the divided areas where the warehousing is performed and the divided areas where the warehousing is performed. However, the color is superimposed on only one of the areas, or the color is changed by a user operation. You may comprise so that the area | region to overlap can be switched. By configuring in this way, it is possible to display only the information that the user wants to know, making it easier to grasp the information.

DESCRIPTION OF SYMBOLS 100 Storage container 101 Image | photographing part 102 Area extraction part 103 Image storage part 105 History information management part 106 Storage part communication part 107 Entry / exit determination part 200,209,210 Display terminal 201 Terminal communication part 202 Storage part 203,207,208 Visualization process Section 204 Change information presentation section 205 Display section 206 Article area extraction section 300 Server

Claims (7)

A storage management system comprising a storage for storing articles, and a display terminal for receiving information transmitted from the storage and displaying an image based on the received information,
The storage is
A shooting section for shooting the inside;
An image storage unit for storing information of an image captured by the imaging unit;
A region extraction unit that extracts a predetermined region by comparing information of an image newly captured by the imaging unit with information of an image stored in the image storage unit;
An information management unit that manages information related to the change history of articles stored in the storage, based on the region extracted by the region extraction unit;
A transmission unit for transmitting information managed by the information management unit or information stored in the image storage unit;
With
The display terminal is
A receiver for receiving information transmitted from the transmitter;
A visualization processing unit that obtains an inventory period of an article stored in the storage from the information received by the reception unit, and performs visualization by image processing according to the inventory period;
A display unit for displaying an image processed by the visualization processing unit;
A storage management system characterized by comprising: Visualization by image processing according to the inventory period is
The warehouse management system according to claim 1, wherein the image is displayed by superimposing a color on the image captured by the imaging unit. Visualization by image processing according to the inventory period is
The change history of the articles stored in the storage is performed so that it is possible to determine whether the storage into the storage is different from the storage from the storage. The storage management system described. Visualization by image processing according to the inventory period is
The warehouse management system according to any one of claims 1 to 3, wherein an image captured by the imaging unit is divided into a plurality of regions. Visualization by image processing according to the inventory period is
The in-box management system according to any one of claims 1 to 4, wherein the information is performed by superimposing and displaying character information on an image photographed by the photographing unit. The change history information management unit of the storage is
When the change area is extracted by the change time information extraction unit, the change time information that is information in the warehouse at that time is stored in association with the change area,
The transmission unit of the storage transmits the change time information,
The receiving unit of the display device receives the change time information,
6. The display device according to claim 1, further comprising a change time information presentation unit that displays the change time information on the display unit in association with each region of the inventory period visualization image. The in-house management system according to claim 1. A storage management method in a storage management system comprising a storage for storing articles, and a display terminal that receives information transmitted from the storage and displays an image based on the received information,
Photographing the inside of the storage with a photographing unit;
Storing information of an image captured by the imaging unit in an image storage unit;
Comparing the information of the image newly captured by the imaging unit with the information of the image stored in the image storage unit, and extracting a predetermined region by the region extraction unit;
Based on the region extracted by the region extraction unit, the information management unit manages information related to the change history of the articles stored in the storage;
A transmission unit transmitting information managed by the information management unit or information stored by the image storage unit;
Receiving the information transmitted from the transmission unit by the display unit at a reception unit;
Obtaining an inventory period of an article stored in the storage from the information received by the receiving unit, and a visualization processing unit performing visualization by image processing according to the inventory period;
A display unit displaying an image processed by the visualization processing unit;
An in-house management method comprising:

Images