JP2015155345A - storage management system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a storage management system with which a storage position of a storage object can be easily recognized, in a storage where a user himself/herself can go in and out.

SOLUTION: A storage management system includes: a camera arranged in a storage inside which a storage object 3 can be stored, to capture an image inside the storage; a control device which stores an image captured by the camera, calculates a change with time of a face width of a user 2 in the stored image, detects that the user 2 stops and stands on the basis of the change with time of the face width of the user 2, estimates a position where the user 2 stops and stands, on the basis of the face width at the time the user 2 stops and stands in the stored image, estimates the storage position of the storage object 3 on the basis of the position where the user 2 stops and stands, and stores the storage position; and a portable terminal capable of searching the storage position stored by the control device.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to a technology of a storage management system that manages storage items stored in a storage, and more particularly to a technology of a storage management system that manages so that the storage position of storage items can be easily grasped. .

  Conventionally, the technology of a storage management system for managing stored items stored in a storage is well known. For example, as described in Patent Document 1.

  Patent Document 1 discloses a plurality of drawers that store stored items (articles), a camera that captures the interior of the drawer that has moved to a predetermined position, and a storage unit that stores an internal image of the drawer captured by the camera as image information. And a storage management system (storage with search function) including a display device that displays image information stored in a storage unit.

  In the storage management system configured as described above, the inside of the drawer in which the stored items are taken in and out is photographed by the camera, and the internal image is stored as image information. Thereby, a drawer in which a desired storage item is stored can be searched based on the image information at a later date, and the storage position of the storage item can be easily grasped.

  However, the technique described in Patent Document 1 can be applied only to a storage that stores relatively small storage items. That is, it cannot be applied to a large storage (so-called walk-in closet or the like) in which a user who takes in and out the stored items can enter and exit. For this reason, in such a large storage where the user himself can go in and out, it is difficult to manage the stored items (ascertain the storage position of the stored items).

JP 2006-103852 A

  The present invention has been made in view of the above situation, and the problem to be solved is storage in which the user can easily grasp the storage position of the stored item in a storage that can be accessed by the user himself / herself. It is to provide a management system.

  The problem to be solved by the present invention is as described above. Next, means for solving the problem will be described.

  That is, according to the first aspect of the present invention, the camera is provided in a storage that can store the storage items therein, the camera that captures the inside of the storage, and the image that is captured by the camera are stored. Calculating a temporal change in the size of the user's face, detecting that the user has stopped based on the temporal change in the size of the user's face, and detecting the user in the stored image A control device that estimates the position where the user has stopped based on the size of the face when the user stops, and estimates and stores the storage position of the stored item based on the position where the user stopped And a search device capable of searching for the storage position stored in the control device.

  According to a second aspect of the present invention, when the position where the user has stopped is estimated, the control device also considers information regarding the orientation of the user's face.

  In Claim 3, the said control apparatus specifies and memorize | stores the user image | photographed with the said camera, The said search apparatus stores the said storage position memorize | stored in the said control apparatus based on the specified user. It is searchable.

  According to a fourth aspect of the present invention, a plurality of the cameras are provided at different positions in the storage, and the control device is configured such that a user's face in an image photographed by the camera is predetermined on the left and right with respect to the front. When the image is rotated by an angle or more, the image is not stored.

  According to a fifth aspect of the present invention, the search device is a portable terminal that can be carried.

  As effects of the present invention, the following effects can be obtained.

  According to the first aspect, the storage position of the storage item can be easily grasped.

  According to the second aspect, the storage position of the stored item can be estimated more accurately.

  According to the third aspect of the present invention, it is possible to search the storage position by specifying the user, and the search efficiency can be improved.

  According to the fourth aspect of the present invention, it is possible to store only images that are highly necessary, and to prevent useless storage capacity from being used.

  According to the fifth aspect, the search device can be used not only in the vicinity of the storage but also in an arbitrary place, and the convenience of the search can be improved.

The side surface cross-sectional schematic diagram which showed the whole structure of the storage management system and storage which concerns on one Embodiment of this invention. Similarly, a plane cross-sectional schematic diagram. The block diagram which showed the storage management system. The side surface cross-section schematic diagram which showed immediately after the user entered the room. Similarly, a plane cross-sectional schematic diagram. The plane cross-sectional schematic diagram which showed a mode that the user is moving in the storage. The side surface cross-sectional schematic diagram which showed immediately after the user left the room. Similarly, a plane cross-sectional schematic diagram. (A) The schematic diagram which showed the width | variety of the user's face. (B) The figure which showed the time change of the width | variety of a user's face. The plane schematic diagram which showed the mode of calculation of the distance from a camera to a user. The plane schematic diagram which showed the position where the said user stopped estimated from the distance from a camera to a user. The plane schematic diagram which showed the position where the said user stopped estimated from direction of a user's face. The plane schematic diagram which showed the estimated position where the user stopped and the storage position. The figure which showed the information displayed on the portable terminal. The plane cross-sectional schematic diagram which showed the modification of the storage. The figure which showed the other example of the time change of the width | variety of a user's face. (A) The figure which showed an example of the user's image immediately after entering a room. (B) The figure which showed an example of the user's image just before leaving.

  Below, based on the arrow shown in the figure, it demonstrates by defining the up-down direction, the front-back direction, and the left-right direction.

  Below, the structure of the storage management system 5 which concerns on one Embodiment of this invention is demonstrated using FIGS. 1-3. The storage management system 5 performs management so that the storage position of the storage items 3 stored in the storage 1 can be easily grasped.

  Here, the storage 1 shown in FIGS. 1 and 2 is a space for storing various items (stored items). The storage 1 is a storage unit (a so-called walk-in closet) provided in a room in a house. The storage 1 is formed in a substantially rectangular parallelepiped box shape that extends long in the front-rear direction. An entrance 1a is formed in the left and right central portion of the rear side of the storage 1. A person (user) 2 who uses the storage 1 can enter and exit the storage 1 through the entrance 1a. In the storage 1, a first storage shelf 1 b and a second storage shelf 1 c are formed. The first storage shelf 1 b is formed along the right side surface of the storage 1. The second storage shelf 1 c is formed along the left side surface of the storage 1. A space in which the user 2 can move is secured in the left and right central portion (between the first storage shelf 1 b and the second storage shelf 1 c) in the storage 1.

  The user 2 is a person who uses the storage 1 and is not limited to a specific person (one person) but may mean a plurality of persons. For example, the user 2 may mean a plurality of people who live in the house (for example, a father, a mother, an eldest son, an eldest daughter, etc.).

  1 to 3 mainly includes a plurality of cameras 10 (first camera 11, second camera 12, third camera 13, fourth camera 14 and fifth camera 15), control device 20 and A portable terminal 30 is provided.

The plurality of cameras 10 are for photographing the inside of the storage 1. The plurality of cameras 10 are arranged near the ceiling in the storage 1.
The first camera 11 is disposed near the front side surface in the storage 1. The first camera 11 is arranged at the left and right center part in the storage 1. The first camera 11 is disposed rearward (that is, toward the entrance 1a).
The second camera 12 is disposed in the vicinity of the right front end in the storage 1. The 2nd camera 12 is arrange | positioned toward the left back and diagonally downward.
The third camera 13 is disposed in the vicinity of the right rear end in the storage 1. The 3rd camera 13 is arrange | positioned toward the left front and diagonally downward.
The fourth camera 14 is disposed in the vicinity of the left front end in the storage 1. The fourth camera 14 is disposed rearward right and obliquely downward.
The fifth camera 15 is arranged in the vicinity of the left rear end in the storage 1. The fifth camera 15 is disposed right front and obliquely downward.

  In the present embodiment, the plurality of cameras 10 all have the same performance (specification). Further, “images” taken by a plurality of cameras 10 mean “moving images”.

  The control device 20 shown in FIG. 3 is for estimating and storing the position of the storage item 3 from the images in the storage 1 photographed by the plurality of cameras 10. The control device 20 may actually be configured such that a CPU, ROM, RAM, HDD, or the like is connected by a bus, or may be configured by a one-chip LSI or the like. Functionally, the control device 20 includes a storage unit 21, a recording instruction unit 22, a face recognition unit 23, a face authentication unit 24, a storage position estimation unit 25, and a search unit 26.

  The storage unit 21 is for storing various information. The memory | storage part 21 can transmit the memorize | stored information as needed while memorize | storing the information received from the other part.

  The recording instruction unit 22 gives an instruction as to whether or not to store (record) images taken by the plurality of cameras 10 in the storage unit 21. A plurality of recording instruction units 22 are provided so as to correspond to the plurality of cameras 10, respectively, and are connected to the corresponding cameras 10. Only when an instruction to record an image is issued by the recording instruction unit 22, an image taken by the camera 10 corresponding to the recording instruction unit 22 is stored in the storage unit 21.

  The face recognition unit 23 detects the face of the user 2 from the image. Further, the face recognition unit 23 detects the orientation and size of the face of the user 2 in the image. The detection of the face of the user 2 and the detection of the orientation and size of the face can be performed using a general program. A plurality of face recognition units 23 are provided to correspond to the plurality of cameras 10, respectively, and are connected to the corresponding cameras 10. Further, the face recognition unit 23 transmits a control signal to the recording instruction unit 22 connected to the same camera 10 as the face recognition unit 23 in a predetermined case.

  The face authentication unit 24 identifies the user 2 (individual) from the face of the user 2 in the image. The user 2 can be identified using a general program. The face authentication unit 24 is provided to correspond to the first camera 11 and is connected to the first camera 11.

  In the present embodiment, the face authentication unit 24 stores in advance the faces of a plurality of people who live in the house (for example, father, mother, eldest son, eldest daughter, etc.) as databases. The face authentication unit 24 determines to which person in the database the face of the user 2 in the image corresponds, and identifies the user 2 by selecting one from the database. .

  The storage position estimation unit 25 estimates the position (storage position) where the storage item 3 is arranged in the storage 1 from various information stored in the storage unit 21.

  The search unit 26 searches for desired information from various information stored in the storage unit 21 based on a predetermined condition (search condition). The search unit 26 is connected to a portable terminal 30 described later.

  In the control device 20 configured as described above, each unit operates according to programs and rules stored in advance in the control device 20.

  The portable terminal 30 is an embodiment of the search device according to the present invention, and is capable of searching for the storage position of the storage item 3 stored in the storage 1. The portable terminal 30 is provided with a touch panel that combines a display device and an input device. The portable terminal 30 is wirelessly connected to the search unit 26 of the control device 20. The portable terminal 30 is not fixed at a predetermined place, and the user 2 can carry it to a desired place.

  Hereinafter, the operation of the storage management system 5 configured as described above will be described with reference to FIGS. 1 to 14. More specifically, the operation of the storage management system 5 when the user 2 stores the storage item 3 in the storage 1 will be described.

  As shown in FIGS. 1 and 2, when the user 2 is not in the storage 1, the control device 20 operates only the face recognition unit 23 and the face authentication unit 24 corresponding to the first camera 11. The operation of the face recognition unit 23 corresponding to the recording instruction unit 22 and the other cameras 10 is stopped (hereinafter, this state is referred to as “standby state”).

  Thus, in a state where the user 2 is not in the storage room 1, energy saving can be achieved by stopping the operations of the recording instruction unit 22 and the face recognition unit 23 as much as possible. On the other hand, when the user 2 enters the storage 1, only the face recognition unit 23 and the face authentication unit 24 corresponding to the first camera 11 are always operated in order to detect the entry of the user 2. .

  In the standby state, it is also possible to further save energy by stopping image capturing by the second camera 12, the third camera 13, the fourth camera 14, and the fifth camera 15.

  As shown in FIGS. 4 and 5, when the user 2 enters the storage 1 from the entrance 1 a, the user 2 appears in the image taken by the first camera 11. In this case, the face recognition unit 23 corresponding to the first camera 11 detects the face of the person (user 2) from the image photographed by the first camera 11. When the face of the user 2 is detected by the face recognition unit 23, the control device 20 activates the other face recognition unit 23 and the recording instruction unit 22 that have been stopped.

  At this time, the user 2 is specified by the face authentication unit 24. That is, the face authentication unit 24 determines to which person in the database stored in advance the face of the user 2 in the image photographed by the first camera 11 corresponds. The face authentication unit 24 stores the specified user 2 in the storage unit 21. For example, when the face authentication unit 24 determines that the user 2 is “mother”, the face authentication unit 24 stores the fact in the storage unit 21.

  As shown in FIG. 6, when the user 2 is moving in the storage 1, images taken by the plurality of cameras 10 are stored (recorded) in the storage unit 21 by the recording instruction unit 22.

  Here, the image stored in the storage unit 21 is the camera 10 (in the state of FIG. 6, the first image in the state of FIG. 6) arranged within a range of 50 degrees to the left and right from the front of the face of the user 2. Only images from one camera 11 and second camera 12). Whether or not each camera 10 is arranged in the range S is determined by the face recognition unit 23.

That is, when the face recognition unit 23 determines that the orientation of the face of the user 2 in the image photographed by the camera 10 is rotated at least 50 degrees to the left or right with respect to the front, The recognizing unit 23 transmits a control signal not to record an image to the corresponding recording instruction unit 22.
On the other hand, when the face recognizing unit 23 determines that the orientation of the face of the user 2 in the image captured by the camera 10 is rotated by less than 50 degrees to the left or right with respect to the front, The face recognition unit 23 transmits a control signal for recording an image to the corresponding recording instruction unit 22. The recording instruction unit 22 that has received the control signal causes the storage unit 21 to record an image captured by the corresponding camera 10.

  Since the user 2 is moving, the orientation of the face of the user 2 changes according to the movement. For this reason, the camera 10 included in the range S may be out of the range S, or the camera 10 out of the range S may be included in the range S. When the camera 10 is out of the range S, the recording of the image of the camera 10 is stopped. When the camera 10 is included in the range S, recording of an image of the camera 10 is started. As described above, the recording of the image and the stop of the recording are repeated in accordance with the change of the range S.

  In this manner, by storing only the image of the camera 10 arranged at a position close to the front of the user 2 in the storage unit 21, it is possible to efficiently store the image. That is, normally, the user 2 does not have the stored item 3 on the back side (back side) of the user 2. For this reason, by storing the image of the camera 10 on the front side of the user 2 in the storage unit 21, what is the stored item 3 that the user 2 has and what the user 2 has stored the stored item 3. It is possible to acquire an image that shows where it is placed. In addition, unnecessary images (images of the user 2 taken from the back side) are not stored in the storage unit 21, so useless storage capacity can be prevented from being used.

  Further, when the user 2 is moving in the storage 1 in this way, the face recognition unit 23 determines the orientation of the face of the user 2 in the image captured by the corresponding camera 10 (to the left and right with respect to the front in front). All changes in angle) are stored in the storage unit 21.

  As shown in FIGS. 7 and 8, when the user 2 leaves the storage 1, the face of the user 2 is not detected from any of the images taken by the plurality of cameras 10. As described above, when any face recognition unit 23 cannot detect the face of the user 2, it is considered that the user 2 has left the storage 1, and the control device 20 returns to the standby state again.

  After the user 2 leaves the storage 1, the control device 20 estimates the position (storage position) where the user 2 has placed the stored item 3. Hereinafter, a method for estimating the storage position will be described in detail.

  First, each face recognizing unit 23 stores changes in the size of the face of the user 2 in the image recorded by the corresponding camera 10 in time series in the storage unit 21. In the present embodiment, as shown in FIG. 9A, the size of the face of the user 2 means the width W of the face of the user 2 in the image. As the user 2 approaches the camera 10, the width W of the face of the user 2 in the image taken by the camera 10 increases. Further, as the user 2 moves away from the camera 10, the width W of the face of the user 2 in the image captured by the camera 10 becomes smaller.

  Next, as shown in FIG. 9B, the storage position estimation unit 25 compares changes in the size of the face of the user 2 (face width W) summarized in time series by each face recognition unit 23. To do.

  Here, in this embodiment, since all the cameras 10 have the same performance (specification), the face width W can be simply compared. If the specifications of the cameras 10 are different from each other, it is necessary to standardize so that the face widths W in the images obtained from the cameras 10 can be compared with each other.

  In the time zone P from time t1 to time t2 shown in the figure, the width W of the face of the user 2 captured by the second camera 12 (detected by the face recognition unit 23 corresponding to the second camera 12). It can be seen that is substantially constant. The fact that the width W of the face of the user 2 is constant means that the user 2 is not moving in the storage 1 and the stored item 3 is placed in the storage 1 at that time. Can do.

  Therefore, the storage position estimation unit 25 estimates the position where the user 2 has stopped using the image captured by the second camera 12.

  Specifically, first, the storage position estimation unit 25, as shown in FIG. 10, the user 2 (user 2a) in the image taken by the second camera 12 immediately after the user 2 enters the storage 1. ) And the face width W of the user 2 (user 2b) in the image taken by the second camera 12 in the time period P, the user 2 is stopped. The distance from the position to the second camera 12 is estimated.

  In the example shown in FIG. 10, the width W of the face of the user 2a is 1/4 with respect to the width 4W of the image taken by the second camera 12 immediately after the user 2 enters the storage 1. On the other hand, the width W of the face of the user 2b is ½ of the width 2W of the image taken by the second camera 12 when the user 2 stops. From this, it is considered that the distance L from the second camera 12 to the user 2b is reduced to ½ of the distance 2L from the second camera 12 to the user 2a.

  The distance 2L from the second camera 12 to the user 2a immediately after entering the storage 1 (that is, the vicinity of the entrance 1a of the storage 1) is determined in advance from the dimensions of the storage 1 and the position where the second camera 12 is disposed. I know. Accordingly, the distance L from the second camera 12 to the user 2b can be calculated (estimated).

  When the distance L is calculated, as shown in FIG. 11, the user 2 (user 2b) stops somewhere in the range R in the vicinity of the arc having the radius L centered on the second camera 12 in plan view. Can be estimated.

  Next, the storage position estimator 25 determines that the user 2 moves from the second camera 12 based on the face orientation of the user 2 when the user 2 stops (time zone P in FIG. 9B). It is estimated which range (range D1, range D2, or range D3 described later) is being viewed.

In the example illustrated in FIG. 12, when the user 2 is standing in the leftward range D <b> 1 when viewed from the second camera 12, there is a high possibility that the user 2 is facing the first storage shelf 1 b side. In this case, the face of the user 2 in the image taken by the second camera 12 is facing left.
In other words, when the face of the user 2 in the image captured by the second camera 12 is facing left, it can be estimated that the user 2 is standing in the range D1.

When the user 2 stands in a rightward range D3 when viewed from the second camera 12, the user 2 is on the second storage shelf 1c side (more specifically, the front end portion from the front and rear center of the second storage shelf 1c). It is highly possible that In this case, the face of the user 2 in the image photographed by the second camera 12 is facing right.
In other words, when the face of the user 2 in the image photographed by the second camera 12 is facing the right, it can be estimated that the user 2 is standing in the range D3.

When the user 2 stands in a range D2 near the left and right center when viewed from the second camera 12, the user 2 is on the second storage shelf 1c side (more specifically, from the front and rear center of the second storage shelf 1c). There is a high possibility that it is facing the rear end). In this case, the face of the user 2 in the image photographed by the second camera 12 faces backward.
In other words, when the face of the user 2 in the image photographed by the second camera 12 is facing backward, it can be estimated that the user 2 is standing in the range D3.
In the present embodiment, as described above, when the face of the user 2 is facing backward (rotated to the left or right by 50 degrees or more with respect to the front), the image is taken by the second camera 12. The recorded image is not stored (recorded) in the storage unit 21.

  Thus, the range in which the user 2 is standing can be estimated from the orientation of the face of the user 2 in the image captured by the second camera 12. Therefore, the storage position estimation unit 25 uses the information when the user 2 stops (time zone P in FIG. 9B) based on the information about the time change of the face direction of the user 2 stored in the storage unit 21. The direction of the face of the person 2 is searched.

  In the present embodiment, it is assumed that the face of the user 2 in the image taken by the second camera 12 when the user 2 stops is facing left. That is, the storage position estimation unit 25 estimates that the user 2 is standing in the range D1.

  Finally, as shown in FIG. 13, the storage position estimation unit 25 estimates that the range F where the range R and the range D1 obtained as described above overlap is the position where the user 2 has stopped. Further, since the storage position estimation unit 25 is considered to have placed the storage 3 in the storage 1 when the user 2 stops, the storage shelf (the first storage shelf 1b and the second storage shelf 1b closest to the range F). In the example of FIG. 13, that is, the shelf 1 c), that is, it is presumed that the stored item 3 is arranged (stored) in the front and rear central portion G of the first storage shelf 1 b.

  As described above, in addition to the distance L from the second camera 12 to the user 2, the position of the user 2 can be estimated more accurately by taking into account the orientation of the face of the user 2. .

  After the position (storage position) at which the user 2 has placed the storage item 3 is estimated, the storage position estimation unit 25 collectively stores information on the storage item 3 in the storage unit 21. Specifically, as shown in FIG. 14, the information about the stored item 3 is the date and time when the user 2 entered the storage 1 (information M1) and who the user 2 is (information M2). The estimated storage position of the stored item 3 (information M3), an image recorded by the first camera 11 immediately after the user 2 enters the room (information M4), and an image recorded by each camera 10 (information M5) Etc.

  In this way, the storage management system 5 causes the storage unit 21 to store the information related to the stored items 3 as described above every time the user 2 enters or leaves the storage 1.

  In the following, with reference to FIG. 3 and FIG. 14, a description will be given of a state when searching for information (specifically, a storage position of a desired storage item 3) related to the storage item 3 stored in the storage unit 21 as described above. To do.

  When the user 2 (for example, “mother” who stores the stored item 3 in the storage 1) searches for the storage position of the stored item 3 at a later date, the search can be performed using the portable terminal 30. Specifically, an application for searching for the storage position of the storage item 3 installed in advance in the portable terminal 30 is activated, and a search condition is input.

  In the present embodiment, information M1 and information M2 are used as search conditions. That is, the user 2 who operates the portable terminal 30 inputs to the portable terminal 30 whether the stored item 3 that is symmetric to the search is stored in the storage 1 for “who (information M2)” and “when (information M1)”. To do. In the present embodiment, it is assumed that the stored item 3 stored by “Mom” on “September 30, 2013” is retrieved.

  When the search condition is input to the portable terminal 30, the search condition is transmitted to the search unit 26 of the control device 20. The search unit 26 searches for information satisfying the search condition from information regarding the stored items 3 stored in the storage unit 21, and transmits the information to the portable terminal 30. As shown in FIG. 14, the portable terminal 30 displays information on the stored item 3 transmitted from the search unit 26 on the touch panel. Below, each displayed information is demonstrated.

  At the top of the touch panel, the date and time (information M1) when the user 2 entered the storage 1 is displayed. Below the information M1 is displayed who is the user 2 who entered the storage 1 (information M2).

  Below the information M2, the storage position (information M3) of the storage item 3 estimated by the storage position estimation unit 25 is displayed. Specifically, a schematic diagram of the storage 1 in plan view is displayed in the field for information M3. Then, in the schematic diagram of the storage 1, the estimated storage position of the storage item 3 is indicated by a circle. By looking at the information M2, the user 2 can know the storage position of the desired storage item 3.

  Below the information M3, an image (information M4) recorded by the first camera 11 immediately after the user 2 enters the room is displayed. Immediately after the user 2 enters the room, the first camera 11 records the user 2 from the front. Accordingly, it is considered that the stored item 3 carried by the user 2 is also reflected in the image (moving image) recorded by the first camera 11. By confirming the contents of the image (information M4), the contents of the storage item 3 can be confirmed.

  Below the information M4, an image (information M5) recorded by each camera 10 is displayed. By confirming each image (moving image) recorded by each camera 10, it is possible to confirm the contents and storage position of the storage item 3.

  Thus, the storage position of the storage item 3 can be easily searched by simply inputting the search condition to the portable terminal 30. In addition, since the contents, storage position, and the like of the stored item 3 can be confirmed from the image, the stored position of the stored item 3 can be grasped more reliably.

As described above, the storage management system 5 according to the present embodiment is
A camera 10 provided in the storage 1 capable of storing the storage 3 therein, and photographing the inside of the storage 1;
Store the image taken by the camera 10,
Calculating a temporal change in the width W (face size) of the user 2 in the stored image;
Detecting that the user 2 has stopped based on the temporal change in the width W of the user 2's face;
Based on the width W of the face when the user 2 stops in the stored image, the position where the user 2 stopped is estimated,
A control device 20 that estimates and stores the storage position of the storage item 3 based on the position where the user 2 stops;
A portable terminal 30 (search device) capable of searching for the storage position stored in the control device 20;
It comprises.
By comprising in this way, the storage position of the stored item 3 can be grasped | ascertained easily.
Further, since the storage position of the storage item 3 is automatically stored, the user 2 does not need to perform any special work.

In addition, the control device 20
When estimating the position where the user 2 has stopped, information regarding the orientation of the face of the user 2 is also taken into consideration.
By comprising in this way, the position where the user 2 stopped can be estimated more correctly, and by extension, the storage position of the stored item 3 can be estimated more accurately.

In addition, the control device 20
The user 2 photographed by the camera 10 is specified and stored,
The portable terminal 30
The storage position stored in the control device 20 can be searched based on the specified user 2.
By comprising in this way, the user 2 can be specified and a storage position can be searched, and the search efficiency can be improved.

The camera 10
A plurality of different positions are provided in the storage 1.
The control device 20
When the face of the user 2 in the image photographed by the camera 10 is rotated by a predetermined angle (50 degrees) or more to the left and right with respect to the front, the image is not stored.
With this configuration, it is possible to store only images that are highly necessary, and it is possible to prevent useless storage capacity from being used.

Further, the search device includes:
This is a portable terminal 30 that can be carried.
By comprising in this way, the portable terminal 30 can be used not only in the vicinity of the storage 1 but in arbitrary places, and the convenience of search can be improved.

  In the present embodiment, the storage 1 has been described as a storage unit (walk-in closet) provided in the house, but the present invention is not limited to this. That is, the storage 1 is not limited to the one provided in the house as long as it forms a storage space where the user 2 can enter and exit. For example, the storage 1 may be a warehouse or the like provided outside independently from a house.

  In the present embodiment, five cameras 10 are provided. However, the present invention is not limited to this, and the number of cameras 10 is not limited. However, it is desirable to arrange a necessary and sufficient number of cameras 10 so that no blind spots are generated in the storage 1.

  In the present embodiment, for convenience of explanation, the control device 20 is functionally classified into a storage unit 21, a recording instruction unit 22, a face recognition unit 23, a face authentication unit 24, a storage position estimation unit 25, and a search unit 26. Although described, the present invention is not limited to this. That is, as long as the control device 20 can perform the processing described in this embodiment, the functional classification is not limited to the above.

  In the present embodiment, the portable terminal 30 includes a touch panel. However, the present invention is not limited to this, and a liquid crystal screen or the like as a display device and an operation tool (button or the like) as an input device. It may be a combination of In addition, the search device according to the present invention is not limited to the portable terminal 30 that can be carried, but may be fixed near the entrance 1 a of the storage 1, for example.

  In the present embodiment, the information M4 and the information M5 displayed on the portable terminal 30 are moving images. However, the present invention is not limited to this, and still images can be displayed. .

  In the present embodiment, the face recognition unit 23 detects the face width W as the size of the face of the user 2, but the present invention is not limited to this. That is, the face recognizing unit 23 may detect the dimensions of other parts of the face (for example, the vertical width) as long as the face sizes can be compared.

  Moreover, in this embodiment, the face authentication part 24 shall identify the said user 2 by judging who the user 2's face corresponds to in the database memorize | stored previously. However, the present invention is not limited to this. For example, a configuration in which the database is not used is also possible. In this case, the face authentication unit 24 identifies the user 2 individual from the face of the user 2 who has entered the storage 1 and classifies them by appropriately naming “Mr. A”, “Mr. B”,. This eliminates the need to update the database even when the number of users 2 (for example, people living in the house) newly increases.

  In the present embodiment, in addition to the distance L from the camera 10 (second camera 12) to the user 2, the position of the user 2 is estimated in consideration of the orientation of the user 2's face. However, the present invention is not limited to this. That is, based on only the distance L from the camera 10 to the user 2, the position where the user 2 stops can be estimated. For example, as shown in FIG. 15, when the storage case 1 has a long and narrow shape, and the storage shelf (first storage shelf 1 b) is formed only on one side surface (right side surface) of the storage case 1. The user 2 hardly moves in the left-right direction. In such a case, the position where the user 2 has stopped can be estimated only by the distance L from the camera 10 to the user 2. Moreover, since the 1st storage shelf 1b is formed only in the one side of the storage 1, the storage position of the stored item 3 can also be estimated from the position where the user 2 stopped.

  Further, in the present embodiment, only the image taken by the second camera 12 is detected that the face width W of the user 2 is substantially constant in the time zone P (FIG. 9). (See (b)), it may be detected by the plurality of cameras 10 that the width W of the face of the user 2 becomes constant at the same time. For example, FIG. 16 shows an example in which the first camera 11 and the second camera 12 detect that the width W of the face of the user 2 is substantially constant in the time zone P. In such a case, paying attention to the camera 10 closest to the user 2, that is, the second camera 12 having the largest face width W, the distance L from the second camera 12 to the user 2, and the Based on the orientation of the face of the user 2 in the image photographed by the second camera 12, the position where the user 2 has stopped is estimated. Thus, the estimation accuracy can be improved by estimating the position where the user 2 has stopped using the camera 10 as close to the user 2 as possible.

  In addition, in the storage management system 5 according to the present embodiment, a user 2 who enters the storage 1 with the storage 3 (a user 2 who intends to store the storage 3 in the storage 1), It cannot be distinguished from a user 2 (user 2 who does not intend to store the stored item 3) who enters the storage 1 without holding the stored item 3. Therefore, it is possible to distinguish the user 2 using the method described below.

  That is, the image of the user 2 immediately after entering the storage 1 by the control device 20 (see FIG. 17A) and the image of the user 2 immediately before leaving the storage 1 (see FIG. 17B). ) And. Specifically, the control device 20 compares the texture (color, pattern, etc.) of a certain range U (generally near the chest of the user 2) below the face of the user 2. Usually, when the user 2 carries the storage item 3, it is expected that the storage item 3 is held in front of the chest of the user 2. Therefore, when there is a certain difference or more in the texture of the range U in the two images, the user 2 can estimate that the stored item 3 is stored in the storage 1.

  On the other hand, if the difference between the textures of the range U in the two images does not exceed a certain level, the user 2 does not store the storage item 3 in the storage unit 1 (stores the storage item 3 in the first place). It can be estimated that the user has not entered the storage room 1 or has entered the room with the storage item 3 but has left the room with the storage item 3 as it is. In this case, the control device 20 deletes all the images obtained by recording the user 2 and the data acquired regarding the entry / exit of the user 2. Thereby, unnecessary data can be prevented from being accumulated.

DESCRIPTION OF SYMBOLS 1 Storage 2 User 3 Storage 5 Storage management system 10 Camera 20 Control apparatus 30 Portable terminal (search apparatus)

Claims (5)

A camera that is provided in a storage capable of storing storage items therein, and that photographs the inside of the storage;
Storing an image taken by the camera;
Calculate the time change of the user's face size in the stored image,
Detecting that the user has stopped based on a temporal change in the size of the user's face;
Based on the size of the face when the user stops in the stored image, estimate the position where the user stopped,
A control device for estimating and storing the storage position of the storage object based on the position where the user has stopped;
A search device capable of searching the storage position stored in the control device;
A storage management system comprising: The controller is
When estimating the position where the user has stopped, also consider information regarding the orientation of the user's face,
The storage management system according to claim 1. The controller is
Identify and store the user photographed by the camera,
The search device includes:
The storage position stored in the control device can be searched based on the identified user.
The storage management system according to claim 1 or 2. The camera
A plurality of different locations are provided in the storage,
The controller is
When the face of the user in the image photographed by the camera is rotated by a predetermined angle or more to the left and right with respect to the front, the image is not stored.
The storage management system according to any one of claims 1 to 3. The search device includes:
A portable device that can be carried around.
The storage management system according to any one of claims 1 to 4.

Images