JP7392986B2 - Article position management device, article position management system, article position management method and program - Google Patents

Description

The present invention relates to an article position management device, an article position management system, an article position management method, and a program.

At production sites such as factories, it is important to manage the location of items such as materials used. This is because it is essential to quickly find the necessary materials for smooth factory operations. As a system for realizing such position management of articles, for example, there is a system that manages the position of articles existing in a management area such as indoors and the history of entering and leaving the warehouse of articles in that area (Patent Document 1 below) .

Patent Document 1 discloses that a signal transmitter that can periodically transmit an identification number wirelessly is attached to all items brought indoors, and a fixed camera installed indoors periodically photographs the inside. The system is described. In this system, a computer installed indoors, for example, based on an identification number periodically sent from a signal transmitter attached to an imported item A, identifies the item A linked to the identification number. recognize. On the other hand, this computer monitors fluctuations (changes in pixel differences) in captured images periodically sent from fixed cameras. When a change is detected, the shape of the difference image is compared with the sample shape of the article A (stored in advance) linked to the above identification number to identify the article A and identify the shape of the difference image. The position of article A is calculated from the photographed image. In other words, this system manages the position of the delivered item A using the identification number periodically transmitted from the delivered item A and the captured images periodically sent from a fixed camera. There is.

Japanese Patent Application Publication No. 2006-103857

However, in the system described in Patent Document 1, as the number of items to be managed indoors increases and the number of items with similar shapes increases, the existence of the items may be recognized based on the regularly sent identification numbers. Although this is possible, the accuracy of detecting the position of the article may be degraded due to errors caused by detecting fluctuations in photographed images and processing for comparing image samples. Specifically, in cases where multiple objects with similar shapes are moved at the same time, or where objects enter the blind spot of a fixed camera as a result of movement, changes in captured images may be detected. Even if there is such a difference, the shape of the difference image becomes complicated, and the detection accuracy deteriorates significantly in the comparison process with the sample shape of the article. As a result, there is a problem in that it is not possible to accurately determine which articles have been moved to where.

In addition, the system described in Patent Document 1 has the problem that in addition to a fixed camera that photographs the indoor situation, it is necessary to prepare a wireless communication means to be attached to all objects, resulting in an expensive system.

SUMMARY OF THE INVENTION An object of the present invention is to provide an article position management device, an article position management system, an article position management method, and a program that can accurately manage the position of articles to be managed at low cost.

The article position management device disclosed in the present application includes a first position specifying unit that specifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order. a second position specifying unit that identifies the position of the article based on image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order; and a position management unit that manages the information. When an article whose position cannot be determined during storage is detected based on the latest positioning result in the first positioning section, the location management section determines whether the article is located based on the first trajectory information. A first determination process is performed to determine whether or not the article has moved outside the storage area, and for articles that cannot be determined to have moved outside the storage area in the first determination process, furthermore, if the article is not determined to have moved outside the storage area, A second determination process is performed to determine whether or not the user has moved out of the storage area based on the position confirmed in the above and the second trajectory information, and in the first or second determination process If it is determined that the article has moved outside of the article, the first trajectory information regarding the article is deleted.

The article position management system disclosed in the present application includes a first position specifying unit that specifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order. a second position specifying unit that identifies the position of the article based on image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order; a position management unit that manages the article position management unit; and an article position management device including the article position management unit. The position management unit of the article position management device stores information in the first trajectory information when an article whose position cannot be determined during storage is detected based on the latest position determination result in the first position determination unit. A first determination process is performed to determine whether the article has moved out of the storage area based on the above-mentioned information, and for articles that cannot be determined to have moved out of the storage area in the first determination process, further , performing a second determination process to determine whether the article has moved out of the storage area based on the last confirmed position and the second trajectory information, and performing the first or second determination process. If it is determined that the article has moved out of the storage area, the first trajectory information regarding the article is deleted.

The article position management method disclosed in the present application includes a first position specifying process that specifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order. and performing a second position specifying process of specifying the position of the article based on the image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order. In addition, when an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article is located in the storage area based on the first trajectory information. A first determination process is performed to determine whether or not the article has been moved outside the storage area, and for items that cannot be determined to have moved outside the storage area in the first determination process, a performing a second determination process for determining whether or not the storage area has moved out of the storage area based on the confirmed position and the second trajectory information; The method includes deleting first trajectory information regarding the article when it is determined that the article has moved out of the area.

The program disclosed in the present application specifies the position of an article based on image data of the entire storage area of the article, and performs a first position specifying process of creating first trajectory information in which the position is accumulated in chronological order. and performing a second position specifying process of specifying the position of the article based on the image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order; When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article moves out of the storage area based on the first trajectory information. A first determination process is performed to determine whether or not the item has been moved, and for items that cannot be determined to have moved out of the storage area in the first determination process, a check is performed to determine whether the item has been moved out of the storage area. performing a second determination process for determining whether or not the user has moved out of the storage area based on the position and the second trajectory information; If it is determined that the article has moved to , the processor is caused to delete the first trajectory information regarding the article.

According to the article position management device, article position management system, article position management method, and program disclosed in the present application, the position of articles to be managed can be managed accurately and at low cost.

FIG. 1 is a diagram showing an example of a schematic configuration of an article position management system according to a first embodiment. FIG. 2 is a diagram showing an example of an AR marker. FIG. 3 is a diagram showing an example of the processing configuration (processing module) of the storage camera device 10 and the access monitoring camera device 20 according to the first embodiment. FIG. 4 is a diagram showing an example of a processing configuration (processing module) of the article position management device 30 according to the first embodiment. FIG. 5 is a diagram showing an example of article marker information. FIG. 6 is a diagram showing an example of camera-related information. FIG. 7 is a diagram showing an example of article position information. FIG. 8 is a diagram showing an example of trajectory information. FIG. 9 is a flowchart illustrating an example of the operation of the first article position specifying unit 302 (position specifying operation using image data acquired from the storage camera device 10). FIG. 10 is a flowchart showing an example of the operation of the article position management section 304. FIG. 11 is a diagram illustrating an example of an article position calculation process. FIG. 12 is a diagram illustrating an example of a schematic configuration of an article position management system according to the second embodiment. FIG. 13 is a diagram showing an example of a processing configuration (processing module) of the article position management device 30 according to the second embodiment. FIG. 14 is a diagram showing an example of reference point marker information. FIG. 15 is a diagram illustrating an example of position calculation processing of the camera device. FIG. 16 is a block diagram showing an example of a schematic configuration of an article position management device according to the third embodiment. FIG. 17 is a diagram showing an example of the hardware configuration of the article position management device 30.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of an article position management device, an article position management system, an article position management method, and a program disclosed in the present application will be described in detail based on the drawings. Note that the present invention is not limited to this embodiment. Further, in the specification and drawings of the present application, elements that can be explained in the same manner are designated by the same reference numerals, so that repeated explanation can be omitted.

[Overview of embodiment]
First, an overview of the embodiment will be explained.

(1) Technical issues For example, a signal transmitter that can periodically transmit an identification number wirelessly is attached to all items brought indoors, and a fixed camera installed indoors periodically monitors the contents inside. There is a system that manages the location of indoor items and the history of entry and exit of the items by taking pictures.

In this system, a computer installed indoors recognizes the item A linked to the identification number based on the identification number sent periodically from a signal transmitter attached to the imported item A. . On the other hand, this computer monitors fluctuations (changes in pixel differences) in captured images periodically sent from fixed cameras, and when it detects fluctuations, it calculates the shape of the difference image and the above-mentioned identification number. The article A is identified by comparing the linked sample shape of the article A (stored in advance), and the position of the imported article A is calculated from the photographed image. In other words, this system manages the position of the delivered item A using the identification number periodically transmitted from the delivered item A and the captured images periodically sent from a fixed camera. There is.

However, in the system described in Patent Document 1, as the number of items to be managed indoors increases and the number of items with similar shapes increases, the existence of the items may be recognized based on the regularly sent identification numbers. Although this is possible, the accuracy of detecting the position of the article may be degraded due to errors caused by detecting fluctuations in photographed images and processing for comparing image samples. Specifically, in cases where multiple objects with similar shapes are moved at the same time, or where objects enter the blind spot of a fixed camera as a result of movement, changes in captured images may be detected. Even if there is such a difference, the shape of the difference image becomes complicated, and the detection accuracy deteriorates significantly in the comparison process with the sample shape of the article. As a result, there is a problem in that it is not possible to accurately determine which articles have been moved to where.

In addition, the system described in Patent Document 1 has the problem that in addition to a fixed camera that photographs the indoor situation, it is necessary to prepare a wireless communication means to be attached to all objects, resulting in an expensive system.

There are also systems that detect the location of items using images taken by camera devices that take pictures of indoor situations, without using the identification numbers sent from signal transmitters. There is a possibility that the object will enter the blind spot of the camera device and the position cannot be determined. For example, even if the location of an item cannot be determined, it is possible to locate the item again if it is within the storage area (if the blind spot is resolved, the location can be determined again). If an item is moved outside the storage area without being able to specify its position, there will be no opportunity to detect the position of the item (whereabouts unknown).

Therefore, the purpose of the present embodiment is to manage the positions of objects to be managed with high precision and at low cost. Furthermore, in an inexpensive system that uses image data sent from a camera device to identify the location of an object without using wireless communication means such as a signal transmitter, the location of an object that has entered the blind spot of the camera device can be detected. It consists in estimating.

(2) Technical Features In this embodiment, a storage camera device capable of photographing an article storage area, an entry/exit monitoring camera device capable of photographing an area around an entrance in the storage area, and an article position management system managing the position of articles are provided. and a device.

The article position management device includes a first article position specifying unit that specifies the position of the article based on the image data acquired from the storage camera device and creates first trajectory information in which the position is accumulated in chronological order. , a second article position specifying unit that identifies the position of the article based on the image data acquired from the entrance/exit monitoring camera device and creates second trajectory information in which the position is accumulated in chronological order; and an article location management section.

Then, when an article whose position cannot be determined during storage is detected, the article position management section determines whether the article has moved out of the storage area based on the first trajectory information. After performing the determination process, if the article cannot be determined to have moved out of the storage area in the first determination process, the article is further moved to the storage area based on the last confirmed position of the article and the second trajectory information. A second determination process is performed to determine whether the article has moved outside the storage area, and if it is determined in the first or second determination process that the article has moved outside the storage area, the first trajectory information regarding the item is delete.

This makes it possible to determine when an item whose location cannot be determined during storage has been moved outside the storage area, making it possible to accurately manage the location of the item being managed. . Further, since there is no need to attach a wireless communication means to the articles to be managed, position management of articles can be realized at low cost, especially as the number of articles in storage increases. Furthermore, it is now possible to determine whether an item whose location could no longer be determined during storage has moved outside the storage area. It can be estimated that the items that have been stored have entered the blind spot of the storage camera device within the storage area.

Note that the above-mentioned technical features are specific examples of the embodiments of the present invention, and the embodiments of the present invention are not limited to the above-mentioned technical features.

Hereinafter, specific embodiments will be described in more detail with reference to the drawings.

[First embodiment]
The first embodiment will be described in more detail using the drawings.

[System configuration]
FIG. 1 is a diagram showing an example of a schematic configuration of an article position management system according to a first embodiment. As shown in FIG. 1, the article position management system includes a storage camera device 10, an entrance/exit monitoring camera device 20, and an article position management device 30.

In the first embodiment, the storage camera device 10 is a camera fixed to a pillar or ceiling in a storage area. The storage camera device 10 is installed, for example, to photograph the entire storage area periodically or at predetermined timing. As shown in Figure 1, the storage area is all areas predefined as areas where items can be stored, including the floor (ground) and space, and may be indoors or outdoors. .

The entrance/exit monitoring camera device 20 is a camera fixed to a pillar or ceiling within the storage area so as to be able to photograph a specific area around the entrance (hereinafter referred to as the area around the entrance) within the range of the storage area. . The entrance/exit monitoring camera device 20 is installed, for example, to photograph the entire area around the entrance/exit periodically or at predetermined timing. The area around the entrance corresponds to the shaded area shown in FIG. 1 within the range of the storage area. Further, in this embodiment, as an example, a part of the storage area is defined as the area around the entrance/exit, but the invention is not limited to this, and these areas may be defined so as not to overlap.

The storage camera device 10, the access monitoring camera device 20, and the article position management device 30 are connected by wire or wirelessly, and are configured to be able to communicate with each other (send and receive data). In the following description, when the term "camera device" is simply used without any special explanation, it refers to the storage camera device 10 or the access monitoring camera device 20.

In the first embodiment, the article position management device 30 identifies and manages the positions of articles placed in the storage area based on information (image data) transmitted from the storage camera device 10. Furthermore, the article position management device 30 detects the position of articles existing in the area around the entrance based on information (image data) transmitted from the entrance/exit monitoring camera device 20.

Further, as shown in FIG. 1, a plurality of articles 40 are arranged in the storage area. The article 40 is, for example, a container stacked on a pallet, a cardboard box, a box, a machine, a device, or the like. An article marker 50 is affixed to each of the plurality of articles 40 arranged in the storage area. Note that the article markers 50 affixed to each article 40 are managed so as not to overlap, and the articles 40 and article markers 50 correspond on a one-to-one basis. In other words, when the article marker 50 is uniquely identified and the position of the article marker 50 is specified, the position of the article 40 is identified.

The article marker 50 can be detected in an image taken by a camera mounted on a camera device, and functions as an identifier (marker) that can calculate the relative position of the camera device and the marker. For example, an AR (Augmented Reality) marker can be used as the article marker 50. However, the purpose is not to limit the article marker 50 to an AR marker, but any marker can be used as long as it can be detected from a photographed image and the relative position between the camera and the marker can be calculated.

Furthermore, the article marker 50 can be a printed matter of the marker. In this embodiment, it is assumed that the printed AR marker is attached, for example, upwardly to the top of the article 40 so that it can be easily seen from the storage camera device 10. Note that the attachment position of the article marker 50 is not limited to this.

As mentioned above, the article marker 50 is an identifier that allows the relative position of the camera device and the marker to be calculated. A method of solving a PnP (Perspective-n-Point) problem (hereinafter referred to as a PnP method) can be used to calculate the relative position. That is, the relative position of article marker 50 is calculated as a solution to the PnP problem.

Identification of AR markers and calculation of relative positions using AR markers will be outlined below.

FIG. 2 is a diagram showing an example of an AR marker. As shown in FIG. 2, the AR marker has a predetermined pattern printed inside a black frame. The AR marker is identified (specified) by performing image processing on image data obtained by photographing the AR marker. Note that details of the image processing performed when identifying an AR marker are described in Reference 1 below. Therefore, detailed explanation regarding image processing will be omitted here.
<Reference 1>
https://docs.google.com/document/d/1QU9KoBtjSM2kF6ITOjQ76xqL7H0TEtXriJX5kwi9Kgc/edit#heading=h.oz0eoslrvt8v

By detecting the black frame included in the image data, the existence of the AR marker itself is recognized, and the AR marker is uniquely identified by the pattern within the black frame. More specifically, a pattern file is prepared in advance in which the inside of the black frame is divided into a plurality of regions and the image in each divided region is digitized. The AR marker is uniquely identified by comparing the pattern file with the pattern extracted from the image data photographed by the camera. That is, by associating the pattern of the AR marker with a numerical value, the AR marker can have an identification function.

Furthermore, in order to apply the PnP method to an AR marker, three or more coordinates in a three-dimensional coordinate system and three or more coordinates in a two-dimensional image corresponding to these coordinates are required. In the first embodiment, the four corner vertices P1 to P4 of the AR marker are used as the feature points.

Note that for many AR markers, a method of obtaining the vertex coordinates of the four corners of a detected marker on a two-dimensional image has been established. Therefore, when installing an AR marker, by recording the marker's position, direction, and size in advance, the coordinates of the vertices of the four corners of the AR marker in the three-dimensional coordinate system can be obtained. The size is the length of one side if the AR marker is square, and the length and width if it is rectangular.

Further, in the first embodiment, it has been explained that the four corners of the AR marker are used as feature points, but the feature points do not necessarily have to be the vertices of the four corners. When using feature points other than the four corners, it is sufficient to record information that allows the coordinates of the feature points in a three-dimensional coordinate system to be obtained.

The solution to the PnP problem will be outlined below.

To solve the PnP problem, information regarding the angle of view and distortion of the camera that acquired the image data is required. More specifically, data called "camera matrix" and "distortion coefficient vector" are required.

The camera matrix is a three-dimensional square matrix containing focal lengths fx, fy expressed in pixels and a principal point (cx, cy), which is usually the center of the image. The camera matrix is matrix A expressed as Equation (1) below.

In the above matrix A, the camera angle of view is expressed by fx and fy. For example, consider a case where the image width is 2000 pixels and the horizontal angle of view is 60 degrees in a general camera (pinhole camera model). In this case, "cx=2000/2=1000" and "fx=1/tan(60/2)*(2000/2)=approximately 1732".

The camera matrix is a matrix necessary to solve the PnP problem and find r11 to r33 and t1 to t3 that minimize the error of the equation expressing perspective projection transformation shown in Equation (2) below.

Regarding the above equation (equation expressing perspective projection transformation), r11 to r33 found as solutions are the orientation of the camera (rotation matrix), and t1 to t3 are the positions of the camera (translation vector).

On the other hand, the distortion coefficient vector is used to obtain ideal point coordinates that have undergone distortion correction from the observed point coordinates. Note that the camera matrix and distortion coefficient vector can be obtained by camera calibration shown in References 2 and 3 below.

<Reference 2>
http://labs.eecs.tottori-u.ac.jp/sd/Member/oyamada/OpenCV/html/py_tutorials/py_c
alib3d/py_calibration/py_calibration.html#calibration
<Reference 3>
http://opencv.jp/opencv-2.1/cpp/camera_calibration_and_3d_reconstruction.html#cv
-calibratecamera

[Summary of system operation]
Next, an outline of the operation of the article position management system will be explained with reference to FIG.

The storage camera device 10 photographs the entire storage area at predetermined timing or periodically. The image data acquired by the storage camera device 10 is transmitted to the article position management device 30 together with the identifier (ID: Identification) of the storage camera device 10, for example.

The article position management device 30 calculates the absolute position of the article 40 (article marker 50) stored in the storage area based on the image data received from the storage camera device 10. Then, the obtained absolute position of the article 40 is associated with the identifier of the article 40 and stored as article position information. The article position management device 30 usually manages the position and movement of the article 40 stored in the storage area based on the accumulation result of the article position information (referred to as first trajectory information). Note that the term "normal" here refers to a situation in which the positions of all articles 40 in storage can be identified. In addition, when an article 40 whose position cannot be specified because it has entered a blind spot of the storage camera device 10 is detected, the article position management device 30 determines whether the article 40 is stored based on the first trajectory information. Determine whether the object has moved out of the area.

The entrance/exit monitoring camera device 20 photographs the entire area around the entrance/exit forming a part of the storage area at a predetermined timing or periodically. The image data acquired by the entry/exit monitoring camera device 20 is transmitted to the article position management device 30 together with the ID of the entry/exit monitoring camera device 20, for example.

The article position management device 30 calculates the absolute position of the article 40 (article marker 50) existing in the area around the entrance based on the image data received from the entrance/exit monitoring camera device 20. Then, the obtained absolute position of the article 40 is associated with the identifier of the article 40 and stored as article position information. The article position management device 30 finally identifies the article 40 whose position cannot be detected due to reasons such as the article 40 being stored entering a blind spot of the storage camera device 10 and whose position cannot be detected. When the position of the article 40 is in the area around the entrance/exit, it is determined whether the article 40 has moved out of the storage area based on the accumulation result of the article position information (referred to as second trajectory information).

Note that the article position management device 30 determines the source of the image data based on, for example, identifiers uniquely associated with the storage camera device 10 and the entrance/exit monitoring camera device 20.

Next, a method of calculating an absolute position using the camera device will be briefly explained. In this embodiment, it is assumed that the coordinate information (absolute position, absolute direction) of the camera device can be grasped by the article position management device 30.

The article position management device 30 that has received the image data determines whether or not the article marker 50 is included in the image data. If the article marker 50 is included, the article position management device 30 calculates the relative position of the article marker 50 with respect to the camera device that took the image. Then, the article position management device 30 calculates the absolute position of the article marker 50 based on the coordinate information of the camera device and the relative position of the article marker 50.

The article position management device 30 stores the information of each article 40 in which the identifier of the article marker 50 (article 40) is associated with the absolute position of the article marker 50 (article 40) every time the storage area or the area around the entrance/exit is photographed. Save article location information.

In this way, the article position management device 30 identifies the position of the stored article 40 based on the article marker 50 included in the image data acquired from the camera device, and stores the position identification result as article position information for each article 40. do. Then, the article position management device 30 executes this position specifying process every time image data is received from the camera device, thereby converting the article position information for each article 40 obtained each time into trajectory information (first or second trajectory information).

Next, each device included in the article position management system will be explained.

[Storage camera device]
FIG. 3 is a diagram showing an example of a processing configuration (processing module) of the storage camera device 10 according to the first embodiment. Referring to FIG. 3, the storage camera device 10 includes a camera control section 201, a data output section 202, and a storage section 203.

The camera control unit 201 is a means for controlling a camera (not shown). The camera control unit 201 controls the camera and acquires image data. Note that the camera control unit 201 may acquire image data periodically or at a predetermined timing.

The camera control unit 201 delivers the acquired image data to the data output unit 202. The data output unit 202 transmits image data to the article position management device 30. The data output unit 202 may transmit the image data to the article position management device 30 in real time, or may accumulate a predetermined amount of image data and then transmit it all at once to the article position management device 30.

The storage unit 203 is a means for storing information and data necessary for the operation of the storage camera device 10. For example, the storage unit 203 stores an identifier assigned to the own device (archive camera device 10). When transmitting the image data to the article position management device 30, the data output unit 202 also transmits the identifier of the own device.

[Enter/exit surveillance camera device]
The processing configuration (processing module) of the entrance/exit monitoring camera device 20 can be the same as the processing configuration of the storage camera device 10, so a description of the access/exit monitoring camera device 20 corresponding to FIG. 3 will be omitted.

[Article position management device]
FIG. 4 is a diagram showing an example of a processing configuration (processing module) of the article position management device 30 according to the first embodiment. Referring to FIG. 4, the article location management device 30 includes a data input section 301, a first article location identification section 302, a second article location identification section 303, an article location management section 304, and a storage section 305. ,including.

The data input unit 301 is a means for inputting (obtaining) image data from camera devices (storage camera device 10, entrance/exit monitoring camera device 20). The data input unit 301 delivers the image data acquired from the storage camera device 10 to the first article position specifying unit 302 and the image data acquired from the access monitoring camera device 20 to the second article position specifying unit 303.

The first article position specifying unit 302 specifies the position of the article 40 based on the image data acquired from the storage camera device 10. Further, the second article position specifying unit 303 specifies the position of the article 40 based on the image data acquired from the entrance/exit monitoring camera device 20.

The article position management unit 304 manages the position and movement of the article 40 in storage. For example, the article position management unit 304 determines whether the article 40 being stored in the storage area has entered the blind spot of the storage camera device 10, whether the article 40 being stored has moved out of the storage area, etc.

The storage unit 305 stores information and data necessary for the operation of the article position management device 30. Specifically, the storage unit 305 stores information regarding the article marker 50, information regarding the camera, and information regarding the position of the article 40. Note that in the following description, information regarding the article marker 50 will be referred to as "article marker information." Information regarding the camera will be referred to as "camera-related information." Information regarding the position of the article 40 will be referred to as "article position information," and information in which article position information is accumulated in chronological order will be referred to as "first trajectory information" or "second trajectory information."

FIG. 5 is a diagram showing an example of article marker information. As shown in FIG. 5, the article marker information is information that associates the identifier of the article to which the article marker is attached, the identifier of the article marker, and the size of the article marker.

The item identifier is, for example, the serial number of the item being managed. Alternatively, the product name may be used as the product identifier.

The identifier of the article marker 50 is, for example, information (for example, a file name, etc.) that specifies the pattern file of the above-mentioned AR marker. The size of the article marker 50 is the length of one side when the marker is square, and the length and width when the marker is rectangular. In FIG. 5, the length of one side of the article marker is shown as a square.

FIG. 6 is a diagram showing an example of camera-related information. As shown in FIG. 6, the camera-related information includes the identifier of the camera device, the angle of view and distortion information of the camera mounted on each camera device, and the coordinate information (absolute position, absolute direction) of the camera device. This is associated information.

For example, an IP (Internet Protocol) address or a MAC (Media Access Control) address can be used as the identifier of the camera device. Furthermore, in this embodiment, since the camera devices are fixed, coordinate information (absolute position, absolute direction) within the storage area where each camera device is installed is registered as camera-related information. Note that, as described above, the camera view angle and distortion information included in the camera-related information are used when detecting the relative position of the article marker 50 using the PnP method. More specifically, information regarding the angle of view (camera matrix) is used as a parameter for solving the PnP problem in the perspective projection transformation equation. Information regarding distortion (distortion coefficient vector) is used to obtain ideal point coordinates that have undergone distortion correction from observed point coordinates.

FIG. 7 is a diagram showing an example of article position information. As shown in FIG. 7, the article position information is information in which the identifier of the article is associated with the absolute position of the article. Note that the "absolute direction" may be added to the article position information. In other words, not only the absolute position of the article but also the direction in which the article is placed may be managed using the article position information.

FIG. 8 is a diagram illustrating an example of trajectory information (first trajectory information or second trajectory information). As shown in FIG. 8, the trajectory information is information in which the above article position information is accumulated in chronological order, and for example, the identifier of the article is associated with the absolute position of the article accumulated in chronological order. It is stored in the same state. In FIG. 8, the absolute position (1) indicates the newest absolute position, and the trajectory of the article is managed in chronological order as (2), (3), and so on. Here, it is shown that the articles 40 with identifiers B_21 and B_22 are not moving, but the article 40 with identifier B_23 is moving. Further, the trajectory information only needs to remain article position information for a predetermined period of time, and is deleted as appropriate by the article position management unit 304. For example, a method may be considered in which information for about one hour going back from the latest absolute position (1) is stored as trajectory information.

[Example of operation of article location management system]
Next, the operations of the first article position specifying section 302 and the second article position specifying section 303 will be explained. In addition, in the operation of the article position management system of this embodiment, it is assumed that the storage camera device 10 and the entrance/exit monitoring camera device 20 are constantly operating, but the invention is not limited to this.

[Positioning operation]
FIG. 9 is a flowchart illustrating an example of the operations of the first article position specifying section 302 and the second article position specifying section 303. Here, as an example, a position specifying operation of the first article position specifying unit 302 using image data acquired from the storage camera device 10 will be described. The operation of the second article position specifying unit 303 is the same except that the position is specified using the image data acquired from the entrance/exit monitoring camera device 20, so a detailed explanation will be omitted.

When the first article position specifying unit 302 receives image data via the data input unit 301 (Step S1, Yes) while waiting for image data from the storage camera device 10 (Step S1, No), the first article position specifying unit 302 It is determined whether the image data includes the article marker 50 (step S2). For example, if it is determined that the image data does not include the article marker 50 (Step S2, No), the first article position specifying unit 302 shifts to a state of waiting for image data (Step S1).

On the other hand, if it is determined that the image data includes the article marker 50 (step S2, Yes), the first article position specifying unit 302 refers to the camera-related information stored in the storage unit 305 and Information regarding the angle of view and distortion of the storage camera device 10 that captured the image data is acquired (step S3).

Further, the first article position specifying unit 302 refers to the article marker information stored in the storage unit 305 and obtains the size of the article marker 50 corresponding to the identifier of the article marker 50 included in the image data (step S4 ).

The first article position specifying unit 302 uses the angle of view and distortion of the storage camera device 10 acquired above and the size of the article marker 50 (marker size) to determine the position of the storage camera device 10 at the time of image data acquisition. The relative position of the article marker 50 is calculated (step S5).

Then, the first article position specifying unit 302 uses the coordinate information (absolute position, absolute direction) of the storage camera device 10 and the relative position of the article marker 50 calculated above to determine the absolute position of the article marker 50. is calculated (step S6). Here, it is assumed that the processes shown in steps S3 to S6 are executed for all article markers 50 determined to be included in the image data in step S2.

The first article position specifying unit 302 stores the absolute positions of all article markers 50 calculated above in the storage unit 305 as article position information (step S7). Here, as an example of article position information, the identifier of the article 40 and the absolute position of the article marker 50 affixed to the article 40 (absolute position of the article 40) are associated and stored in the storage unit 305.

Thereafter, the first article position specifying unit 302 repeatedly executes the processes from step S1 to step S7. Thereby, the article position information is accumulated in chronological order, and the accumulated result is stored in the storage unit 305 as first trajectory information. Furthermore, by repeatedly executing the processes from step S1 to step S7, article position information is accumulated in chronological order, even for articles newly brought to the storage area, for example. Note that the second article position specifying unit 303 also executes the processes of steps S1 to S7 in parallel to accumulate article position information in chronological order, and this accumulated result is used as second trajectory information. It is stored in the storage unit 305.

[Location management operation]
FIG. 10 is a flowchart showing an example of the operation of the article position management section 304.

In the state of waiting for the first article position specifying section 302 to save the article position information (Step S11, No), if the latest article position information has been saved (Step S11, Yes), the article position management section 304 selects the first article position information. The article position specifying unit 302 compares the latest article position information stored with the previously saved article position information (comparison based on the first trajectory information), and identifies articles whose position cannot be specified among the stored articles 40. 40 is present (step S12). Here, the article 40 whose position cannot be specified is an article 40 whose position was specified in the previous position specifying process, but whose position could not be specified this time. Note that the article position management unit 304 can also detect the article 40 newly brought to the storage area by executing the above comparison process. This makes it possible to manage the position of the article 40 newly brought to the storage area.

If it is determined that there is no article 40 whose position could not be specified (step S12, No), the article position management unit 304 shifts to a state of waiting for image data (step S1).

On the other hand, if it is determined that there is an article 40 whose position could not be specified (step S12, Yes), the article position management unit 304 refers to the first trajectory information and stores the article 40 whose position could not be specified. It is determined whether the position could not be specified because it was moved outside the area or whether the position could not be specified because it entered the blind spot of the storage camera device 10 (step S13). As a judgment criterion, for example, if the position of the article 40 identified last with reference to the first trajectory information reaches the position of the entrance/exit of the storage area (coordinates in the absolute coordinate system within the storage area), It is determined that the article 40 has moved outside the storage area. On the other hand, if the article 40 has not reached the entrance/exit position of the storage area, it is determined that the article 40 has entered the blind spot of the storage camera device 10 (it is highly likely that the article 40 exists somewhere within the storage area). Note that the judgment criteria are not limited to this, for example, by referring to the first trajectory information to check the article position information for the past several times, and determining the position of the article 40 from the moving direction or from the calculation results of the moving direction and moving speed. You may decide to move outside.

For example, if it is determined that the position of the article 40 could not be specified because it was moved outside the storage area (step S14, Yes), the article position management unit 304 stores the article position information of the article 40 that has been moved outside the storage area. (first trajectory information regarding this article 40) is deleted from the storage unit 305 (step S15), and the process shifts to a state of waiting for image data (step S11).

On the other hand, if it is determined that the position of the article 40 could not be specified because it entered the blind spot of the storage camera device 10 (step S14, No), the article position management unit 304 determines the position of the article 40 based on the first trajectory information. It is determined whether the last location where the article 40 entered is in the area around the entrance/exit (step S16). If the last identified position is outside the area around the entrance/exit (step S16, No), the article position management unit 304 maintains the article position information of the article 40 that entered the blind spot outside the area around the entrance, and The first trajectory information is updated by storing information indicating that the item position information (for example, the absolute position of the item position information is "Not found") as the latest item position information in the storage unit 305. state (step S11).

Furthermore, if the last position identified based on the first trajectory information is in the area around the entrance/exit (step S16, Yes), the article position management unit 304 checks the second trajectory information, Regarding the article 40 whose position could not be specified after the last confirmation, it is determined whether the article 40 has been moved outside the storage area (step S17). This judgment criterion, for example, refers to the second trajectory information, and determines whether the position of the article 40 whose position could not be specified after the last confirmation in the area around the entrance/exit is the location of the entrance/exit of the storage area (absolute coordinates within the storage area). coordinates in the system), it is determined that the article 40 has moved outside the storage area. On the other hand, in other cases, it is determined that the article 40 has entered the blind spot of the storage camera device 10 (it may exist somewhere within the storage area). Other cases include a case where the article 40 has not reached the entrance/exit position of the storage area, a case where the article 40 is not photographed by either the storage camera device 10 or the entrance/exit monitoring camera device 20, etc. include. Note that the judgment criteria is not limited to this, for example, by referring to the second trajectory information to check the article position information for the past several times, and determining the position of the article 40 from the moving direction or from the calculation results of the moving direction and moving speed. You may decide to move outside.

For example, when it is determined that the article 40 has been moved outside the storage area (step S18, Yes), the article position management unit 304 stores the article position information of the article 40 that has been moved outside the storage area (the first (trajectory information) from the storage unit 305 (step S15), and shifts to a state of waiting for image data (step S11). Further, if it is determined that the article 40 has not been moved outside the storage area (step S18, No), the article position management unit 304 maintains the article position information of the article 40 that has entered the blind spot in the area around the entrance, Information indicating that the object has entered the blind spot ("Not found") is stored in the storage unit 305 as the latest article position information, thereby updating the first trajectory information and transitioning to a state of waiting for image data (step S11). .

In this manner, by performing steps S1 to S7 and steps S11 to S18, the position of the article 40 being stored in the storage area can be precisely specified. Furthermore, even if the position of the article 40 being stored cannot be specified in the storage area, by referring to the first trajectory information and the second trajectory information regarding the article 40, the location of the article 40 can be determined. It becomes possible to estimate the location with high accuracy.

[Specific example of position calculation process]
Here, the article position calculation process described in steps S3 to S6 in FIG. 9 will be specifically explained with reference to FIG. 11. Here, as an example, position calculation processing by the first article position specifying unit 302 using image data acquired from the storage camera device 10 will be described. The position calculation process of the second article position specifying unit 303 is the same except that the position calculation process is performed using the image data acquired from the entrance/exit monitoring camera device 20, so a detailed explanation will be omitted.

First, the storage camera device 10 photographs the article marker 50.

The first article position specifying unit 302 acquires image data in which the article marker 50 is photographed.

The first article position specifying unit 302 reads the details of the storage camera device 10 and the article marker 50 that captured the image data from the storage unit 305 (camera-related information, article marker information), and uses the PnP method to locate the storage camera device 10 and the article marker 50. The relative position aB of the position B of the article marker 50 with respect to the position a is calculated.

Next, the first article position specifying unit 302 uses the coordinate information (absolute position, absolute direction) of the storage camera device 10 to move from the position a based on the absolute direction of the storage camera device 10 to the position B of the article marker 50. Find the direction vector of. Specifically, the first article position specifying unit 302 calculates the direction vector raB from the position a to the position B based on the absolute direction of the storage camera device 10 using the following equation (3).
raB=rAmat*aB…(3)
Note that rAmat in equation (3) is a rotation matrix indicating the absolute direction of the storage camera device 10.

Next, the first article position specifying unit 302 calculates the absolute position rB of the article marker 50 using the following equation (4).
rB=rA+raB...(4)
Note that rA in equation (4) is the absolute position of the camera device (absolute coordinates of position a). Furthermore, when registering the absolute direction of the article marker 50 in the article position information, the first article position specifying unit 302 uses "raB" in the above equation (4).

As described above, the article position management device 30 according to the first embodiment includes a first article position specifying section 302, a second article position specifying section 303, and an article position management section 304. The first article position specifying unit 302 specifies the position of the article 40 based on the image data of the entire storage area, and creates first trajectory information in which the position is accumulated in chronological order. Further, the second article position specifying unit 303 specifies the position of the article 40 based on the image data of the area around the entrance and exit, and creates second trajectory information in which the position is accumulated in chronological order. The article position management unit 304 then manages the position of the article 40 based on the first trajectory information and the second trajectory information.

More specifically, when an article 40 whose position cannot be determined during storage is detected, the article position management unit 304 determines whether the article 40 has moved out of the storage area based on the first trajectory information. A first judgment process is performed. Furthermore, for articles 40 that cannot be determined to have moved out of the storage area in the first determination process, the article position management unit 304 determines the position of the article 40 based on the last confirmed position and the second trajectory information. A second judgment process is performed to judge whether the object has moved outside the storage area. If it is determined in the first or second determination process that the article 40 has moved out of the storage area, the first trajectory information regarding the article 40 is deleted.

Thereby, the article position management unit 304 can accurately manage the position of the article 40 based on the first trajectory information when there is no article 40 whose position cannot be specified. Furthermore, even if an article 40 whose position cannot be specified during storage is detected, it can be determined that it has moved outside the storage area, so the number of articles 40 that has been moved outside the storage area can be determined. By knowing this, you can accurately manage the quantity of items in storage.

In addition, the article location management unit 304 is able to determine whether or not the article 40 whose position cannot be specified during storage has moved outside the storage area. Regarding the articles 40 that are determined not to have moved outside, it can be estimated that the articles 40 have entered the blind spot of the storage camera device 10 within the storage area.

Furthermore, in the article position management system of this embodiment, there is no need to attach a wireless communication means to the article 40 to be managed. Furthermore, the article markers 50 necessary for managing the positions of the articles 40 of this embodiment are available as printed matter, and even if the number of articles 40 increases, the cost required to acquire the markers is limited. Therefore, position management of the article 40 can be realized at low cost.

[Second embodiment]
The second embodiment will be described in detail using the drawings. In this embodiment, elements that can be explained in the same manner as in the first embodiment are given the same reference numerals so that repeated explanation can be omitted.

In the first embodiment, it is assumed that the camera device is a fixed type, and that the article position management device 30 can grasp the coordinate information (absolute position, absolute direction) of the camera device by referring to camera-related information. In the second embodiment, for example, when the position of the camera device is moved, the coordinate information (absolute position, absolute direction) of the camera device after the movement is calculated and the camera-related information is updated to determine the article position. The management device 30 grasps the coordinate information of the camera device after the movement. Thereby, even if the position of the camera device is changed, the article position management device 30 can manage the position of the stored article 40 and the movement of the article 40 in the same manner as in the first embodiment. .

FIG. 12 is a diagram illustrating an example of a schematic configuration of an article position management system according to the second embodiment. As shown in FIG. 12, the article position management system includes a storage camera device 10, an entrance/exit monitoring camera device 20, and an article position management device 30. Note that although FIG. 12 shows, as an example, the case where the position of the storage camera device 10 that can photograph the entire storage area is changed, the same operation is possible when the position of the access monitoring camera device 20 is changed. .

[Reference point marker]
As shown in FIG. 12, the storage area includes at least one reference point marker 60. As for the reference point marker 60, an AR marker can be used similarly to the article marker 50.

The reference point marker 60 has coordinate information (absolute position, absolute direction) and marker size. For example, although one reference point marker 60 is illustrated in FIG. 12, a plurality of reference point markers may be included. When a plurality of reference point markers 60 are prepared, each of the reference point markers 60 has different coordinate information. Note that regarding the size of the reference point marker 60 (marker size), the size of each marker may be the same or different. Any size can be used for each marker as long as it is predetermined.

Further, the coordinate information possessed by the reference point marker 60 is coordinate information (absolute position, absolute direction) in an absolute coordinate system within the storage area. The reference point marker 60 is an identifier that can calculate the relative position between the marker and the camera device, and the PnP method can be used to calculate the relative position. That is, the relative position of the camera device is calculated as a solution to the PnP problem. Note that calculation of the relative position using the AR marker is the same as in the first embodiment.

[Article position management device]
FIG. 13 is a diagram showing an example of a processing configuration (processing module) of the article position management device 30 according to the second embodiment. As shown in FIG. 13, in addition to the configuration of the first embodiment, the article position management device 30 of the second embodiment further includes a camera position specifying section 306, and the storage section 305 includes a reference point marker 60. Information regarding is stored. Note that in the following description, information regarding the reference point marker 60 will be referred to as "reference point marker information."

FIG. 14 is a diagram showing an example of reference point marker information. As shown in FIG. 14, the reference point marker information is information in which the identifier of the reference point marker, the coordinate information (absolute position, absolute direction) of the reference point marker, and the size of the reference point marker are associated with each other. Note that FIG. 14 illustrates a case where a plurality of reference point markers 60 are prepared in the storage area.

The identifier of the reference point marker 60 is, for example, information (for example, a file name, etc.) that specifies the pattern file of the above-mentioned AR marker. The absolute position and absolute direction of the reference point marker 60 are coordinate information in the absolute coordinate system within the storage area. The size of the reference point marker 60 is the length of one side when the marker is square, and the length and width when the marker is rectangular. In FIG. 14, the reference point marker is a square and the length of one side is described.

[Specific example of position calculation process]
FIG. 15 is a diagram illustrating an example of position calculation processing of the camera device. The position calculation process of the camera device when the position of the camera device is changed will be specifically described with reference to FIG. 15.

First, the camera device photographs the reference point marker 60.

The camera position specifying unit 306 acquires image data captured by the reference point marker 60.

Next, the camera position specifying unit 306 reads the details of the camera device that captured the image data and the reference point marker 60 from the storage unit 305 (camera related information, reference point marker information), and uses the PnP method to locate the reference point marker 60. The relative position Ca of the position a of the camera device with respect to the position C of is calculated.

Next, the camera position specifying unit 306 uses the coordinate information (absolute position, absolute direction) of the reference point marker 60 to determine a direction vector from the position C to the position a of the camera device based on the absolute direction of the reference point marker 60. demand. Specifically, the camera position specifying unit 306 calculates the direction vector rCa from the position C to the position a based on the absolute direction of the reference point marker 60 using the following equation (5).
rCa=rCmat*Ca...(5)
Note that rCmat in equation (5) is a rotation matrix indicating the absolute direction of the reference point marker 60.

Next, the camera position specifying unit 306 uses the following equation (6) to find the absolute position ra of the camera device.
ra=rC+rCa...(6)
Note that rC in equation (6) is the absolute position of the reference point marker 60 (absolute coordinates of position C). Furthermore, when registering the absolute direction of the camera device in the camera-related information, the camera position specifying unit 306 uses "rCa" in the above equation (6).

As described above, in the second embodiment, when at least one of the storage camera device 10 and the access monitoring camera device 20 is moved, the camera position specifying unit 306 determines the position of the camera device after the movement (storage area (coordinate information in the absolute coordinate system within). Specifically, first, the camera device photographs the reference point marker 60. Next, the camera position specifying unit 306 acquires image data including the reference point marker 60 from the camera device. Then, the camera position specifying unit 306 calculates the relative position of the camera device with respect to the reference point marker 60 whose coordinates in the absolute coordinate system within the storage area are predetermined, and uses the coordinate information of the reference point marker 60 and the camera device. The relative position of the camera device is used to identify the position of the camera device after the movement.

Thereby, the position of the camera device can be freely changed according to the changing amount and arrangement of the articles 40. Furthermore, if the coordinate information in the absolute coordinate system within the storage area of the camera device can be specified, then the article position management device 30 manages the position of the article 40 within the storage area in the same manner as in the first embodiment. be able to.

[Modified example]
Note that in the first and second embodiments described above, the embodiment is described in which image data is acquired from two camera devices, but the invention is not limited to this, and for example, images acquired from one storage camera device 10 It is also conceivable to manage the position of the article 40 based on data.

For example, the storage camera device 10 switches and photographs the storage area and the area around the entrance/exit. Specifically, when photographing the area around the entrance/exit, the zoom function or the like of the storage camera device 10 is used.

The first article position specifying unit 302 specifies the position of the article by executing the processes of steps S1 to S7 shown in FIG. 9 based on the image data of the entire storage area acquired from the storage camera device 10, First trajectory information is created in which the positions are accumulated in chronological order.

On the other hand, the second article position specifying unit 303 executes the processes of steps S1 to S7 shown in FIG. The second trajectory information is created by chronologically accumulating the specified positions.

Then, the article position management unit 304 manages the position of the article 40 by executing steps S11 to S18 shown in FIG. 10, as in the first embodiment.

Further, when the storage camera device 10 is moved, it is also possible to specify the position of the storage camera device 10 after the movement by performing position calculation processing in the same manner as in the second embodiment.

In the above modification, the storage camera device 10 switches to take pictures of the storage area and the area around the entrance/exit. However, for example, a modification may be made in which the storage camera device 10 constantly takes pictures of the storage area without switching the captured images. Examples are also possible. In this case, the second article position specifying unit 303 extracts image data of the area around the entrance from the image data of the entire storage area acquired from the storage camera device 10 (for example, performs image processing such as cropping), and extracts the image data around the extracted entrance and exit. The position of the article 40 is specified based on the image data of the area (second trajectory information is created). Other operations are the same as above.

[Third embodiment]
Next, a third embodiment of the present invention will be described with reference to FIG. 16. The first embodiment described above is a specific embodiment, but the third embodiment is a more generalized embodiment.

[Configuration of article position management device]
FIG. 16 is a block diagram illustrating an example of a schematic configuration of an article position management device according to the third embodiment. Referring to FIG. 16, the article location management device 100 includes a first location identification section 101, a second location identification section 102, and a location management section 103.

[Technical features]
Next, technical features of the third embodiment will be explained.

In the third embodiment, the article position management device 100 (first position specifying unit 101) specifies the position of the article based on image data of the entire storage area of the article, and stores the position in a time-series manner. Create trajectory information for 1.

The article position management device 100 (second position specifying unit 102) specifies the position of the article based on the image data of the area around the entrance in the storage area, and generates second trajectory information that stores the position in chronological order. Create.

The article position management device 100 (position management section 103) manages the position of articles. Further, the article position management device 100 (position management unit 103) is configured to detect an article whose position cannot be specified during storage based on the latest position specification result in the first position specification unit 101. A first determination process is performed to determine whether or not the article has moved out of the storage area based on the first trajectory information, and for articles that cannot be determined to have moved out of the storage area in the first determination process, , further performing a second determination process of determining whether the article has moved out of the storage area based on the last confirmed position and the second trajectory information; If it is determined in the determination process that the article has moved outside the storage area, the first trajectory information regarding the article is deleted.

For example, the article position management device 100 may perform the operations of the article position management device 30 according to the first embodiment.

The third embodiment has been described above. According to the third embodiment, for example, it is possible to manage the position of an article to be managed with high accuracy and at low cost.

[Hardware configuration of article position management device]
Next, the hardware of each device constituting the article position management system will be explained. FIG. 17 is a diagram showing an example of the hardware configuration of the article position management device 30.

The article position management device 30 can be configured by an information processing device (computer), and has a configuration illustrated in FIG. 17 . For example, the article location management device 30 includes a processor 311, a memory 312, an input/output interface 313, a communication interface 314, and the like. The components such as the processor 311 are connected by an internal bus or the like and are configured to be able to communicate with each other.

However, the configuration shown in FIG. 17 is not intended to limit the hardware configuration of the article position management device 30. The article position management device 30 may include hardware (not shown), and may not include the input/output interface 313 if necessary. Furthermore, the number of processors 311 and the like included in the article position management device 30 is not limited to the example shown in FIG. 17; for example, a plurality of processors 311 may be included in the article position management device 30.

The processor 311 is, for example, a programmable device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). Alternatively, the processor 311 may be a device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The processor 311 executes various programs including an operating system (OS).

The memory 312 is a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. The memory 312 stores OS programs, application programs, and various data.

The input/output interface 313 is an interface for a display device or input device (not shown). The display device is, for example, a liquid crystal display. The input device is, for example, a device such as a keyboard or a mouse that receives user operations.

The communication interface 314 is a circuit, module, etc. that communicates with other devices. For example, the communication interface 314 includes a NIC (Network Interface Card).

The functions of the article position management device 30 are realized by various processing modules. The processing module is realized, for example, by the processor 311 executing a program stored in the memory 312. Further, the program can be recorded on a computer-readable storage medium. The storage medium can be non-transitory, such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, etc. That is, the present invention can also be implemented as a computer program product. Furthermore, the above program can be updated via a network or by using a storage medium that stores the program. Furthermore, the processing module may be realized by a semiconductor chip.

Note that the control element (control module) such as a camera device can also be configured by an information processing device in the same way as the article position management device 30, and its basic hardware configuration is not different from the article position management device 30. The explanation will be omitted.

Part or all of the above embodiments may be described as in the following supplementary notes, but are not limited to the following.

(Additional note 1)
a first position specifying unit that identifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order;
a second position specifying unit that specifies the position of the article based on the image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order;
a location management department that manages the location of items;
Equipped with
The location management unit includes:
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning section, the article moves out of the storage area based on the first trajectory information. Perform a first judgment process to judge whether or not the object has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. Perform a second judgment process to judge whether the object has moved,
If it is determined in the first or second determination process that the article has moved outside the storage area, the first trajectory information regarding the article is deleted;
Article position management device.

(Additional note 2)
The location management unit includes:
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
The article position management device according to Supplementary Note 1.

(Additional note 3)
The location management unit includes:
managing the position of the article newly transported to the storage area based on the first trajectory information;
The article position management device according to supplementary note 1 or 2.

(Additional note 4)
The first and second position specifying units specify the position of the article based on the article marker when the image data includes an article marker that can identify the article.
The article position management device according to appendix 1, 2 or 3.

(Appendix 5)
The first position specifying unit is
Calculate the relative position of the article marker with respect to a first camera device that has photographed the entire storage area, and use the coordinate information of the first camera device and the relative position of the article marker to determine the position of the article in the storage area. identify,
The second position specifying unit is
The relative position of the article marker with respect to the second camera device that photographed the area around the entrance is calculated, and the position of the article in the area around the entrance is calculated using the coordinate information of the second camera device and the relative position of the article marker. pinpoint location,
The article position management device according to appendix 4.

(Appendix 6)
When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. a camera position specifying unit that calculates and specifies the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further comprising,
The article position management device according to appendix 5.

(Appendix 7)
The first position specifying unit is
Calculate the relative position of the article marker included in the image data of the entire storage area with respect to a camera device that can switch between the storage area and the area around the entrance and exit, and calculate the relative position of the article marker included in the image data of the entire storage area, and calculate the coordinate information of the camera device and the article marker. locating the item in the storage area using the relative position;
The second position specifying unit is
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
The article position management device according to appendix 4.

(Appendix 8)
The first position specifying unit is
Calculating the relative position of the article marker included in the image data of the entire storage area with respect to a camera device capable of photographing the entire storage area, and using the coordinate information of the camera device and the relative position of the article marker, locating the item in the storage area;
The second position specifying unit is
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
The article position management device according to appendix 4.

(Appendix 9)
When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. a camera position specifying unit that specifies the position of the camera device after movement using the relative position of the device;
further comprising,
The article position management device according to appendix 7 or 8.

(Appendix 10)
a first position specifying unit that identifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order;
a second position specifying unit that specifies the position of the article based on the image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order;
a location management department that manages the location of items;
An article position management device including:
The position management unit of the article position management device includes:
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning section, the article moves out of the storage area based on the first trajectory information. Perform a first judgment process to judge whether or not the object has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. Perform a second judgment process to judge whether the object has moved,
If it is determined in the first or second determination process that the article has moved outside the storage area, the first trajectory information regarding the article is deleted;
Goods location management system.

(Appendix 11)
The position management unit of the article position management device includes:
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
Article position management system according to appendix 10.

(Appendix 12)
The position management unit of the article position management device includes:
managing the position of the article newly transported to the storage area based on the first trajectory information;
The article position management system according to appendix 10 or 11.

(Appendix 13)
The first and second position specifying units of the article position management device specify the position of the article based on the article marker when the image data includes an article marker that can identify the article.
The article position management system according to appendix 10, 11 or 12.

(Appendix 14)
a first camera device capable of photographing the entire storage area;
a second camera device capable of photographing the area around the entrance;
Furthermore,
The first position specifying unit of the article position management device includes:
calculating the relative position of the article marker with respect to the first camera device, and identifying the position of the article in the storage area using the coordinate information of the first camera device and the relative position of the article marker;
The second position specifying unit of the article position management device includes:
calculating the relative position of the article marker with respect to the second camera device, and identifying the position of the article in the area around the entrance using the coordinate information of the second camera device and the relative position of the article marker;
Article position management system according to appendix 13.

(Appendix 15)
The article position management device includes:
When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. a camera position specifying unit that calculates and specifies the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further comprising,
Article position management system according to appendix 14.

(Appendix 16)
a camera device capable of switching and photographing the storage area and the area around the entrance;
More prepared,
The first position specifying unit of the article position management device includes:
The relative position of the article marker included in the image data of the entire storage area with respect to the camera device is calculated, and the position of the article in the storage area is calculated using the coordinate information of the camera device and the relative position of the article marker. identify,
The second position specifying unit of the article position management device includes:
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
Article position management system according to appendix 13.

(Appendix 17)
a camera device capable of photographing the entire storage area;
Furthermore,
The first position specifying unit of the article position management device includes:
The relative position of the article marker included in the image data of the entire storage area with respect to the camera device is calculated, and the position of the article in the storage area is calculated using the coordinate information of the camera device and the relative position of the article marker. identify,
The second position specifying unit of the article position management device includes:
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
Article position management system according to appendix 13.

(Appendix 18)
The article position management device includes:
When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. a camera position specifying unit that specifies the position of the camera device after movement using the relative position of the device;
further comprising,
The article position management system according to appendix 16 or 17.

(Appendix 19)
performing a first position specifying process of specifying the position of the article based on image data of the entire storage area of the article and creating first trajectory information in which the position is accumulated in chronological order;
performing a second position specifying process of specifying the position of the article based on image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order;
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article moves out of the storage area based on the first trajectory information. performing a first determination process to determine whether the vehicle has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. performing a second determination process to determine whether or not the vehicle has moved;
deleting first trajectory information regarding the article if it is determined in the first or second determination process that the article has moved outside the storage area;
Article location management method including.

(Additional note 20)
moreover,
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
including,
Article position management method according to appendix 19.

(Additional note 21)
managing the position of the article newly transported to the storage area based on the first trajectory information;
including,
The article position management method according to appendix 19 or 20.

(Additional note 22)
In the first and second position specifying processes, when the image data includes an article marker by which the article can be identified, the position of the article is specified based on the article marker.
The article position management method according to appendix 19, 20 or 21.

(Additional note 23)
In the first location specifying process,
Calculate the relative position of the article marker with respect to a first camera device that has photographed the entire storage area, and use the coordinate information of the first camera device and the relative position of the article marker to determine the position of the article in the storage area. identify,
In the second location specifying process,
The relative position of the article marker with respect to the second camera device that photographed the area around the entrance is calculated, and the position of the article in the area around the entrance is calculated using the coordinate information of the second camera device and the relative position of the article marker. pinpoint location,
Article position management method according to appendix 22.

(Additional note 24)
When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. calculating and identifying the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further including,
Article position management method according to appendix 23.

(Additional note 25)
In the first location specifying process,
Calculate the relative position of the article marker included in the image data of the entire storage area with respect to a camera device that can switch between the storage area and the area around the entrance and exit, and calculate the relative position of the article marker included in the image data of the entire storage area, and calculate the coordinate information of the camera device and the article marker. locating the item in the storage area using the relative position;
In the second location specifying process,
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
Article position management method according to appendix 22.

(Additional note 26)
In the first location specifying process,
Calculating the relative position of the article marker included in the image data of the entire storage area with respect to a camera device capable of photographing the entire storage area, and using the coordinate information of the camera device and the relative position of the article marker, locating the item in the storage area;
In the second location specifying process,
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
Article position management method according to appendix 22.

(Additional note 27)
When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. specifying the position of the camera device after the movement using the relative position of the device;
further including,
The article position management method according to appendix 25 or 26.

(Additional note 28)
performing a first position specifying process of specifying the position of the article based on image data of the entire storage area of the article and creating first trajectory information in which the position is accumulated in chronological order;
performing a second position specifying process of specifying the position of the article based on image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order;
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article moves out of the storage area based on the first trajectory information. performing a first determination process to determine whether the vehicle has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. performing a second determination process to determine whether or not the vehicle has moved;
deleting first trajectory information regarding the article if it is determined in the first or second determination process that the article has moved outside the storage area;
A program that causes a processor to execute.

Note that the form of attachment 28 can be developed into the forms of attachment 2 to attachment 9, similar to the form of attachment 1.

10 Storage camera device 20 Entry/exit monitoring camera device 30, 100 Article position management device 40 Article 50 Article marker 60 Reference point marker 101 First position specifying section 102 Second position specifying section 103 Position management section 201 Camera control section 202 Data output Units 203 and 305 Storage unit 301 Data input unit 302 First article position specifying unit 303 Second article position specifying unit 304 Article position management unit 306 Camera position specifying unit 311 Processor 312 Memory 313 Input/output interface 314 Communication interface

Claims (28)

a first position specifying unit that identifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order;
a second position specifying unit that specifies the position of the article based on the image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order;
a location management department that manages the location of items;
Equipped with
The location management unit includes:
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning section, the article moves out of the storage area based on the first trajectory information. Perform a first judgment process to judge whether or not the object has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. Perform a second judgment process to judge whether the object has moved,
If it is determined in the first or second determination process that the article has moved outside the storage area, the first trajectory information regarding the article is deleted;
Article position management device. The location management unit includes:
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
The article position management device according to claim 1. The location management unit includes:
managing the position of the article newly transported to the storage area based on the first trajectory information;
The article position management device according to claim 1 or 2. The first and second position specifying units specify the position of the article based on the article marker when the image data includes an article marker that can identify the article.
The article position management device according to claim 1, 2, or 3. The first position specifying unit is
The relative position of the article marker with respect to a first camera device capable of photographing the entire storage area is calculated, and the coordinate information of the first camera device and the relative position of the article marker are used to determine the position of the article in the storage area. identify the location,
The second position specifying unit is
The relative position of the article marker with respect to a second camera device capable of photographing the area around the entrance is calculated, and the coordinate information of the second camera device and the relative position of the article marker are used to calculate the position of the article marker in the area around the entrance. locate the
The article position management device according to claim 4. When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. a camera position specifying unit that calculates and specifies the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further comprising,
The article position management device according to claim 5. The first position specifying unit is
Calculate the relative position of the article marker included in the image data of the entire storage area with respect to a camera device that can switch between the storage area and the area around the entrance and exit, and calculate the relative position of the article marker included in the image data of the entire storage area, and calculate the coordinate information of the camera device and the article marker. locating the item in the storage area using the relative position;
The second position specifying unit is
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
The article position management device according to claim 4. The first position specifying unit is
Calculating the relative position of the article marker included in the image data of the entire storage area with respect to a camera device capable of photographing the entire storage area, and using the coordinate information of the camera device and the relative position of the article marker, locating the item in the storage area;
The second position specifying unit is
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
The article position management device according to claim 4. When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. a camera position specifying unit that specifies the position of the camera device after movement using the relative position of the device;
further comprising,
The article position management device according to claim 7 or 8. a first position specifying unit that identifies the position of the article based on image data of the entire storage area of the article and creates first trajectory information in which the position is accumulated in chronological order;
a second position specifying unit that specifies the position of the article based on the image data of the area around the entrance in the storage area and creates second trajectory information in which the position is accumulated in chronological order;
a location management department that manages the location of items;
An article position management device including:
The position management unit of the article position management device includes:
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning section, the article moves out of the storage area based on the first trajectory information. Perform a first judgment process to judge whether or not the object has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. Perform a second judgment process to judge whether the object has moved,
If it is determined in the first or second determination process that the article has moved outside the storage area, the first trajectory information regarding the article is deleted;
Goods location management system. The position management unit of the article position management device includes:
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
The article position management system according to claim 10. The position management unit of the article position management device includes:
managing the position of the article newly transported to the storage area based on the first trajectory information;
The article position management system according to claim 10 or 11. The first and second position specifying units of the article position management device specify the position of the article based on the article marker when the image data includes an article marker that can identify the article.
The article position management system according to claim 10, 11 or 12. a first camera device capable of photographing the entire storage area;
a second camera device capable of photographing the area around the entrance;
Furthermore,
The first position specifying unit of the article position management device includes:
calculating the relative position of the article marker with respect to the first camera device, and identifying the position of the article in the storage area using the coordinate information of the first camera device and the relative position of the article marker;
The second position specifying unit of the article position management device includes:
calculating the relative position of the article marker with respect to the second camera device, and identifying the position of the article in the area around the entrance using the coordinate information of the second camera device and the relative position of the article marker;
The article position management system according to claim 13. The article position management device includes:
When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. a camera position specifying unit that calculates and specifies the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further comprising,
The article position management system according to claim 14. a camera device capable of switching and photographing the storage area and the area around the entrance;
More prepared,
The first position specifying unit of the article position management device includes:
The relative position of the article marker included in the image data of the entire storage area with respect to the camera device is calculated, and the position of the article in the storage area is calculated using the coordinate information of the camera device and the relative position of the article marker. identify,
The second position specifying unit of the article position management device includes:
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
The article position management system according to claim 13. a camera device capable of photographing the entire storage area;
Furthermore,
The first position specifying unit of the article position management device includes:
The relative position of the article marker included in the image data of the entire storage area with respect to the camera device is calculated, and the position of the article in the storage area is calculated using the coordinate information of the camera device and the relative position of the article marker. identify,
The second position specifying unit of the article position management device includes:
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
The article position management system according to claim 13. The article position management device includes:
When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. a camera position specifying unit that specifies the position of the camera device after movement using the relative position of the device;
further comprising,
The article position management system according to claim 16 or 17. performing a first position specifying process of specifying the position of the article based on image data of the entire storage area of the article and creating first trajectory information in which the position is accumulated in chronological order;
performing a second position specifying process of specifying the position of the article based on image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order;
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article moves out of the storage area based on the first trajectory information. performing a first determination process to determine whether the vehicle has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. performing a second determination process to determine whether or not the vehicle has moved;
deleting first trajectory information regarding the article if it is determined in the first or second determination process that the article has moved outside the storage area;
Article location management method including. moreover,
Among the articles whose location cannot be determined during the storage, the location of the articles other than those determined to have moved outside the storage area in the first determination process and the second determination process is determined to be the same. adding information indicating that it has not been completed to the first trajectory information;
including,
The article position management method according to claim 19. managing the position of the article newly transported to the storage area based on the first trajectory information;
including,
The article position management method according to claim 19 or 20. In the first and second position specifying processes, when the image data includes an article marker by which the article can be identified, the position of the article is specified based on the article marker.
The article position management method according to claim 19, 20 or 21. In the first location specifying process,
The relative position of the article marker with respect to a first camera device capable of photographing the entire storage area is calculated, and the coordinate information of the first camera device and the relative position of the article marker are used to determine the position of the article in the storage area. identify the location,
In the second location specifying process,
The relative position of the article marker with respect to a second camera device capable of photographing the area around the entrance is calculated, and the coordinate information of the second camera device and the relative position of the article marker are used to calculate the position of the article marker in the area around the entrance. locate the
The article position management method according to claim 22. When at least one of the first and second camera devices is moved, the relative position of the moved camera device with respect to a reference point marker whose coordinates in an absolute coordinate system are predetermined within the storage area is determined. calculating and identifying the position of the moved camera device using the coordinate information of the reference point marker and the relative position of the moved camera device;
further including,
The article position management method according to claim 23. In the first location specifying process,
Calculate the relative position of the article marker included in the image data of the entire storage area with respect to a camera device that can switch between the storage area and the area around the entrance and exit, and calculate the relative position of the article marker included in the image data of the entire storage area, and calculate the coordinate information of the camera device and the article marker. locating the item in the storage area using the relative position;
In the second location specifying process,
Calculate the relative position of the article marker included in the image data of the area around the entrance with respect to the camera device, and use the coordinate information of the camera device and the relative position of the article marker to calculate the position of the article in the area around the entrance. pinpoint location,
The article position management method according to claim 22. In the first location specifying process,
Calculating the relative position of the article marker included in the image data of the entire storage area with respect to a camera device capable of photographing the entire storage area, and using the coordinate information of the camera device and the relative position of the article marker, locating the item in the storage area;
In the second location specifying process,
Calculate the relative position of the article marker included in the image data of the area around the entrance and exit extracted from the image data of the entire storage area with respect to the camera device, and calculate the relative position of the article marker and the coordinate information of the camera device. identifying the position of the article in the area around the entrance using
The article position management method according to claim 22. When the camera device is moved, the relative position of the camera device with respect to a reference point marker whose coordinates in the absolute coordinate system are predetermined within the storage area is calculated, and the coordinate information of the reference point marker and the camera are calculated. specifying the position of the camera device after the movement using the relative position of the device;
further including,
The article position management method according to claim 25 or 26. performing a first position specifying process of specifying the position of the article based on image data of the entire storage area of the article and creating first trajectory information in which the position is accumulated in chronological order;
performing a second position specifying process of specifying the position of the article based on image data of the area around the entrance in the storage area and creating second trajectory information in which the position is accumulated in chronological order;
When an article whose position cannot be specified during storage is detected based on the latest positioning result in the first positioning process, the article moves out of the storage area based on the first trajectory information. performing a first determination process to determine whether the vehicle has moved;
For articles that cannot be determined to have moved out of the storage area in the first determination process, the article is further moved out of the storage area based on the last confirmed position of the article and the second trajectory information. performing a second determination process to determine whether or not the vehicle has moved;
deleting first trajectory information regarding the article if it is determined in the first or second determination process that the article has moved outside the storage area;
A program that causes a processor to execute.

Images