JP6817643B2 - Information processing device - Google Patents

Description

The technology disclosed herein relates to an information processing device used to display information relating to the arrangement of luggage loaded in the luggage compartment.

For example, Japanese Patent Application Laid-Open No. 2014-151978 (hereinafter referred to as Patent Document 1) discloses a system for calculating an arrangement when a cargo is loaded on a trailer. This system has a data input unit that inputs baggage information and trailer information, a loading calculation unit that calculates the placement of the baggage in the trailer based on the input baggage information and trailer information, and a baggage based on the calculation result. It is provided with a result output unit that outputs an arrangement image of the trailer. The efficiency of the loading work is improved by having the workers load the luggage on the actual trailer according to the arrangement image.

In the system of Patent Document 1, the arrangement image is generated as an image of an aspect including a trailer image of a trailer and a luggage image of a luggage. The trailer image and the luggage image included in the arrangement image are output in a manner in which the arrangement of each luggage on the trailer can be grasped from a bird's-eye view, for example, as shown in a perspective view.

However, the viewpoint of the worker when performing the loading work on the actual trailer is often different from the bird's-eye view that can be grasped by the arrangement image. When a worker who should have grasped the image of the arrangement of luggage by looking at the arrangement image actually performs the loading work toward the actual trailer, the appearance inside the trailer from the worker's point of view is the trailer of the arrangement image. Since it is different from the image, it may be difficult to concretely imagine the arrangement of luggage with respect to the actual trailer. In such a case, the efficiency of the loading work by the worker may not be improved.

This specification discloses a technique that allows an operator to specifically grasp the arrangement of luggage with respect to an actual luggage compartment and to improve the efficiency of loading work.

The information processing apparatus disclosed by the present specification includes a spatial information measuring unit, a display unit, and a control unit. The control unit includes a first acquisition unit, a second acquisition unit, and a display control unit. The first acquisition unit is spatial information measured by the spatial information measurement unit, and acquires spatial information including the dimensions and orientation of the luggage compartment. The second acquisition unit acquires the arrangement information. The placement information includes luggage information related to the luggage compartment, luggage information for each of one or more luggage to be loaded in the luggage compartment, and luggage information including at least one of the dimensions and weight of the luggage. It is the result of calculating the arrangement of one or more luggage in each luggage compartment based on. The display control unit causes the display unit to display one or more object images indicating each of the one or more packages based on the acquired spatial information and the acquired arrangement information. Each of the one or more object images is displayed in a manner of being superimposed on the actual luggage compartment at the position calculated as the arrangement position in the luggage compartment of the luggage indicated by the object image.

According to the above configuration, the display control unit superimposes each of one or more object images on the display unit at the position calculated as the arrangement position of the luggage indicated by the object image in the luggage compartment in the actual luggage compartment. It can be displayed in such a manner. Therefore, a worker who loads luggage into the actual luggage compartment can arrange the luggage with respect to the actual luggage compartment by looking at one or more object images displayed in a manner superimposed on the actual luggage compartment. It becomes easier to imagine concretely. Therefore, according to the above-mentioned information processing device, it is possible to make the worker concretely grasp the arrangement of the luggage with respect to the actual luggage compartment and improve the efficiency of the loading work.

Here, the "luggage compartment" includes any chamber for loading luggage such as a container, a trailer, and a warehouse. The "spatial information measuring unit" includes an arbitrary device capable of measuring spatial information including the luggage compartment, such as a camera and an infrared sensor. Further, the "object image" may be either a still image or a moving image (that is, an image accompanied by movement) as long as it is an image showing luggage.

The display control unit displays a first object image showing the first luggage on the display unit in a situation where the second luggage should be placed after the first luggage in the luggage compartment, and the first object image is displayed. While displayed on the display unit, the second object image indicating the second luggage is displayed on the display unit in the first case where the first luggage is correctly arranged at the position indicated by the first object image. You may let it.

According to this configuration, the worker who sees the display unit may arrange the first baggage at the position indicated by the first object image while the first object image is displayed on the display unit. While the second object image is displayed on the display unit, it can be easily grasped that the second baggage should be arranged at the position indicated by the second object image. In a situation where the second baggage should be placed after the first baggage, the worker can properly place the first baggage and the second baggage without having to remember the position and order of the placement of each baggage. Can be placed. Therefore, it is not necessary for the worker to memorize the placement position and the placement order, the burden on the worker can be reduced, and the work efficiency can be improved.

The display control unit is a second object in the second case where the first baggage is not correctly arranged at the position indicated by the first object image while the first object image is displayed on the display unit. The image may not be displayed on the display unit.

According to this configuration, when the worker places the luggage at the position indicated by the first object image while the first object image is displayed on the display unit, the worker places the next luggage (that is, the first luggage). If the second object image indicating) is not displayed, it can be recognized that the placed luggage was not placed correctly. As a result, it is possible to suppress a situation in which each package is arranged in an arrangement different from the arrangement indicated by the arrangement information.

Here, in the "second case where the first baggage is not correctly arranged at the position indicated by the first object image", the baggage different from the first baggage is arranged (that is, the type of baggage is Incorrect) and incorrect position and orientation of the first package being placed, including at least one of them.

The information processing device may further include a notification unit. The control unit may further include a notification control unit that causes the notification unit to execute the notification operation in the second case.

According to this configuration, the notification unit executes the notification operation, so that the worker can more appropriately recognize that the arranged luggage has not been arranged correctly. It is possible to more appropriately suppress the situation where each baggage is arranged in an arrangement different from the arrangement indicated by the arrangement information.

The information processing device may include a terminal device and a server capable of communicating with the terminal device. The terminal device may include a spatial information measuring unit, a display unit, and a terminal control unit. The terminal control unit may include a first acquisition unit, a second acquisition unit, and a display control unit. The second acquisition unit may acquire the arrangement information by receiving the arrangement information from the server. The server may include a storage device and a server control unit. The storage device may store the arrangement information. The server may include an information transmission unit that transmits the arrangement information in the storage device to the terminal device.

According to this configuration, the server includes a storage device for storing the arrangement information in advance, and the terminal device acquires a predetermined arrangement information from the server. That is, the terminal device does not have to have a storage device for storing in advance. Therefore, a terminal device having a relatively small storage capacity can be used.

The terminal control unit may further include a notification unit that transmits a completion notification to the server when the arrangement of one or more packages corresponding to the one or more object images is completed.

As a result, the user of the server (for example, the manager of the worker) can know that the loading work in the luggage compartment has been completed.

The terminal device may further include a frame that can be worn on the user's head. The display unit may include a display device mounted on the frame and arranged at a position facing at least one of the right eye and the left eye of the user wearing the frame.

According to this configuration, the worker (that is, the user) can perform the loading work while looking at the object image displayed on the display unit with the terminal device attached to the head. In this case, since the object image is displayed on the display unit in a manner superimposed on the actual luggage compartment in the worker's field of view, the worker can arrange each luggage in the luggage compartment from his / her own point of view. Can be grasped properly.

The above-mentioned information processing device, terminal device, control method for realizing a server, a computer program, and a computer-readable medium for storing the computer program are also new and useful.

The outline of the management system is shown. The block diagram of the image display device is shown. The block diagram of the server is shown. The flowchart of the simulation process executed by the control part of the server is shown. A flowchart of the loading control process executed by the control unit of the image display device is shown. An example of the display part where the first object image is displayed is shown. An example of the display part where the second object image is displayed is shown.

(Example)
(Configuration of management system 2; FIG. 1)
The management system 2 shown in FIG. 1 shows the efficiency of the luggage in the luggage compartment with respect to the worker (for example, the worker W) who performs the loading work of loading the luggage (for example, the luggage P) in the luggage compartment (for example, the container C). It is a system for instructing the appropriate placement and managing the loading work of workers. The management system 2 includes an image display device 10 and a server 50. The image display device 10 and the server 50 can mutually execute wireless communication (specifically, Wi-Fi communication) via the Internet 4.

(Structure of image display device 10; FIGS. 1 and 2)
The image display device 10 is an image display device (so-called head-mounted display) that is used by being attached to the head of a user (for example, worker W). As shown in FIG. 1, the image display device 10 includes a frame 12, a right display unit 14R, a left display unit 14L, a right projection unit 15R, a left projection unit 15L, a right camera 16R, and a left camera 16L. , A control box 18 and a speaker 19 are provided.

The frame 12 is a member in the shape of a spectacle frame. The user can wear the image display device 10 on the head by wearing the frame 12 as if wearing glasses.

The right display unit 14R and the left display unit 14L are translucent display members, respectively. When the user attaches the image display device 10 to the head, the right display unit 14R is arranged at a position facing the user's right eye, and the left display unit 14L is arranged at a position facing the left eye. Hereinafter, the right display unit 14R and the left display unit 14L may be collectively referred to as "display unit 14".

The right projection unit 15R and the left projection unit 15L are members that project an image onto the right display unit 14R and the left display unit 14L, respectively. The right projection unit 15R and the left projection unit 15L are provided on the side portions of the right display unit 14R and the left display unit 14L, respectively. Hereinafter, the right projection unit 15R and the left projection unit 15L may be collectively referred to as “projection unit 15”. In this embodiment, the projection unit 15 projects a predetermined virtual image (hereinafter, referred to as an “object image”) showing the luggage P on the display unit 14 according to the instruction of the control unit 30. As a result, the user can see the object in the real world (for example, container C) or / or the object in the real world as if the object image is superimposed on the predetermined position in the space through the display unit 14. You can see objects and / and spaces and object images. Hereinafter, in the present specification, when explaining that the control unit 30 causes the display unit 14 to display a desired image by instructing the projection unit 15 to project an image, the operation of the projection unit 15 will be described. It may be omitted and simply expressed as "the control unit 30 causes the display unit 14 to display a desired image".

The right camera 16R is a camera arranged at a position above the right display unit 14R (that is, a position corresponding to above the user's right eye) in the frame 12. On the other hand, the left camera 16L is a camera arranged at a position above the left display unit 14L (that is, a position corresponding to above the left eye of the user) in the frame 12. Each of the right camera 16R and the left camera 16L can shoot a range corresponding to the field of view of the user who wears the image display device 10 (hereinafter, may be referred to as a "specific range") from different angles. That is, it can be said that the right camera 16R captures the visual field range of the user's right eye, and the left camera 16L captures the visual field range of the user's left eye. Further, hereinafter, the right camera 16R and the left camera 16L may be collectively referred to as "camera 16".

The control box 18 is a control device built in a part of the frame 12. Each element that controls the control system of the image display device 10 is housed in the control box 18. Specifically, as shown in FIG. 2, the control box 18 houses the Wi-Fi interface 20, the control unit 30, and the memory 32. In the following, the interface will be referred to as "I / F". In another example, the control box 18 may be provided separately from the frame 12. In that case, each component (Wi-Fi interface 20, control unit 30, memory 32) in the control box 18 and each component (display unit 14, projection unit 15, camera 16, etc.) provided in the frame 12 Should be electrically connected by a cable or the like.

The Wi-Fi I / F 20 is an I / F for executing Wi-Fi communication with an external device (for example, a server 50) via the Internet 4.

The control unit 30 executes various processes including a stacking control process (see FIG. 5) described later according to a program stored in the memory 32. Further, the control unit 30 is electrically connected to the display unit 14, the projection unit 15, the camera 16, the Wi-Fi I / F20, and the memory 32, and can control the operation of each of these elements.

The memory 32 stores various programs. Further, the memory 32 is provided with a storage area for temporarily storing various information acquired in the execution process of the loading control process described later. Further, although not shown, the memory 32 also stores a device ID for identifying the image display device 10.

The speaker 19 is an output unit for outputting sound.

(Configuration of server 50; FIGS. 1 and 3)
The server 50 shown in FIG. 1 is a server installed by an administrator of the image display device 10 (for example, a management company for loading work). As shown in FIG. 3, the server 50 includes a display unit 52, an operation unit 54, a Wi-Fi I / F 56, a control unit 60, and a memory 60.

The display unit 52 is a display capable of displaying various information. The operation unit 54 includes a keyboard and a mouse. The user of the server 50 can input various instructions to the server 50 by operating the operation unit 54. The Wi-Fi I / F 56 is an I / F for executing Wi-Fi communication with an external device (for example, an image display device 10) via the Internet 4. The control unit 60 executes various processes including a simulation process (see FIG. 4) described later according to the program stored in the memory 62. The memory 62 stores various programs. Further, the memory 62 is provided with an arrangement information storage area 64 for storing the arrangement information generated by the control unit 60 executing the simulation process. The arrangement information is information indicating an appropriate arrangement and loading order of each package when loading one or more packages into a designated luggage compartment (for example, a container, a warehouse, etc.). By performing the loading work according to the arrangement and loading order of each luggage indicated by the arrangement information, each luggage can be efficiently stored in the luggage compartment.

(Simulation processing; Fig. 4)
A simulation process that is executed in advance by the control unit 60 of the server 50 of this embodiment will be described before the loading work by the worker is actually performed. The simulation process is a process for simulating an appropriate arrangement position and order of each package when one or more packages are loaded into a designated luggage compartment (for example, a container, a warehouse, etc.). .. When the user of the server 50 inputs a predetermined simulation start operation in the operation unit 52, the control unit 60 starts the process of FIG.

In S10, the control unit 60 monitors that the luggage compartment information is input. In the operation unit 52, the user of the server 50 has various information (that is, luggage compartment information) related to the luggage compartment to be loaded with luggage (hereinafter, referred to as “target luggage compartment”; for example, container C in FIG. 1). Can be entered. The luggage compartment information includes various information related to the luggage compartment, such as identification information of the target luggage compartment, dimensions of the target luggage compartment, and restrictions on the target luggage compartment. In another example, the memory 62 may store the luggage compartment information corresponding to a plurality of types of luggage compartments in advance. In that case, in S10, the control unit 60 may display a list showing each luggage compartment information in the memory 62 on the display unit 52 and allow the user to select one luggage compartment information. When the luggage compartment information of the target luggage compartment is input by the user, the control unit 60 determines YES in S10 and proceeds to S12.

In S12, the control unit 60 monitors that the baggage information is input. In the operation unit 52, the user of the server 50 obtains various information related to each of one or more packages to be loaded in the target luggage compartment (hereinafter, referred to as “target luggage”, for example, luggage B in FIG. 1). You can enter it. The baggage information includes various information related to the target baggage, such as identification information of the target baggage, dimensions, weight, contents, restrictions (for example, loading impossible, designation of loading direction, etc.). The identification information of the target package is, for example, an identification number of a label attached to the target package. When the user inputs the baggage information of one or more target packages, the control unit 60 determines YES in S12 and proceeds to S14.

In S14, the control unit 60 executes the arrangement simulation by using the luggage compartment information input in S10 and the luggage information input in S12. Specifically, in S14, the control unit 60 calculates the arrangement position and the arrangement order for efficiently loading one or more target luggages indicated by the luggage information in the target luggage compartment indicated by the luggage compartment information.

In the following S16, the control unit 60 generates placement information. Specifically, in S16, the control unit 60 generates placement information which is information indicating the placement position and the placement order calculated as a result of the placement simulation in S14.

In the following S18, the control unit 60 stores the generated arrangement information in the arrangement information storage area 64 of the memory 62. After finishing S18, the control unit 60 ends the simulation process of FIG.

(Loading control process; Fig. 5)
Subsequently, the loading control process executed by the control unit 30 of the image display device 10 when the worker wearing the image display device 10 performs the loading work will be described. By the time the loading control process of FIG. 5 is started, the simulation process of FIG. 4 has already been completed. A worker (for example, worker W in FIG. 1) is determined by performing a predetermined gesture within the shooting range (that is, a specific range) of the camera 16 while wearing the image display device 10 whose power is turned on. You can enter the loading control process start operation of. In that case, the control unit 30 starts the process of FIG.

In S30, the control unit 30 monitors that the identification information of the target luggage compartment is input. Specifically, first, the control unit 30 causes the display unit 14 to display a keypad image showing a keypad for inputting identification information. The worker can input the identification information of the target luggage compartment (for example, the container C in FIG. 1) by performing a gesture of operating the keypad indicated by the keypad image within a specific range. The identification information of the target luggage compartment is described, for example, on the outer surface of the target luggage compartment. In another example, an information code (bar code, two-dimensional code, etc.) in which identification information of the target luggage compartment is recorded may be displayed on the outer surface of the target luggage compartment. In that case, the operator may have the camera 16 take a picture of the information code by looking at the information code on the outer surface of the target luggage compartment (that is, putting the identification information within a specific range). In that case, the control unit 30 may acquire the identification information by decoding the captured information code. Further, as in the example of FIG. 1, when the target luggage compartment is a container C mounted on a vehicle, the identification information of the target luggage compartment may be the number written on the license plate of the vehicle. The method of inputting the identification information is not limited to each of the above methods, and any other method can be used. When the identification information of the target luggage compartment is input, the control unit 30 determines YES in S30 and proceeds to S32.

In S32, the control unit 30 transmits the identification information input in S30 to the server 50. When the control unit 60 of the server 50 receives the identification information (S32) from the image display device 10, it refers to the arrangement information storage area 64 of the memory 62, and receives one or more arrangement information stored in the arrangement information storage area 64. From among them, the arrangement information (hereinafter referred to as "target arrangement information") corresponding to the luggage compartment indicated by the received identification information is specified. Then, the control unit 60 reads the target arrangement information from the arrangement information storage area 64 and transmits the target arrangement information to the image display device 10.

In S34, the control unit 30 monitors the reception of the target arrangement information from the server 50. When the control unit 30 receives the target placement information from the server 50, it determines YES in S34 and proceeds to S36. On the other hand, if a predetermined period (for example, 1 minute) elapses without receiving the target arrangement information after transmitting the identification information in S32, the control unit 30 executes error processing and ends the processing of FIG.

In S36, the control unit 30 determines the value of N to 1.

In the following S38, the control unit 30 acquires the spatial information at this time. The spatial information includes various information related to each article within the field of view of the worker (for example, dimensions, orientation, distance to the image display device 10, etc.). Specifically, in S38, first, the control unit 30 acquires a camera image currently captured by the camera 16 from the camera 16. As described above, the specific range (that is, the shooting range of the camera 16) corresponds to the field of view of the worker who wears the image display device 10. When the worker looks at the target luggage compartment (for example, container C in FIG. 1) to perform the loading work, the camera image includes an image of the target luggage compartment. Then, the control unit 30 extracts coordinate information for identifying the feature points of each article included in the camera image based on the camera image acquired from the camera 16. Next, the control unit 30 calculates various information such as the dimensions of each article included in the camera image, the orientation of each article, and the distance between each article and the image display device 10 based on the extracted coordinate information. The various information calculated here is an example of spatial information. When the above camera image includes an image of the target luggage compartment, the spatial information includes the actual dimensions, orientation, and image display device 10 of at least a part of the target luggage compartment existing in the worker's field of view. Various information such as distance is included. By executing each of the above processes, the control unit 30 acquires spatial information in S38.

In the following S40, the control unit 30 has the Nth target baggage (that is, the arrangement order specified by the arrangement information is the Nth) based on the spatial information acquired in S38 and the arrangement information acquired in S34. An object image showing the target baggage) is displayed on the display unit 14. In the first S40, since N is set to 1, the control unit 30 causes the display unit 14 to display an object image indicating the first target baggage. At this time, the control unit 30 displays the object image in a manner of being superimposed on the actual target luggage compartment at the position specified in the arrangement information as the position where the Nth target luggage should be arranged. Further, the control unit 30 can display various character information such as identification information, weight, and constraint information of the target package at the same time as the object image.

FIG. 6 shows an example of displaying an object image in S40. In the example of FIG. 6, when the worker W of FIG. 1 performs a loading operation for loading a plurality of luggage B, which is the target luggage, into the container C, which is the target luggage compartment, the control unit 30 is set to 1. The state of displaying the object image 100 which shows the item | target baggage is shown. In the example of FIG. 6, the object image 100 is displayed on the display unit 14 in a manner of being superimposed on the innermost position of the container C when viewed from the worker W. Further, in addition to the object image 100, character information (identification information, weight, remarks, etc.) 110 related to the first target baggage is also displayed. By looking at the display unit 14 of FIG. 6, the worker W can know that the first target baggage should be placed at the innermost position of the container C as viewed from himself / herself. Further, the worker W can also know the direction of the baggage (that is, the direction seen from himself / herself) when arranging the target baggage by looking at the object image 100 displayed on the display unit 14. If the worker W arranges the first target baggage so as to overlap the object image 100 while looking at the object image 100 displayed on the display unit 14, the arrangement position and arrangement indicated by the arrangement information calculated in advance The first target baggage can be appropriately arranged according to the order. The worker W does not have to think about the arrangement position and arrangement order of the luggage with respect to the container C by himself / herself, and can efficiently perform the loading work.

In the following S42, the control unit 30 determines at this point whether or not the luggage is arranged at the position indicated by the displayed object image. Specifically, in S42, the control unit 30 acquires a camera image currently being captured from the camera 16, and based on the acquired camera image, the position indicated by the displayed object image (that is, the Nth position). It is determined whether or not some kind of luggage has been placed at the position specified in the placement information as the position where the target luggage should be placed. At this point, if the camera image indicates that some kind of luggage (hereinafter, may be referred to as "placed luggage") is placed at the position indicated by the displayed object image, the control unit 30 Determines YES in S42 and proceeds to S44. On the other hand, at this point, if the camera image indicates that nothing is arranged at the position indicated by the displayed object image, the control unit 30 determines NO in S42 and returns to S38. Acquire the current spatial information (that is, the latest spatial information). In this case, the control unit 30 repeatedly executes each of the processes S38 to S42 until it determines YES in S42.

In S44, the control unit 30 determines whether or not the Nth target baggage is correctly arranged at the position indicated by the object image. Specifically, in S44, the control unit 30 first analyzes the camera image at the time when YES is determined in S42, and determines whether or not the arranged baggage is the Nth target baggage. More specifically, the control unit 30 identifies the identification information printed on the label attached in advance on the outer surface of the arranged luggage, compares it with the identification information of the target luggage, and if both match, the arranged luggage is N. Judged as the first target baggage. In another example, whether or not the placed baggage is the Nth target baggage may be determined by another method. For example, the control unit 30 may determine whether or not the arranged baggage is the Nth target baggage based on the outer shape and color of the arranged baggage. When the control unit 30 determines that the arranged luggage is the Nth target luggage, it subsequently determines whether or not the arranged luggage (that is, the Nth target luggage) is arranged in the correct direction. .. More specifically, the control unit 30 further analyzes the camera image and determines whether or not the orientation of the arranged luggage in the camera image matches the orientation of the Nth target luggage specified by the arrangement information. When the control unit 30 determines that the orientation of the placed luggage (that is, the Nth target luggage) in the camera image matches the orientation of the Nth target luggage specified by the placement information, the control unit 30 determines YES in S44. Then proceed to S48. On the other hand, when the control unit 30 determines that the arranged baggage is not the Nth target baggage, or the arranged baggage is the Nth target baggage, the orientation of the arranged baggage is N specified by the arrangement information. If it is determined that the orientation of the target baggage does not match, it is determined as NO in S44, and the process proceeds to S46.

In S46, the control unit 30 performs an error operation. Specifically, the control unit 30 displays an error message (for example, "the type of luggage is different", "the orientation of the luggage is incorrect", an icon indicating an error, etc.) on the display unit 14 and displays the error message. , The speaker 19 is made to output a notification sound. Since the error operation is performed in S46, the worker can appropriately recognize that the arranged luggage has not been arranged correctly. It is possible to appropriately suppress the situation where each baggage is arranged in an arrangement different from the arrangement indicated by the arrangement information. When an error operation is performed in S46, the control unit 30 returns to S38.

On the other hand, in S48, the control unit 30 determines whether or not all the target packages whose placement positions are specified in the placement information have been placed. If all the target packages have been arranged at this point, the control unit 30 determines YES in S48 and proceeds to S52. On the other hand, when the arrangement of all the target packages is not completed, the control unit 30 determines NO in S48 and proceeds to S50.

In S50, the control unit 30 increments the value of N by 1. Next, the control unit 30 returns to S38 and acquires the spatial information (that is, the latest spatial information) at this time. In the following S40, the control unit 30 causes the display unit 14 to display an object image showing the Nth target baggage according to the value of N after being incremented in S50.

FIG. 7 shows an example of displaying the object image in S40 for the second time. FIG. 7 is a display example of the object image 200 displayed in the second S40 after the object image 100 of FIG. 6 is displayed. In the example of FIG. 7, the first target baggage B1 (that is, the target baggage corresponding to the object image 100 of FIG. 6) is already arranged at the innermost position of the container C. In the example of FIG. 7, the object image 200 is placed on the display unit 14 at the innermost position of the container C when viewed from the worker, and is superimposed on the position on the target luggage B1 already arranged. Is displayed. Further, in addition to the object image 200, character information (identification information, weight, remarks, etc.) 210 related to the first target baggage is also displayed. By looking at the display unit 14 of FIG. 7, the worker W can know that the second target baggage should be loaded and arranged on the first target baggage arranged earlier. In addition, the worker W can also know the orientation of the luggage (that is, the orientation as seen from himself / herself) when arranging the second target luggage by looking at the object image 200 displayed on the display unit 14. If the worker W arranges the second target baggage so as to overlap the object image 200 while looking at the object image 200 displayed on the display unit 14, the arrangement position and arrangement indicated by the arrangement information calculated in advance The second target baggage can be arranged appropriately according to the order. The worker does not have to think about the arrangement position and arrangement order of the luggage with respect to the container C by himself / herself, and can efficiently perform the loading work.

In S52, the control unit 30 transmits a completion notification to the server 50. The completion notification includes identification information of the target luggage compartment and information indicating that the loading work has been completed. After finishing S52, the control unit 30 ends the loading control process of FIG.

When the control unit 60 of the server 50 receives the completion notification (S52) from the image display device 10, it stores the target arrangement information in the arrangement information storage area 64 of the memory 62 in association with the predetermined completion information.

The configuration and operation of the management system 2 of this embodiment have been described above. As described above, in the present embodiment, the control unit 30 uses the object image showing the Nth target baggage as the placement position in the target luggage compartment of the Nth target baggage based on the spatial information and the placement information. It is displayed at the calculated position so as to be superimposed on the actual target luggage compartment (see S40, FIG. 6 and FIG. 7 in FIG. 5). Therefore, a worker who loads luggage into the actual luggage compartment can arrange the luggage with respect to the actual luggage compartment by looking at one or more object images displayed in a manner superimposed on the actual luggage compartment. It becomes easier to imagine concretely. Therefore, according to the management system 2 of the present embodiment, it is possible to make the worker concretely grasp the arrangement of the luggage with respect to the actual luggage compartment and improve the efficiency of the loading work.

In particular, in the present embodiment, the control unit 30 uses the spatial information acquired based on the camera image taken by the camera 16 to grasp the feature points of the actual luggage compartment and then superimposes the luggage compartment on the luggage compartment. The object image is displayed on the display unit 14 in this manner. That is, in this embodiment, the control unit 30 can display the object image in a manner that matches the line of sight of the worker. Workers can more specifically grasp the arrangement of luggage.

Further, in the present embodiment, the control unit 30 displays the Nth object image indicating the target baggage on the display unit 14 (S40 in FIG. 5), and then at the position indicated by the object image, the Nth object image is displayed. When it is determined whether or not the target baggage is correctly arranged (S42, S44) and it is determined that the Nth target baggage is correctly arranged at the position indicated by the object image (YES in S42 and S44). , N + 1 Displays an object image showing the target baggage (next S40). Therefore, in particular, as in the examples of FIGS. 6 and 7, the worker W looks at the object image 100 (200) displayed on the display unit 14 and overlaps the object image 100 (200) with the target baggage. If the above is arranged, the target luggage can be appropriately arranged according to the arrangement position and the arrangement order indicated by the arrangement information calculated in advance. In this way, the worker does not have to think about the arrangement of the luggage in the luggage compartment and the arrangement order of each luggage by himself / herself, and can efficiently perform the loading work.

Further, in the present embodiment, the control unit 30 displays the Nth object image indicating the target baggage on the display unit 14 (S40 in FIG. 5), and then at the position indicated by the object image, the Nth object image is displayed. When it is determined that the target baggage is not arranged correctly (NO in S44), an error operation is performed without displaying the object image showing the N + 1th target baggage (S46). By performing the error operation, the worker can properly recognize that the placed luggage was not placed correctly. It is possible to appropriately suppress the situation where each baggage is arranged in an arrangement different from the arrangement indicated by the arrangement information.

Further, in this embodiment, the management system 2 includes an image display device 10 and a server 50. The control unit 60 of the server 50 generates placement information and stores it in the memory 62. The control unit 30 receives the arrangement information from the server 50. The control unit 30 of the image display device 10 displays an object image based on the spatial information and the arrangement information. That is, in this embodiment, it is not necessary to store the arrangement information in advance in the memory 32 of the image display device 10.

Further, in the present embodiment, when the arrangement of all the target packages is completed (YES in S48 of FIG. 4), the control unit 30 transmits a completion notification to the server 50 (S52). Therefore, the user of the server 50 (for example, the manager of the worker) can know that the loading work in the target luggage compartment has been completed.

Further, in this embodiment, the image display device 10 is a head-mounted display including a frame 12 that can be attached to the head of a user (worker W). When the user wears the frame 12, the display unit 14 (that is, the right display unit 14R and the left display unit 14L) is arranged at a position facing the user's right eye and left eye. Therefore, in the present embodiment, the worker (that is, the user) performs the loading work using both hands while looking at the object image displayed on the display unit 14 with the image display device 10 attached to the head. be able to. In this case, since the object image is displayed on the display unit 14 in a manner superimposed on the actual luggage compartment in the worker's field of view (see FIGS. 6 and 7), the worker sees from his / her own eyes. It is possible to properly grasp the arrangement of each luggage in the luggage compartment.

The correspondence between this embodiment and the claims will be described. The management system 2 is an example of an "information processing device". The image display device 10 is an example of a “terminal device”. The camera 16 is an example of a “spatial information measuring unit”. The object image 100 of FIG. 6 and the object image 200 of FIG. 7 are examples of the “first object image” and the “second object image”, respectively.
The case of YES in S42 and S44 of FIG. 4 is an example of the “first case”. The case of NO in S44 is an example of the "second case". The speaker 19 and the display unit 14 are examples of the “notification unit”. The error operation of S46 is an example of the "notification operation".

Although the examples have been described in detail above, these are merely examples and do not limit the scope of claims. The techniques described in the claims include various modifications and modifications of the specific examples illustrated above. For example, the following modification may be adopted.

(Modification 1) In the above embodiment, the control unit 30 of the image display device 10 displays the object image indicating the Nth target baggage on the display unit 14 (S40 in FIG. 5), and then the object image. It is determined whether or not the Nth target baggage is correctly arranged at the position indicated by (S42, S44), and it is determined that the Nth target baggage is correctly arranged at the position indicated by the object image. In the case (YES in S42 and S44), an object image showing the N + 1th target baggage is displayed (next S40). Not limited to this, the control unit 30 may display two or more object images out of a plurality of object images at the same time.

(Modification 2) Arrangement information related to the loading work performed by the worker may be stored in advance in the memory 32 of the image display device 10. In that case, the control unit 30 omits the process of transmitting the identification information of the target luggage compartment to the server 50 and receiving the arrangement information from the server 50 (S30 to S34 in FIG. 4). The control unit 30 may display each object image on the display unit 14 based on the arrangement information and the spatial information stored in advance in the memory 32. In this modification, the image display device 10 is an example of an “information processing device”.

(Modification 3) In the above embodiment, the control unit 60 of the server 50 executes the simulation process (FIG. 4). Not limited to this, the control unit 30 of the image display device 10 may execute the simulation process (FIG. 4). In that case, the control unit 30 may store the generated arrangement information in the memory 32 of the image display device 10.

(Modification 4) In the above embodiment, the display unit 14 of the image display device 10 is a translucent display. Not limited to this, the display unit of the image display device 10 may be a light-shielding display, which may block the user's field of view when the user wears the image display device 10. In that case, the control unit 30 may display the camera image taken by the camera 16 (that is, the image corresponding to the user's field of view) on the display unit. Further, in the stacking control process (see FIG. 5), the control unit 30 may superimpose the object image on the camera image displayed on the display unit and display the object image.

(Modification 5) In the above embodiment, the control unit 30 acquires spatial information based on the camera image captured by the camera 16 (S38 in FIG. 4). Not limited to this, the control unit 30 may acquire spatial information from information other than the camera image. For example, the image display device 10 may further include an infrared sensor capable of detecting each article within the user's field of view. In that case, the control unit 30 may acquire spatial information based on the detection result of the infrared sensor.

(Modification 6) In the above embodiment, when the arrangement of all the target packages is completed (YES in S48 of FIG. 4), the control unit 30 transmits a completion notification to the server 50 (S52). Not limited to this, the control unit 30 does not have to send the completion notification to the server 50.

(Modification 7) In each of the above embodiments, the image display device 10 has a substantially spectacle-shaped frame, and can be worn on the user's head as if wearing spectacles. Not limited to this, the image display device may have an arbitrary support frame such as a hat shape or a helmet shape as long as it can be worn on the user's head.

(Modification 8) The image display device is formed by attaching a camera, a control box, etc. to eyewear (glasses, sunglasses, etc.) generally used for applications such as vision correction and eye protection. May be done. In that case, the lens portion of the eyewear may be used as a display unit.

(Modification 9) In the above embodiment, the image display device 10 has two cameras, a right camera 16 and a left camera 16, but the number of cameras may be one. Further, the image display device 10 may have only one of the right display unit 14R and the left display unit 14L.

(Modification 10) The image display device is not limited to the one that can be worn on the user's head, and may be of any form. For example, the image display device may be a mobile terminal that can be carried by the user.

(Modification 11) In each of the above embodiments, the object images 100 and 200 (FIGS. 6 and 7) are all displayed as still images. Not limited to this, the object image may be displayed as a moving image (that is, an image at least partially animated).

In addition, the technical elements described in the present specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the techniques illustrated in the present specification or drawings achieve a plurality of objectives at the same time, and achieving one of the objectives itself has technical usefulness.

Claims (7)

Spatial information measurement unit and
Display and
An information processing device equipped with a control unit.
The control unit
The first acquisition unit that acquires the spatial information including the dimensions and orientation of the luggage compartment, which is the spatial information measured by the spatial information measurement unit,
A second acquisition unit for acquiring arrangement information, the arrangement information is luggage information related to the luggage compartment and luggage information for each of one or more luggage to be loaded in the luggage compartment. The second acquisition unit, which is the result of calculating the arrangement of each of the one or more packages in the luggage compartment based on the luggage information including at least one of the dimensions and the weight of the luggage.
A display control unit for displaying one or more object images indicating each of the one or more packages on the display unit based on the acquired spatial information and the acquired arrangement information. Each of the above object images is a virtual image that virtually represents the corresponding luggage, and is superimposed on the actual luggage compartment at the position calculated as the arrangement position of the luggage indicated by the object image in the luggage compartment. The display control unit, which is displayed in the above mode, is provided.
Information processing device. The display control unit
In a situation where a second luggage should be placed after the first luggage in the luggage compartment, a first object image showing the first luggage is displayed on the display unit.
In the first case where the first luggage is correctly arranged at the position indicated by the first object image while the first object image is displayed on the display unit, the second luggage is placed. The second object image to be shown is displayed on the display unit.
The information processing device according to claim 1. The display control unit
In the second case where the first baggage is not correctly arranged at the position indicated by the first object image while the first object image is displayed on the display unit, the second object. Do not display the image on the display unit,
The information processing device according to claim 2. Equipped with a notification unit
The control unit further
In the second case, the notification control unit is provided so that the notification unit executes the notification operation.
The information processing device according to claim 3. The information processing device includes a terminal device and a server capable of communicating with the terminal device.
The terminal device includes the spatial information measuring unit, the display unit, and a terminal control unit.
The terminal control unit includes the first acquisition unit, the second acquisition unit, and the display control unit.
The second acquisition unit acquires the arrangement information by receiving the arrangement information from the server, and obtains the arrangement information.
The server includes a storage device and a server control unit.
The storage device stores the arrangement information and
The server includes an information transmission unit that transmits the arrangement information in the storage device to the terminal device.
The information processing device according to any one of claims 1 to 4. The terminal control unit further
A notification unit for transmitting a completion notification to the server when the arrangement of the one or more packages corresponding to the one or more object images is completed is provided.
The information processing device according to claim 5. The terminal device further comprises a frame that can be worn on the user's head.
The display unit includes a display device mounted on the frame and arranged at a position facing at least one of the right eye and the left eye of the user who wears the frame.
The information processing device according to claim 5 or 6.

Images