JP2023181782A - Estimation system, learning system, estimation method, learning method, and program - Google Patents

Abstract

To provide an estimation system capable of grasping to which position an operation is performed in the height direction.SOLUTION: A estimation system includes: a sensor information acquisition unit for, by vibration of a vibration unit, which is installed at any position of a fixture having a plurality of shelf plates in the height direction, in response to application of vibration generated through an operation of placing or taking out an object to any of the shelf plates in the fixture, acquiring sensor information obtained from a sensor for detecting vibration in which at least one of the duration or intensity of vibration generated according to the operation is changed; and an estimation unit for obtaining position information including a position in the height direction of the fixture operated by inputting the acquired sensor information into a learned model having learned the relation between the sensor information and the position in the fixture operated.SELECTED DRAWING: Figure 1

Description

The present invention relates to an estimation system, a learning system, an estimation method, a learning method, and a program.

In recent years, the use of smart shelf systems has been considered. In situations such as product management in stores and product inventory management in warehouses, smart shelf systems can be used to manage products in response to actions such as placing or removing products on the shelves of a product shelf. It is possible to detect the location of the operation and the content of the operation (place or take). Demand for such smart shelf systems is increasing.
Furthermore, in the context of smart homes, there is a demand for smart shelves that can naturally understand daily life activities and present appropriate information.

In order to widely deploy smart shelves, it is necessary to have low introduction costs and to be able to apply them to a wide range of target fixtures. A configuration that is simple and can be installed on existing fixtures is required.
Non-Patent Document 1 discloses a technique for estimating motion on a desk by using a load sensor without using a camera.

Junnan Imai, Kazuya Murao, Tsutomu Terada, Masahiko Tsukamoto, "Design and implementation of a desk motion recognition system using load sensors," Multimedia, Distributed Coordination and Mobile Symposium 2013 Proceedings, 2013, P1180-P1187

However, although it is becoming possible to estimate the position of shelves, desks, etc. on a flat surface, some shelves have multiple shelves, and in this case, it is difficult to estimate which position in the height direction the operation is being performed. It has not been possible to estimate the

The present invention has been made in view of the above circumstances, and its purpose is to provide an estimation system, a learning system, an estimation method, a learning method, and a learning system capable of grasping at which position in the height direction an operation has been performed. The goal is to provide programs.

In order to solve the above-mentioned problems, one aspect of the present invention is to provide a vibration unit at any position of a fixture in which a plurality of shelf boards are provided in the height direction, The vibration unit vibrates in response to the application of vibrations that occur in response to an operation in which an object is placed or taken out, thereby reducing at least either the duration or the intensity of the vibrations that occur in response to the operation. a sensor information acquisition unit that acquires sensor information obtained from a sensor that detects vibrations that have been changed; and a trained model that has learned the relationship between the sensor information and the position in the operated fixture. The estimation system includes an estimation unit that inputs sensor information and obtains position information including a position in the height direction of the operated fixture.

Further, in one aspect of the present invention, a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. Vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of vibration generated in response to the operation. a learning data acquisition unit that acquires sensor information obtained from a sensor that detects a sensor and position information including a position in the height direction of the operated fixture; and a learning data acquisition unit that acquires sensor information obtained from a sensor that detects the The learning system includes a learning unit that generates a trained model that has learned a relationship with position information.

Further, in one aspect of the present invention, a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. Vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of vibration generated in response to the operation. The acquired sensor information is input to a trained model that has learned the relationship between the sensor information and the position of the operated fixture, and the operated fixture is This is an estimation method for obtaining position information including the position in the height direction.

Further, in one aspect of the present invention, a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. Vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of vibration generated in response to the operation. and position information including the position in the height direction of the operated fixture, and use the acquired learning data to determine the relationship between the sensor information and the position information. This is a learning method that generates a trained model that has been trained.

Further, in one aspect of the present invention, a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. Vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of vibration generated in response to the operation. The acquired sensor information is input to a trained model that has learned the relationship between the sensor information and the position of the operated fixture, and the operated fixture is This is a program that causes a computer to obtain position information including the position in the height direction.

Further, in one aspect of the present invention, a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. Vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of vibration generated in response to the operation. and position information including the position in the height direction of the operated fixture, and use the acquired learning data to determine the relationship between the sensor information and the position information. This is a program that causes a computer to generate a trained model that has learned the following.

As described above, according to the present invention, it is possible to ascertain at which position in the height direction the operation has been performed.

FIG. 1 is a perspective view showing an example of fixtures used in the estimation system according to an embodiment of the present invention. FIG. 3 is a perspective view showing a case where a vibration unit 25a and a vibration unit 25b are attached to a shelf T. It is a side view showing the case where the vibration unit 25a and the vibration unit 25b are seen from the side. 3 is a plan view of the vortex member 250. FIG. It is a side view of the vortex member 250 seen from the side. FIG. 3 is a waveform chart showing an example of vibration characteristics of a vibration unit. FIG. 7 is a perspective view of another example of the vibration unit. FIG. 3 is a diagram illustrating directions in which the vibration unit 35 can vibrate. 5A is a sectional view illustrating the internal structure of the vibration unit 35 taken along line AA in FIG. 5A. FIG. 5C is a cross-sectional view illustrating the internal structure of the vibration unit 36 in a configuration different from that in FIG. 5C. FIG. 5C is a cross-sectional view illustrating the internal structure of the vibration unit 37 in a configuration different from that in FIG. 5C. FIG. FIG. 3 is a cross-sectional view showing a cross section when a plurality of vibration units are connected. FIG. 3 is a cross-sectional view showing a cross section when a plurality of vibration units are connected. It is a schematic block diagram explaining the function of management system S. It is a block diagram showing an example of the functional configuration of a learning system 40 according to the present embodiment. It is a block diagram showing an example of the functional composition of estimation system 50 concerning this embodiment. It is a figure which represents the simulation result in the case where a vibration unit is not provided in the shelf as an image. It is a figure which represents the simulation result when a vibration unit is provided in the shelf as an image. It is a figure explaining the flow of processing of management system S. It is a flowchart showing an example of the operation of the vibration occurrence position estimation system 4 according to the present embodiment. 6 is a diagram showing an example of an estimation result displayed on the display screen of the user terminal 2 or the display device 6. FIG. It is a graph showing a detection result detected by a sensor device.

Hereinafter, an estimation system according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a perspective view showing an example of fixtures used in an estimation system according to an embodiment of the present invention.
The shelf T is an example of a fixture, and has three shelf boards: a lower shelf, a middle shelf, and an upper shelf.
The shelf board 11a is the lower stage, the shelf board 11b is the middle stage, and the shelf board 11c is the upper stage, and it is possible to place an object on the upper surface of each of them. Posts 12a, 12b, 12c, and 12d are provided at the four corners of the shelf T, and shelf boards 11a, 11b, and 11c are attached to these posts.
On the lower surface of the shelf board 11a, a sensor 14a is provided at a position corresponding to the support column 12a, a sensor 14b is provided at a position corresponding to the support column 12b, and a sensor 14d is provided at a position corresponding to the support column 12d. It will be done. Thereby, the sensor 14a, the sensor 14b, and the sensor 14c are each located between the support column and the floor surface. Hereinafter, when the sensor 14a, the sensor 14b, and the sensor 14c are not particularly identified, they may be simply referred to as the sensor 14.

Sensor 14a, sensor 14b, and sensor 14c each detect the force applied to shelf T. These sensors 14a, 14b, and 14c detect the force applied to the shelf T, for example, by detecting vibrations caused by the force applied to the shelf T. The sensor 14a, the sensor 14b, and the sensor 14c transmit detected information (hereinafter also referred to as "sensor information") to the vibration occurrence position estimation system 4 (described later). Here, the shelf T is designed such that when an object is placed on one of the shelf boards or an object is taken out from the shelf board, the vibration generated by that operation is propagated to the bottom of the shelf. It has become. Sensor 14 detects this vibration.

The sensors 14a, 14b, and 14c are, for example, any one of a vibration sensor, a load sensor, a pressure sensor, or an acceleration sensor. The vibration sensor detects the generation of force applied to the shelf T by changes in vibration. The load sensor detects the force applied to the shelf T by changes in vibration. The pressure sensor detects the force applied to the shelf T by changes in pressure. The acceleration sensor detects the force applied to the shelf T by changes in acceleration.
Here, the force applied to the shelf T is the operation of placing an object (for example, object 18a, object 18b) on one of the shelves T, and the This occurs in response to at least one of the operations of taking out an object placed on the object. Therefore, the sensor 14a, the sensor 14b, and the sensor 14c each detect vibrations corresponding to the operation on the shelf T.

Sensor information is information with an arbitrary number of dimensions. The number of dimensions is determined depending on the type and number of sensor devices 3. For example, when two 1-axis sensor devices 3 are provided, the number of dimensions of sensor information is two-dimensional (1 axis x 2). Further, for example, when three two-axis sensor devices 3 are provided, the number of dimensions of sensor information is six (two axes x three).

The vibration unit 15a is attached to the lower surface of the shelf board 11b near the support column 12a, and the vibration unit 15b is attached to the lower surface of the shelf board 11b to the vicinity of the support column 12d.
The vibration unit 15c is attached to the lower surface of the shelf board 11c near the support column 12a, and the vibration unit 15d is attached to the lower surface of the shelf board 11c to the vicinity of the support column 12d.
The vibration units 15a, 15b, 15c, and 15d may be simply referred to as the vibration unit 15 unless specifically identified.
The vibration unit 15 vibrates in response to the application of vibrations that occur when an object is placed on or taken out from one of the shelves in the fixture (for example, the shelf T). By vibrating, the vibration unit 15 can amplify the vibration caused by the operation on the shelf T. That is, the vibration unit 15 can emphasize vibrations caused by operations on the shelf T. Such a vibration unit 15 may be any device as long as it can generate vibrations.

In this figure, an object 18a is placed on the top surface of the shelf board 11b, and an object 18b is placed on the top surface of the shelf board 11c. The user U can operate an object on any of the shelf boards 11a, 11b, and 11c. The operation of an object is at least one of an operation of placing an object on a shelf board and an operation of taking out an object placed on a shelf board.
Any object among a plurality of types of objects may be placed on the shelves 11a, 11b, and 11c, or the types of objects that can be placed on each shelf are predetermined. It's okay.
Further, among the shelf boards 11a, 11b, and 11c, only one object or a plurality of objects may be placed on one shelf board.

FIG. 2 is a diagram showing another example of the configuration of the vibration unit 15. FIG. 2A is a perspective view showing the case where the vibration unit 25a and the vibration unit 25b are attached to the shelf T, and FIG. 2B is a side view showing the case where the vibration unit 25a and the vibration unit 25b are viewed from the side.
In FIGS. 2A and 2B, the vibration unit 25a and the vibration unit 25b are provided integrally with the base member 20. The height of the base member 20 corresponds to the height of the shelf T from the lower stage to the upper stage. The base member 20 may be attached to the shelf T as a back plate of the shelf T, for example.

The vibration unit 25a is provided with a vortex member 25a1. Here, by forming a spiral groove in the base member 220, the spiral member 25a1 is formed. Here, a groove is formed so as to penetrate from one main surface (front surface) to the other main surface (back surface) of the base member 20, and the vortex member 25a1 is formed on one side by applying vibration from the outside. The center of the vortex of the vortex member 25a1 can vibrate in the direction (horizontal direction) passing through the main surface of the base member 20 and the other main surface, and the vortex can vibrate in the height direction (vertical direction) of the base member 20. The center of the vortex of the member 25a1 can vibrate. That is, the vortex member 25a1 and other vortex members can vibrate in any direction (three-dimensional direction) in three-dimensional space in response to externally applied vibrations. Here, a weight may be attached to the center of the vortex member 25a1 in the radial direction, and by changing the weight of this weight, the vibration characteristics of the vortex member 25a1 can be changed. Such a weight can be attached not only to the vortex member 25a1 but also to other vortex members.

Further, the vibration unit 25a is provided with a connecting member 25a3 that connects the center of the vortex member 25a1 and the base member 20. One end of the connecting member 25a3 is attached to the base member 20 near the outer periphery of the vortex member 25a1, and the other end of the connecting member 25a3 is attached to the center of the vortex member 25a1 and connected by a pin 25a2. has been done.
In the vibration unit 25a, the center portion of the vortex member 25a1 and the base member 20 are connected by the connecting member 25a3, so that vibrations of the vortex member 25a1 are more easily attenuated than when they are not connected. This connecting member 25a3 may be provided as necessary, and may not be provided if the vibration of the vortex member 25a1 is not damped. Furthermore, by changing the material, thickness, length, etc. of the connecting member 25a3, the degree of attenuation may be increased (easier to attenuate), or the degree of attenuation may be lowered (easier to attenuate). You can also do this.

The vibration unit 25b is configured by combining a first unit 251b and a second unit 252b.
The first unit 251b has at least one of the characteristics of extending the duration of vibration and increasing the intensity of vibration.
The second unit 252b has at least one of the characteristics of shortening the duration of vibration and reducing the intensity of vibration.
By combining the first unit 251b and the second unit 252b, it is possible to edit characteristics that change at least one of the duration and intensity of vibrations generated in response to operations on the shelf T.
For example, the vibration unit 25b may be configured to combine one type of first unit among multiple types of first units with different characteristics, and one type of second unit among multiple types of second units with different characteristics. It consists of a combination of.

For example, the first unit has characteristics that greatly extend the duration of vibration and the characteristics that increase the intensity of vibration, and the second unit has characteristics that slightly shorten the duration of vibration and characteristics that increase vibration intensity. If the characteristics have little effect on the strength, when these are combined, the duration of the vibration of the first unit is slightly shortened, but the strength of the vibration remains the same. It is possible to obtain a vibrating unit.
Further, for example, the first unit has a characteristic of slightly extending the duration of vibration and a characteristic of increasing the intensity of vibration, and the second unit does not have much influence on the duration of vibration. However, if they have the characteristic of slightly reducing the vibration intensity, when these are combined, the duration of the vibration of the first unit does not change much, but the vibration intensity is slightly reduced. You can get an edited vibration unit.

Here, the center portions of the vortex member 251b1 of the first unit 251b and the vortex member 252b1 of the second unit 252b are connected by a connecting member 25b3. One end of the connecting member 25b3 is connected to the center of the vortex member 251b1 by a mounting pin 251b2, and the other end of the connecting member 25b3 is connected to the center of the vortex member 252b1 by a mounting pin 252b2.
Such vortex members 25a1, 251b1, and 252b1 may be produced integrally with the base member 20 by cutting the base member 20 itself, or may be produced using a 3D printer. Therefore, when using a vortex member, there is an advantage that processing and manufacturing are easier than when using a coil spring.
Although the case where two vortex members are connected is illustrated here, three or more vortex members may be connected.

FIG. 3 is a diagram showing the configuration of the vortex member 250 (vortex member 25a1, vortex member 251b1, and vortex member 252b1). FIG. 3A is a plan view of the vortex member 250, and FIG. 3B is a side view of the vortex member 250 when viewed from the side.
The vortex member 250 has a starting end 255 in the vicinity of a center point 257, and has a spiral shape (algebraic spiral shape) in which the diameter gradually increases from the starting end 255 to the terminal end 256, and has approximately the same width t from the starting end 255 to the terminal end 256. At the same time, it is formed to have a thickness of W. Further, in the radial direction of the vortex member 250, the distances P between the centers of adjacent vortices in the spiral shape are formed at equal intervals. In the width direction of the starting end 255, the distance between the side closer to the center point 257 of the vortex and the center point 257 is set to da/2, and the distance between the center point 257 and the terminal end 256 is set to do/2. It is set.

Here, when producing the vortex member 250, the vortex member 250 can be produced by directly processing a plate-like member to form a groove, and the processing is easy. In addition, in the vortex member 250, the width t, distance P, distance da/2, distance do/2, thickness W, etc., a line connecting the starting end 255 and the center point 257, a line connecting the terminal end 256 and the center point 257, etc. The vibration characteristics can be changed by arbitrarily changing at least one of the variables, such as the angle of , the ease of elastic deformation depending on the material of the plate member to be processed, and the like. Furthermore, when changing at least one of these, the vibration characteristics (elasticity) of the vortex member 250 can be arbitrarily designed by simulating the frequency at which the vortex member 250 vibrates. Therefore, the vibration characteristics of the vortex member 250 can be easily designed.
In this way, the use of a vortex-shaped elastic body as the elastic member has the advantage of ease of design and manufacture.

The vibration unit using such a vortex member may be provided not only on the back plate of the shelf, but also on at least one of the side plate, support, shelf board, etc. of the shelf. Here, when a vibration unit using a vortex member is installed on a shelf, the vortex member is spaced apart so that it does not come into contact with any part of the shelf or objects placed on the shelf when vibrating. Alternatively, the vortex member may be provided with a cover.

FIG. 4 is a waveform diagram showing an example of the vibration characteristics of the vibration unit. In FIG. 4, the vertical axis represents signal strength, and the horizontal axis represents time.
Reference numeral 400 is a waveform diagram when the vibration intensity and vibration duration are average characteristics among a plurality of types of vibration units.
Reference numeral 410 is a waveform diagram of a case where among the plurality of types of vibration units, the vibration intensity is average, but the vibration characteristic is damped so that the duration of the vibration is shortened.
Then, a combination vibration unit can be obtained by connecting a vibration unit having vibration characteristics shown by reference numeral 400 as a first unit and a vibration unit having vibration characteristics shown by reference numeral 410 as a second unit.
Reference numeral 420 is a diagram showing vibration characteristics of a combination vibration unit including such a first unit and a second unit. In this case, the combined vibration unit is created by combining the vibrations of the first unit and the second unit, so that the vibration of the first unit and the vibration of the second unit are combined, so that the vibration of the first unit and the vibration of the second unit are combined. In the section, a resonance state occurs, and as a result, the amplitude becomes large, and after time t2, the amplitude becomes small, and the vibration almost stops at time t3. Regarding the duration of vibration, the first unit has a longer vibration duration than the second unit, but the combination vibration unit has a longer vibration duration than the first unit. There is.
In this way, by combining vibration units with different characteristics, it is possible to obtain a vibration unit with new characteristics whose vibrations can be edited.

Depending on the characteristics of the shelf, such as the overall weight of the shelf, the number of shelves, the number of supports, the width and thickness of the supports, and the mounting position of the supports relative to the shelf, even if an object is placed on the shelf, In some cases, the detection result of vibration is such that it is not possible to identify which shelf board the object is placed on. In contrast, as mentioned above, by editing the coupled vibration according to the characteristics of vibration intensity and vibration duration, it is possible to It is possible to create a combination that provides vibration characteristics that make position detection easier. Further, for example, even if a relatively light object is placed on the shelf board, it is possible to easily detect at which position in the height direction the object is placed.

In addition, even if there are a large number of shelves, such as 10 or 20, the first unit and the second unit can be arbitrarily combined to make it easier to identify each shelf, and the combination vibration unit The variation of coupled vibrations can be increased. With this, it is possible to arbitrarily produce a combination vibration unit having individual coupled vibrations corresponding to the stages of the shelf board, and to attach it to any position on the shelf. When attaching the combination unit to a shelf, it may be attached by gluing or by screws.
Furthermore, for example, in a shelf with 10 shelves, 10 different types of combination vibration units may be manufactured and one may be provided for each of the 10 shelves. The combination vibration unit may be provided on the upper surface side of the shelf board, or may be provided on the lower surface side. Further, the combination vibration unit may be provided at any position on the surface of the shelf board (front side, back side, right side, left side, center). Furthermore, the combination vibration unit may be provided on the support instead of the shelf board. In addition, the combination units may be installed in each stage, or they may be installed in the upper five stages and not installed in the lower five stages, or they may be installed every other stage. It may be provided at any number of stages.

The optimal mounting position for a combination unit or vibration unit varies depending on the structure of the shelf (number of tiers, number of supports, mounting position of the supports relative to the shelf board, etc.), the material that makes up the shelf, and the nature of the object to be placed on it. It is preferable that the vibration occurrence position estimation system 4 (described later) be installed at a position where vibration occurs so that the position in the height direction can be easily identified. For example, the vibration unit may be placed anywhere on the shelf, but it is best to place it above the shelf (for example, on the top of the shelf, on the top of the support, on the top of the side plate, on the top of the back, etc.). In some cases, it is possible to obtain a detection result from the sensor device that makes it easier to identify the position in the height direction according to the operation.

In addition, here, we have explained the case where the vibration units are connected and then installed on the shelf, but if the vibration generated according to the operation performed on the shelf can be changed, it is possible to do so without connecting the vibration units. A single vibration unit may be individually provided on the shelf. In particular, when changing vibrations generated in response to operations, it is desirable to make changes that emphasize the characteristics of the vibrations. When emphasizing the characteristics of vibrations, for example, when the amplitude becomes larger than the overall vibration, when the duration of the vibration becomes longer, when the amplitude of a specific frequency component included in the vibration becomes larger, when the specific frequency component included in the vibration It may be the case that the amplitude of a frequency component becomes smaller, or the duration of vibration of a specific frequency component becomes shorter. In this way, when an operation is performed on any position in the height direction, the operation is influenced by the relative position of each vibration unit with respect to the position where the operation was performed, the characteristics of the shelf, etc. Depending on the position in the height direction, the intensity of the vibration transmitted to the sensor 14 and the duration of the vibration become a characteristic vibration that is detected by the sensor. This makes it possible to identify at which position in the height direction the operation has been performed.

FIG. 5 is a diagram showing another example of the vibration unit.
FIG. 5A is a perspective view of another example of the vibration unit.
The external shape of the vibration unit 35 is approximately cubic. An elastic body such as a spring is provided inside the vibration unit 35, and the spring inside the vibration unit 35 vibrates in response to vibrations applied from the outside of the vibration unit 35. A slide switch 351 is provided on one of the outer peripheral surfaces of the vibration unit 35. By sliding the slide switch 351, the elastic body inside the vibration unit 35 can selectively switch between a state where it can vibrate and a state where it cannot vibrate.

FIG. 5B is a diagram illustrating directions in which the vibration unit 35 can vibrate. The vibration unit 35 can vibrate in a total of six directions, each of the vertical direction, the horizontal direction, the front side, and the back side, in response to vibrations applied from the outside. Here, depending on the applied vibration, it is also possible to vibrate in an oblique direction.

FIG. 5C is a sectional view illustrating the internal structure of the vibration unit 35 taken along line AA in FIG. 5A.
A weight 35a is housed inside the vibration unit 35, and is attached to the base member via springs in the vertical direction, left and right direction, and on the front side and the back side. For example, the weight 35a is attached to the base member 35c1 via the spring 35b1 in the upward direction, and attached to the base member 35c2 via the spring 35b2 in the left direction. The shape of the weight 35a is approximately spherical. The springs 35b1 and 35b2 are, for example, coil springs. The base members 35c1 and 35c2 are each fixed by being adhered or adhered to the inner peripheral surface. This allows the weight 35a to vibrate in any of the six directions or in an oblique direction.
Such a vibration unit 35 can be used alone, or can be used as a first unit in combination with a second unit.
In addition, since the weight 35a of the vibration unit 36 is connected to a spring, the weight 35a vibrates in response to the vibration generated in response to the operation performed on the shelf T. Vibration can be amplified (emphasized). Thereby, the vibration caused by the operation can be detected by the sensor 14 as a more characteristic vibration characteristic, and the position of the shelf T not only in the plane direction but also in the height direction can be detected by the vibration generation position estimation system 4 (described later). It becomes possible to identify.

FIG. 5D is a cross-sectional view illustrating the internal structure of the vibration unit 36 in a configuration different from that in FIG. 5C.
In FIG. 5C, a coil spring is used in the vibration unit 35, but in FIG. 5D, a lattice spring is used in the vibration unit 36.
A weight 36a is housed inside the vibration unit 36, and is attached to the base member via a lattice spring in the vertical direction, left and right direction, and on the front side and the back side. For example, the weight 36a is attached to the base member 36c1 via the lattice spring 36b1 in the upward direction, and is attached to the base member 36c2 via the lattice spring 36b2 in the left direction. The shape of the weight 36a is approximately cubic. The base members 36c1 and 36c2 are each fixed by being adhered or adhered to the inner circumferential surface. This allows the weight 36a to vibrate in any of the six directions or in an oblique direction.
Such a vibration unit 36 can be used alone, or can be used as a first unit in combination with a second unit.

FIG. 5E is a cross-sectional view illustrating the internal structure of the vibration unit 37 in a configuration different from that in FIG. 5C.
In FIG. 5C, a coil spring is used in the vibration unit 35, but in FIG. 5E, a damper is used in the vibration unit 37.
A damper is housed inside the vibration unit 37, and is attached via a base member to each of the vertical direction, the horizontal direction, the front side, and the back side.
For example, in FIG. 5E, damper 37a, damper 37b, damper 37c, and damper 37d are illustrated. The damper 37a is attached to the inner peripheral surface of the right side via the base member 38a, and damps vibrations in the left-right direction. The damper 37b is attached to the inner peripheral surface of the upper surface via the base member 38b, and damps vibrations in the vertical direction.
The damper 37c is attached to the inner peripheral surface of the left side via the base member 38c, and damps vibrations in the left-right direction. The damper 37d is attached to the inner peripheral surface of the lower surface via the base member 38d, and damps vibrations in the vertical direction.
The base members 38a, 38b, 38c, and 38d are each fixed by adhesion or adhesion to the inner peripheral surface. This allows the weight 36a to vibrate in any of the six directions or in an oblique direction.
Although a case has been described in which the vibration unit 37 is provided with a plurality of dampers, the vibration unit 37 may be provided with only one damper.
Such a vibration unit 37 can be used alone, or can be used as a second unit in combination with the first unit. Further, a plurality of vibration units 37 may be connected.

FIG. 5F is a sectional view showing a cross section when a plurality of vibration units are connected.
In this figure, the outer peripheral surface of one of the plurality of vibration units 35A and the outer peripheral surface of one of the vibration units 35B are connected by adhesion or adhesive. The natural vibration characteristics of the vibration unit 35A and the vibration unit 35B may be the same or different. By connecting the vibration unit 35A and the vibration unit 35B, it is possible to edit the vibration characteristics as a combined vibration unit in which the two vibration units are connected. When the natural vibration characteristics of the vibration unit 35A and the vibration unit 35B are made to differ, the weights of the weights may be made different, and the length and hardness of the springs may be changed to provide different spring constants. Good too.
Each weight vibrates due to externally applied vibrations, and a resonance state occurs depending on the externally applied vibrations. For example, vibrations in the connected directions result in a resonant state. Thereby, at the mounting position on the shelf, vibrations from the connected vibration unit can be applied in addition to vibrations applied from the outside.
Although this figure shows a case in which two vibration units are connected, three or more vibration units may be connected.

FIG. 5G is a sectional view showing a cross section when a plurality of vibration units are connected.
In this figure, the outer peripheral surface of one of the plurality of vibration units 35 and the outer peripheral surface of one of the vibration units 37A are connected by adhesion or adhesive. The vibration unit 35 has a weight attached to the inner circumferential surface via a coil spring, and the vibration unit 37A is attached with a damper that damps vibrations in the left-right direction (here, the connection direction). By connecting the vibration unit 35 and the vibration unit 37A, the vibration characteristics as a combined vibration unit in which the two vibration units are connected are edited.
The weight vibrates due to externally applied vibrations, but the damper also drives, so the damper is driven to attenuate the vibrations of the weight. As a result, vibrations based on vibration characteristics depending on the combination of the vibration unit 35 and the vibration unit 37A are applied at the attachment position to the shelf, separately from vibrations applied from the outside.
In this figure, a case has been described in which one vibration unit 35 and one vibration unit 37A are connected, but a plurality of at least one of the vibration units 35 and 37A may be connected. .

The above-mentioned vibration units 15, 25, 35, 36, 37, 35A, 35B, and 37A may be pre-installed in the fixture, or may be adjusted depending on the vibration characteristics of the fixture, the weight of the item placed on the fixture, and the position where it is placed. It may be installed later depending on the situation. In the case of installing the fixture later, it is possible to select and install a vibration unit that makes it easier to detect operations on the fixture (which can be detected with an advantage). In addition, even if the number of shelf boards of fixtures changes or the installation position of the shelf boards changes, the position of the vibration unit should be changed or replaced with a different vibration unit according to the change. You can do it like this.
Further, the vibration unit can be attached to the fixture at any position. Therefore, for example, when performing an operation such as taking out an article from the shelf T or placing an article thereon, it can be attached at a position where it does not get in the way. For example, when attaching the vibration unit to the back of the shelf, there is no need to install the vibration unit on the top of the shelf, so it is possible to secure an area where it can be placed on the shelf, and it is easy to place the vibration unit. It is possible to secure a wide area in which the extraction operation can be performed, and improve operability.
In addition, when installing the vibration unit on the back side of the shelf, it is possible to avoid the flow line when taking out products from the front of the shelf or putting products in from the front of the shelf. Since it can be installed, operability can be improved.
In addition, when the vibration unit is provided at a position that is difficult to see from the front side of the shelf T, the design can be maintained without impairing the design even if the vibration unit is attached to the shelf.

FIG. 6 is a schematic block diagram illustrating the functions of the management system S.
In this embodiment, an example will be described in which the management system S is a smart shelf system. Note that the display shelves in the smart shelf system are an example of fixtures. The display shelf is a shelf having multiple shelf boards. Furthermore, the products in the smart shelf system are examples of objects placed on fixtures and objects taken out from fixtures.
Here, the display shelf may be a shelf T shown in FIG. 1. Moreover, a vibration unit is provided on the display shelf. As the vibration unit, at least one of the above-mentioned vibration units 15, 25, 35, 36, 37, 35A, 35B, 37A, etc. can be used.

The management system S includes a user terminal 2, a sensor device 3, a vibration occurrence position estimation system 4, a product management system 5, and a display device 6. The user terminal 2 and the vibration occurrence position estimation system 4 are communicably connected. The sensor device 3 and the vibration occurrence position estimation system 4 are communicably connected. The vibration occurrence position estimation system 4 and the product management system 5 are communicably connected. The product management system 5 and the display device 6 are communicably connected.

The user terminal 2 is a terminal operated by a user. The user terminal 2 is, for example, a PC (personal computer), a smartphone, a tablet terminal, or the like. The user inputs and confirms various information using an application installed on the user terminal 2, a web browser, or the like.

For example, the user operates the user terminal 2 to input learning data that the vibration occurrence position estimation system 4 uses for machine learning. A trained model used to estimate the position where vibration occurs when a predetermined force is applied to the display shelf (hereinafter also referred to as "vibration occurrence position") through machine learning using the learning data. is generated. The vibration generation position is also the position where a predetermined force is applied. The predetermined force is, for example, a force that is applied to the display shelf when the product is placed on the display shelf, or a force that is applied to the display shelf when the product is removed from the display shelf.
Further, the user can operate the user terminal 2 to confirm the estimation result of the vibration occurrence position by the vibration occurrence position estimation system 4.

The sensor device 3 detects the force applied to the display shelf. This sensor device 3 can use the above-mentioned sensor 14, for example. The sensor device 3 transmits the detection result as sensor information to the vibration occurrence position estimation system 4.

The vibration occurrence position estimation system 4 includes a learning system 40 and an estimation system 50.
The learning system 40 is a system that generates a trained model. The learning system 40 is realized by, for example, a server device, a PC, or the like.
The learning system 40 generates a trained model using learning data received from the user terminal 2. Further, the learning system 40 inputs the sensor information received from the estimation system 50 into the learned model, and transmits the output position information indicating the vibration occurrence position to the estimation system 50.

The estimation system 50 is a system that estimates the position where vibration occurs on a display shelf. The estimation system 50 is realized by, for example, a server device, a PC, or the like.
The estimation system 50 estimates the vibration occurrence position using the trained model generated by the learning system 40. Specifically, the estimation system 50 acquires sensor information from the sensor device 3 and transmits it to the learning system 40, receives position information output from the learned model from the learning system 40, and estimates the vibration occurrence position.
The estimation system 50 transmits the estimation results to the user terminal 2 and the product management system 5.

The product management system 5 is a system that manages products. For example, the product management system 5 manages the inventory of products displayed on display shelves, and displays product descriptions and related advertisements regarding products picked up by customers on the display device 6. The product management system 5 has, for example, product information that associates product location information, inventory information, product descriptions, and related advertisements. Based on the location information indicated by the estimation result received from the estimation system 50, the product management system 5 updates inventory information associated with the location information, acquires product descriptions and related advertisements from the product information, and displays the information on the display device. 6.

The display device 6 is a display device that displays various information. The display device 6 is, for example, a liquid crystal display, a plasma display, an organic EL (Organic Electro-Luminescence) display, a signage display, or the like. Further, the display device 6 is not limited to a display, and may be a projector, for example. In the case of a projector, various information is displayed by projecting content from the projector onto a screen, wall, or the like.
The display device 6 is provided in a store, for example, to display information for customers. The display device 6 displays product descriptions and related advertisements received from the product management system 5 as information for customers.

FIG. 7 is a block diagram showing an example of the functional configuration of the learning system 40 according to this embodiment.
As shown in FIG. 7, the learning system 40 includes a communication section 401, a storage section 402, a learning data acquisition section 403, a learning section 404, an output processing section 405, and a control section 406.

The communication unit 110 has a function of transmitting and receiving various information. For example, the communication unit 110 receives learning data from the user terminal 2. The communication unit 110 also receives sensor information from the estimation system 50. Furthermore, the communication unit 110 transmits position information of the vibration occurrence position output from the learned model to the estimation system 50.
Note that the communication by the communication unit 110 may be wireless communication, wired communication, or a combination of wireless communication and wired communication.

The storage unit 402 has a function of storing various information.
The storage unit 402 includes a storage area for storing learning data and a storage area for storing trained models.
The learning data is a set of position information indicating the position where vibration occurs when a predetermined force is applied to an arbitrary position on the display shelf, and sensor information corresponding to the vibration where the vibration is changed. It is.
The learning data is prepared by the user performing measurements or the like in advance. The user sets the position information of the learning data according to the shape of the display shelf (for example, the number of shelves) and the position that the application wants to recognize. The position information is given as (x, y, L) by, for example, information L indicating which shelf among a plurality of shelves it is, and two-dimensional coordinates (x, y) on the surface of the shelf. . Note that the position information may be given as three-dimensional coordinates (x, y, z) on the display shelf. Here, as the position in the height direction, information identifying which shelf level the shelf board is on may be used as L, or the height based on the bottom of the shelf may be used as z.
The user performs an operation such as placing an object at a position indicated by the set position information or taking out an object, and causes the sensor device 3 to detect sensor information at that time. The user acquires the set position information and the sensor information detected by the sensor device 3 as one learning data. The user inputs the acquired learning data into the user terminal 2 and causes it to be transmitted to the learning system 40. Thereby, the relationship between position information including information in the height direction and detected vibrations can be learned as learning data. The greater the number of learning data, the more accurate the output results of the trained model can be.

A trained model is stored in the trained model storage area.

The storage unit 402 includes a storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), or a RAM ( Random Access read/write Memory), ROM (Read Only Memory), or any combination of these storage media.

The learning data acquisition unit 403 acquires learning data. For example, the learning data acquisition unit 403 can perform an operation in which a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and an object is placed on or taken out from one of the shelves in the fixture. The vibration unit vibrates in response to the application of vibrations generated in response to the operation, and detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to the operation is changed. Sensor information obtained from the sensor and position information including the position in the height direction of the operated fixture are acquired. The learning data acquisition unit 403 may acquire sensor information including the result of detecting vibrations in which both the duration and intensity of vibrations generated in response to an operation are changed.
The learning data acquisition unit 403 acquires the learning data transmitted from the user terminal 2 via the communication unit 110, and writes and stores the learning data in the learning data storage area of the storage unit 402.

The learning unit 404 generates a learned model that has learned the relationship between sensor information and position information using the acquired learning data. For example, the learning unit 404 generates a learned model by machine learning using the learning data acquired by the learning data acquisition unit 403. Specifically, the learning unit 404 performs machine learning on the relationship between sensor information and position information using the learning data acquired by the learning data acquisition unit 403, and performs learning to output position information when sensor information is input. Generate a completed model.
The learning unit 404 writes the generated trained model to the trained model storage area of the storage unit 402 and stores it.

The learning unit 404 uses the learning data to perform machine learning on a neural network model using deep learning. The neural network model is, for example, LSTM (Long Short-Term Memory), which is a type of RNN (Recurrent Neural Network) suitable for inputting time-series signals.
Note that the learning unit 404 may perform preprocessing such as standardization and normalization on the learning data before inputting the learning data to the model.

The output processing unit 405 has a function of controlling output of various information. For example, the output processing unit 405 causes the output result obtained from the learned model to be transmitted to the estimation system 50 via the communication unit 110.

The control unit 406 controls the overall operation of the learning system 40. The control unit 406 is realized, for example, by causing a CPU (Central Processing Unit) included in the learning system 40 as hardware to execute a program.

FIG. 8 is a block diagram showing an example of the functional configuration of the estimation system 50 according to this embodiment.
The estimation system 50 includes a communication section 501, a storage section 502, a sensor information acquisition section 503, an estimation section 504, a weight calculation section 505, an output processing section 506, and a control section 507.

The communication unit 501 transmits and receives various information. For example, the communication unit 501 receives sensor information from the sensor device 3. Furthermore, the communication unit 501 transmits sensor information to the learning system 40. Furthermore, the communication unit 501 receives position information of the vibration occurrence position from the learning system 40 . Further, the communication unit 501 transmits the estimation result of the vibration occurrence position to the user terminal 2 or the product management system 5.
Note that the communication by the communication unit 501 may be wireless communication, wired communication, or a combination of wireless communication and wired communication.

The storage unit 502 is configured by a storage medium, for example, an HDD, SSD, flash memory, EEPROM, RAM, ROM, or any combination of these storage media.

The sensor information acquisition unit 503 acquires sensor information. The sensor information acquisition unit 503 is configured such that a vibration unit is provided at any position of a fixture in which a plurality of shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture. The vibration unit vibrates in response to the application of the vibration generated in response to the operation, and the sensor detects vibration in which at least either the duration or the intensity of the vibration generated in response to the operation is changed. Obtain the sensor information.
The sensor information acquisition unit 503 may detect vibrations in which both the duration and intensity of vibrations that occur in response to an operation are changed. In this case, the sensor information acquisition unit 503 detects a continuous vibration intensity change pattern within the duration by detecting a vibration in which both the duration and intensity of the vibration generated in response to the operation are changed. be able to. That is, the sensor information acquisition unit 503 may detect a change in intensity as a change amount pattern within a duration.
For example, the sensor information acquisition unit 503 acquires sensor information that the communication unit 501 receives from the sensor device 3. This sensor information may include not only the result of detecting vibration but also the result of detecting weight. When the sensor information acquisition unit 503 acquires the result of detecting weight, a weight calculation unit 505, which will be described later, can calculate the difference in weight.

Estimating section 504 estimates the vibration occurrence position. For example, the estimation unit 504 estimates the vibration occurrence position using the learned model. The estimation unit 504 inputs the acquired sensor information into a trained model that has learned the relationship between sensor information and the position of the operated fixture, and calculates position information including the position in the height direction of the operated fixture. By obtaining this information, the position of the operated fixture is estimated.
Specifically, the estimation unit 504 transmits the sensor information acquired by the sensor information acquisition unit 503 from the communication unit 210 to the learning system 40, and causes the sensor information to be input into the learned model stored in the storage unit 402. The estimation unit 504 acquires position information output from the trained model from the learning system 40 via the communication unit 501. Then, the estimation unit 504 estimates the vibration occurrence position based on the position information indicated by the obtained output result. Specifically, the estimation unit 504 estimates the position indicated by the position information indicated by the obtained output result as the vibration occurrence position.

The weight calculation unit 505 calculates the difference in weight of the fixture from the detection results of the weight sensor obtained before and after the operation.

The output processing unit 506 has a function of controlling output of various information. For example, the output processing unit 506 causes the communication unit 501 to transmit the estimation result by the estimation unit 504 to the user terminal 2 or the product management system 5.

The control unit 507 has a function of controlling the overall operation of the estimation system 50. The control unit 507 is realized, for example, by causing a CPU included in the estimation system 50 as hardware to execute a program.

FIGS. 9 and 10 are diagrams showing, as images, the calculation results obtained by calculating the vibration propagated to the shelf when an object is placed on the shelf. FIG. 9 shows the simulation results when the shelf is not provided with a vibration unit, and FIG. 10 shows the simulation result when the shelf is provided with the vibration unit.
Here, the results of an FEM (Finite Element Method) analysis of changes in vibration characteristics when an operation is performed on a shelf are shown.

FIG. 9 shows the results of an FEM analysis for a case where an object is placed at position A, approximately in the center of the top surface of the top shelf of a shelf where no vibration unit is provided. As a result, the vibration propagated to the lowest part of the shelf is propagated in a line-symmetrical relationship with the vibration point as the center on the plane.
On the other hand, the shelf in FIG. 10 has the same shape and structure as the shelf in FIG. 9, but differs in that a vibration unit is provided. Here, the vibration units, which are vortex members, are arranged so that 2 of them are arranged in the vertical direction and 12 of them are arranged in the horizontal direction on each of the four pillars of the shelf, and an object is placed at position A, which is the same as in Fig. 9. The results of FEM analysis are shown for this case. As a result, a result was obtained in which the vibration propagated to the lowest part of the shelf showed a propagation situation different from the propagation situation in FIG. 9 (symbol B).

As described above, the detection result of vibration detected by the sensor is different when a vibration unit is provided than when there is no vibration unit. Here, a single vibration unit may be provided on the shelf, or a plurality of vibration units may be combined to edit the vibration characteristics and provided on the shelf. By editing the vibration characteristics of the vibration unit so that the operated position in the height direction can be easily identified, and by arbitrarily changing the mounting position with respect to the shelf, you can have a high degree of freedom in design. The operated position in the height direction can be detected. Regarding the ease of identifying the position in the height direction, for example, when the stages to be operated are different, the higher the degree of difference in the results of FEM analysis, the easier it is to identify. Therefore, by changing various conditions through FEM analysis in advance and checking the analysis results, it is possible to examine the ease of identifying the position in the height direction.

In this way, by installing the vibration unit, the vibration occurrence position estimation system 4 can easily classify the position not only in the horizontal direction but also in the height direction using the trained model, and identify the position in the height direction. Improves accuracy. That is, the trained model is used to classify the position in the direction based on the detection result of the sensor device, but in order to clarify the classification pattern of the trained model, a vibration unit is used. There is.

FIG. 11 is a diagram illustrating the processing flow of the management system S, and FIG. 12 is a flowchart illustrating an example of the operation of the vibration occurrence position estimation system 4 according to the present embodiment.
《Learning phase》
First, the operator places an object on one of the shelves T or takes out an object placed on one of the shelves T. As a result, a force is applied to the shelf T, and vibrations are propagated to the shelf T. The vibration unit vibrates in response to the vibration generated on the shelf T (FIG. 11: Step S10). The sensor device detects vibrations including vibrations applied in response to operations on the shelf T and vibrations generated by the vibration unit (FIG. 11: Step S11).

On the other hand, the operator inputs operation position data indicating the number of shelves on which the operation has been performed on the shelf T from the user device. The control unit 406 of the learning system 40 obtains the detection results output from the sensor device and the operation position data input to the user device through the communication unit 401, and writes them into the storage unit 402 (FIG. 11: Step S12).
Here, the operation is performed on the shelf T multiple times. Every time the operator performs an operation on the shelf T, the operator inputs the position in the height direction at which the operation was performed from the user device as operation position data. As a result, each time an operation is performed on the shelf T, a combination of the detection result output from the sensor device and the operation position data is acquired by the learning system 40, and written by the control unit 406 to the storage unit 402 as learning data. .

The learning data acquisition unit 131 of the learning system 40 reads the learning data from the storage unit 402 when a certain amount of learning data is written in the storage unit 402 (FIG. 11: Step S13, FIG. 12: Step S101).
When the learning data is read, the learning unit 404 performs machine learning using the learning data and generates a learned model (FIG. 11: Step S14, FIG. 12: Step S102).

The learning unit 404 writes and stores the generated trained model in the trained model storage area of the storage unit 402 (FIG. 11: Step S15, FIG. 12: Step S103).

《Execution phase》
Once the trained model is generated, the estimation system 50 can perform position information estimation processing.
Here, when an operation is performed to place an object on the shelf T or an operation is performed to take out an object placed on the shelf T, the sensor device detects the vibration caused by the operation performed on the shelf T and the vibration unit. Outputs detection results according to the sensor information as sensor information.
The sensor information acquisition unit 503 of the estimation system 50 acquires sensor information output from the sensor device (FIG. 12: Step S104). Specifically, the sensor information acquisition unit 503 acquires sensor information detected by the sensor device 3 and received by the communication unit 210 of the estimation system 50.
Next, the estimation unit 232 of the estimation system 50 transmits the sensor information to the learning system 40 (FIG. 12: Step S105). Specifically, the estimation unit 504 transmits the sensor information acquired by the sensor information acquisition unit 503 from the communication unit 210 to the learning system 40, and causes the sensor information to be input into the learned model.

Next, the control unit 130 of the learning system 40 inputs sensor information to the learned model (FIG. 11: Step S16, FIG. 12: Step S106). Specifically, the control unit 130 inputs the sensor information that the communication unit 110 receives from the estimation system 50 to the learned model stored in the storage unit 402.
Next, the control unit 130 causes the communication unit 110 to transmit the output result from the learned model to the estimation system 50 (FIG. 12: Step S107).

Next, the communication unit 210 of the estimation system 50 receives the output result transmitted from the learning system 40 (FIG. 12: Step S108).
Next, the estimation unit 232 estimates the vibration occurrence position based on the output result (FIG. 11: Step S17, FIG. 12 Step S109). Specifically, the estimation unit 232 estimates the position indicated by the position information of the output result as the vibration occurrence position.
Next, the output processing unit 233 of the estimation system 50 outputs the estimation result (FIG. 11: Step S18, FIG. 12 Step S110). Specifically, the output processing unit 233 transmits the estimation result from the communication unit 210 to the user terminal 2 or the product management system 5.

FIG. 13 is a diagram showing an example in which the estimation results are displayed on the display screen of the user terminal 2 or the display device 6.
The display screen displays the shelf boards 11a, 11b, and 11c of the shelf T, and also displays how the three sensors 14 are arranged on the lower surface side of the shelf board 11a. Then, when an estimation result is obtained in which the coordinates in the horizontal direction and the position in the height direction are estimated, a figure representing the object 18 is displayed on the display screen at a position according to the estimation result. Is displayed. Here, if the estimated position in the height direction is that the object 18 is placed on the second stage from the top, a figure representing the object 18 is displayed on the upper surface side of the shelf board 11b. Here, the horizontal position where the object 18 is displayed is displayed based on the horizontal position obtained as an estimation result.

Further, when the object 18 is taken out from above the shelf board, the displayed figure of the object 18 is hidden from view based on the result of estimating the position at which the object 18 was taken out. This makes it possible to understand on the screen that the object 18 that has been taken out is not placed.
Note that whether the object 18 is placed on the shelf T or taken out from the shelf T is determined by the weight calculation unit 505 based on the weight detection result obtained from the sensor device, based on the weight detection result before the operation is performed. and the detected weight after the operation is performed. The estimating unit 504 can estimate that the article has been taken out if the weight has decreased before and after the operation, and can estimate that the article has been placed when the weight has increased before and after the operation.

By displaying such estimation results on the display device 6, you can, for example, see what products are placed on the product display shelves, or which shelf board they are placed on, without having to take an image of the product shelf with a camera. It is possible to display on the display device 6 whether or not a product is placed. As a result, for example, a customer or a store employee can check the product display status on the display screen of the display device 6 even if he or she does not directly check the product shelf or is located at a distance from the product shelf. .
Furthermore, if the display device 6 is installed in an area where store employees work, such as in the backyard of a store, it is possible to check whether or not the product shelves need to be replenished with products. Further, when the object to be managed is not a product shelf but a shelf in a warehouse, the store clerk can grasp the inventory status by checking the display screen of the display device 6.

In the embodiments described above, the case where the smart shelf system is installed in a store has been described, but it may also be installed in a general household or various facilities such as public facilities. For example, a pump-type container capable of containing liquids such as shampoo and liquid soap can be placed on the shelf. In this case, when the user presses the pump part of the pump-type container to dispense liquid detergent, it is possible to detect vibrations generated in response to this operation. This makes it possible to identify which container on the shelf the container is operated on. Additionally, when the liquid detergent is discharged by pushing the pump, the weight before and after the container is operated is detected, and by finding the difference, the amount of liquid used in the container can be determined. . In this case, not only the position information including the operated position in the height direction is displayed on the display device, but also the remaining amount of liquid in addition to the calculated usage amount may be displayed on the display device. In this case, the display device may be installed near the shelf. This makes it easy to check the remaining amount when using a pump type container.
Here, if the sensor device 3 is a weight sensor, it is possible to not only obtain information that can detect the position in the height direction, but also to detect the difference in weight (amount used) without adding a new sensor. can be detected.

According to the embodiment described above, for example, as shown in FIG. 1, the shelf has three shelf boards: an upper, a middle, and a lower shelf, and the vibration unit shown in FIG. 5F is attached to the lower surface of the uppermost shelf. 35A and the vibration unit 35B is provided, and when the combination vibration unit of the vibration unit 35 and the vibration unit 37A shown in FIG. When this is done, the vibration is transmitted to the vibration unit provided on the lower surface of the upper shelf board, and this vibration unit vibrates. Thereby, the vibration of the entire shelf is strengthened starting from the upper shelf board, and the vibration is sustained, and the vibration can be detected by the sensor device provided at the bottom of the shelf.

Further, when an article is operated on the middle shelf, vibrations are transmitted to a vibration unit provided on the lower surface of the middle shelf, and this vibration unit vibrates. Here, since a damper is provided as the vibration unit 37A, it acts to suppress the vibration of the entire shelf.
In this way, the vibrations detected by the sensor device can be varied depending on the position (step) in the height direction of the shelf board, depending on the difference in the characteristics of the vibration unit and the position at which the vibration unit is attached. Thereby, the learning system 40 can learn so that the position in the height direction can be easily identified, and the position in the height direction can be made easier to identify.

For example, if the horizontal axis is represented as a graph with the maximum amplitude value and the horizontal axis as the duration of vibration, and plotted based on the detection results detected by the sensor device, it can be represented as shown in FIG. 14, for example. By clustering these detection results, it may be possible to identify which stage the operation is for.

In the embodiments described above, the surface on which objects are placed or taken out on the shelf may be a smooth surface, but may also be formed with minute irregularities or may be roughened. Thereby, it is possible to increase the vibration that occurs when the operation is performed.

The learning system 40 and the estimation system 50 in the embodiments described above may each be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Note that the "computer system" herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, a "computer-readable recording medium" refers to a storage medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that retains a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. Further, the above-mentioned program may be one for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configuration is not limited to these embodiments, and includes designs within the scope of the gist of the present invention.

2... User terminal, 3... Sensor device, 4... Vibration occurrence position estimation system, 5... Product management system, 6... Display device, 11a, 11b, 11c... Shelf board, 12a, 12b, 12c, 12d... Support column, 14, 14a, 14b, 14c, 14d... Sensor, 15, 15a, 15b, 15c, 15d... Vibration unit, 18, 18a, 18b... Object, 20, 35c1, 35c2... Base member, 25, 25a, 25b, 35, 35A, 35B, 36... Vibration unit, 25a1, 251b1, 250, 252b1... Vortex member, 25a3, 25b3... Connection member, 36c1, 36c2, 38a, 38b, 38c, 38d, 220... Base member, 37, 37A... Vibration unit, 37a , 37b, 37c, 37d...Damper, 251b...First unit, 251b2, 25a2, 252b2...Pin, 252b...Second unit, 255...Start end, 256...Terminal end, 257...Center point, 351...Slide switch, 40...Learning System, 50... Estimation system, 403... Learning data acquisition unit, 401, 501... Communication unit, 402, 502... Storage unit, 403... Learning data acquisition unit, 404... Learning unit, 405, 506... Output processing unit, 406, 507...Control unit, 503...Sensor information acquisition unit, 504...Estimation unit, 505...Weight calculation unit, T...Shelf, S...Management system

Claims (10)

A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. a sensor information acquisition unit that acquires the
Inputting the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position of the operated fixture, and estimating the position information including the position in the height direction of the operated fixture. Department and
An estimation system with The estimation system according to claim 1, wherein the number of the vibration units is one or more, and the vibration units are installed at arbitrary positions with respect to the fixture. The vibration unit includes a first unit having at least one of the following characteristics: lengthening the vibration duration and increasing the vibration intensity; and shortening the vibration duration and decreasing the vibration intensity. The second unit having at least one of the characteristics can be combined to make it possible to edit the characteristic of changing at least one of the duration and the intensity of the vibration generated in response to the operation. estimation system. The vibration unit is
At least one of the plurality of types of first units having different characteristics,
At least one of the plurality of types of second units having different characteristics,
The estimation system according to claim 3, wherein: is combined. The sensor is a weight sensor,
Any one of claims 1 to 3, wherein the estimation system includes a weight calculation unit that calculates a difference in weight of the fixture from detection results of the weight sensor obtained before and after the operation. The estimation system described in Section. A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. and a learning data acquisition unit that acquires position information including the position in the height direction of the operated fixture;
a learning unit that generates a learned model that has learned the relationship between the sensor information and the position information using the acquired learning data;
A learning system with A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. get
Inputting the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position on the operated fixture to obtain position information including the position in the height direction of the operated fixture.Estimation Method. A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. and position information including the position in the height direction of the operated fixture,
A learning method that generates a trained model that has learned the relationship between the sensor information and the position information using the acquired learning data. A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. get
inputting the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position of the operated fixture to obtain position information including the position in the height direction of the operated fixture; A program that causes a computer to execute. A vibration unit is provided at any position of a fixture in which multiple shelves are provided in the height direction, and when an object is placed on or taken out from one of the shelves in the fixture, Sensor information obtained from a sensor that detects a vibration in which at least one of the duration or intensity of the vibration generated in response to the operation is changed by the vibration unit vibrating in response to the application of the generated vibration. and position information including the position in the height direction of the operated fixture,
A program that causes a computer to generate a learned model that has learned a relationship between the sensor information and the position information using the acquired learning data.

Images