JP7477839B2 - Object state estimation system and object state estimation method - Google Patents

Description

This invention relates to technology for determining changes in the state of an object, and more specifically, to an object state estimation system that estimates the time when the state of an object changes based on multiple images, and an object state estimation method using the same.

Along with steel, concrete is one of the most important construction materials, and is used in a variety of structures, including civil engineering structures such as dams, tunnels, and bridges, as well as architectural structures such as apartment buildings and office buildings. These concrete structures may be manufactured in advance in a factory and transported to the designated location, but in the case of civil engineering and architectural structures, they are often constructed directly at the designated location (site). In any case, concrete structures are constructed by pouring concrete (fresh concrete), which is a mixture of cement, water, aggregate, etc., into a formwork, waiting for the concrete to harden, and then removing the formwork.

Fresh concrete is poured into the formwork by pouring it through a chute from an agitator truck, dropping it from a hose using a concrete pump truck, or in some cases by a worker splashing it with a shovel. The fresh concrete poured into the formwork is then compacted using a vibrator. When the vibrator vibrates the concrete, it liquefies, and as it liquefies further, the air bubbles within the concrete rise and escape to the outside, and the aggregate and mortar within the concrete are rearranged, resulting in the compaction of the concrete.

Conventionally, the judgement of the degree of compaction of concrete (i.e., whether it is sufficiently compacted or not) was left to the judgement of the operator (worker) based on various factors such as subtle changes in the concrete surface and compaction time. The Standard Specifications for Concrete [Construction] also states that the judgement should be made when "a line of cement paste appears on the contact surface between the concrete and the sheathing" and "the concrete volume is not reduced, the surface is almost horizontal, and the surface appears glossy." In other words, the judgement of the degree of compaction of concrete, and by extension the quality of the concrete structure, depended on the experience and knowledge of the vibrator operator. However, there are only a limited number of operators who can accurately judge the compaction of concrete, and in addition, due to the chronic labor shortage problem in recent years, it is becoming difficult to secure such operators in a timely manner.

Therefore, several techniques have been proposed to objectively evaluate the compaction of concrete without relying on the operator's qualitative judgment. For example, Patent Document 1 proposes a technique that continuously captures images of concrete being compacted and uses machine learning to determine the degree of compaction from the images.

JP 2020-41290 A

The technology shown in Patent Document 1 is extremely useful in that it can make objective judgments without relying on the qualitative judgment of the operator. It is not necessary to secure an operator who can properly judge the degree of concrete compaction, and because the judgment results are recorded, it is also possible to carry out post-mortem verification and accountability.

When judging the degree of concrete compaction from images, conventional techniques including Patent Document 1 mainly judge the degree of concrete compaction on an image-by-image basis. In other words, each image captured in succession during concrete compaction work is assessed as either "sufficiently compacted (hereinafter, for convenience, referred to as "completed assessment")" or "not yet sufficiently compacted (hereinafter, for convenience, referred to as "incomplete assessment")," and concrete compaction is judged to be complete based on the time of acquisition of the first image to which a "completed assessment" is assigned.

However, the current situation is that automatic image recognition technology, such as that using machine learning, is still unable to obtain a completely correct answer. It merely outputs the assessment of completion or incomplete, whichever has the higher likelihood, as the correct answer. In particular, when it comes to concrete compaction, even experienced operators will be cautious in their judgments before and after compaction is complete, so that image recognition technology will not give a superior likelihood of either the assessment of completion or incomplete, meaning that it is prone to outputting the incorrect answer.

The objective of the present invention is to solve the problems of the conventional technology, that is, to provide an object state estimation system that can objectively determine that the state of an object (e.g., fresh concrete) has changed (e.g., sufficient compaction has been achieved) without relying on the judgment of a person (operator), and that can make a more appropriate judgment than in the past, and an object state estimation method using the same.

The present invention focuses on the point that, rather than determining that the state of an object has changed based on a single image among multiple images captured in succession of an object whose state is changing, the change in the state of the object is determined comprehensively using multiple images, and is an invention based on an unprecedented concept.

The object state estimation system of the present invention is a system that estimates a change in the state of an object based on a plurality of images of the object captured at a predetermined capture time interval, and includes an image classification means, a state value calculation means, and a change time extraction means. The image classification means performs image recognition using a "trained model" constructed by machine learning to determine whether or not the object has changed for each image, and classifies images determined to have changed in the object as "changed images." The state value calculation means calculates a "state value," which is the cumulative number of changed images counted in order of capture time from the start of capture of the object, and the change time extraction means extracts the capture time of the image when the state value exceeds a predetermined threshold as the "change time" of the object. It is then estimated that the state of the object has changed at the change time extracted by the change time extraction means.

The object state estimation system of the present invention can also estimate a state change of an object by using the proportion of the number of changed images in an "image group" as a state value. Here, an image group is a collection of multiple images obtained by dividing images arranged in order of shooting time into multiple groups. In this case, the state value calculation means calculates the proportion of the number of changed images in the image group as the state value. Furthermore, the change time extraction means selects the image group with the earliest shooting time from among the image groups whose state value exceeds the threshold, and extracts the shooting time of the image in the selected image group with a predetermined shooting order (for example, the last image in the image group) as the change time.

The object state estimation system of the present invention can also estimate the state change of an object by using the proportion of the number of changed images in an "image set" as a state value. Here, an image set is a collection of multiple images obtained by combining a certain number of images arranged in order of shooting time. In this case, the state value calculation means moves through the image sets and calculates the proportion of the number of changed images in the image set as the state value. Furthermore, the change time extraction means selects the image set with the earliest shooting time from among the image sets whose state value exceeds the threshold, and extracts the shooting time of the image in the selected image set with a predetermined shooting order as the change time.

The object state estimation system of the present invention can also estimate the degree of compaction for fresh concrete that is being continuously compacted. In this case, the image classification means classifies an image in which it is determined that the fresh concrete has been compacted as a post-change image.

The object state estimation system of the present invention may further include a display means and a display parameter setting means. The display means is a means for displaying "split images" obtained by splitting an image into a plurality of parts, and the display parameter setting means is a means for setting "display parameters" for each split image according to a state value. Examples of the display parameters include the transparency when the split image is displayed, the display color for highlighting the split image, and the display color of the frame line displayed around the split image. In this case, the image classification means determines whether or not there is a change in the object for each split image, and classifies the split image determined to have a change in the object into a changed image, and the state value calculation means calculates a state value for each split image. The display means then displays the split image based on the display parameters set by the display parameter setting means.

The object state estimation method of the present invention is a method for estimating a change in the state of an object using the object state estimation system of the present invention, and is a method including an image acquisition step, an image classification step, a state value calculation step, and a change time extraction step. In the image acquisition step, the object is photographed at a shooting time interval to acquire a plurality of images, and in the image classification step, an image determined by the image classification means to have changed in the object is classified as a changed image. In the state value calculation step, a state value is calculated by the state value calculation means, and in the change time extraction step, the change time of the object is extracted by the change time extraction means. It is then estimated that the state of the object has changed at the change time extracted by the change time extraction step.

The object state estimation system and the object state estimation method of the present invention have the following advantages.
(1) For example, it is possible to objectively and instantly (in real time) determine whether or not compaction of concrete (object) is complete (change in state) without relying on the judgment of an operator who operates the vibration vibrator.
(2) By absorbing errors in image recognition using machine learning, it is possible to more accurately determine the degree of compaction of concrete than was previously possible.
(3) The present invention can be applied to various cases, such as not only cases where a vibration vibrator is operated by a person, but also cases where a vibration vibrator attached to construction machinery or the like is operated.

A model diagram showing a schematic diagram of multiple images that make up a video or a series of still images. FIG. 1 is a block diagram showing a main configuration of an object state estimation system according to the present invention. FIG. 2 is a flowchart showing a main process flow of the object state estimation system of the present invention. FIG. 11 is a model diagram for explaining a procedure for calculating a first state value. FIG. 11 is a model diagram for explaining a procedure for calculating a second state value. FIG. 11 is a model diagram for explaining a procedure for calculating a third state value. FIG. 2A is a front view showing an image in which one image is displayed as one region, and FIG. 2B is a front view showing an image made up of a plurality of divided regions. 1A is a front view showing the state of concrete in the early stages of compaction in terms of transparency, and FIG. 1B is a front view showing the state of concrete after compaction has progressed to a certain extent in terms of transparency. (a) is a front view in which the state of concrete at the early stage of compaction is shown by the display color of the image frame, and (b) is a front view in which the state of concrete after compaction has progressed to a certain extent is shown by the display color of the image frame. FIG. 13 is a model diagram showing images of interest selected from an image group or set of images of interest based on a predetermined shooting order. FIG. 2 is a flowchart showing the main process flow of the object state method of the present invention.

An example of the implementation of the object state estimation system and object state estimation method of the present invention will be described with reference to the drawings. The object state estimation system and object state estimation method of the present invention are techniques for estimating a specific change point (changing state) of an object whose state is changing, and therefore can be used for anything whose state changes. For example, it can be used to estimate the state change of various objects, such as estimating the state of fresh concrete that has hardened to a certain extent, estimating the state of an object that is being compacted as an embankment, or estimating the state of a slope that is at risk of landslide or collapse when harmful changes have occurred. For convenience, an example of estimating the state of fresh concrete that is being compacted (hereinafter referred to as the "compaction completion state") will be described here.

When the object state estimation system and object state estimation method of the present invention are used to estimate the completed compaction state of concrete, they can be implemented not only in cast-in-place concrete work, but also in the production of precast concrete in factories and the production of precast concrete (so-called site PC) at construction sites.

1. Overall Overview The present invention is a technology for estimating the compaction completion state based on images of concrete undergoing continuous compaction work, for example. More specifically, concrete that changes due to compaction is photographed in sequence, and a provisional judgment is made for each of the multiple images obtained as a result as to whether or not the compaction is complete, and the time when the compaction completion state is reached is estimated comprehensively from the multiple images for which the judgment was made. Therefore, in order not to miss the target change state (compaction completion state), it is preferable that the time interval (hereinafter referred to as the "photographing time interval") for acquiring images of the target object (concrete) is relatively short, and it is preferable to photograph the target object as a video or continuous still images as shown in FIG. 1. Note that both the video and the continuous still images are composed of images (frames) acquired at extremely short intervals, but since a video is generally composed of images of 24 to 60 frames per second (i.e., 24 to 60 fps), here, continuous still images are particularly composed of images of less than 24 fps or more than 60 fps.

2. Object State Estimation System The object state estimation system of the present invention will be described in detail. Note that the object state estimation method of the present invention is a method for estimating a change in the state of an object using the object state estimation system of the present invention. Therefore, the object state estimation system of the present invention will be described first, and then the object state estimation method of the present invention will be described.

Figure 2 is a block diagram showing the main components of the object state estimation system 100 of the present invention. As shown in this figure, the object state estimation system 100 of the present invention is configured to include an image classification means 110, a state value calculation means 120, and a change time extraction means 130, and can also be configured to include a display specification setting means 140, a display means 150 such as a display, an image storage means 160, and a trained model storage means 170.

Of the main components constituting the object state estimation system 100, the image classification means 110, state value calculation means 120, change time extraction means 130, and display specification setting means 140 can be manufactured as dedicated components, or a general-purpose computer device can be used. This computer device is equipped with a processor such as a CPU, memory such as ROM and RAM, input means such as a mouse and keyboard, and a display, and can be configured from a personal computer (PC), a server, a tablet PC such as an iPad (registered trademark), or a mobile terminal including a smartphone. When using a computer device equipped with a display, it is recommended that the display be used as the display means 150.

The image storage means 160 and the trained model storage means 170 can be implemented as a general-purpose computer storage device, or as a database server. If implemented as a database server, they can be placed on a local network (LAN: Local Area Network), or can be a cloud server that stores data via the Internet (i.e., wireless communication).

Next, the main process flow when using the object state estimation system 100 of the present invention will be described with reference to Figure 3. Figure 3 is a flow diagram showing the main process flow of the object state estimation system 100, with the central column showing the process to be performed, the left column showing the input information required for that process, and the right column showing the output information resulting from that process.

First, the image classification means 110 (Fig. 2) reads an image from the image storage means 160 (Fig. 2) and judges whether the state of the concrete in the image is "compacted" or not (Step 110 in Fig. 3). The image storage means 160 stores a plurality of images (i.e., videos or continuous still images) of concrete that changes due to compaction taken at a predetermined shooting time interval, associated (linked) with the shooting time, that is, these images are given information of the shooting time. The image classification means 110 then judges whether each read image is in a compacted state or not. However, this judgment is only a provisional judgment made on an image-by-image basis, and does not ultimately judge the state of the concrete. For convenience, an image that is judged to have changed in the object, that is, an image that is judged to be in a compacted state, is referred to as a "changed image", and an image that is not judged to be in a compacted state is referred to as a "pre-change image". In other words, the image classification means 110 classifies each image read from the image storage means 160 into a post-transformation image or a pre-transformation image.

When the image classification means 110 classifies images, a "trained model" stored in the trained model storage means 170 (FIG. 2) is used. This trained model is constructed by machine learning, and more specifically, it is a model that extracts features of the transformed image by learning from "images determined to be transformed images" and "images determined to be pre-transformation images" as teacher data, and outputs whether or not an arbitrary image corresponds to the transformed image when it is input. Note that the machine learning used to construct the trained model can employ various conventional machine learning techniques, including deep learning including CNN (Convolutional Neural Network), which is often used for image recognition, and NIN (Network In Network), which is said to recognize images with higher accuracy than CNN.

Once the image classification means 110 classifies all the images read out as either post-change images or pre-change images, the state value calculation means 120 (FIG. 2) calculates a "state value" indicating the state of the concrete (Step 120 in FIG. 3). This state value is determined according to the number and proportion of post-change images, and can be broadly classified into a state value calculated based on the cumulative number of post-change images (hereinafter referred to as the "first state value"), a state value calculated based on the proportion of post-change images in a set of a certain number of images (hereinafter referred to as the "image group") (hereinafter referred to as the "second state value"), and a state value calculated based on the proportion of post-change images in a set of images included in a moving area (hereinafter referred to as the "image set") (hereinafter referred to as the "third state value"). Below, the procedures for calculating the first state value, the second state value, and the third state value will each be described in detail.

Figure 4 is a model diagram for explaining the procedure for calculating the first state value. In this diagram, each image is represented by a line segment (vertical line), and the images are arranged in order of shooting time (i.e., in chronological order) starting from the image at which shooting began (the image at the leftmost end in the diagram). In addition, the before-change image and the after-change image are distinguished by the shade of the line segment; that is, the before-change image is represented by a pale (thin) line segment, and the after-change image is represented by a dark line segment.

As shown in Figure 4, the more time passes, i.e., the longer the compaction time, the more images are classified as changed images. When the first changed image appears, the frequency of changed images is low, but in the latter half (right side of the figure), the frequency of changed images increases. Therefore, if the first changed image is determined to be the "compaction complete state" as in the conventional technology, the chance of the features in the image cannot be eliminated, and the reliability of the judgment is low. On the other hand, as the number of changed images increases, the frequency of their appearance also increases, and it can be sufficiently estimated that the "compaction complete state" has been reached. In other words, the number of changed images can be considered an index of the compaction state of the concrete. Therefore, the "first state value" is calculated as the cumulative number of changed images counted in order of shooting time from the start of shooting.

Figure 5 is a model diagram for explaining the procedure for calculating the second state value. As in Figure 4, before-change images are indicated by light lines and after-change images are indicated by dark lines, arranged in order of their shooting time. To calculate the second state value, the images arranged in order of their shooting time are first divided into a number of image groups. For example, in the case of Figure 5, the images are divided into five image groups, from the first image group to the fifth image group. The second state value is calculated as the number of images constituting the image group, that is, the ratio (percentage) of the number of after-change images to the total number of before-change images and after-change images. For example, the second state value for an image group containing 21 before-change images and 9 after-change images is 30%, and the second state value for an image group containing 3 before-change images and 27 after-change images is 90%. In this way, the second state value is calculated for each image group.

The image group can be divided into a predetermined number of images, and for example, an image group can be set to include 30 images. Of course, the division is not limited to 30 images, and the number of images can be appropriately set according to the state of the concrete, such as 15 images or 60 images. Also, considering that the images are acquired at a predetermined shooting time interval, the image group can be said to be a set (group) of images divided into a certain time interval. For example, if images are acquired at 30 fps, if the image group is divided into 30 images, the divided group can be considered to have a length of 1 second. In other words, the image group is, so to speak, a time width for determining the state of the concrete, and the second state value is an index that indicates the state of the concrete determined for each time width. Conventionally, an operator determines the degree of compaction in units of 1 to several seconds, so it is advisable to divide the image group so that the time width is 1 to several seconds.

Figure 6 is a model diagram for explaining the procedure for calculating the third state value. In this diagram, as in Figure 4, the pre-change images shown by light lines and the post-change images shown by dark lines are arranged in order of shooting time. When calculating the third state value, an image set is first set for the images arranged in order of shooting time. This image set is composed of multiple images (e.g., 30 images) like an image group. However, while an image group is a static collection of images that does not move in the time direction, an image set is a dynamic collection of images that move in the time direction. Specifically, as shown in Figure 6, an image set is a collection of images included in an area that is set to include a predetermined number of images (e.g., 30 images) and that moves in the direction of time (hereinafter referred to as a "moving area"). Note that the unit amount by which the moving area moves at one time can be designed arbitrarily, and it is recommended to set it as a multiple (natural number multiple) of the shooting time interval, for example.

The third state value is calculated as the ratio (percentage) of the number of images included in the image set, that is, the number of post-change images to the total number of before-change images and after-change images. For example, the third state value in an image set containing 15 before-change images and 15 after-change images is 50%, and the third state value in an image set containing 12 before-change images and 18 after-change images is 60%. The third state value is calculated each time the movement area that forms the image set moves.

When the state value calculation means 120 calculates the state value, the display specification setting means 140 (FIG. 2) sets the "display specification" according to the state value (Step 130 in FIG. 3), and the display means 150 (FIG. 2) displays the image according to the display specification (Step 140 in FIG. 3). The display specification here refers to requirements for displaying an image, such as the transparency of the image, the display color for coloring the image (hereinafter referred to as the "image display color"), and the line color and line type of the frame line representing the periphery of the image (hereinafter referred to as the "image frame"). The display specification also has the role of indicating the state of the concrete, and therefore its value changes according to the state value. For example, it can be designed with various specifications, such as a specification in which the transparency becomes higher (or lower) as the state value indicates a larger value, a specification in which the image display color is changed (e.g., from red to green) when the state value exceeds a predetermined value, a specification in which the display color of the image frame is changed (e.g., from white to black) when the state value exceeds a predetermined value, or a specification in which a combination of multiple requirements (such as transparency and image display color) is changed according to the state value.

The object state estimation system 100 of the present invention can be designed to process one image as one region, or to process one image as composed of multiple small regions (hereinafter referred to as "split regions SP") by dividing it. When the object state estimation system 100 processes an image composed of multiple split regions SP, the split regions SP are treated as individual images. That is, the image classification means 110 (FIG. 2) classifies each split region SP into a pre-change image or a post-change image (Step 110 in FIG. 3), the state value calculation means 120 (FIG. 2) calculates a state value for each split region SP (Step 120 in FIG. 3), and the display parameter setting means 140 (FIG. 2) sets display parameters for each split region SP (Step 130 in FIG. 3). The display means 150 displays an image composed of multiple split regions SP (6 x 8 in the figure) as shown in FIG. 7(b). Note that FIG. 7(a) shows the image that is displayed when the object state estimation system 100 processes one image as one region.

Figure 8 shows an example in which the display parameter setting means 140 sets "transparency" as a display parameter for each divided area SP, and the display means 150 displays the divided areas SP according to that transparency, where (a) shows the state of concrete in the early stages of compaction, and (b) shows the state of concrete after compaction has progressed to a certain extent. In this figure, the transparency increases as the state value increases, and in Figure 8(a), since compaction is in the early stages, many divided areas SP are colored black (i.e., have low transparency). In contrast, in Figure 8(b), compaction has progressed to a certain extent, so most of the divided areas SP except for the four corners are in a transparent state, which means that compaction is progressing overall.

Figure 9 shows an example in which the display parameter setting means 140 sets the "image frame display color" as a display parameter for each divided area SP, and the display means 150 displays the divided area SP according to the image frame display color, where (a) shows the state of concrete in the early stages of compaction, and (b) shows the state of concrete after compaction has progressed to a certain extent. In this figure, the display color of the image frame is set to change from white to black when the state value exceeds a predetermined value, and since Fig. 9(a) is in the early stages of compaction, there are many divided areas SP with white image frames. In contrast, in Fig. 9(b), compaction has progressed to a certain extent, so most of the divided areas SP except for the four corners have black image frames, which indicates that compaction is progressing overall.

When the number of acquired images increases by a predetermined number or when a certain amount of time has passed (i.e., periodically), the image classification means 110 classifies the images (or divided areas SP), the state value calculation means 120 calculates a state value for each image (or divided area SP), and the display specification setting means 140 can set display specifications for each image (or divided area SP). The display means 150 can also change (switch) the image to be displayed when the display specifications are set by the display specification setting means 140, or periodically.

When the state value calculation means 120 calculates the state value, the change time extraction means 130 (FIG. 2) compares the state value with a predetermined threshold value (hereinafter referred to as the "state threshold value") (Step 150 in FIG. 3). If the state value falls below the state threshold value (No in Step 150), the conditions are changed to obtain the state value (Step 120 in FIG. 3). For example, in the case of the first state value shown in FIG. 4, the time is advanced and the cumulative number of images after the change is calculated, in the case of the second state value shown in FIG. 5, the image group is changed (third image group → fourth image group) to obtain the proportion of the number of images after the change, and in the case of the third state value shown in FIG. 6, the image set is changed (the movement area is moved) to obtain the proportion of the number of images after the change.

On the other hand, when the state value exceeds the state threshold (Yes in Step 150), an image or divided area SP that can be estimated to be in a "compaction completed state" (hereinafter referred to as "image of interest") is selected (Step 160). This image of interest may be selected according to the type of state value to be sought. For example, when the first state value is sought, the image when the cumulative number of images after the change exceeds the state threshold for the first time can be selected as the image of interest. In the case of FIG. 4, the state threshold is set to 40, and the image when the first state value becomes 40 is selected as the image of interest. Of course, the state threshold is not limited to 40, and can be set appropriately according to the state of the concrete, such as 90 or 120. In addition, the object state estimation system 100 of the present invention may be provided with a "state threshold input means" so that the operator can input a desired state threshold.

When determining the second state value, it is advisable to first select an image group including an image of interest (hereinafter referred to as the "image group of interest"). This image group of interest can be selected as the image group with the earliest shooting time among the image groups whose second state value exceeds the state threshold. In the case of FIG. 5, the state threshold is set to 90%, and the fifth image group when the second state value reaches 90% is selected as the image group of interest. Of course, the state threshold is not limited to 90%, and can be set appropriately according to the state of the concrete, such as 60% or 80%. Note that when the state threshold is set to 60% in FIG. 5, the state value of the fourth image group (60%) and the state value of the fifth image group (90%) are both above the state threshold, but since the shooting time of the fourth image group is earlier than that of the fifth image group, the fourth image group is selected as the image group of interest.

When a target image group is selected, a target image is selected from among the images (or divided area SP) contained in the target image group. At this time, it is advisable to determine in advance the ordinal number (hereinafter referred to as the "shooting order") of the images in the target image group, starting with the image taken earliest, and to select the target image based on this shooting order. For example, in the case shown in FIG. 10, an image group is made up of 20 images, and the image taken 12th (third from the last) is selected as the target image. Of course, the shooting order of the image selected as the target image is not limited to the third from the last, and it can be the last image in the target image group, an intermediate image in the target image group, or any other image appropriate to the condition of the concrete, etc.

When determining the third state value, as with the second state value, it is advisable to first select an image set including an image of interest (hereinafter referred to as an "image set of interest"). This image set of interest can be selected as the image set with the earliest shooting time for which the third state value exceeds the state threshold. Once the image set of interest is selected, an image of interest is selected from among the images (or divided area SP) included in the image set of interest. At this time, as with the second state value, it is advisable to select an image of interest from the image set of interest based on a predetermined shooting order.

When an image of interest is selected, the change time extraction means 130 (Figure 2) extracts the shooting time (hereinafter referred to as the "change time") associated with the image of interest (Step 170 in Figure 3). As described above, the image storage means 160 stores images along with their shooting times. Therefore, for example, by querying the image storage means 160 with the identifier of the image of interest, the shooting time (i.e., the change time) of the image of interest can be obtained. Then, it is estimated that the target concrete has reached a "compaction complete state" at this change time.

3. Object State Estimation Method Next, the object state estimation method of the present invention will be described with reference to Fig. 11. Note that the object state estimation method of the present invention is a method for estimating a change in the state of an object using the object state estimation system 100 described up to this point, and therefore, explanations that overlap with the contents described in the object state estimation system 100 will be avoided, and only the contents unique to the object state estimation method of the present invention will be described. In other words, the contents not described here are the same as those described in "2. Object State Estimation System".

Figure 11 is a flow diagram showing the main steps of the object state estimation method of the present invention. As shown in this figure, first, a video or continuous still images of fresh concrete undergoing compaction are taken (Step 10 in Figure 11). When the images of the concrete are acquired, the image classification means 110 of the object state estimation system 100 of the present invention is used to classify each image (or divided area SP) into a pre-change image or a post-change image (Step 20 in Figure 11), and the state value calculation means 120 of the object state estimation system 100 is used to calculate a state value for each image (or divided area SP) (Step 30 in Figure 11). Then, the change time extraction means 130 of the object state estimation system 100 is used to select an image of interest and extract the change time (Step 40 in Figure 11), and it is estimated that the target concrete has reached a "compaction completion state" at this change time.

The object state estimation system and object state estimation method of the present invention can be used to determine changes in the state of various objects whose states change, such as determining the compaction of concrete, the hardening of concrete, and the compaction of embankments. Considering that the present invention can provide, for example, appropriately compacted, high-quality concrete structures, it can be said to be an invention that can be expected to not only be used industrially, but also to make a significant contribution to society.

100 Object state estimation system of the present invention 110 Image classification means (of object state estimation system) 120 State value calculation means (of object state estimation system) 130 Change time extraction means (of object state estimation system) 140 Display specification setting means (of object state estimation system) 150 Display means (of object state estimation system) 160 Image storage means (of object state estimation system) 170 Learned model storage means (of object state estimation system) SP Divided area

Claims (7)

A system for estimating a change in a state of an object based on a plurality of images of the object captured at a predetermined capture time interval, comprising:
An image classification means for determining whether or not there is a change in the object for each of the images by performing image recognition using a trained model constructed by machine learning, and classifying the image for which it is determined that there is a change in the object as a changed image;
a state value calculation means for calculating a state value which is a cumulative number of the post-change images counted in order of shooting time from the start of shooting the object;
a change time extraction means for extracting, as a change time of the object, a photographing time of the image when the state value exceeds a predetermined threshold value,
It is estimated that a state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system comprising: A system for estimating a change in a state of an object based on a plurality of images of the object captured at a predetermined capture time interval, comprising:
An image classification means for determining whether or not the object has changed for each image by performing image recognition using a trained model constructed by machine learning, and classifying the image for which it is determined that the object has changed as a changed image;
a state value calculation means for dividing the images arranged in order of shooting time into a plurality of image groups and calculating a state value which is a ratio of the number of the changed images in each image group;
a change time extraction means for selecting an image group having the earliest shooting time from among the image groups whose state value has exceeded a predetermined threshold value, and extracting, from the selected image group, the shooting time of an image having a predetermined shooting order as the change time of the object,
It is estimated that a state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system comprising: A system for estimating a change in a state of an object based on a plurality of images of the object captured at a predetermined capture time interval, comprising:
An image classification means for determining whether or not the object has changed for each image by performing image recognition using a trained model constructed by machine learning, and classifying the image for which it is determined that the object has changed as a changed image;
a state value calculation means for setting an image set of a certain number of the images arranged in order of shooting time, and calculating a state value that is a ratio of the number of the changed images in the image set while moving through the image set;
a change time extraction means for selecting an image set having the earliest photographing time from among the image sets in which the state value exceeds a predetermined threshold value, and extracting, from the selected image set, a photographing time of the image having a predetermined photographing order as a change time of the object,
It is estimated that a state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system comprising: The object is fresh concrete that is being continuously compacted,
The image classification means classifies the image in which it is determined that the fresh concrete has been compacted into the changed image.
4. The object state estimation system according to claim 1, wherein the object state estimation system is a system for estimating an object state based on the state of the object. a display means for displaying a plurality of divided images obtained by dividing the image;
a display parameter setting means for setting display parameters for each of the divided images according to the state value,
the image classification means determines whether or not there is a change in the object for each of the divided images, and classifies the divided images for which it is determined that there is a change in the object into the changed image;
the state value calculation means calculates the state value for each of the divided images;
The display means displays the divided image based on the display specifications set by the display specification setting means.
5. The object state estimation system according to claim 1, the display parameter setting means sets a transparency, a display color, or a frame color as the display parameter based on the state value of the divided image;
6. The object state estimation system according to claim 5. A method for estimating a change in a state of an object using the object state estimation system according to any one of claims 1 to 6, comprising:
an image acquisition step of photographing the object at the photographing time interval and acquiring a plurality of the images;
an image classification step of classifying the image determined to have a change in the object by the image classification means into the changed image;
a state value calculation step of calculating the state value by the state value calculation means;
a change time extraction step of extracting the change time of the object by the change time extraction means,
It is estimated that a state of the object has changed at the change time extracted by the change time extraction step.
4. A method for estimating an object state comprising:

Images