JP2022015003A - Object state estimation system and object state estimation method - Google Patents

Abstract

To provide an object state estimation system capable of determining the state of the object has changed without relying on determination of an operator, as well as, determining it more appropriately then ever, and to provide an object state estimation method using the system, so as to solve the problem of the prior art.SOLUTION: An object state estimation system of the present invention estimates a state change of an object based on images, and is provided with image classification means, state value calculation means, and change time extraction means. Of these, the state value calculation means calculates a "state value", which is a cumulative number of changed images counted in order of shooting time points from a start of shooting of the object, and the change time extraction means extracts, as a "change time" of the object, a shooting time point of the image when the state value exceeds a predetermined threshold value.SELECTED DRAWING: Figure 2

Description

The present invention is a technique for determining a state change of an object, and more specifically, an object state estimation system that estimates a time point when the state of an object changes based on a plurality of images, and an object state estimation system using the same. It relates to an object state estimation method.

Concrete is one of the most important construction materials along with steel, and is used in various structures such as civil engineering structures such as dams, tunnels, and bridges, and building structures such as apartment buildings and office buildings. This concrete structure may be manufactured in advance at a factory or the like and transported to a predetermined place, but in the case of a civil engineering structure or a building structure, it is often directly constructed at a predetermined place (site). In any case, the concrete structure is constructed by putting concrete (fresh concrete) in a state where cement, water, aggregate, etc. are kneaded into the formwork, waiting for the concrete to harden, and then removing the formwork. ..

Fresh concrete is poured into the formwork by pouring it from an agitator truck through a chute, dropping it from a hose by a concrete pump truck, or in some cases by a worker shoveling it. Then, the fresh concrete put into the formwork is compacted by a vibrator (vibration vibrator). When vibration is applied by the vibration vibrator, the concrete is liquefied, and the liquefaction causes the bubbles in the concrete to rise and escape to the outside, and the aggregate and mortar in the concrete are rearranged, resulting in the concrete. It is compacted.

Conventionally, in determining the appropriateness of the degree of compaction of concrete (that is, whether or not it has been sufficiently compacted), an operator (worker) based on various factors such as subtle changes in the concrete surface and compaction time. ) Was left to the judgment. In the concrete standard specification [Construction], "a line of cement paste appears on the contact surface between the concrete and the dam" and "the volume of the concrete is not reduced, and the surface becomes almost horizontal. It is decided to make a judgment after confirming that "gloss appears on the surface". In other words, the appropriateness of the degree of compaction of concrete, and the quality of the concrete structure, depended on the experience and knowledge of the operator of the vibration vibrator. However, the number of operators who can accurately judge the suitability of compaction is limited, and it is difficult to secure such operators in a timely manner due to the problem of chronic labor shortage in recent years. ..

Therefore, some techniques for objectively evaluating the compaction of concrete without relying on the operator's so-called qualitative judgment have been proposed so far. For example, Patent Document 1 proposes a technique for continuously acquiring images of concrete to be compacted and utilizing machine learning to determine the degree of compaction from the images.

Japanese Unexamined Patent Publication No. 2020-41290

The technique shown in Patent Document 1 is extremely useful in that it can be objectively determined without relying on the qualitative judgment of the operator. It is not always necessary to secure an operator who can properly judge the degree of compaction of concrete, and since the judgment result remains, it is possible to fulfill post-mortem verification and accountability.

By the way, when the degree of compaction of concrete is judged from an image, in the prior art including Patent Document 1, it is the mainstream to judge by the acquired image unit. That is, for each of the individual images continuously acquired during the concrete compaction work, "evaluation that sufficient compaction has been performed (hereinafter referred to as" completion evaluation "for convenience)" or "still sufficient". One of the "evaluations" (hereinafter referred to as "unfinished evaluation" for convenience) "is given to the concrete if it has not been compacted, and the concrete is given the acquisition time of the image to which the" completion evaluation "was first given from the image acquisition. It was judged that the compaction was completed.

However, the current situation is that it is still not possible to obtain a complete correct answer with a technology that automatically recognizes images, such as by utilizing machine learning. To the last, the one with the higher likelihood of the completed evaluation or the incomplete evaluation is output as the correct answer. Especially in the case of concrete compaction, the likelihood of either completion evaluation or incomplete evaluation does not prevail in the judgment by image recognition technology so that even a skilled operator can make a careful judgment before and after the completion of compaction. In other words, it tends to output incorrect answers.

The object of the present invention is to solve the problem of the prior art, that is, the state of the object (for example, fresh concrete) is objectively changed without relying on the judgment of a person (operator) (for example, sufficient). It is to provide an object state estimation system that can be determined to have been compacted) and that can be appropriately determined as compared with the conventional case, and an object state estimation method using the system.

The present invention uses a plurality of images, rather than determining that the state of the object has changed due to a single image, among a plurality of images continuously acquired for the object whose state changes. So to speak, it was made focusing on the point that the state change of the object is comprehensively determined, and it is an invention made based on an unprecedented idea.

The object state estimation system of the present invention is a system that estimates a change in the state of an object based on a plurality of images of the object taken at a predetermined shooting time interval, and is an image classification means, a state value calculation means, and a change. It is equipped with a time extraction means. Of these, the image classification means determines whether or not there is a change in the object for each image by performing image recognition using the "trained model" constructed by machine learning, and it is determined that the object has a change. It is a means to classify the image as a "post-change image". Further, the state value calculation means is a means for calculating a "state value" which is the cumulative number of changed images counted in order of shooting time from the start of shooting of the object, and the change time extraction means is a means for extracting the state value to a predetermined threshold value. This is a means for extracting the shooting time of the image when the value exceeds the above as the "change time" of the object. Then, it is estimated that the state of the object has changed at the change time extracted by the change time extraction means.

The object state estimation system of the present invention can also estimate the state change of the object by using the ratio of the number of changed images in the "image group" as the state value. Here, the image group is a set of a plurality of images obtained as a result of dividing the images arranged in order of shooting time into a plurality of images. In this case, the state value calculating means calculates the ratio of the number of changed images in the image group as the state value. Further, the change time extraction means selects the image group related to the earliest shooting time among the image groups whose state values exceed the threshold value, and also selects a predetermined shooting order (for example, the end of the image group) among the selected image groups. ) Is extracted as the change time.

The object state estimation system of the present invention can also estimate the state change of the object by using the ratio of the number of changed images in the "image set" as the state value. Here, the image set is a set consisting of a plurality of images obtained as a result of collecting a certain number of images arranged in order of shooting time. In this case, the state value calculating means calculates the ratio of the number of changed images in the image set as the state value while moving the image set. Further, the change time extraction means selects the image set related to the earliest shooting time from the image sets whose state values exceed the threshold value, and also selects the shooting time related to the images of the predetermined shooting order from the selected image sets. Extract as change time.

The object state estimation system of the present invention can also estimate the degree of compaction of fresh concrete that has been continuously compacted. In this case, the image classification means classifies the image determined to be compacted with fresh concrete into the post-change image.

The object state estimation system of the present invention may further include display means and display specification setting means. This display means is a means for displaying a "divided image" in which an image is divided into a plurality of parts, and a display specification setting means is a means for setting a "display specification" according to a state value for each divided image. Here, the display specifications can be increased such as the transparency when displaying the divided image, the display color for highlighting the divided image, and the display color of the frame line displayed around the divided image. In this case, the image classification means determines whether or not there is a change in the object for each divided image, classifies the divided image determined to have a change in the object into the changed image, and the state value calculation means The state value is calculated for each divided image. Then, the display means displays the divided image based on the display specifications set by the display specification setting means.

The object state estimation method of the present invention is a method of estimating a state change of an object by using the object state estimation system of the present invention, and is an image acquisition step, an image classification step, a state value calculation step, and a change time extraction step. It is a method equipped with. Of these, in the image acquisition step, an object is photographed at shooting time intervals to acquire a plurality of images, and in the image classification step, images determined by the image classification means to have a change in the object are classified into post-change images. do. Further, in the state value calculation step, the state value is calculated by the state value calculation means, and in the change time extraction step, the change time of the object is extracted by the change time extraction means. Then, it is estimated that the state of the object has changed at the change time extracted by the change time extraction step.

The object state estimation system and the object state estimation method of the present invention have the following effects.
(1) For example, it is possible to objectively and immediately (real-time) determine the completion of compaction (state change) of concrete (object) without relying on the determination of the operator who operates the vibration vibrator.
(2) As a result of absorbing the error of image recognition by machine learning, it is possible to appropriately judge the degree of compaction of concrete as compared with the conventional case.
(3) Not only when the vibration vibrator is operated by a person, but also when the vibration vibrator attached to a construction machine or the like is operated, it can be applied to various cases.

A model diagram schematically showing a plurality of images constituting a moving image or a continuous still image. The block diagram which shows the main structure of the object state estimation system of this invention. The flow chart which shows the main processing flow of the object state estimation system of this invention. A model diagram for explaining a procedure for calculating a first state value. A model diagram for explaining a procedure for calculating a second state value. A model diagram for explaining a procedure for calculating a third state value. (A) is a front view showing an image displayed with one image as one area, and (b) is a front view showing an image composed of a plurality of divided areas. (A) is a front view showing the state of concrete at the initial stage of compaction by permeability, and (b) is a front view showing the state of concrete in which compaction has progressed to some extent by permeability. (A) is a front view showing the state of concrete at the initial stage of compaction by the display color of the image frame, and (b) is a front view showing the state of concrete in which compaction has progressed to some extent by the display color of the image frame. A model diagram schematically showing an attention image selected based on a predetermined shooting order from an attention image group or an attention image set. The flow chart which shows the main processing flow of the object state method of this invention.

An example of the implementation of the object state estimation system and the object state estimation method of the present invention will be described with reference to the drawings. The object state estimation system and the object state estimation method of the present invention are techniques for estimating a specific change point (change state) of an object whose state changes, and therefore any state changes. You can target things. For example, it is estimated that the object is hardened to a certain extent as fresh concrete, it is estimated that the object is sufficiently compacted as an embankment where compaction work is being performed, and slopes where there is a risk of landslides and collapse. It can be used to estimate the state change of various objects, such as estimating the state in which a harmful change has occurred. For the sake of convenience, here, an example of estimating a state in which concrete is sufficiently compacted (hereinafter referred to as "compacting completed state") will be described for fresh concrete in which compaction work is being performed.

Further, when the object state estimation system and the object state estimation method of the present invention are used for estimating the compaction completion state of concrete, in addition to cast-in-place concrete work, precast concrete production in factories, etc., and construction sites It can be carried out by manufacturing precast concrete (so-called site PC) in.

1. 1. Overall Overview The present invention is, for example, a technique for estimating the compaction completion state based on an image of concrete in which the compaction work is continuously performed. More specifically, the concrete that changes due to compaction is photographed in sequence, and for each of the multiple images obtained as a result, it is tentatively determined whether or not the compaction is completed, and the multiple images for which the determination is made. Estimate the time when the compaction is completed comprehensively from. Therefore, it is better that the time interval for acquiring the image of the object (concrete) (hereinafter referred to as "shooting time interval") is relatively short so as not to miss the target change state (compacting completed state). As shown in FIG. 1, it is advisable to shoot an object as a moving image or a continuous still image. Both moving images and continuous still images are composed of images (frames) acquired at extremely short intervals, but generally, images of 24 to 60 images per second (that is, 24 to 60 fps) are regarded as moving images. Therefore, here, a continuous still image is defined as an image composed of an image of less than 24 fps or more than 60 fps.

2. 2. Object state estimation system The object state estimation system of the present invention will be described in detail. The object state estimation method of the present invention is a method of estimating the change state of the object by using the object state estimation system of the present invention. Therefore, first, the object state estimation system of the present invention will be described, and then the object state estimation method of the present invention will be described.

FIG. 2 is a block diagram showing a main configuration of the object state estimation system 100 of the present invention. As shown in this figure, the object state estimation system 100 of the present invention includes an image classification means 110, a state value calculation means 120, and a change time extraction means 130, and further includes display specification setting means 140, a display, and the like. It can also be configured to include a display means 150, an image storage means 160, and a trained model storage means 170.

Among the main elements constituting the object state estimation system 100, the image classification means 110, the state value calculation means 120, the change time extraction means 130, and the display specification setting means 140 can be manufactured as dedicated ones. A general-purpose computer device can also be used. This computer device is equipped with a processor such as a CPU, a memory such as a ROM and a RAM, an input means such as a mouse and a keyboard, and a display. A personal computer (PC), a server, a tablet PC such as an iPad (registered trademark), and a smartphone. It can be configured by a mobile terminal or the like including. When a computer device provided with a display is used, the display may be used as the display means 150.

Further, the image storage means 160 and the trained model storage means 170 can use a storage device of a general-purpose computer or can be built on a database server. When building on a database server, it can be placed on a local network (LAN: Local Area Network), or it can be a cloud server that stores via the Internet (that is, wireless communication).

Next, the main processing flow when the object state estimation system 100 of the present invention is used with reference to FIG. 3 will be described. FIG. 3 is a flow chart showing the main processing flow of the object state estimation system 100. The central column shows the processing to be performed, the left column shows the input information required for the processing, and the right column shows the input information required for the processing. The output information generated from that process is shown.

First, the image classification means 110 (FIG. 2) reads an image from the image storage means 160 (FIG. 2), and determines whether or not the state of the concrete contained in the image is the "compacting completed state" (FIG. 3). Step 110). In the image storage means 160, a plurality of images (that is, moving images and continuous still images) obtained by shooting concrete that changes due to compaction at predetermined shooting time intervals are associated (linked) with the shooting time. That is, these images are given information called the shooting time. Then, the image classification means 110 determines whether or not the compaction is completed for each of the read images. However, the judgment here is only a tentative judgment if it is to be made on an image-by-image basis, and is not a final judgment on the state of concrete. For convenience, here, the image determined to have a change in the object, that is, the image determined to be in the compaction completed state is referred to as the "post-change image", and the image not determined to be in the compaction completed state is referred to as "the image in the compaction completed state". It is called "image before change". In other words, the image classification means 110 classifies each image read from the image storage means 160 into a post-change image or a pre-change image.

When the image classification means 110 classifies images, the "trained model" stored in the trained model storage means 170 (FIG. 2) is used. This trained model was constructed by machine learning, and more specifically, by learning the "image as a post-change image" and the "image as a pre-change image" as teacher data, the post-change image can be obtained. It is a model that extracts the feature amount and outputs whether or not it corresponds to the changed image when an arbitrary image is input. Machine learning for constructing a trained model includes deep learning including CNN (Convolutional Neural Network), which is often used for image recognition, and NIN (Network In Network), which is said to recognize images with higher accuracy than CNN. In addition to (deep learning), various conventionally used machine learning techniques can be adopted.

When all the images read by the image classification means 110 are classified into a post-change image or a pre-change image, the state value calculation means 120 (FIG. 2) calculates a "state value" indicating the state of concrete (FIG. 3). Step120). This state value is a value determined according to the number and ratio of the changed images, and is a state value (hereinafter referred to as "first state value") obtained by the cumulative number of changed images, or a fixed number. A state value (hereinafter referred to as "second state value") obtained by the ratio of changed images in an image set (hereinafter referred to as "image group"), and an image set included in a moving area (hereinafter referred to as "image"). It can be roughly classified into a state value (hereinafter, referred to as a "third state value") obtained by the ratio of the changed image in the "set"). Hereinafter, the procedure for obtaining the first state value, the second state value, and the third state value will be described in detail.

FIG. 4 is a model diagram for explaining a procedure for calculating the first state value. In this figure, each image is represented by a line segment (vertical line), and an image in which shooting is started (FIG. 4). Then, the images are arranged in order of shooting time (that is, in order of passage of time) from the leftmost image). Further, the pre-change image and the post-change image are distinguished by the shade of the line segment, that is, the pre-change image is shown by a light (light) line segment, and the post-change image is shown by a dark line segment.

As shown in FIG. 4, as time elapses, that is, as the compaction time becomes longer, the number classified into the changed image increases. When the changed image first appears, the appearance frequency of the changed image is low, and the appearance frequency of the changed image is high in the latter half (right side of the figure). Therefore, it is not possible to eliminate the contingency of the feature amount in the image to determine the "compacting completed state" when the image after the change first appears as in the prior art, and the reliability of the judgment is high. It can be said that it is low. On the other hand, as the number of appearances of the image after the change increases, the frequency of appearance also increases, and it can be fully estimated that the "compacting completed state" has been reached. That is, the number of appearances of the changed image can be considered as an index showing the compacted state of concrete. Therefore, the "first state value" is determined as the cumulative number of changed images counted in order of shooting time from the start of shooting.

FIG. 5 is a model diagram for explaining the procedure for calculating the second state value. Similar to FIG. 4, this figure also captures a pre-change image shown by a light line segment and a post-change image shown by a dark line segment. They are arranged in chronological order. In calculating the second state value, first, the images arranged in order of shooting time are divided into a plurality of image groups. For example, in the case of FIG. 5, the first image group to the fifth image group are divided into five image groups. The second state value is obtained as the number of images constituting the image group, that is, the ratio (percentage) of the number of changed images to the total number of pre-change images and post-change images. For example, the second state value in the image group containing 21 pre-change images and 9 post-change images is 30%, and the second in the image group containing 3 pre-change images and 27 post-change images. The state value of 2 is 90%. In this way, the second state value is calculated for each image group.

The image group can be divided according to a predetermined number of images, and the image group can be set so as to include, for example, 30 images. Of course, it is not limited to the case of dividing in units of 30 sheets, but it can be divided in units of 15 sheets or 60 sheets, which is appropriately set according to the state of concrete and the like. Further, considering that images are acquired at predetermined shooting time intervals, an image group can be said to be a set (group) of images divided at regular time intervals. For example, when an image is acquired at 30 fps, if an image group is divided into 30 images, the divided group can be considered to have a length of 1 second. That is, the image group is, so to speak, a time width for determining the state of concrete, and the second state value is an index indicating the state of concrete determined for each time width. Conventionally, since the operator determines the degree of compaction in units of 1 second to several seconds, it is preferable to divide the image group so that the time width is 1 second to several seconds.

FIG. 6 is a model diagram for explaining a procedure for calculating a third state value. Similar to FIG. 4, this figure also captures a pre-change image shown by a light line segment and a post-change image shown by a dark line segment. They are arranged in chronological order. In calculating the third state value, first, an image set is set for the images arranged in the order of shooting time. This image set is composed of a plurality of images (for example, 30 images) as in the image group. However, while an image group is a so-called static image set that does not move in the time direction, an image set is a so-called dynamic image set that moves in the time direction. Specifically, as shown in FIG. 6, an area set to include a predetermined number of images (for example, 30 images) and moving in a direction in which time elapses (hereinafter, “moving area””. The image set included in) is an image set. The unit amount for moving the moving area at one time can be arbitrarily designed, and may be set, for example, as a multiple of the shooting time interval (natural number multiple).

The third state value is obtained as the number of images included in the image set, that is, the ratio (percentage) of the number of changed images to the total number of pre-change images and post-change images. For example, the third state value in an image set containing 15 pre-change images and 15 post-change images is 50%, and the third in an image set containing 12 pre-change images and 18 post-change images. The state value of 3 is 60%. The third state value is calculated each time the moving region forming the image set moves.

When the state value calculating means 120 calculates the state value, the display specification setting means 140 (FIG. 2) sets the "display specifications" according to the state value (Step 130 in FIG. 3), and the display means 150 (FIG. 2). 2) displays the image according to the display specifications (Step 140 in FIG. 3). Here, the display specifications are the transparency of the image, the display color for coloring the image (hereinafter referred to as "image display color"), and the frame line representing the periphery of the image (hereinafter referred to as "image frame". ) Is a requirement for displaying an image, such as the line color and line type. In addition, the display specifications have a role of indicating the state of concrete, and therefore the value changes according to the state value. For example, the specification may be such that the transparency becomes higher (or lower) as the state value indicates a larger value, or the image display color is changed (for example, red → green) when the state value exceeds a predetermined value. , Change the display color of the image frame when the state value exceeds a predetermined value (for example, white → black), or combine multiple requirements (transparency and image display color, etc.) according to the state value. It can be designed with various specifications, such as specifications that can be changed.

By the way, the object state estimation system 100 of the present invention may be specified to process one image as one area, or a plurality of small areas obtained by dividing one image (hereinafter referred to as "divided area SP"). It can also be a specification to process as if it is composed of). When the object state estimation system 100 processes as an image composed of a plurality of divided regions SP, the divided region SP is treated as an individual image. That is, the image classification means 110 (FIG. 2) classifies the image before change or the image after change for each division area SP (Step 110 in FIG. 3), and the state value calculation means 120 (FIG. 2) has a state value for each division area SP. (Step 120 in FIG. 3), and the display specification setting means 140 (FIG. 2) sets the display specifications for each division area SP (Step 130 in FIG. 3). Then, as shown in FIG. 7B, the display means 150 displays an image composed of a plurality of (6 × 8 images in the figure) divided regions SP. Note that FIG. 7A shows an image displayed when the object state estimation system 100 processes one image as one area.

FIG. 8 is a diagram showing an example in which the display specification setting means 140 sets “transparency” as the display specification for each divided area SP, and the display means 150 displays the divided area SP according to the transparency. (A) represents the state of concrete at the initial stage of compaction, and (b) represents the state of concrete in which compaction has progressed to some extent. In the case of this figure, the specification is such that the larger the state value is, the higher the transmittance is. In FIG. 8A, since it is in the initial stage of compaction, the divided region is colored black (that is, the transmittance is low). Many SPs are seen. On the other hand, in FIG. 8B, since the compaction has progressed to some extent, most of the divided regions SP except the four corners are in a transparent state, that is, it can be seen that the compaction has progressed as a whole. ..

FIG. 9 shows an example in which the display specification setting means 140 sets the “display color of the image frame” as the display specification for each divided area SP, and the display means 150 displays the divided area SP according to the display color of the image frame. (A) shows the state of concrete at the initial stage of compaction, and (b) shows the state of concrete in which compaction has progressed to some extent. In the case of this figure, the specification is to change the display color of the image frame from white to black when the state value exceeds a predetermined value, and in FIG. 9A, the image frame is white because it is in the initial stage of compaction. There are many divided areas SP. On the other hand, in FIG. 9B, since compaction has progressed to some extent, most of the divided region SPs except the four corners are black image frames, that is, it can be seen that compaction has progressed as a whole. ..

It should be noted that the image classification means 110 classifies the images (or the divided area SP) at the timing when the acquired images increase by a predetermined number or at the timing when a certain time elapses (that is, periodically), and the state. The value calculating means 120 can calculate the state value for each image (or the divided area SP), and the display specification setting means 140 can set the display specifications for each image (or the divided area SP). Further, the display means 150 can also change (switch) the image to be displayed at the timing when the display specifications are set by the display specification setting means 140 or periodically.

When the state value calculating means 120 calculates the state value, the change time extracting means 130 (FIG. 2) compares the state value with a predetermined threshold value (hereinafter referred to as “state threshold value”) (Step 150 in FIG. 3). Then, when the state value falls below the state threshold value (No in Step 150), the condition is changed to obtain the state value (Step 120 in FIG. 3). For example, in the case of the first state value shown in FIG. 4, the cumulative number of changed images is recorded by advancing the time, and in the case of the second state value shown in FIG. 5, the image group is changed (the first). 3 Image group → 4th image group) Obtain the ratio of the number of changed images, and in the case of the third state value shown in FIG. 6, change the image set (move the moving area) to determine the number of changed images. Find the ratio.

On the other hand, when the state value exceeds the state threshold value (Yes in Step 150), an image that can be presumed to be in the “compacting completed state” or the divided region SP (hereinafter referred to as “attention image”) is selected (Step 160). This attention image may be selected according to the type of the desired state value. For example, when obtaining the first state value, the image when the cumulative number of changed images exceeds the state threshold value for the first time can be selected as the image of interest. In the case of FIG. 4, the state threshold value is set to 40 images, and the image when the first state value is 40 images is selected as the image of interest. Of course, the state threshold value is not limited to 40 sheets, and can be appropriately set according to the state of concrete such as 90 sheets or 120 sheets. Further, the object state estimation system 100 of the present invention may be provided with the "state threshold value input means", and the specifications may be such that the operator inputs a desired state threshold value.

When obtaining the second state value, it is advisable to first select an image group (hereinafter, referred to as "attention image group") including the image of interest. This attention image group can be selected as the one related to the earliest shooting time among the image groups whose second state value exceeds the state threshold value. In the case of FIG. 5, the state threshold value is set to 90%, and the fifth image group when the second state value becomes 90% is selected as the attention image group. Of course, the state threshold value is not limited to 90%, but can be appropriately set to 60%, 80%, etc. according to the state of concrete and the like. When the state threshold value is 60% in FIG. 5, the state value (60%) of the 4th image group and the state value (90%) of the 5th image group are each equal to or higher than the state threshold value, but the 5th image. Since the shooting time of the 4th image group is earlier than that of the group, the 4th image group is selected as the image group of interest.

When the attention image group is selected, the attention image is selected from the images (or the divided area SP) included in the attention image group. At this time, it is advisable to determine in advance the order number (hereinafter referred to as “shooting order”) when arranging the images in the attention image group from the earliest shooting time, and select the attention images based on this shooting order. For example, in the case shown in FIG. 10, an image group is composed of 20 images, and the image having the twelfth shooting order (third from the last) is selected as the image of interest. Of course, the shooting order selected as the attention image is not limited to the third from the last, and it may be the last image in the attention image group, the middle image in the attention image group, or set appropriately according to the state of concrete. can do.

When obtaining the third state value, as in the case of the second state value, it is advisable to first select an image set (hereinafter, referred to as “attention image set”) including the image of interest. This attention image set can be selected as the image set having the earliest shooting time among the image sets in which the third state value exceeds the state threshold value. Then, when the attention image set is selected, the attention image is selected from the images (or the divided region SP) included in the attention image set. At this time, as in the case of the second state value, the attention image may be selected based on a predetermined shooting order in the attention image set.

When the image of interest is selected, the change time extraction means 130 (FIG. 2) extracts the shooting time (hereinafter referred to as “change time”) related to the image of interest (Step 170 in FIG. 3). As described above, the image storage means 160 stores an image together with the shooting time. Therefore, for example, by inquiring the image storage means 160 with the identifier of the attention image, the shooting time (that is, the change time) of the attention image can be obtained. Then, at this change time, it is estimated that the concrete, which is the object, has reached the "compacting completed state".

3. 3. Object state estimation method Next, the object state estimation method of the present invention will be described with reference to FIG. The object state estimation method of the present invention is a method of estimating the change state of the object by using the object state estimation system 100 described so far, and therefore, the content described in the object state estimation system 100. Overlapping explanations will be avoided, and only the contents peculiar to the object state estimation method of the present invention will be explained. That is, the contents not described here are the same as those described in "2. Object state estimation system".

FIG. 11 is a flow chart showing the main steps of the object state estimation method of the present invention. As shown in this figure, first, a moving image or a continuous still image of fresh concrete being compacted is taken (Step 10 in FIG. 11). When the image of the concrete is acquired, the image classification means 110 of the object state estimation system 100 of the present invention is used to classify each image (or the divided region SP) into a pre-change image or a post-change image (Step 20 in FIG. 11). , The state value is calculated for each image (or the divided region SP) by using the state value calculating means 120 of the object state estimation system 100 (Step 30 in FIG. 11). Then, using the change time extraction means 130 of the object state estimation system 100, the image of interest is selected and the change time is extracted (Step 40 in FIG. 11), and the concrete that is the object at this change time is in the “compacting completed state”. It is estimated that it became.

The object state estimation system and the object state estimation method of the present invention determine the state change of various objects whose state changes, such as concrete compaction judgment, concrete hardening judgment, and embankment compaction judgment. Can be used when Considering that the invention of the present application provides, for example, an appropriately compacted high-quality concrete structure, it can be said that the invention is not only industrially applicable but also can be expected to make a great contribution to society.

100 Object state estimation system of the present invention 110 Image classification means (of object state estimation system) 120 State value calculation means 130 (object state estimation system) Change time extraction means 140 (object) Display specification setting means (of object state estimation system) 150 Display means (of object state estimation system) 160 Image storage means (of object state estimation system) 170 (Learned model storage means of object state estimation system) SP division region

Claims (7)

A system that estimates changes in the state of an object based on a plurality of images of the object taken at predetermined shooting time intervals.
By performing image recognition using a trained model constructed by machine learning, it is determined whether or not there is a change in the object for each image, and after the image determined to have a change in the object is changed. Image classification means for classifying images and
A state value calculation means for calculating a state value which is a cumulative number of the changed images counted in order of shooting time from the start of shooting of the object.
A change time extraction means for extracting the shooting time of the image when the state value exceeds a predetermined threshold value as the change time of the object is provided.
It is presumed that the state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system characterized by this. A system that estimates changes in the state of an object based on a plurality of images of the object taken at predetermined shooting time intervals.
By performing image recognition using a trained model constructed by machine learning, it is determined whether or not there is a change in the object for each image, and after the image determined to have a change in the object is changed. Image classification means for classifying images and
A state value calculation means for dividing the images arranged in order of shooting time into a plurality of image groups and calculating a state value which is a ratio of the number of changed images in the image group.
The image group related to the earliest shooting time among the image groups whose state value exceeds a predetermined threshold is selected, and the shooting time related to the image having a predetermined shooting order among the selected image groups is selected. Is provided with a change time extraction means for extracting the change time of the object.
It is presumed that the state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system characterized by this. A system that estimates changes in the state of an object based on a plurality of images of the object taken at predetermined shooting time intervals.
By performing image recognition using a trained model constructed by machine learning, it is determined whether or not there is a change in the object for each image, and after the image determined to have a change in the object is changed. Image classification means for classifying images and
A state value calculation that calculates a state value that is a ratio of the number of changed images in the image set while moving the image set while setting a fixed number of image sets for the images arranged in order of shooting time. Means and
The image set related to the earliest shooting time of the image sets whose state value exceeds a predetermined threshold is selected, and the shooting time of the selected image set having a predetermined shooting order is related to the image. Is provided with a change time extraction means for extracting the change time of the object.
It is presumed that the state of the object has changed at the change time extracted by the change time extraction means.
An object state estimation system characterized by this. The object is fresh concrete that has been continuously compacted.
The image classification means classifies the image determined to be compacted with the fresh concrete into the post-change image.
The object state estimation system according to any one of claims 1 to 3, wherein the object state estimation system is characterized in that. A display means for displaying the divided image obtained by dividing the image into a plurality of parts, and
Further provided with a display specification setting means for setting display specifications according to the state value for each divided image.
The image classification means determines whether or not there is a change in the object for each divided image, and classifies the divided image determined to have a change in the object into the post-change image.
The state value calculating means calculates the state value for each of the divided images, and obtains the state value.
The display means displays the divided image based on the display specifications set by the display specification setting means.
The object state estimation system according to any one of claims 1 to 4, wherein the object state estimation system is characterized in that. The display specification setting means sets the transparency, the display color, or the border color as the display specification based on the state value of the divided image.
The object state estimation system according to claim 5. A method of estimating a change in the state of an object by using the object state estimation system according to any one of claims 1 to 6.
An image acquisition step of photographing the object at the photographing time interval and acquiring a plurality of the images, and an image acquisition step.
An image classification step of classifying the image determined to have a change in the object into the post-change image by the image classification means.
The state value calculation step of calculating the state value by the state value calculation means, and
The change time extraction means comprises a change time extraction step of extracting the change time of the object.
It is estimated that the state of the object has changed at the change time extracted by the change time extraction step.
An object state estimation method characterized by this.

Images