CN115984745A

CN115984745A - Moisture control method for black garlic fermentation

Info

Publication number: CN115984745A
Application number: CN202211724315.1A
Authority: CN
Inventors: 张黎明; 张明永
Original assignee: Jiangsu Bio Tech Ltd Fu Duomei
Current assignee: Jiangsu Bio Tech Ltd Fu Duomei
Priority date: 2022-12-30
Filing date: 2022-12-30
Publication date: 2023-04-18

Abstract

The application relates to the field of intelligent control, and particularly discloses a moisture control method for black garlic fermentation, which extracts state change characteristic information focused on garlic bulbs in a monitoring video of fermentation liquor by adopting an artificial intelligent control technology based on deep learning, extracts multi-scale dynamic characteristics of the fermentation environment humidity value, and further performs real-time control on the fermentation environment humidity value at the current time point by means of relevance characteristic representation of the fermentation environment humidity value and the garlic bulb. Therefore, the self-adaptive control of the water content in the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic bulbs is ensured, and the quality of the black garlic after fermentation is further ensured.

Description

Moisture control method for black garlic fermentation

Technical Field

The present application relates to the field of intelligent control, and more particularly, to a moisture control method for black garlic fermentation.

Background

The black garlic is a food prepared by fermenting fresh raw garlic bulbs with skins in a fermentation box for 90-120 days, can retain the original components of the raw garlic, and has higher content of trace elements, sour and sweet taste and no garlic flavor.

The black garlic is heated to lose part of water in the fermentation process, and then the garlic can be fermented into the black garlic in a constant-humidity constant-temperature fermentation environment for a long time. In order to ensure the humidity required by the fermentation of the garlic in the fermentation process, the garlic needs to be humidified occasionally in the fermentation process. For example, chinese patent application CN207707244U discloses a black garlic fermentation humidifying device, which is provided with a black garlic fermentation machine humidifying base, a black garlic fermentation heating plate and a water adding drawer at the bottom of a fermentation machine, so that the black garlic fermentation humidifying device can achieve a better humidifying effect during fermentation.

However, during the operation of the humidifying device, it is found that the humidity change in the fermentation environment is dynamic due to the different amounts and activities of the fermentation tubes at different stages, and thus, the condition of over-heavy humidity or over-light humidity often occurs in the process of creating the constant-humidity fermentation environment by using the humidifying device.

Therefore, an optimized moisture control scheme for fermentation of black garlic is desired.

Disclosure of Invention

The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides a moisture control method for black garlic fermentation, which extracts state change characteristic information focusing on garlic bulbs in a monitoring video of fermentation liquor by adopting an artificial intelligence control technology based on deep learning, also extracts multi-scale dynamic characteristics of the fermentation environment humidity value, and further performs real-time control on the fermentation environment humidity value at the current time point by means of relevance characteristic representation of the garlic bulbs and the multi-scale dynamic characteristics. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

According to an aspect of the present application, there is provided a moisture control method for fermentation of black garlic, which includes: acquiring a fermentation liquid monitoring video acquired by a camera within a preset time period and humidity values of a fermentation environment at a plurality of preset time points within the preset time period; extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; arranging the plurality of fermentation liquor monitoring key frames into a three-dimensional input tensor, and then obtaining a fermentation liquor state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; passing the fermentation liquor state change characteristic diagram through a spatial attention module to obtain a spatial enhancement fermentation liquor state change characteristic diagram; performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space-enhanced fermentation broth to obtain a state change feature vector of the space-enhanced fermentation broth; arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension, and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module; performing correlation coding on the space-enhanced fermentation liquor state change eigenvector and the humidity eigenvector to obtain a classification characteristic matrix; performing characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and passing the corrected classification characteristic matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

In the above method for controlling moisture in black garlic fermentation, the extracting a plurality of fermentation liquid monitoring key frames from the fermentation liquid monitoring video includes: and sampling the fermentation liquid monitoring video at a preset sampling frequency to obtain a plurality of fermentation liquid monitoring key frames.

In the above method for controlling moisture in black garlic fermentation, after arranging the plurality of fermentation liquid monitoring key frames into a three-dimensional input tensor, obtaining a fermentation liquid state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel, the method includes: performing, using layers of the first convolutional neural network model using a three-dimensional convolution kernel, in forward pass of layers, input data separately: carrying out convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a local feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolution neural network using the three-dimensional convolution kernel is the fermentation liquid state change characteristic diagram, and the input of the first layer of the first convolution neural network using the three-dimensional convolution kernel is the three-dimensional characteristic tensor.

In the above method for controlling moisture in black garlic fermentation, the step of passing the characteristic diagram of state change of fermentation broth through a spatial attention module to obtain a characteristic diagram of state change of spatially enhanced fermentation broth includes: calculating the global mean value of each characteristic matrix of the fermentation liquid state change characteristic diagram along the space dimension to obtain a space characteristic vector; inputting the spatial feature vector into a Softmax activation function to obtain a spatial attention weight feature vector; and respectively weighting each characteristic matrix of the fermentation liquid state change characteristic diagram along the spatial dimension by taking the characteristic value of each position in the spatial attention weight characteristic vector as a weight so as to obtain the spatial enhancement fermentation liquid state change characteristic diagram.

In the above moisture control method for black garlic fermentation, the multi-scale neighborhood feature extraction module includes: the convolutional encoder comprises a first convolutional layer, a second convolutional layer which is parallel to the first convolutional layer, and a cascade layer which is connected with the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale.

In the above moisture control method for black garlic fermentation, the step of arranging the humidity values of a plurality of predetermined time points in the predetermined time into a humidity input vector according to a time dimension and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module includes: inputting the humidity input vector into a first convolution layer of the multi-scale neighborhood region feature extraction module to obtain a first neighborhood region scale humidity feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; inputting the humidity input vector into a second convolution layer of the multi-scale neighborhood region feature extraction module to obtain a second neighborhood dimension humidity feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the first neighborhood scale humidity characteristic vector and the second neighborhood scale humidity characteristic vector to obtain the humidity characteristic vector.

In the above method for controlling moisture in black garlic fermentation, the performing associated encoding on the state change eigenvector of the spatially enhanced fermentation broth and the humidity eigenvector to obtain a classification characteristic matrix includes: performing correlation coding on the state change characteristic vector of the spatially enhanced fermentation broth and the humidity characteristic vector by using the following formula to obtain a classification characteristic matrix; wherein the formula is:

wherein V _c Representing the characteristic vector of the state change of the space-enhanced fermentation liquid, V representing the humidity characteristic vector, and M representing the classification characteristic matrix.

In the above method for controlling moisture in black garlic fermentation, the performing feature distribution correction on the classification feature matrix to obtain a corrected classification feature matrix includes: performing characteristic distribution correction on the classification characteristic matrix according to the following formula to obtain the corrected classification characteristic matrix;

wherein the formula is:

wherein M is _c And M is the classification featureA matrix and said corrected classification feature matrix, reLU (-) representing a ReLU activation function,

representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix.

In the above moisture control method for black garlic fermentation, the step of passing the classification feature matrix through a classifier to obtain a classification result, where the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased, includes: processing the corrected classification feature matrix using the classifier with the following formula to obtain a classification result, wherein the formula is:

O＝softmax{(W _n ,B _n ):…:(W ₁ ,B ₁ ) L Project (F), where Project (F) represents the projection of the corrected sorted feature matrix as a vector, W ₁ To W _n As a weight matrix for each fully connected layer, B ₁ To B _n Representing the bias vectors of the fully connected layers of each layer.

According to another aspect of the present application, there is provided a moisture control system for fermentation of black garlic, comprising: the information acquisition module is used for acquiring a fermentation liquid monitoring video acquired by a camera within a preset time period and humidity values of a fermentation environment at a plurality of preset time points within the preset time period; the key frame extraction module is used for extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; the convolution module is used for arranging the fermentation liquid monitoring key frames into a three-dimensional input tensor and then obtaining a fermentation liquid state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; the spatial attention module is used for enabling the fermentation liquor state change characteristic diagram to pass through the spatial attention module so as to obtain a spatial enhancement fermentation liquor state change characteristic diagram; the pooling module is used for performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space enhanced fermentation broth to obtain a state change feature vector of the space enhanced fermentation broth; the multi-scale feature extraction module is used for arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension and then obtaining a humidity feature vector through the multi-scale neighborhood feature extraction module; the correlation coding module is used for performing correlation coding on the state change characteristic vector of the space enhanced fermentation liquor and the humidity characteristic vector to obtain a classification characteristic matrix; the correction module is used for carrying out characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and the classification result generation module is used for enabling the corrected classification characteristic matrix to pass through a classifier to obtain a classification result, and the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory in which computer program instructions are stored, which, when executed by the processor, cause the processor to perform the moisture control method for black garlic fermentation as described above.

According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to execute the moisture control method for black garlic fermentation as described above.

Compared with the prior art, the moisture control method for black garlic fermentation provided by the application extracts the state change characteristic information focused on garlic heads in the monitoring video of the fermentation liquid by adopting the artificial intelligence control technology based on deep learning, also extracts the multi-scale dynamic characteristic of the fermentation environment humidity value, and further performs real-time control on the fermentation environment humidity value at the current time point by using the relevance characteristic representation of the two. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally indicate like parts or steps.

Fig. 1 is an application scenario diagram of a moisture control method for black garlic fermentation according to an embodiment of the present application;

FIG. 2 is a flow chart of a moisture control method for fermentation of black garlic according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a water control method for fermenting black garlic according to an embodiment of the present disclosure;

FIG. 4 is a flowchart of a first convolutional neural network coding process in a moisture control method for black garlic fermentation according to an embodiment of the present application;

FIG. 5 is a flow chart illustrating a spatial enhancement process in a moisture control method for fermentation of black garlic according to an embodiment of the present application;

FIG. 6 is a flowchart of a multi-scale neighborhood feature extraction process in a moisture control method for black garlic fermentation according to an embodiment of the present application;

FIG. 7 is a block diagram of a moisture control system for fermentation of black garlic according to an embodiment of the present application;

fig. 8 is a block diagram of an electronic device according to an embodiment of the application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.

Overview of a scene

As mentioned in the background, since it is found during the operation of the actual humidifying device that the amount and activity of the fermentation tubes are different in different stages, the humidity change in the fermentation environment is dynamic, which causes the situation that the humidity is too heavy or too light in the process of creating the constant-humidity fermentation environment by using the humidifying device. Therefore, an optimized moisture control scheme for fermentation of black garlic is desired.

At present, deep learning and neural networks have been widely applied in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks also exhibit a level close to or even exceeding that of humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.

In recent years, deep learning and development of neural networks provide new solutions and schemes for controlling the moisture of black garlic fermentation.

Accordingly, since the moisture control is to control the moisture in the fermentation environment, i.e. the humidity control of the fermentation environment, it is considered that when the humidity control of the black garlic fermentation environment is actually performed, the humidity value of the fermentation environment should be adaptively controlled according to the state change of the black garlic fermentation liquid. That is, since the fermentation state of the fermentation liquid is different due to the difference in the amount and activity of the fermentation tubes at different stages, it is necessary to extract the correlation characteristic distribution information between the state change characteristic of the fermentation liquid and the humidity dynamic characteristic of the fermentation environment to perform the actual moisture control. Specifically, in the technical scheme of the application, an artificial intelligence control technology based on deep learning is adopted to extract the state change characteristic information focusing on garlic bulbs in the monitoring video of the fermentation liquid, and also extract the multi-scale dynamic characteristic of the fermentation environment humidity value, and further the relevance characteristics of the two are used for representing to perform real-time control on the fermentation environment humidity value at the current time point. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

Specifically, in the technical scheme of the application, firstly, a camera is used for collecting a fermentation liquor monitoring video in a preset time period, and a humidity sensor is used for acquiring humidity values of a fermentation environment at a plurality of preset time points in the preset time period. Next, considering that when the environmental humidity control in the black garlic fermentation process is actually performed, the state change characteristic of the fermentation liquid can be represented by the difference between adjacent monitoring frames in the monitoring video of the fermentation liquid, that is, the state change condition of the fermentation liquid is represented by the image representation of the adjacent image frames. However, considering that the difference between adjacent frames in the fermentation broth monitoring video is small and a large amount of data redundancy exists, in order to reduce the calculation amount and avoid adverse effects on detection caused by the data redundancy, key frame sampling is performed on the fermentation broth monitoring video at a preset sampling frequency, and then a plurality of fermentation broth monitoring key frames are extracted from the fermentation broth monitoring video.

Then, a convolutional neural network model with excellent performance in the aspect of image implicit feature extraction is used for feature mining of the fermentation liquor monitoring key frames, and particularly, in consideration of feature distribution of the fermentation liquor monitoring key frames with relevance in a time dimension, dynamic change feature distribution information of the fermentation liquor state in a time sequence needs to be mined in order to improve accuracy of real-time control of the fermentation environment humidity value at the current time point. Therefore, after the plurality of fermentation liquid monitoring key frames are further arranged into a three-dimensional input tensor, the state change characteristics of the fermentation liquid are extracted by using a first convolution neural network model of a three-dimensional convolution kernel, and thus a fermentation liquid state change characteristic diagram is obtained.

Furthermore, in order to ensure the humidity required by the fermentation of the garlic bulbs during the fermentation of the black garlic, the garlic bulbs need to be humidified at random during the fermentation process. Therefore, when controlling the humidity of fermentation, much attention needs to be paid to the state change characteristics spatially related to the garlic bulbs. Specifically, in the technical scheme of the application, the characteristic diagram of the state change of the fermentation broth is subjected to characteristic enhancement of spatial position in a spatial attention module to extract state change characteristic information focused on garlic bulbs in the fermentation broth, so that the characteristic diagram of the state change of the spatially enhanced fermentation broth is obtained. Here, the spatially enhanced fermentation broth state change characteristics extracted by the spatial attention reflect the weight of the characteristic difference of the fermentation broth state in spatial dimension, and are used for suppressing or enhancing the characteristics of different spatial positions to highlight the state change characteristics of garlic bulbs.

The humidity value of the fermentation environment has volatility and uncertainty in the time dimension, and thus, it shows different dynamic change characteristics under different time spans. Based on this, in the technical scheme of the application, the humidity values of a plurality of preset time points in the preset time period are arranged into a humidity input vector according to the time dimension, and then are encoded in the multi-scale neighborhood feature extraction module, so that the multi-scale neighborhood associated features of the humidity values under different time spans in the preset time period are extracted, and the humidity feature vector is obtained. Here, the multi-scale neighborhood feature extraction module includes: the convolutional encoder comprises a first convolutional layer, a second convolutional layer which is parallel to the first convolutional layer, and a cascade layer which is connected with the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale.

And then, carrying out correlation coding on the state change characteristic vector of the space-enhanced fermentation liquor and the humidity characteristic vector to express correlation characteristic information between state change characteristic information focused on garlic bulbs in the fermentation liquor and multi-scale dynamic characteristics of the fermentation environment humidity value, and carrying out classification processing in a classifier by taking the correlation characteristic information as a classification characteristic matrix so as to obtain a classification result which expresses that the humidity value of the fermentation environment at the current time point should be increased or decreased. Therefore, the real-time control of the black garlic fermentation moisture can be carried out based on the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic bulbs is ensured.

Particularly, in the technical solution of the present application, when the spatial enhancement fermentation liquid state change feature vector and the humidity feature vector are subjected to correlation coding to obtain the classification feature matrix, because the spatial enhancement fermentation liquid state change feature vector expresses the correlation of the image semantic features of the surveillance video in the channel dimension, and there is a difference in expression dimension with the time sequence multi-scale correlation of the humidity value expressed by the humidity feature vector, when the spatial enhancement fermentation liquid state change feature vector and the position-by-position correlation value of the humidity feature vector are calculated to obtain the classification feature matrix, a negative correlation value relative to the global feature distribution is introduced into the local feature distribution of the classification feature matrix, thereby affecting the classification accuracy of the classification feature matrix.

Therefore, the applicant of the present application modifies the classification feature matrix by using a full orthographic nonlinear re-weighting method, which is expressed as:

M _c and M is the classification characteristic matrix after and before correction, respectively, and the division between the numerator matrix and the denominator matrix is the division of the characteristic values of the matrix according to the position.

Here, the full forward projection nonlinear re-weighting guarantees full forward of projection by the ReLU function to avoid aggregating negatively correlated information, and simultaneously introduces a nonlinear re-weighting mechanism to aggregate the eigenvalue distribution of the classification feature matrix, so that the modified intrinsic structure of the classification feature matrix can penalize long-distance connection to strengthen local coupling. In this way, a synergistic effect of spatial feature transformation (feature transform) corresponding to full forward projection reweighting of the classification feature matrix in a high-dimensional feature space is achieved, so that negative correlation values in local feature distribution of the classification feature matrix relative to global feature distribution are eliminated, and the classification accuracy of the classification feature matrix is improved. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

Based on this, the present application proposes a moisture control method for black garlic fermentation, which comprises: acquiring a fermentation liquid monitoring video acquired by a camera within a preset time period and humidity values of a fermentation environment at a plurality of preset time points within the preset time period; extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; arranging the plurality of fermentation liquor monitoring key frames into a three-dimensional input tensor, and then obtaining a fermentation liquor state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; passing the fermentation liquor state change characteristic diagram through a spatial attention module to obtain a spatial enhanced fermentation liquor state change characteristic diagram; performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space-enhanced fermentation broth to obtain a state change feature vector of the space-enhanced fermentation broth; arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension, and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module; performing correlation coding on the space-enhanced fermentation liquor state change eigenvector and the humidity eigenvector to obtain a classification characteristic matrix; performing characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and passing the corrected classification feature matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

Fig. 1 is an application scenario diagram of a moisture control method for black garlic fermentation according to an embodiment of the present application. As shown in fig. 1, in this application scenario, a fermentation broth monitoring video is acquired by a camera (e.g., C as illustrated in fig. 1) for a predetermined period of time, and humidity values of the fermentation environment at a plurality of predetermined time points within the predetermined period of time are acquired by a humidity sensor (e.g., H as illustrated in fig. 1). Then, the above information is inputted into a server (e.g., S in fig. 1) deployed with a moisture control algorithm for black garlic fermentation, wherein the server can process the above inputted information with the moisture control algorithm for black garlic fermentation to generate a classification result indicating that the humidity value of the fermentation environment at the current time point should be increased or decreased.

Having described the general principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.

Exemplary method

Fig. 2 is a flowchart of a moisture control method for fermentation of black garlic according to an embodiment of the present application. As shown in fig. 2, the moisture control method for black garlic fermentation according to the embodiment of the present application includes the steps of: s110, acquiring a fermentation liquid monitoring video acquired by a camera within a preset time period and humidity values of a fermentation environment at a plurality of preset time points within the preset time period; s120, extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; s130, arranging the fermentation liquor monitoring key frames into a three-dimensional input tensor, and then obtaining a fermentation liquor state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; s140, passing the fermentation liquor state change characteristic diagram through a space attention module to obtain a space-enhanced fermentation liquor state change characteristic diagram; s150, performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space-enhanced fermentation broth to obtain a state change feature vector of the space-enhanced fermentation broth; s160, arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension, and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module; s170, performing correlation coding on the state change characteristic vector of the space-enhanced fermentation liquor and the humidity characteristic vector to obtain a classification characteristic matrix; s180, performing characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and S190, passing the corrected classification characteristic matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

Fig. 3 is a schematic structural diagram of a moisture control method for fermentation of black garlic according to an embodiment of the present disclosure. As shown in fig. 3, in the network structure, first, a monitoring video of a fermentation broth in a predetermined time period and humidity values of a fermentation environment at a plurality of predetermined time points in the predetermined time period are acquired; extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; arranging the plurality of fermentation liquor monitoring key frames into a three-dimensional input tensor, and then obtaining a fermentation liquor state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; then, the fermentation liquor state change characteristic diagram is processed by a space attention module to obtain a space enhanced fermentation liquor state change characteristic diagram; performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space-enhanced fermentation broth to obtain a state change feature vector of the space-enhanced fermentation broth; then, arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension, and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module; performing correlation coding on the state change characteristic vector of the space enhanced fermentation liquor and the humidity characteristic vector to obtain a classification characteristic matrix; performing characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and passing the corrected classification characteristic matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

Specifically, in step S110, a fermentation broth monitoring video of a predetermined time period collected by a camera and humidity values of the fermentation environment at a plurality of predetermined time points within the predetermined time period are acquired. It should be understood that, in the actual operation process of the humidifying device, the amount and activity of the fermentation tubes are different in different stages, and therefore, the humidity change in the fermentation environment is dynamic, which often causes the situation of over-heavy humidity or over-light humidity in the process of creating a constant-humidity fermentation environment by using the humidifying device. In the technical scheme of the application, the fermentation state of the fermentation liquid is different due to different amounts and activities of the fermentation bacteria at different stages, so that correlation characteristic distribution information between the state change characteristic of the fermentation liquid and the humidity dynamic characteristic of the fermentation environment needs to be mined to perform actual water control. More specifically, a fermentation broth monitoring video of a predetermined time period can be obtained through a camera, and humidity values of the fermentation environment at a plurality of predetermined time points in the predetermined time period can be obtained through a humidity sensor.

Specifically, in step S120, a plurality of fermentation broth monitoring key frames are extracted from the fermentation broth monitoring video. Considering that when the environmental humidity control in the black garlic fermentation process is actually performed, the state change characteristics of the fermentation liquid can be represented by the difference between adjacent monitoring frames in the monitoring video of the fermentation liquid, that is, the state change condition of the fermentation liquid is represented by the image representation of the adjacent image frames. However, considering that the difference between adjacent frames in the fermentation broth monitoring video is small and a large amount of data redundancy exists, in order to reduce the calculation amount and avoid adverse effects on detection caused by the data redundancy, key frame sampling is performed on the fermentation broth monitoring video at a preset sampling frequency, and then a plurality of fermentation broth monitoring key frames are extracted from the fermentation broth monitoring video.

Specifically, in step S130, after arranging the plurality of fermentation liquid monitoring key frames into a three-dimensional input tensor, a fermentation liquid state change feature map is obtained by using a first convolution neural network model of a three-dimensional convolution kernel. In the technical scheme of the application, a convolutional neural network model with excellent performance in the aspect of image implicit feature extraction is used for feature mining of the plurality of fermentation liquor monitoring key frames, and particularly, in consideration of feature distribution of the plurality of fermentation liquor monitoring key frames with relevance in a time dimension, dynamic change feature distribution information of the state of the fermentation liquor in a time sequence needs to be mined in order to improve the accuracy of real-time control of the humidity value of the fermentation environment at the current time point. Therefore, after the plurality of fermentation liquid monitoring key frames are further arranged into a three-dimensional input tensor, the state change characteristics of the fermentation liquid are extracted by using a first convolution neural network model of a three-dimensional convolution kernel, and therefore a fermentation liquid state change characteristic diagram is obtained. In a specific example, the first convolution network model is essentially a three-dimensional convolution neural network model, and except for the difference of the convolution kernels, the network structure of other parts is consistent with the structure of the one-dimensional convolution kernel neural network model. In one specific example, each layer of the first convolutional neural network performs, in a forward pass of the layer, input data separately: performing three-dimensional convolution coding on input data by using a convolution module through a three-dimensional convolution kernel to obtain a convolution characteristic diagram; pooling the convolution feature map by using a pooling module to obtain a pooled feature map; and using an activation module to carry out nonlinear activation on the characteristic values of all positions of the pooling characteristic diagram to obtain a baking state change characteristic diagram; the input of the first layer of the second convolutional neural network is a three-dimensional feature tensor obtained by arranging the plurality of differential feature maps, and the output of the last layer of the second convolutional neural network is the state change feature map of the fermentation liquor. Specifically, the input data of the first convolution layer of the first convolution neural network is the three-dimensional input tensor, and each layer of the first convolution neural network performs convolution processing based on a three-dimensional convolution kernel, mean pooling processing based on an eigen matrix, and nonlinear activation processing on the input data in forward transmission of the layer, so as to output the state change characteristic diagram of the fermentation liquid from the last layer of the first convolution neural network.

Fig. 4 is a flowchart of a first convolution neural network coding process in a moisture control method for black garlic fermentation according to an embodiment of the present application. As shown in fig. 4, in the first convolutional neural network coding process, the method includes: performing, using layers of the first convolutional neural network model using a three-dimensional convolution kernel, in forward pass of layers, input data separately: s210, performing convolution processing on input data to obtain a convolution characteristic diagram; s220, pooling the convolution characteristic diagram based on a local characteristic matrix to obtain a pooled characteristic diagram; and S230, carrying out nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolution neural network using the three-dimensional convolution kernel is the fermentation liquid state change characteristic diagram, and the input of the first layer of the first convolution neural network using the three-dimensional convolution kernel is the three-dimensional characteristic tensor.

Specifically, in step S140 and step S150, the fermentation broth state change feature map is processed by a spatial attention module to obtain a spatially enhanced fermentation broth state change feature map, and then global mean pooling is performed on each feature matrix along the channel dimension of the spatially enhanced fermentation broth state change feature map to obtain a spatially enhanced fermentation broth state change feature vector. In the process of fermenting the black garlic, in order to ensure the humidity required by the fermentation of the garlic, the garlic needs to be humidified at random in the fermentation process. Therefore, in controlling the humidity of fermentation, much attention needs to be paid to the state change characteristics spatially related to the garlic bulbs. Specifically, in the technical scheme of the application, the characteristic diagram of the state change of the fermentation broth is subjected to characteristic enhancement of spatial position in a spatial attention module to extract state change characteristic information focused on garlic bulbs in the fermentation broth, so that the characteristic diagram of the state change of the spatially enhanced fermentation broth is obtained. Here, the spatially enhanced fermentation broth state change characteristics extracted by the spatial attention reflect the weight of the characteristic difference of the fermentation broth state in spatial dimension, and are used for suppressing or enhancing the characteristics of different spatial positions to highlight the state change characteristics of garlic bulbs. And further performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the spatially enhanced fermentation broth to obtain a state change feature vector of the spatially enhanced fermentation broth.

Fig. 5 is a flowchart illustrating a spatial enhancement process in a moisture control method for fermenting black garlic according to an embodiment of the present application. As shown in fig. 5, the spatial enhancement process includes: s310, calculating the global mean value of each feature matrix of the fermentation liquor state change feature map along the space dimension to obtain a space feature vector; s320, inputting the space feature vector into a Softmax activation function to obtain a space attention weight feature vector; and S330, respectively weighting each feature matrix of the fermentation liquid state change feature map along the spatial dimension by taking the feature value of each position in the spatial attention weight feature vector as a weight so as to obtain the spatially enhanced fermentation liquid state change feature map.

Specifically, in step S160, the humidity values at a plurality of predetermined time points in the predetermined time are arranged as a humidity input vector according to a time dimension, and then pass through a multi-scale neighborhood feature extraction module to obtain a humidity feature vector. It will be appreciated that the humidity value for the fermentation environment is fluctuating and uncertain in the time dimension and, therefore, exhibits different dynamic characteristics over different time spans. Based on this, in the technical scheme of the application, the humidity values of a plurality of preset time points in the preset time period are arranged into a humidity input vector according to the time dimension, and then are encoded in the multi-scale neighborhood feature extraction module, so that the multi-scale neighborhood associated features of the humidity values under different time spans in the preset time period are extracted, and the humidity feature vector is obtained. Here, the network structure of the multi-scale neighborhood feature extraction module is a network structure including a plurality of one-dimensional convolution layers in parallel, and more specifically includes: the convolutional encoder comprises a first convolutional layer, a second convolutional layer which is parallel to the first convolutional layer, and a cascade layer which is connected with the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional kernel uses a one-dimensional convolutional kernel with a second scale.

Fig. 6 is a flowchart of a multi-scale neighborhood feature extraction process in a moisture control method for black garlic fermentation according to an embodiment of the present application. As shown in fig. 6, in the multi-scale neighborhood feature extraction process, the method includes: s410, inputting the humidity input vector into a first convolution layer of the multi-scale neighborhood region feature extraction module to obtain a first neighborhood region scale humidity feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; s420, inputting the humidity input vector into a second convolution layer of the multi-scale neighborhood characteristic extraction module to obtain a second neighborhood scale humidity characteristic vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and S430, cascading the first neighborhood scale humidity characteristic vector and the second neighborhood scale humidity characteristic vector to obtain the humidity characteristic vector. Wherein the inputting the humidity input vector into the first convolution layer of the multi-scale neighborhood region feature extraction module to obtain a first neighborhood region scale humidity feature vector comprises: performing one-dimensional convolution coding on the humidity input vector by using a first convolution layer of the multi-scale neighborhood characteristic extraction module according to the following formula to obtain a first neighborhood scale humidity characteristic vector; wherein the formula is:

wherein a is the width of the first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of the first convolution kernel, and X represents the humidity input vector; and inputting the humidity input vector into a second convolution layer of the multi-scale neighborhood feature extraction module to obtain a second neighborhood scale humidity feature vector, comprising: performing one-dimensional convolutional coding on the humidity input vector by using a second convolutional layer of the multi-scale neighborhood characteristic extraction module according to the following formula to obtain a second neighborhood scale humidity characteristic vector; wherein the formula is:

wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the humidity input vector. More specifically, the concatenating the first neighborhood scale humidity characteristic vector and the second neighborhood scale humidity characteristic vector to obtain the humidity characteristic vector includes: fusing the first neighborhood scale humidity eigenvector and the second neighborhood scale humidity eigenvector to obtain the humidity eigenvector by the following formula; wherein the formula is:

V _c ＝oncat[V ₁ ，V ₂ ]

wherein, V ₁ Representing the first neighborhood scale humidity feature vector, V ₂ Represents the second neighborhood scale humidity feature vector, concat [, ]]Representing a cascade function, V _c Representing the humidity feature vector.

Specifically, in step S170, the feature vector of the state change of the spatially enhanced fermentation broth and the humidity feature vector are subjected to correlation coding to obtain a classification feature matrix. Namely, the state change characteristic vector and the humidity characteristic vector of the spatially enhanced fermentation broth are subjected to correlation coding to express correlation characteristic information between state change characteristic information focusing on garlic bulbs in the fermentation broth and multi-scale dynamic characteristics of the fermentation environment humidity value so as to obtain a classification characteristic matrix. In a specific example of the present application, the spatial enhancement fermentation broth state change feature vector and the humidity feature vector are subjected to correlation coding by the following formula to obtain a classification feature matrix; wherein the formula is:

wherein V _c Representing the characteristic vector of the state change of the space enhancement fermentation liquor, V representing the characteristic vector of the humidity, and M representing the classification characteristic matrix.

Specifically, in step S180, the feature distribution of the classification feature matrix is corrected to obtain a corrected classification feature matrix. Particularly, in the technical solution of the present application, when the classification feature matrix is obtained by performing association coding on the spatial enhancement fermentation broth state change feature vector and the humidity feature vector, because the spatial enhancement fermentation broth state change feature vector expresses the association of the image semantic features of the surveillance video in the channel dimension, and there is a difference in expression dimension with the time sequence multi-scale association of the humidity value expressed by the humidity feature vector, when the location-by-location association values of the spatial enhancement fermentation broth state change feature vector and the humidity feature vector are calculated to obtain the classification feature matrix, a negative correlation value relative to global feature distribution is introduced into the local feature distribution of the classification feature matrix, thereby affecting the classification accuracy of the classification feature matrix. Therefore, the applicant of the present application modifies the classification feature matrix by using a full orthographic nonlinear re-weighting method, which is expressed as:

wherein M is _c And M is the classification feature matrix and the corrected classification feature matrix, respectively, reLU (-) represents a ReLU activation function,

representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix. Here, the full forward projection nonlinear re-weighting guarantees full forward of projection by the ReLU function to avoid aggregating negatively correlated information, and simultaneously introduces a nonlinear re-weighting mechanism to aggregate the eigenvalue distribution of the classification feature matrix, so that the modified intrinsic structure of the classification feature matrix can penalize long-distance connection to strengthen local coupling. In this way, the synergistic effect of spatial feature transformation (feature transform) corresponding to full orthographic projection reweighting of the classification feature matrix in the high-dimensional feature space is realized, so that the negative correlation value of the local feature distribution of the classification feature matrix relative to the global feature distribution is eliminated, and the classification accuracy of the classification feature matrix is improved. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

Specifically, in step S190, the corrected classification feature matrix is passed through a classifier to obtain a classification result, which indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased. In the technical scheme of the application, the corrected classification feature matrix is used as a classification feature matrix to be classified in a classifier, so that a classification result indicating that the humidity value of the fermentation environment at the current time point should be increased or decreased is obtained. Therefore, the real-time control of the black garlic fermentation moisture can be carried out based on the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic bulbs is ensured. In a specific example of the present application, the corrected classification feature matrix is processed by the classifier with the following formula to obtain a classification result, where the formula is:

O＝softmax{(W _n ,B _n ):…:(W ₁ ,B ₁ ) L Project (F), where Project (F) represents the projection of the corrected sorted feature matrix as a vector, W ₁ To W _n As a weight matrix for all connected layers of each layer, B ₁ To B _n Representing the bias vectors of the fully connected layers of each layer. In particular, the classifier includes a plurality of fully-connected layers and a Softmax layer cascaded with a last fully-connected layer of the plurality of fully-connected layers. Wherein, in the classification processing of the classifier, the corrected classification feature matrix is first projected as a vector, for example, in a specific example, the corrected classification feature matrix is expanded as a classification feature vector along a row vector or a column vector; then, carrying out multiple full-connection coding on the classification characteristic vector by using multiple full-connection layers of the classifier to obtain a coding classification characteristic vector; further, inputting the encoded classification feature vector into a Softmax layer of the classifier, namely, classifying the encoded classification feature vector by using the Softmax classification function to obtain a first probability value that the humidity value of the encoded classification feature vector belonging to the current time point should be increased and a second probability value that the humidity value of the encoded classification feature vector belonging to the current time point should be decreased; then, the label corresponding to the larger one of the first probability value and the second probability value is determined as the classification result, that is, if the first probability value is larger than the second probability value, the classification result is whenAnd if the second probability value is greater than the first probability value, the classification result is that the humidity value in the fermentation environment at the current time point is increased.

In summary, the moisture control method for black garlic fermentation according to the embodiment of the present application is elucidated, which extracts the state change characteristic information focusing on garlic in the monitoring video of the fermentation liquid by using the artificial intelligence control technology based on deep learning, and also extracts the multi-scale dynamic characteristic of the fermentation environment humidity value, and further performs real-time control of the fermentation environment humidity value at the current time point by using the relevance characteristic representation of the two. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

Exemplary System

Fig. 7 is a block diagram of a moisture control system for fermentation of black garlic according to an embodiment of the present application. As shown in fig. 7, a moisture control system 300 for fermentation of black garlic according to an embodiment of the present application includes: an information acquisition module 310; a key frame extraction module 320; a convolution module 330; a spatial attention module 340; a pooling module 350; a multi-scale feature extraction module 360; an association encoding module 370; a correction module 380; and a classification result generation module 390.

The information obtaining module 310 is configured to obtain a fermentation broth monitoring video acquired by a camera in a predetermined time period and humidity values of a fermentation environment at a plurality of predetermined time points in the predetermined time period; the key frame extracting module 320 is configured to extract a plurality of fermentation liquid monitoring key frames from the fermentation liquid monitoring video; the convolution module 330 is configured to arrange the plurality of fermentation liquid monitoring key frames into a three-dimensional input tensor, and then obtain a state change characteristic diagram of the fermentation liquid by using a first convolution neural network model of a three-dimensional convolution kernel; the spatial attention module 340 is configured to pass the fermentation broth state change feature map through a spatial attention module to obtain a spatially enhanced fermentation broth state change feature map; the pooling module 350 is configured to perform global mean pooling on each feature matrix along a channel dimension of the spatial enhanced fermentation broth state change feature map to obtain a spatial enhanced fermentation broth state change feature vector; the multi-scale feature extraction module 360 is configured to arrange humidity values of a plurality of predetermined time points in the predetermined time into a humidity input vector according to a time dimension, and then obtain a humidity feature vector through the multi-scale neighborhood feature extraction module; the correlation coding module 370 is configured to perform correlation coding on the state change eigenvector of the spatially enhanced fermentation broth and the humidity eigenvector to obtain a classification feature matrix; the correction module 380 is configured to perform feature distribution correction on the classification feature matrix to obtain a corrected classification feature matrix; and the classification result generating module 390 is configured to pass the corrected classification feature matrix through a classifier to obtain a classification result, where the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

In one example, in the moisture control system 300 for black garlic fermentation described above, the key frame extraction module 320 is further configured to: and sampling the fermentation liquid monitoring video at a preset sampling frequency to obtain a plurality of fermentation liquid monitoring key frames.

In one example, in the above moisture control system 300 for black garlic fermentation, the convolution module 330 is further configured to: performing, respectively, input data in forward pass of layers using layers of the first convolutional neural network model using a three-dimensional convolutional kernel: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a local feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolution neural network using the three-dimensional convolution kernel is the fermentation liquid state change characteristic diagram, and the input of the first layer of the first convolution neural network using the three-dimensional convolution kernel is the three-dimensional characteristic tensor.

In one example, in the moisture control system 300 for black garlic fermentation described above, the spatial attention module 340 is further configured to: calculating the global mean value of each characteristic matrix of the fermentation liquid state change characteristic diagram along the space dimension to obtain a space characteristic vector; inputting the spatial feature vector into a Softmax activation function to obtain a spatial attention weight feature vector; and respectively weighting each characteristic matrix of the fermentation liquid state change characteristic diagram along the spatial dimension by taking the characteristic value of each position in the spatial attention weight characteristic vector as a weight so as to obtain the spatial enhancement fermentation liquid state change characteristic diagram.

In one example, in the moisture control system 300 for black garlic fermentation described above, the multi-scale feature extraction module 360 is further configured to: inputting the humidity input vector into a first convolution layer of the multi-scale neighborhood region feature extraction module to obtain a first neighborhood region scale humidity feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; inputting the humidity input vector into a second convolution layer of the multi-scale neighborhood region feature extraction module to obtain a second neighborhood dimension humidity feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the first neighborhood scale humidity characteristic vector and the second neighborhood scale humidity characteristic vector to obtain the humidity characteristic vector. Wherein, the multi-scale neighborhood feature extraction module comprises: the convolutional encoder comprises a first convolutional layer, a second convolutional layer which is parallel to the first convolutional layer, and a cascade layer which is connected with the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional layer uses a one-dimensional convolutional kernel with a second scale.

In one example, in the above moisture control system 300 for black garlic fermentation, the association coding module 370 is further configured to: performing correlation coding on the state change characteristic vector of the spatially enhanced fermentation broth and the humidity characteristic vector by using the following formula to obtain a classification characteristic matrix;

wherein the formula is:

In one example, in the moisture control system 300 for fermenting black garlic, the correction module 380 is further configured to: performing characteristic distribution correction on the classification characteristic matrix according to the following formula to obtain the corrected classification characteristic matrix; wherein the formula is:

wherein, M _c And M is the classification feature matrix and the corrected classification feature matrix, respectively, reLU (-) represents a ReLU activation function,

In one example, in the moisture control system 300 for fermenting black garlic, the classification result generating module 390 is further configured to: processing the corrected classification feature matrix using the classifier with the following formula to obtain a classification result, wherein the formula is:

In summary, the moisture control system 300 for black garlic fermentation according to the embodiment of the present application is illustrated, which extracts the state change feature information focused on garlic in the monitoring video of the fermentation liquid by using the artificial intelligence control technology based on deep learning, and also extracts the multi-scale dynamic feature of the fermentation environment humidity value, and further performs real-time control of the fermentation environment humidity value at the current time point by using the relevance feature representation of the two. Therefore, the self-adaptive control of the water content of the black garlic fermentation can be accurately carried out in real time on the basis of the actual humidity condition of the fermentation environment, so that the fermentation effect of the garlic is ensured, and the quality of the black garlic after fermentation is further ensured.

As described above, the moisture control system for black garlic fermentation according to the embodiment of the present application may be implemented in various terminal devices. In one example, the moisture control system 300 for fermentation of black garlic according to the embodiment of the present application may be integrated into a terminal device as a software module and/or a hardware module. For example, the moisture control system 300 for black garlic fermentation may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the moisture control system 300 for black garlic fermentation can also be one of the hardware modules of the terminal device.

Alternatively, in another example, the moisture control system 300 for black garlic fermentation and the terminal device may be separate devices, and the moisture control system 300 for black garlic fermentation may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information according to the agreed data format.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 8.

FIG. 8 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.

As shown in fig. 8, the electronic device 10 includes one or more processors 11 and memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 11 to implement the functions of the moisture control method for black garlic fermentation of the various embodiments of the present application described above and/or other desired functions. Various contents such as a classification feature matrix may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

The input device 13 may include, for example, a keyboard, a mouse, and the like.

The output device 14 can output various information including the classification result to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 8, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions of the moisture control method for black garlic fermentation according to various embodiments of the present application described in the "exemplary methods" section above in this specification.

The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions of the moisture control method for black garlic fermentation according to various embodiments of the present application described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.

The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

It should also be noted that in the devices, apparatuses, and methods of the present application, each component or step can be decomposed and/or re-combined. These decompositions and/or recombinations are to be considered as equivalents of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A water control method for black garlic fermentation is characterized by comprising the following steps: acquiring a fermentation liquid monitoring video acquired by a camera within a preset time period and humidity values of a fermentation environment at a plurality of preset time points within the preset time period; extracting a plurality of fermentation liquor monitoring key frames from the fermentation liquor monitoring video; arranging the plurality of fermentation liquor monitoring key frames into a three-dimensional input tensor, and then obtaining a fermentation liquor state change characteristic diagram by using a first convolution neural network model of a three-dimensional convolution kernel; passing the fermentation liquor state change characteristic diagram through a spatial attention module to obtain a spatial enhancement fermentation liquor state change characteristic diagram; performing global mean pooling on each feature matrix along the channel dimension of the state change feature map of the space-enhanced fermentation broth to obtain a state change feature vector of the space-enhanced fermentation broth; arranging humidity values of a plurality of preset time points in the preset time into a humidity input vector according to a time dimension, and then obtaining a humidity characteristic vector through a multi-scale neighborhood characteristic extraction module; performing correlation coding on the space-enhanced fermentation liquor state change eigenvector and the humidity eigenvector to obtain a classification characteristic matrix; performing characteristic distribution correction on the classification characteristic matrix to obtain a corrected classification characteristic matrix; and passing the corrected classification characteristic matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased.

2. The moisture control method for black garlic fermentation according to claim 1, wherein the extracting a plurality of fermentation broth monitoring key frames from the fermentation broth monitoring video comprises: and sampling the fermentation liquid monitoring video at a preset sampling frequency to obtain a plurality of fermentation liquid monitoring key frames.

3. The method for controlling moisture in black garlic fermentation according to claim 2, wherein the step of obtaining the state change characteristic diagram of the fermentation liquid by using a first convolution neural network model of a three-dimensional convolution kernel after arranging the plurality of fermentation liquid monitoring key frames as a three-dimensional input tensor comprises: performing, using layers of the first convolutional neural network model using a three-dimensional convolution kernel, in forward pass of layers, input data separately: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a local feature matrix to obtain a pooled feature map; and carrying out nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolution neural network using the three-dimensional convolution kernel is the fermentation liquid state change characteristic diagram, and the input of the first layer of the first convolution neural network using the three-dimensional convolution kernel is the three-dimensional characteristic tensor.

4. The method for controlling moisture content in black garlic fermentation according to claim 3, wherein the step of passing the characteristic diagram of state change of fermentation broth through a spatial attention module to obtain a characteristic diagram of state change of spatially enhanced fermentation broth comprises: calculating the global mean value of each characteristic matrix of the fermentation liquid state change characteristic diagram along the space dimension to obtain a space characteristic vector; inputting the spatial feature vector into a Softmax activation function to obtain a spatial attention weight feature vector; and respectively weighting each characteristic matrix of the fermentation liquid state change characteristic diagram along the spatial dimension by taking the characteristic value of each position in the spatial attention weight characteristic vector as a weight so as to obtain the spatial enhancement fermentation liquid state change characteristic diagram.

5. The moisture control method for black garlic fermentation according to claim 4, wherein the multi-scale neighborhood feature extraction module comprises: the convolutional encoder comprises a first convolutional layer, a second convolutional layer which is parallel to the first convolutional layer, and a cascade layer which is connected with the first convolutional layer and the second convolutional layer, wherein the first convolutional layer uses a one-dimensional convolutional kernel with a first scale, and the second convolutional kernel uses a one-dimensional convolutional kernel with a second scale.

6. The moisture control method for black garlic fermentation according to claim 5, wherein the step of arranging the humidity values of the plurality of predetermined time points in the predetermined time into a humidity input vector according to a time dimension and then obtaining a humidity feature vector through a multi-scale neighborhood feature extraction module comprises: inputting the humidity input vector into a first convolution layer of the multi-scale neighborhood region feature extraction module to obtain a first neighborhood region scale humidity feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; inputting the humidity input vector into a second convolution layer of the multi-scale neighborhood feature extraction module to obtain a second neighborhood scale humidity feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel of a second length, and the first length is different from the second length; and cascading the first neighborhood scale humidity characteristic vector and the second neighborhood scale humidity characteristic vector to obtain the humidity characteristic vector.

7. The moisture control method for black garlic fermentation according to claim 6, wherein the performing the correlation coding on the state change eigenvector and the humidity eigenvector of the spatially enhanced fermentation broth to obtain a classification feature matrix comprises: performing correlation coding on the state change characteristic vector of the spatially enhanced fermentation broth and the humidity characteristic vector by using the following formula to obtain a classification characteristic matrix; wherein the formula is:

8. The method for controlling moisture in black garlic fermentation according to claim 7, wherein the performing feature distribution correction on the classification feature matrix to obtain a corrected classification feature matrix comprises: performing characteristic distribution correction on the classification characteristic matrix according to the following formula to obtain the corrected classification characteristic matrix; wherein the formula is:

9. The method for controlling moisture in black garlic fermentation according to claim 8, wherein the step of passing the classification feature matrix through a classifier to obtain a classification result, wherein the classification result indicates that the humidity value of the fermentation environment at the current time point should be increased or decreased comprises: processing the corrected classification feature matrix using the classifier with the following formula to obtain the classification result, wherein the formula is: o = softmax { (W) _n ,B _n ):…:(W ₁ ,B ₁ ) L Project (F), where Project (F) represents the projection of the corrected sorted feature matrix as a vector, W ₁ To W _n As a weight matrix for each fully connected layer, B ₁ To B _n Representing the bias vectors of the fully connected layers of each layer.