CN113567439A - Pork freshness detection method based on color and smell data fusion - Google Patents

Pork freshness detection method based on color and smell data fusion Download PDF

Info

Publication number
CN113567439A
CN113567439A CN202110860659.4A CN202110860659A CN113567439A CN 113567439 A CN113567439 A CN 113567439A CN 202110860659 A CN202110860659 A CN 202110860659A CN 113567439 A CN113567439 A CN 113567439A
Authority
CN
China
Prior art keywords
pork
freshness
color
image
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202110860659.4A
Other languages
Chinese (zh)
Inventor
于兴军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Yanhuang Industrial Design Co ltd
Original Assignee
Shandong Yanhuang Industrial Design Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Yanhuang Industrial Design Co ltd filed Critical Shandong Yanhuang Industrial Design Co ltd
Priority to CN202110860659.4A priority Critical patent/CN113567439A/en
Publication of CN113567439A publication Critical patent/CN113567439A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands

Abstract

A pork freshness detection method based on color and smell data fusion mainly comprises the following steps: acquiring pork image information through a machine vision acquisition device, selecting an optimal threshold value by adopting an Otsu algorithm, and performing image segmentation on the gray level image by using the threshold value to complete background segmentation of the image; layering the image by using the depth value of red in the image according to the relevant characteristics of pork color and freshness, and acquiring the characteristic information of the pork image according to the color area ratio; collecting and normalizing pork odor information by using an electronic nose collection system, analyzing the color and odor information of pork by a principal component analysis method, and extracting principal components for judging freshness; and establishing a nonlinear relation model between the main components of the pork and the volatile basic nitrogen by using a support vector machine, and judging the freshness level of the pork according to the estimated value of the volatile basic nitrogen to finish the task of detecting the freshness of the pork. The method has better accuracy and real-time performance, and can accurately, quickly and nondestructively complete the detection task.

Description

Pork freshness detection method based on color and smell data fusion
The invention relates to a divisional application of an invention patent with the application number of 2018116112965, wherein the application number is 'pork freshness detection method based on color and smell data fusion'.
Technical Field
The invention relates to a pork freshness detection method based on color and smell data fusion, and belongs to the technical field of image information, sensor technology and intelligent information.
Background
The pork is rotten and deteriorated, namely, the pork is subjected to qualitative change under the pollution action of various microorganisms, a large amount of sulfide and ammonia gas are generated, and the edible value is finally lost. The deteriorated and putrefactive pork has changed nutritive value and palatability, even contains toxic substances, and is harmful to the health of eaters. At present, meat products in the market mainly adopt a sensory inspection method as a main method, and are judged for freshness by combining rapid detection methods of various inspection instruments, but detection results of the detection methods are non-quantitative and lack of accuracy, detection can be completed only by related personnel with rich experience and good training, and high-precision real-time detection cannot be realized.
Disclosure of Invention
In order to solve the above problems, the present invention aims to provide a pork freshness detection method based on color and smell data fusion, which adopts a machine vision and electronic nose detection system to obtain color and smell information of pork, performs qualitative and quantitative analysis on the freshness information by principal component analysis and stochastic resonance method, extracts principal components therein, and detects the content of volatile basic nitrogen in the pork by using a support vector machine, thereby accurately, rapidly and nondestructively completing the detection task.
The technical scheme adopted for solving the problems comprises the following steps:
acquiring pork image information through a machine vision acquisition device, selecting an optimal threshold value by adopting an Otsu algorithm, and performing image segmentation on the gray level image by using the threshold value to complete background segmentation of the image;
layering the image by using the depth value of red in the image according to the relevant characteristics of pork color and freshness, and acquiring the characteristic information of the pork image according to the color area ratio;
collecting and normalizing pork odor information by using an electronic nose collection system, analyzing the color and odor information of pork by a principal component analysis method, and extracting principal components for judging freshness;
and establishing a nonlinear relation model between the main components of the pork and the volatile basic nitrogen by using a support vector machine, and judging the freshness level of the pork according to the estimated value of the volatile basic nitrogen to finish the task of detecting the freshness of the pork.
The invention has the beneficial effects that:
in a task of detecting the freshness of the pork with high complexity and various detection contents, the machine vision and electronic nose detection system is adopted to obtain color and smell information of the pork, qualitative and quantitative analysis is carried out on the freshness information by a principal component analysis and stochastic resonance method, principal components in the freshness information are extracted, and a support vector machine is used for detecting the content of volatile basic nitrogen in the pork, so that the detection task is completed, and the beneficial effects of accuracy, rapidness and no damage are produced.
Drawings
FIG. 1 is an overall flow chart of a pork freshness detection method based on color and smell data fusion;
FIG. 2 is a schematic view of a pork freshness detection device;
fig. 3 is an RGB color space model.
Detailed Description
Referring to fig. 1, the method of the present invention comprises the steps of:
A. acquiring pork image information through a machine vision acquisition device, selecting an optimal threshold value by adopting an Otsu algorithm, and performing image segmentation on the gray level image by using the threshold value to complete background segmentation of the image;
acquiring original image information of pork by using the machine vision acquisition device in FIG. 2, assuming that an image is f and an image gray level is L, wherein the number of pixels with a gray value of g is niAnd if the total number of pixels of the image is N, the probability of each gray value is:
Figure BDA0003185205710000021
setting a threshold value T to divide an image into two types of areas, namely a background area A and a target area B, wherein the background area A comprises pixels with the gray scale of 1-T, the target area B comprises pixels with the gray scale of T + 1-L, and the probabilities of the area A and the area B are respectively as follows:
Figure BDA0003185205710000022
Figure BDA0003185205710000023
the average gray levels are respectively:
Figure BDA0003185205710000024
Figure BDA0003185205710000025
the gray variance of regions a and B is:
Figure BDA0003185205710000026
wherein v (k) ═ vADenotes the probability of the region a, u denotes the average gray-scale value of the entire image, and u (k) is uAThe average gray level of the region A is represented, the threshold value T is changed from 1 to L according to the maximum between-class variance criterion, and the variance sigma is calculated2So that σ2Acquiring the maximum value T as the optimal threshold value for region segmentation, and performing binarization processing on the gray level image by using the threshold value, namely completing background segmentation;
B. layering the image by using the depth value of red in the image according to the relevant characteristics of pork color and freshness, and acquiring the characteristic information of the pork image according to the color area ratio;
(3) since pork red color change is related to freshness, layering images according to the shade of red in an RGB color space;
establishing an RGB color model in a Cartesian product coordinate system to generate an RGB color space model:
in the RGB color space, a cube near the coordinate point (0,255,0) is taken, the color contained therein is taken as the target color, the rest of the colors are taken as the background colors, and according to the corresponding determination criteria:
Figure BDA0003185205710000031
carrying out image segmentation on a red area in a color space, wherein mu represents an image layering threshold value, and R, G, B represents the components of red, green and blue components of an image in an RGB color space respectively;
(4) calculating the color area ratio R of the image according to the ratio of the pixel points of the color layering area in the layered image to the total number of the image pixel points:
Figure BDA0003185205710000032
wherein M represents the total number of pixel points of the target image obtained in the step A, and M represents the number of pixel points of the color layering region obtained by color segmentation according to the step B- (1); setting a color area ratio threshold T in an actual color detection processRClassifying the pork according to the threshold value, and classifying the pork adjacent to the threshold value TRAs characteristic information;
C. collecting and normalizing pork odor information by using an electronic nose collection system, analyzing the color and odor information of pork by a principal component analysis method, and extracting principal components for judging freshness;
(3) in pork corruption process, mainly produce a certain amount of hydrogen sulfide gas and ammonia, adopt the electron nose sensor array of constituteing by two kinds of sensors to gather the volatility gas composition of pork, judge the fresh degree change of pork according to gaseous concentration, consequently the concentration of two kinds of gases is as the characteristic information that the pork freshness detected, and the original characteristic information value of pork that utilizes the electron nose sensor array to gather is great, at first carries out normalization to it:
Figure BDA0003185205710000033
where s represents the original feature information,
Figure BDA0003185205710000034
representing normalized characteristic information values, smaxAnd sminRespectively a maximum value and a minimum value of the characteristic information;
(4) carrying out qualitative analysis on pork freshness detection data by adopting principal component analysis;
assuming that the number of pork features obtained by an electronic nose acquisition system and a machine vision acquisition system is n, normalizing the initial feature value by using an equation (a) to form an n-dimensional random vector X ═ X1,x2,...,xm}TMaking the mean value of each variable 0 and the variance 1 to obtain the covariance C of the characteristic informationx
Cx=E[(X-E[X])(X-E[X]T)]
Calculating a covariance matrix U:
U=E[XXT]
according to the covariance matrix U, calculating the eigenvalue gamma and the corresponding eigenvector Y:
UYi=γiYii=1,2,...,n
assuming a feature value gamma1≥γ2≥…≥γnThen, then
hi=Yi TXi=1,2,...,n
The projection of the input feature information to the feature vector, i.e. the ith principal component representing X, is transformed by the feature vector matrix Y into the component H in the feature space vector H, HiIs the ith principal component of input X;
② diagonal element u of covariance matrix according to vector XiiAnd the first G characteristic values gamma corresponding to the principal componentsi(i ═ 1, 2.., G) the contribution ratio of each component was calculated:
Figure BDA0003185205710000041
determining the number of the principal components according to the contribution rates of the characteristic values, and assuming that the sum of the contribution rates of the first m principal components exceeds 90%, determining that the first m principal components can replace the characteristic information of pork freshness, and completing the qualitative analysis of the pork freshness;
D. and establishing a nonlinear relation model between the main components of the pork and the volatile basic nitrogen by using a support vector machine, and judging the freshness level of the pork according to the estimated value of the volatile basic nitrogen to finish the task of detecting the freshness of the pork.
(3) For the pork detection standard, the pork freshness is measured by the national volatile basic nitrogen content, and the corresponding relationship between the pork freshness grade and the volatile basic nitrogen content is shown in table 1:
TABLE 1 correspondence of volatile basic nitrogen content to freshness grade
Volatile basic nitrogen content Pork freshness grade
15mg/100g Class A (fresh meat)
15~30mg/100g Grade B (sub-fresh meat)
>30mg/100g Grade C (meat)
(4) When a support vector machine is adopted to carry out intelligent detection on pork freshness, the method selects hydrogen sulfide, ammonia gas, color and volatile basic nitrogen content as evaluation indexes, takes the extracted main component as the input of the support vector machine, takes the volatile basic nitrogen content as expected output, and maps the input sample data to a high-dimensional linear feature space by the support vector machine through a nonlinear function F (x) and by utilizing a radial basis function J (x, x'):
Figure BDA0003185205710000042
wherein x isiRepresenting a sample factor of a support vector machine, wherein x is a forecasting factor, and tau represents a nuclear parameter;
constructing a regression estimation function, and establishing a nonlinear relation model:
Figure BDA0003185205710000051
wherein, λ*(0≤λ≤z,0≤λ*And less than or equal to z) is a Lagrange multiplier, l is the number of training samples, z is a penalty factor, k represents a bias, and the optimal model parameters are obtained through learning and training: penalty factor parameter, kernel parameter, offset; and obtaining corresponding volatile basic nitrogen content Z by using the model, and determining the pork freshness grade according to the comparison of the Z value and the value in the table 1 to finish the pork freshness detection task.
In conclusion, the pork freshness detection method based on the fusion of color and smell data is realized. In a task of detecting the freshness of the pork with high complexity and various detection contents, the machine vision and electronic nose detection system is adopted to obtain color and smell information of the pork, qualitative and quantitative analysis is carried out on the freshness information by a principal component analysis and stochastic resonance method, principal components in the freshness information are extracted, and a support vector machine is used for detecting the content of volatile basic nitrogen in the pork, so that the detection task is completed, and the beneficial effects of accuracy, rapidness and no damage are produced.

Claims (2)

1. A pork freshness detection method based on color and smell data fusion is characterized by comprising the following steps:
A. acquiring pork image information through a machine vision acquisition device, selecting an optimal threshold value by adopting an Otsu algorithm, and performing image segmentation on the gray level image by using the threshold value to complete background segmentation of the image;
B. layering the image by using the depth value of red in the image according to the relevant characteristics of pork color and freshness, and acquiring the characteristic information of the pork image according to the color area ratio;
C. collecting and normalizing pork odor information by using an electronic nose collection system, analyzing the color and odor information of pork by a principal component analysis method, and extracting principal components for judging freshness;
D. establishing a non-linear relation model between main components of pork and volatile basic nitrogen by using a support vector machine, and judging the freshness level of the pork according to an estimated value of the volatile basic nitrogen to complete a task of detecting the freshness of the pork; the step B comprises the following steps:
(1) since pork red color change is related to freshness, layering images according to the shade of red in an RGB color space;
establishing an RGB color model in a Cartesian product coordinate system to generate an RGB color space model: in the RGB color space, a cube close to the coordinate point (0,255,0) is taken, the color contained therein is taken as the target color, the rest of the colors are taken as the background colors, and according to the corresponding determination criteria:
Figure FDA0003185205700000011
carrying out image segmentation on a red area in a color space, wherein mu represents an image layering threshold value, and R, G, B represents the components of red, green and blue components of an image in an RGB color space respectively;
(2) calculating the color area ratio R of the image according to the ratio of the pixel points of the color layering area in the layered image to the total number of the image pixel points:
Figure FDA0003185205700000012
wherein M represents the total number of pixel points of the target image obtained in the step A, and M represents the number of pixel points of the color layering region obtained by color segmentation according to the step B- (1); setting a color area ratio threshold T in an actual color detection processRClassifying the pork according to the threshold value, and classifying the pork adjacent to the threshold value TRAs characteristic information;
the step C comprises the following steps:
(1) in pork corruption process, mainly produce a certain amount of hydrogen sulfide gas and ammonia, adopt the electron nose sensor array of constituteing by two kinds of sensors to gather the volatility gas composition of pork, judge the fresh degree change of pork according to gaseous concentration, consequently the concentration of two kinds of gases is as the characteristic information that the pork freshness detected, and the original characteristic information value of pork that utilizes the electron nose sensor array to gather is great, at first carries out normalization to it:
Figure FDA0003185205700000021
where s represents the original feature information,
Figure FDA0003185205700000022
representing normalized characteristic information values, smaxAnd sminRespectively a maximum value and a minimum value of the characteristic information;
(2) carrying out qualitative analysis on pork freshness detection data by adopting principal component analysis;
assuming that the number of pork features obtained by an electronic nose acquisition system and a machine vision acquisition system is n, normalizing the initial feature value by using an equation (a) to form an n-dimensional random vector X ═ X1,x2,...,xm}TLet the mean value of each variable be 0 and the variance be 1 to obtain the characteristic
Covariance C of characteristic informationx
Cx=E[(X-E[X])(X-E[X]T)]
Calculating a covariance matrix U:
U=E[XXT]
according to the covariance matrix U, calculating the eigenvalue gamma and the corresponding eigenvector Y:
UYi=γiYi i=1,2,...,n
assuming a feature value gamma1≥γ2≥…≥γnThen, then
hi=Yi TX i=1,2,...,n
The projection of the input feature information to the feature vector, i.e. the ith principal component representing X, is transformed by the feature vector matrix Y into the component H in the feature space vector H, HiIs the ith principal component of input X;
② diagonal element u of covariance matrix according to vector XiiAnd the first G characteristic values gamma corresponding to the principal componentsi(i ═ 1, 2.., G) the contribution ratio of each component was calculated:
Figure FDA0003185205700000023
and determining the number of the main components according to the contribution rates of the characteristic values, and assuming that the sum of the contribution rates of the first m main components exceeds 90%, determining that the first m main components can replace the characteristic information of pork freshness, and completing the qualitative analysis of the pork freshness.
2. The pork freshness detection method based on color and odor data fusion of claim 1, wherein the step D comprises:
(1) for the pork detection standard, the pork freshness is measured by the national volatile basic nitrogen content, and the corresponding relationship between the pork freshness grade and the volatile basic nitrogen content is shown in table 1:
TABLE 1 correspondence of volatile basic nitrogen content to freshness grade
Figure FDA0003185205700000024
Figure FDA0003185205700000031
(2) When a support vector machine is adopted to carry out intelligent detection on pork freshness, the method selects hydrogen sulfide, ammonia gas, color and volatile basic nitrogen content as evaluation indexes, takes the extracted main component as the input of the support vector machine, takes the volatile basic nitrogen content as expected output, and maps the input sample data to a high-dimensional linear feature space by the support vector machine through a nonlinear function F (x) and by utilizing a radial basis function J (x, x'):
Figure FDA0003185205700000032
wherein x isiRepresenting a sample factor of a support vector machine, wherein x is a forecasting factor, and tau represents a nuclear parameter; constructing a regression estimation function, and establishing a nonlinear relation model:
Figure FDA0003185205700000033
wherein, λ*(0≤λ≤z,0≤λ*And less than or equal to z) is a Lagrange multiplier, l is the number of training samples, z is a penalty factor, k represents a bias, and the optimal model parameters are obtained through learning and training: penalty factor parameter, kernel parameter, offset; and obtaining corresponding volatile basic nitrogen content Z by using the model, and determining the pork freshness grade according to the comparison of the Z value and the value in the table 1 to finish the pork freshness detection task.
CN202110860659.4A 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion Withdrawn CN113567439A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110860659.4A CN113567439A (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110860659.4A CN113567439A (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion
CN201811611296.5A CN109490306B (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201811611296.5A Division CN109490306B (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion

Publications (1)

Publication Number Publication Date
CN113567439A true CN113567439A (en) 2021-10-29

Family

ID=65712671

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201811611296.5A Active CN109490306B (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion
CN202110860659.4A Withdrawn CN113567439A (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201811611296.5A Active CN109490306B (en) 2018-12-27 2018-12-27 Pork freshness detection method based on color and smell data fusion

Country Status (1)

Country Link
CN (2) CN109490306B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110133049B (en) * 2019-04-10 2020-09-18 浙江大学 Electronic nose and machine vision-based rapid nondestructive testing method for tea grade
CN110210680A (en) * 2019-06-11 2019-09-06 北京农业信息技术研究中心 A kind of fish body Noninvasive Measuring Method of Freshness and device based on temperature change
CN110659579B (en) * 2019-08-23 2024-05-03 平安科技(深圳)有限公司 Deteriorated article identification method, apparatus, device and medium
CN111612742B (en) * 2020-04-23 2023-10-20 山东中医药高等专科学校 Mildew detection method, detection system and computer readable storage medium for traditional Chinese medicine
WO2023004573A1 (en) * 2021-07-27 2023-02-02 合肥华凌股份有限公司 Odor removal control method and device applied to refrigerator, and refrigerator
CN116976675B (en) * 2023-08-09 2024-04-05 南京龟兔赛跑软件研究院有限公司 Cold chain transportation risk early warning method and system based on data monitoring of Internet of things

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101144780B (en) * 2006-09-14 2011-05-11 北京工商大学 Pork freshness intelligent detection device
CN102269710A (en) * 2011-06-17 2011-12-07 中国农业大学 Rapid lossless prediction device of fresh port validity based on multispectral imaging
CN103424428A (en) * 2013-06-05 2013-12-04 浙江工商大学 Method for quickly detecting pork freshness based on electronic nose
CN103278609B (en) * 2013-06-27 2015-09-30 山东商业职业技术学院 A kind of Examination of Meat Fresh Level merged based on multi-source perception information
CN203745367U (en) * 2013-12-10 2014-07-30 六安职业技术学院 Intelligent pork freshness detector
CN103900972B (en) * 2014-04-04 2017-01-18 江南大学 Multi-feature fusion-based meat freshness hyperspectral image visual detection
CN106556602A (en) * 2016-12-05 2017-04-05 湖南理工学院 A kind of detection method and its device of fish body freshness

Also Published As

Publication number Publication date
CN109490306A (en) 2019-03-19
CN109490306B (en) 2021-07-23

Similar Documents

Publication Publication Date Title
CN109490306B (en) Pork freshness detection method based on color and smell data fusion
CN110634080B (en) Abnormal electricity utilization detection method, device, equipment and computer readable storage medium
CN105044298B (en) A kind of Eriocheir sinensis class grade of freshness detection method based on machine olfaction
CN104598908B (en) A kind of crops leaf diseases recognition methods
CN108960315B (en) Intelligent evaluation system and method for quality of cooked meat product
CN107239795A (en) SAR image change detecting system and method based on sparse self-encoding encoder and convolutional neural networks
CN105954412B (en) For the sensor array optimization method of hickory nut freshness detection
CN111126511B (en) Vegetation index fusion-based LAI quantitative model establishment method
CN113049509B (en) Agricultural product detection management system based on spectrum technology
CN115994907B (en) Intelligent processing system and method for comprehensive information of food detection mechanism
CN110428121A (en) Hidden Markov model food quality appraisal procedure based on grey correlation analysis
CN107590816B (en) Water body information fitting method based on remote sensing image
CN116362570A (en) Multi-dimensional pollution analysis method and system based on big data platform
Gurubasava et al. Analysis of agricultural soil pH using digital image processing
CN109523514A (en) To the batch imaging quality assessment method of Inverse Synthetic Aperture Radar ISAR
CN109784345B (en) Agricultural pest detection method based on non-scale depth network
CN107610119A (en) The accurate detection method of steel strip surface defect decomposed based on histogram
CN111401444B (en) Method and device for predicting red wine origin, computer equipment and storage medium
CN112445849B (en) Report form analysis method and device
CN116310842B (en) Soil saline-alkali area identification and division method based on remote sensing image
CN112184627A (en) Citrus fresh-keeping quality detection method based on image processing and neural network and application
CN109784417B (en) Black hair pork image identification method
CN114970675A (en) Artificial nose refrigerator food freshness detection system and method based on feature selection
CN112465821A (en) Multi-scale pest image detection method based on boundary key point perception
CN112581467A (en) Intelligent security inspection method based on suspected hazardous article evaluation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20211029

WW01 Invention patent application withdrawn after publication