WO2023029311A1 - 水果成熟度检测方法、装置、设备及存储介质 - Google Patents

水果成熟度检测方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023029311A1
WO2023029311A1 PCT/CN2021/141480 CN2021141480W WO2023029311A1 WO 2023029311 A1 WO2023029311 A1 WO 2023029311A1 CN 2021141480 W CN2021141480 W CN 2021141480W WO 2023029311 A1 WO2023029311 A1 WO 2023029311A1
Authority
WO
WIPO (PCT)
Prior art keywords
fruit
audio
maturity
percussion
audio data
Prior art date
Application number
PCT/CN2021/141480
Other languages
English (en)
French (fr)
Inventor
杨永健
曹志宇
Original Assignee
合肥美的电冰箱有限公司
合肥华凌股份有限公司
美的集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 合肥美的电冰箱有限公司, 合肥华凌股份有限公司, 美的集团股份有限公司 filed Critical 合肥美的电冰箱有限公司
Publication of WO2023029311A1 publication Critical patent/WO2023029311A1/zh

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
    • G01N29/04—Analysing solids
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
    • G01N29/04—Analysing solids
    • G01N29/12—Analysing solids by measuring frequency or resonance of acoustic waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
    • G01N29/44—Processing the detected response signal, e.g. electronic circuits specially adapted therefor

Definitions

  • the present application relates to the field of computer technology, in particular to a fruit maturity detection method, device, equipment and storage medium.
  • the main purpose of this application is to provide a fruit ripeness detection method, device, equipment and storage medium, aiming to solve the technical problems of relatively complicated development and implementation of detection in the prior art and low detection accuracy.
  • the application provides a method for detecting fruit maturity, which includes the following steps:
  • the maturity of the fruit to be detected is detected by the maturity detection model according to the knocking audio feature.
  • a preset machine learning model is trained according to the training sample data set to obtain a maturity detection model.
  • said constructing a training sample data set according to said audio data includes:
  • the percussion audio data is divided based on the label corresponding to the percussion audio data, to obtain ripe fruit percussion audio data and raw fruit percussion audio data;
  • the training of the preset machine learning model according to the training sample data set to obtain the maturity detection model includes:
  • the intercepting a corresponding tapping audio segment from the tapping audio data according to a preset time window includes:
  • the corresponding tap audio clip is intercepted from the audio clip.
  • said acquiring the tap audio data corresponding to the fruit to be detected includes:
  • the maturity detection model after detecting the maturity of the fruit to be detected through the maturity detection model according to the tapping audio feature, it further includes:
  • the maturity score is displayed.
  • a fruit maturity detection device which includes:
  • An acquisition module configured to acquire percussion audio data corresponding to the fruit to be detected, and obtain the fruit type of the fruit to be detected;
  • An intercepting module configured to intercept a corresponding percussion audio segment from the percussion audio data according to a preset time window
  • the extraction module is used to extract the corresponding percussion audio feature of the fruit to be detected from the intercepted percussion audio segment;
  • a detection module configured to detect the maturity of the fruit to be detected through the maturity detection model according to the tap audio features.
  • the present application also proposes a fruit maturity detection device, which includes: a memory, a processor, and a fruit stored in the memory and operable on the processor.
  • a maturity detection program, the fruit maturity detection program is configured to implement the fruit maturity detection method as described above.
  • the present application also proposes a storage medium on which a fruit maturity detection program is stored, and when the fruit maturity detection program is executed by a processor, the fruit maturity detection program as described above is realized. Detection method.
  • the present application obtains the knocking audio data corresponding to the fruit to be detected, and obtains the fruit type of the fruit to be detected; intercepts the corresponding knocking audio segment from the knocking audio data according to the preset time window; from the intercepted Extracting the tap audio feature corresponding to the fruit to be detected from the tap audio clip; determining the corresponding maturity detection model according to the fruit type; and detecting the fruit to be detected through the maturity detection model according to the tap audio feature
  • the ripeness of the fruit is detected based on the audio data without high computing power, and the audio features extracted based on the time window can improve the accuracy of the ripeness detection.
  • Fig. 1 is the structural representation of the fruit ripeness detection equipment of the hardware running environment that scheme of the embodiment of the present application relates to;
  • Fig. 2 is the schematic flow sheet of the first embodiment of the fruit ripeness detection method of the present application
  • Fig. 3 is a schematic diagram of audio segment interception in an embodiment of the fruit maturity detection method of the present application.
  • Fig. 4 is the schematic flow sheet of the second embodiment of the fruit ripeness detection method of the present application.
  • Fig. 5 is a schematic diagram of the original audio without human voice removal in an embodiment of the fruit maturity detection method of the present application
  • Fig. 6 is the percussion audio schematic diagram that carries out human voice removal in an embodiment of the fruit maturity detection method of the present application
  • Fig. 7 is the schematic flow chart of the third embodiment of the fruit maturity detection method of the present application.
  • Fig. 8 is a schematic diagram showing the maturity score in an embodiment of the fruit maturity detection method of the present application.
  • Fig. 9 is a structural block diagram of the first embodiment of the fruit ripeness detection device of the present application.
  • FIG. 1 is a schematic structural diagram of a fruit ripeness detection device in a hardware operating environment involved in an embodiment of the present application.
  • the fruit maturity detection device may include: a processor 1001 , such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 .
  • the communication bus 1002 is used to realize connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface).
  • Wi-Fi Wireless-Fidelity
  • Memory 1005 can be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as disk memory.
  • RAM Random Access Memory
  • NVM Non-Volatile Memory
  • the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .
  • Figure 1 does not constitute a limitation to the fruit ripeness detection device, and may include more or less components than those shown in the illustration, or combine certain components, or arrange different components .
  • the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a fruit maturity detection program.
  • the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 in the fruit maturity detection device of the present application
  • the memory 1005 can be set in the fruit maturity detection device, and the fruit maturity detection device calls the fruit maturity detection program stored in the memory 1005 through the processor 1001, and executes the fruit maturity detection method provided in the embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a first embodiment of a method for detecting fruit maturity of the present application.
  • described fruit maturity detection method comprises the following steps:
  • Step S10 Obtain the tapping audio data corresponding to the fruit to be detected, and obtain the fruit type of the fruit to be detected.
  • the execution subject of this embodiment may be a fruit maturity detection device, and the fruit maturity detection device may be an electronic device such as a personal computer or a server, or other controllers that can realize the same or similar functions.
  • the embodiment does not limit this.
  • the fruit maturity detection method of the present application is described by taking the fruit maturity detection equipment as an example.
  • the fruit to be detected is a fruit that needs to be tested for maturity.
  • the fruit to be detected can be watermelon or muskmelon, etc., which can be judged by tapping.
  • the image of the fruit to be detected is obtained by a camera Information, and the sound pickup equipment to obtain the knocking sound of the fruit to be detected, combined with the image information and the knocking sound to detect the maturity of the fruit to be detected, in the prior art, it is necessary to combine the image information with the knocking sound, what is adopted is the need A deep learning model with high computing power, and mainly based on image information.
  • the maturity of the fruit to be detected is determined through image features such as the color, size, and texture of the fruit to be detected in the image information, and the calculation requirements are relatively high. Moreover, the maturity detection results obtained by the image information-based detection method are not accurate enough.
  • the maturity detection of the fruit to be detected is performed, it is not necessary to obtain the image information of the fruit to be detected, and the maturity detection can be realized by the knocking sound corresponding to the obtained fruit to be detected, and the maturity of the fruit to be detected by the knocking sound
  • the detection accuracy is high, and because it does not involve the processing of image information, the demand for model computing power is greatly reduced.
  • the fruit ripeness detection device is taken as an example to illustrate the mobile terminal.
  • the mobile terminal includes devices such as mobile phones or tablets, and an application program is installed in the mobile terminal.
  • the user activates the recording function of the mobile terminal by operating the application program.
  • the corresponding knocking audio data is obtained through the recording function of the mobile terminal, and the recording function of the mobile terminal is turned off after the knocking audio data is obtained, wherein the opening time of the recording function can be determined according to Actual detection requirements are set accordingly, which is not limited in this embodiment.
  • the fruit type corresponding to the fruit to be detected is determined according to the fruit information input by the user, such as watermelon or muskmelon.
  • the camera of the mobile terminal can be turned on through the application program, and the image information of the fruit to be detected can be captured by the camera, and the fruit to be detected can be determined based on the grain characteristics and color characteristics of the fruit to be detected in the image information.
  • the specific method for detecting the type of fruit can be selected according to the detection requirements, which is not limited in this embodiment.
  • the collected fruit percussion will be affected by many factors. Even for the same fruit, if the percussion force, percussion technique and percussion parts are different, the collected percussion sound will be affected by many factors. The click sound will be different. Moreover, the recording components or system settings of different mobile terminals are different, and the collected audio parameters will also be different. Therefore, it is necessary to re-sample all the collected audio to unify to the same sampling rate. In this embodiment, in order to filter out knocking sounds with the same sampling frequency and improve detection accuracy, it may be implemented in the following manner.
  • the reference percussion audio data corresponding to the moment under the same sampling frequency is used as the percussion audio data of the fruit to be detected, so as to ensure that the acquired percussion audio data is at the same sampling frequency, wherein the preset sampling frequency can be Corresponding settings are made according to actual audio sampling requirements, which is not limited in this embodiment.
  • Step S20 intercepting a corresponding tapping audio segment from the tapping audio data according to a preset time window.
  • the audio data needs to be segmented to accurately distinguish the knocking time period in order to be correct. Get its audio characteristics.
  • the interception of the tapping audio segment is carried out, as shown in Figure 3, the raw fruit tapping audio segment and the ripe fruit tapping audio segment can be intercepted through the preset time window, in Fig. 3, A and B is two time points corresponding to the preset time window, wherein the preset time window can be set correspondingly according to actual needs, which is not limited in this embodiment.
  • Step S30 Extracting the tap audio feature corresponding to the fruit to be detected from the intercepted tap audio clip.
  • the sound of fruit percussion has the characteristics of rapid change in a short period of time, high loudness and weak periodicity. Therefore, in the time domain, the short-term energy, root mean square energy, zero-crossing rate and Loudness standard deviation and other features are more significant. Ripe fruit has a more dull sound than raw fruit because of its abundant water, and its frequency is lower. The aforementioned audio features can be extracted by tapping an audio segment.
  • Step S40 Determine the corresponding maturity detection model according to the fruit type.
  • a deep learning algorithm is used to build a model, which requires high computing power, and there is a black box problem in the modeling process.
  • a machine learning algorithm is used to replace the deep learning algorithm used in the prior art.
  • the maturity detection model is obtained by training a large number of sample data. The sample data obtained for different types of fruits are different, so the obtained maturity detection models are also different, based on the difference between the fruit type and the maturity detection model. The corresponding relationship can obtain a maturity detection model that matches the fruit type, so as to improve the detection accuracy.
  • the maturity detection model is stored in the server, and the server and the mobile terminal establish a communication connection through the Internet.
  • the server in this embodiment Including but not limited to computers, network hosts, single network servers, multiple network server sets or cloud servers composed of multiple servers, wherein the cloud server is composed of a large number of computers or network servers based on cloud computing (Cloud Computing).
  • cloud computing Cloud Computing
  • Step S50 Detect the maturity of the fruit to be detected through the maturity detection model according to the tapping audio features
  • the tapping audio data is input into the maturity detection model, and the maturity detection model can output the fruit corresponding to the fruit to be detected according to the input tapping audio data. maturity.
  • the tapping audio data corresponding to the fruit to be detected by obtaining the tapping audio data corresponding to the fruit to be detected, and obtaining the fruit type of the fruit to be detected; intercepting the corresponding tapping audio segment from the tapping audio data according to the preset time window; Extract the percussion audio feature corresponding to the fruit to be detected from the percussion audio clip; determine the corresponding maturity detection model according to the fruit type; and detect the to-be-detected by the maturity detection model according to the percussion audio feature
  • the ripeness of the fruit, detecting the ripeness of the fruit based on the audio data does not require high computing power, and the audio features extracted based on the time window can improve the accuracy of the ripeness detection.
  • FIG. 4 is a schematic flowchart of a second embodiment of a method for detecting fruit maturity in the present application.
  • the fruit maturity detection method of the present embodiment also includes before the step S40:
  • Step S401 Obtain audio data corresponding to the fruit type.
  • the acquisition of audio data adopts the method of big data.
  • the setting of the variables can also be adjusted accordingly according to the actual model construction requirements, which is not limited in this embodiment.
  • Step S402 Construct a training sample data set according to the audio data.
  • the audio data can be processed by means of data integration, division, and grouping, so as to obtain a training sample data set that can be used for model training.
  • the training sample data set may be constructed in the following manner.
  • the human voice is the main influencing factor in the process of the fruit being knocked.
  • the existence of the human voice will greatly affect the interception of the fruit knocking sound and the extraction of audio features.
  • human voice is equivalent to additive noise. Therefore, before feature extraction, it is necessary to denoise the audio, that is, separate and filter the human voice.
  • the separation methods include but are not limited to the use of Deezer's open-source Spleeter model, which can separate the human voice from the background sound, and the percussion audio It is the background music, and the purpose of noise reduction can be achieved after separating the two.
  • Fig. 5 shows Yuan Shu's audio data without human voice filtering. After the human voice is separated, the audio data shown in Fig. 6 can be obtained, that is, the percussion audio data.
  • the percussion audio data can be divided into mature fruit percussion audio data and raw fruit percussion audio data based on the label corresponding to the percussion audio data.
  • the training sample data set It includes a positive sample data set and a negative sample data set, wherein the negative sample data set is raw fruit percussion audio data, and the positive sample data set is ripe fruit percussion audio data as the positive sample data set.
  • the tags in this embodiment include the color of the fruit, the sweetness and taste of the food, etc. The tags are set by the user, and the classification settings of the tags can be adjusted according to actual needs, which is not limited in this embodiment.
  • Step S403 Train a preset machine learning model according to the training sample data set to obtain a maturity detection model.
  • the model used in this embodiment is a machine learning model, and the model type used by the preset machine learning model can be set correspondingly according to actual training requirements, which is not limited in this embodiment.
  • the XGBoost algorithm can be used to train the preset machine learning model.
  • XGBoost is a decision tree-based scalable supervised learning algorithm.
  • the training of the model includes parameter adjustment.
  • K Folded cross-validation and grid search algorithms adjust the parameters of the preset machine learning model.
  • the generation of the model is actually based on the Flask framework, Gunicorn HTTP server and Nginx load balancing. It should be emphasized that the selection of the above algorithms, frameworks and servers can be Corresponding adjustments are made according to actual model building requirements, which is not limited in this embodiment.
  • the audio data needs to be segmented to accurately distinguish the knocking time period in order to be correct. Get its audio characteristics.
  • the interception of the tapping audio segment is carried out, as shown in Figure 3, the raw fruit tapping audio segment and the ripe fruit tapping audio segment can be intercepted through the preset time window, in Fig.
  • a and B is the two time points corresponding to the preset time window, and then based on the intercepted raw fruit percussion audio clips and ripe fruit percussion audio clips, the percussion audio features when the raw fruit is percussed, and the percussion audio characteristics when the ripe fruit is percussed Finally, the raw percussion audio feature vector and the mature fruit percussion audio feature vector are constructed, and the raw percussion audio feature vector and the ripe fruit percussion audio feature vector are input into the preset machine model, so as to predict the The model parameters of the machine model are adjusted, and the adjusted target model parameters are used as the current model parameters of the preset machine learning model to obtain the maturity detection model.
  • the machine learning model is trained to obtain a maturity detection model, which reduces the computing power requirements for maturity detection and improves the accuracy of maturity detection.
  • FIG. 7 is a schematic flow chart of a third embodiment of a fruit maturity detection method of the present application.
  • the step S20 in this embodiment includes:
  • Step S201 Obtain the start time of the audio segment corresponding to the tapping audio data.
  • Step S202 Determine the target end time according to the start time and the preset time window.
  • the tapping audio data contains multiple audio clips, as shown in Figure 3, each audio clip has a corresponding starting moment, for example, point A in Figure 3 is the starting point of the audio clip
  • point A in Figure 3 is the starting point of the audio clip
  • an audio clip includes a start moment and an end moment, and the duration of the audio clip can be determined according to the start moment and the end moment. Different durations affect the extraction results of audio features.
  • the preset time window is used as Tap the duration corresponding to the audio clip, and add the start time to the preset time window to get the end time corresponding to the tap audio clip, that is, the target end time.
  • the preset time window can be set accordingly according to actual needs. This is not limited in the embodiments.
  • Step S203 According to the start time and the target end time, a corresponding tapping audio segment is intercepted from the audio segment.
  • the target end time can be determined, that is, the end time corresponding to point B, and then According to the start time corresponding to point A and the end time corresponding to point B, the AB tapping audio segment can be intercepted.
  • step S50 described in this embodiment it also includes:
  • Step S60 Generate a maturity score of the fruit to be detected according to the detected maturity.
  • Step S70 Display the maturity score.
  • the corresponding maturity score can also be determined according to the maturity of the fruit to be detected.
  • the maturity score is displayed to the user in a graphical manner, as shown in FIG. 8 .
  • the target end time is determined according to the start time and the preset time window by obtaining the start time of the audio segment corresponding to the tapping audio data, and the target end time is determined according to the start time and the target end time
  • accurate percussion audio clips can be obtained based on the preset time window to improve the accuracy of maturity detection, and at the same time, the ripeness of the fruit and the corresponding ripeness can be intuitively displayed through scoring. quality and improve user experience.
  • the embodiment of the present application also proposes a storage medium on which a fruit maturity detection program is stored, and when the fruit maturity detection program is executed by a processor, the fruit maturity detection method as described above is realized. step.
  • the storage medium adopts all the technical solutions of all the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.
  • FIG. 9 is a structural block diagram of the first embodiment of the fruit ripeness detection device of the present application.
  • the fruit maturity detection device that the embodiment of the present application proposes comprises:
  • the acquiring module 10 is configured to acquire the tapping audio data corresponding to the fruit to be detected, and acquire the fruit type of the fruit to be detected.
  • the executive subject of this embodiment may be a fruit maturity detection device, and the fruit maturity detection device may be an electronic device such as a personal computer or a server, or other controllers that can realize the same or similar functions.
  • the embodiment does not limit this.
  • the fruit maturity detection method of the present application is described by taking the fruit maturity detection device as an example.
  • the fruit to be detected is a fruit that needs to be tested for maturity.
  • the fruit to be detected can be watermelon or muskmelon, etc., which can be judged by tapping.
  • the image of the fruit to be detected is obtained by a camera Information, and the sound pickup equipment to obtain the knocking sound of the fruit to be detected, combined with the image information and the knocking sound to detect the maturity of the fruit to be detected, in the prior art, it is necessary to combine the image information with the knocking sound, what is adopted is the need A deep learning model with high computing power, and mainly based on image information.
  • the maturity of the fruit to be detected is determined through image features such as the color, size, and texture of the fruit to be detected in the image information, and the calculation requirements are relatively high. Moreover, the maturity detection results obtained by the image information-based detection method are not accurate enough.
  • the maturity detection of the fruit to be detected is performed, it is not necessary to obtain the image information of the fruit to be detected, and the maturity detection can be realized by the knocking sound corresponding to the obtained fruit to be detected, and the maturity of the fruit to be detected by the knocking sound
  • the detection accuracy is high, and because it does not involve the processing of image information, the demand for model computing power is greatly reduced.
  • the fruit ripeness detection device is taken as an example to illustrate the mobile terminal.
  • the mobile terminal includes devices such as mobile phones or tablets, and an application program is installed in the mobile terminal.
  • the user activates the recording function of the mobile terminal by operating the application program.
  • the corresponding knocking audio data is obtained through the recording function of the mobile terminal, and the recording function of the mobile terminal is turned off after the knocking audio data is obtained, wherein the opening time of the recording function can be determined according to Actual detection requirements are set accordingly, which is not limited in this embodiment.
  • the fruit type corresponding to the fruit to be detected is determined according to the fruit information input by the user, such as watermelon or muskmelon.
  • the camera of the mobile terminal can be turned on through the application program, and the image information of the fruit to be detected can be captured by the camera, and the fruit to be detected can be determined based on the grain characteristics and color characteristics of the fruit to be detected in the image information.
  • the specific method for detecting the type of fruit can be selected according to the detection requirements, which is not limited in this embodiment.
  • the collected fruit percussion will be affected by many factors. Even for the same fruit, if the percussion force, percussion technique and percussion parts are different, the collected percussion sound will be affected by many factors. The click sound will be different. Moreover, the recording components or system settings of different mobile terminals are different, and the collected audio parameters will also be different. Therefore, it is necessary to re-sample all the collected audio to unify to the same sampling rate. In this embodiment, in order to filter out knocking sounds with the same sampling frequency and improve detection accuracy, it may be implemented in the following manner.
  • the reference percussion audio data corresponding to the moment under the same sampling frequency is used as the percussion audio data of the fruit to be detected, so as to ensure that the acquired percussion audio data is at the same sampling frequency, wherein the preset sampling frequency can be Corresponding settings are made according to actual audio sampling requirements, which is not limited in this embodiment.
  • the intercepting module 20 is configured to intercept a corresponding tapping audio segment from the tapping audio data according to a preset time window.
  • the audio data needs to be segmented to accurately distinguish the knocking time period in order to be correct. Get its audio characteristics.
  • the interception of the tapping audio segment is carried out, as shown in Figure 3, the raw fruit tapping audio segment and the ripe fruit tapping audio segment can be intercepted through the preset time window, in Fig. 3, A and B is two time points corresponding to the preset time window, wherein the preset time window can be set correspondingly according to actual needs, which is not limited in this embodiment.
  • the extraction module 30 is configured to extract the percussion audio feature corresponding to the fruit to be detected from the intercepted percussion audio segment.
  • the sound of fruit percussion has the characteristics of rapid change in a short period of time, high loudness and weak periodicity. Therefore, in the time domain, the short-term energy, root mean square energy, zero-crossing rate and Loudness standard deviation and other features are more significant. Ripe fruit has a more dull sound than raw fruit because of its abundant water, and its frequency is lower. The aforementioned audio features can be extracted by tapping an audio segment.
  • the building block 40 is used to determine the corresponding maturity detection model according to the fruit type.
  • a deep learning algorithm is used to build a model, which requires high computing power, and there is a black box problem in the modeling process.
  • a machine learning algorithm is used to replace the deep learning algorithm used in the prior art.
  • the maturity detection model is obtained by training a large number of sample data. The sample data obtained for different types of fruits are different, so the obtained maturity detection models are also different, based on the difference between the fruit type and the maturity detection model. The corresponding relationship can obtain a maturity detection model that matches the fruit type, so as to improve the detection accuracy.
  • the maturity detection model is stored in the server, and the server and the mobile terminal establish a communication connection through the Internet.
  • the server in this embodiment Including but not limited to computers, network hosts, single network servers, multiple network server sets or cloud servers composed of multiple servers, wherein the cloud server is composed of a large number of computers or network servers based on cloud computing (Cloud Computing).
  • cloud computing Cloud Computing
  • the detection module 50 is configured to detect the maturity of the fruit to be detected through the maturity detection model according to the tapping audio features.
  • the tapping audio data is input into the maturity detection model, and the maturity detection model can output the fruit corresponding to the fruit to be detected according to the input tapping audio data. maturity.
  • the tapping audio data corresponding to the fruit to be detected by obtaining the tapping audio data corresponding to the fruit to be detected, and obtaining the fruit type of the fruit to be detected; intercepting the corresponding tapping audio segment from the tapping audio data according to the preset time window; Extract the percussion audio feature corresponding to the fruit to be detected from the percussion audio clip; determine the corresponding maturity detection model according to the fruit type; and detect the to-be-detected by the maturity detection model according to the percussion audio feature
  • the ripeness of the fruit, detecting the ripeness of the fruit based on the audio data does not require high computing power, and the audio features extracted based on the time window can improve the accuracy of the ripeness detection.

Landscapes

  • Physics & Mathematics (AREA)
  • Analytical Chemistry (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Engineering & Computer Science (AREA)
  • Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)

Abstract

一种水果成熟度检测方法、装置、设备及存储介质，检测方法包括：获取待检测水果对应的敲击音频数据，并获取待检测水果的水果种类（S10）；按照预设时间窗口从敲击音频数据中截取相应的敲击音频片段（S20）；从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征（S30）；根据水果种类确定相应的成熟度检测模型（S40）；以及根据敲击音频特征通过成熟度检测模型检测待检测水果的成熟度（S50）。基于音频数据检测水果的成熟度无需借助较高的计算能力，并且基于时间窗口所提取到音频特征能够提高成熟度检测的精度。

Description

水果成熟度检测方法、装置、设备及存储介质
本申请要求于2021年8月31日申请的、申请号为202111019373.X的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种水果成熟度检测方法、装置、设备及存储介质。
 背景技术
当前常应用于农业的水果成熟度检测方法需要专业的大型仪器和设备，成本高且使用不便；而应用于生活中的已有技术是结合图像及敲击音频，通过深度学习的方法建立模型，此种方法涉及图像及敲击音频对计算能力需求高，建模过程是黑箱的问题，既不便于快速开发和实现，检测精度也不高。
上述内容仅用于辅助理解本申请的技术方案，并不代表承认上述内容是现有技术。
 技术问题
本申请的主要目的在于提供一种水果成熟度检测方法、装置、设备及存储介质，旨在解决现有技术检测开发与实现较为复杂且检测精度不高的技术问题。
 技术解决方案
为实现上述目的，本申请提供了一种水果成熟度检测方法，所述水果成熟度检测方法包括以下步骤：
按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；
从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；
根据所述水果种类确定相应的成熟度检测模型；以及
根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度。
在一实施例中，所述根据所述水果种类确定相应的成熟度检测模型之前，还包括：
获取所述水果种类对应的音频数据；
根据所述音频数据构建训练样本数据集；以及
根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型。
在一实施例中，所述根据所述音频数据构建训练样本数据集，包括：
对所述音频数据进行人声分离，以得到敲击音频数据；
基于所述敲击音频数据对应的标签对所述敲击音频数据进行划分，以得到成熟水果敲击音频数据和生水果敲击音频数据；
将所述生水果敲击音频数据作为负样本数据集，将所述成熟水果敲击音频数据作为正样本数据集；以及
根据所述负样本数据集和所述正样本数据集构建训练样本数据集。
在一实施例中，所述根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型，包括：
按照预设时间窗口从所述负样本数据集中获取生水果敲击音频片段，以及从所述正样本数据集中获取熟水果敲击音频片段；
从所述生水果敲击音频片段中提取生水果敲击音频特征，以及从所述熟水果敲击音频片段提取熟水果敲击音频特征；
根据所述生水果敲击音频特征和所述熟水果敲击音频特征对应的特征向量对预设机器学习模型的模型参数进行调整，以得到目标模型参数；以及
将所述目标模型参数输入至所述预设机器学习模型，以得到成熟度检测模型。
在一实施例中，所述按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段，包括：
获取所述敲击音频数据对应的音频片段的起始时刻；
根据所述起始时刻和预设时间窗口确定目标结束时刻；以及
按照所述起始时刻和所述目标结束时刻从所述音频片段中截取相应的敲击音频片段。
在一实施例中，所述获取待检测水果对应的敲击音频数据，包括：
获取待检测水果在多个时刻的参考敲击音频数据；以及
从多个所述时刻的参考敲击音频数据中筛选出符合预设采样频率的参考敲击音频数据，将所述符合预设采样频率的参考敲击音频数据作为所述待检测水果对应的敲击音频数据。
在一实施例中，所述根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度之后，还包括：
根据检测的成熟度生成所述待检测水果的成熟度分值；以及
对所述成熟度分值进行展示。
此外，为实现上述目的，本申请还提出一种水果成熟度检测装置，所述水果成熟度检测装置包括：
获取模块，用于获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；
截取模块，用于按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；
提取模块，用于从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；
构建模块，用于根据所述水果种类确定相应的成熟度检测模型；以及
检测模块，用于根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度。
此外，为实现上述目的，本申请还提出一种水果成熟度检测设备，所述水果成熟度检测设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的水果成熟度检测程序，所述水果成熟度检测程序配置为实现如上文所述的水果成熟度检测方法。
此外，为实现上述目的，本申请还提出一种存储介质，所述存储介质上存储有水果成熟度检测程序，所述水果成熟度检测程序被处理器执行时实现如上文所述的水果成熟度检测方法。
 有益效果
本申请通过获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；根据所述水果种类确定相应的成熟度检测模型；以及根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度，基于音频数据检测水果的成熟度无需借助较高的计算能力，并且基于时间窗口所提取到音频特征能够提高成熟度检测的精度。
 附图说明
图1是本申请实施例方案涉及的硬件运行环境的水果成熟度检测设备的结构示意图；
图2为本申请水果成熟度检测方法第一实施例的流程示意图；
图3为本申请水果成熟度检测方法一实施例中音频片段截取示意图；
图4为本申请水果成熟度检测方法第二实施例的流程示意图；
图5为本申请水果成熟度检测方法一实施例中未进行人声去除的原始音频示意图；
图6为本申请水果成熟度检测方法一实施例中进行人声去除的敲击音频示意图；
图7为本申请水果成熟度检测方法第三实施例的流程示意图；
图8为本申请水果成熟度检测方法一实施例中成熟度分值展示的示意图；
图9为本申请水果成熟度检测装置第一实施例的结构框图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
 本发明的实施方式
当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
参照图1，图1为本申请实施例方案涉及的硬件运行环境的水果成熟度检测设备结构示意图。
如图1所示，该水果成熟度检测设备可以包括：处理器1001，例如中央处理器（Central Processing Unit，CPU），通信总线1002、用户接口1003，网络接口1004，存储器1005。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏（Display）、输入单元比如键盘（Keyboard），可选用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可选的可以包括标准的有线接口、无线接口（如无线保真（Wireless-Fidelity，Wi-Fi）接口）。存储器1005可以是高速的随机存取存储器（Random Access Memory，RAM）存储器，也可以是稳定的非易失性存储器（Non-Volatile Memory，NVM），例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。
本领域技术人员可以理解，图1中示出的结构并不构成对水果成熟度检测设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图1所示，作为一种存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及水果成熟度检测程序。
在图1所示的水果成熟度检测设备中，网络接口1004主要用于与网络服务器进行数据通信；用户接口1003主要用于与用户进行数据交互；本申请水果成熟度检测设备中的处理器1001、存储器1005可以设置在水果成熟度检测设备中，所述水果成熟度检测设备通过处理器1001调用存储器1005中存储的水果成熟度检测程序，并执行本申请实施例提供的水果成熟度检测方法。
本申请实施例提供了一种水果成熟度检测方法，参照图2，图2为本申请一种水果成熟度检测方法第一实施例的流程示意图。
本实施例中，所述水果成熟度检测方法包括以下步骤：
步骤S10：获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类。
在本实施例中，本实施例的执行主体可以是水果成熟度检测设备，水果成熟度检测设备可以是个人电脑或服务器等电子设备，还可以为其他可实现相同或相似功能的控制器，本实施例对此不加以限制，在本实施例及下述各实施例中，以水果成熟度检测设备为例对本申请水果成熟度检测方法进行说明。
需要说明的是，待检测水果为需要进行成熟度检测的水果，待检测水果可以为西瓜或香瓜等可以通过敲击判断成熟度的水果，现有技术中是通过摄像装置获取待检测水果的图像信息，以及拾音设备获取待检测水果的敲击声，结合图像信息和敲击声对待检测水果进行成熟度检测，现有技术中需要将图像信息与敲击声进行结合，所采取的是需要较高计算能力的深度学习模型，并且主要以图像信息为主，通过图像信息中所包含的待检测水果的颜色、体积大小以及纹路等图像特征确定待检测水果的成熟度，计算要求较高，并且以图像信息为主的检测方式所得到的成熟度检测结果不够准确。本实施例中在对待检测水果进行成熟度检测时，无需获取待检测水果的图像信息，通过获取到的待检测水果对应的敲击声即可实现成熟度检测，通过敲击声进行的成熟度检测精确度较高，并且由于不涉及图像信息的处理，大大降低了对模型计算能力的需求。
在具体实施中，以水果成熟度检测设备为移动终端为例进行说明，移动终端包括手机或平板等设备，移动终端中安装有应用程序，用户通过操作该应用程序开启移动终端的录音功能，在用户或其他人对待检测水果进行敲击时，通过移动终端的录音功能获取相应的敲击音频数据，在获取到敲击音频数据之后关闭移动终端的录音功能，其中，录音功能的开启时长可以根据实际检测需求进行相应地设置，本实施例中对此不加以限制。进一步地，在获取敲击音频数据之后，根据用户所输入的水果信息确定待检测水果对应的水果种类，例如西瓜或香瓜等。此外，本实施例中还可以为了简化用户的操作，通过应用程序开启移动终端摄像头，利用摄像头拍摄待检测水果的图像信息，基于图像信息中所包含的待检测水果的纹路特征和颜色特征确定待检测水果的种类，具体方式可以根据检测需求进行相应地选择，本实施例中对此不加以限制。
需要说明的是，所采集的水果敲击声会受到多种因素的影响，即使对于同一个水果来说，如果敲击的力度、敲击的手法以及敲击的部位不同，所采集到的敲击声音都会不同。并且，不同移动终端之间的录音元件或系统设定有所不同，采集的音频参数也会有所区别，因此需要对所有采集的音频进行再采样，统一至相同的采样率。本实施例中为了筛选出同一采样频率的敲击声，提高检测准确性，可以按照如下方式实现。
在具体实现中，先采集若干个待检测水果的敲击音频数据，然后检测各个敲击音频数据对应的时刻并进行相应的时刻标记，从而得到多个时刻下待检测水果的参考敲击音频数据，该时刻为接收到音频数据的时刻，然后按照预设采样频率筛选出处于同一采样频率下的时刻，例如假设预设频率为T，获取到T 0时刻、T 1时刻、T 2时刻以及T 3时刻的参考敲击音频，又假设T 1-T 0=T，T 2-T 1小于T，T 3-T 2小于T，T 3-T 1=T，可以得到T 0时刻、T 1时刻以及T 3时刻处于同一采样频率，T 0时刻、T 1时刻以及T 3时刻对应的参考音频数据即可作为待检测水果的音频数据。最后将处于同一采样频率下的时刻所对应的参考敲击音频数据作为待检测水果的敲击音频数据，即可保证所获取到的敲击音频数据处于同一采样频率，其中，预设采样频率可以根据实际音频采样需求进行相应地设置，本实施例中对此不加以限制。
步骤S20：按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段。
需要说明的是，音频当中存在次数不等的敲击声，且敲击声所占整个音频的时长较短，在特征提取前需要对音频数据进行分割处理，准确区分出敲击时段，才能正确获取其音频特征。本实施例中按照预设时间窗口进行敲击音频片段的截取，如图3所示，通过预设时间窗口可以截取到生水果敲击音频片段和熟水果敲击音频片段，图3中A和B为预设时间窗口对应的两个时间点，其中，预设时间窗口可以根据实际需求进行相应地设置，本实施例中对此不加以限制。
步骤S30：从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征。
需要说明的是，水果敲击声相对与外界噪声具有短时间内变化快，响度高和周期性弱的特点，因此在时间域上其音频的短时能量、均方根能量、过零率和响度标准差等特征较为显著。而成熟水果因为其水分充沛，敲击声音相较生水果更为沉闷，其频率更低，在频域上其音频的频谱质心、声谱衰减和梅尔频率倒谱系数等特征较为明显，基于敲击音频片段可以提取出上述音频特征。
步骤S40：根据所述水果种类确定相应的成熟度检测模型。
需要说明的是，现有技术中采用深度学习算法建立模型，计算能力需求高，建模过程中存在黑箱问题，本实施例中利用机器学习算法替换现有技术中所采用的深度学习算法。成熟度检测模型是经过大量的样本数据进行训练得到的，不同种类的水果所得到的样本数据是不同的，因此所得到的成熟度检测模型也是不同的，基于水果种类与成熟度检测模型之间的对应关系可以获取到与水果种类匹配的成熟度检测模型，以提高检测的精度。本实施例中成熟度检测模型存储在服务器中，服务器与移动终端通过互联网形式建立通信连接，移动终端在进行成熟度检测时，可从服务器中获取到成熟度检测模型，本实施例中的服务器包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云服务器，其中，云服务器由基于云计算(Cloud Computing)的大量计算机或网络服务器构成。
步骤S50：根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度
在具体实施中，在确定成熟度检测模型和敲击音频数据之后，将敲击音频数据输入至成熟度检测模型中，成熟度检测模型根据输入的敲击音频数据即可输出待检测水果所对应的成熟度。
本实施例通过获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；根据所述水果种类确定相应的成熟度检测模型；以及根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度，基于音频数据检测水果的成熟度无需借助较高的计算能力，并且基于时间窗口所提取到音频特征能够提高成熟度检测的精度。
参考图4，图4为本申请一种水果成熟度检测方法第二实施例的流程示意图。
基于上述第一实施例，本实施例水果成熟度检测方法在所述步骤S40之前，还包括：
步骤S401：获取所述水果种类对应的音频数据。
需要说明的是，在通过成熟度检测模型进行成熟度检测之前，需要先构建相应的成熟度检测模型，而模型的构建需要大量的样本数据，本实施例中通过获取该水果种类所对应的音频数据得到模型构建所需要的样本数据。本实施例中音频数据的获取采取大数据的方式，在获取音频数据时，为保证样本数据之间的差异性，设置水果品种、水果体积大小、敲击者性别、敲击位置及敲击手法等多个变量来进行音频数据的获取，当然，变量的设置也可以根据实际模型构建需求进行相应地调整，本实施例中对此不加以限制。
步骤S402：根据所述音频数据构建训练样本数据集。
需要说明的是，在得到音频数据之后，可以采取对音频数据进行数据整合、划分以及分组的方式对音频数据进行处理，从而得到可以用于模型训练的训练样本数据集。
进一步地，本实施例中为了提高成熟度检测模型的精度，可以按照如下方式构建训练样本数据集。
在具体实现中，水果受到敲击过程中，人声是主要的影响因素，人声的存在会极大程度地影响水果敲击声音的截取及音频特征的提取，对于实际的敲击声音来说，人声相当于加性噪声。因此，在特征提取前需要对音频降噪即人声的分离滤除，分离所采取的方法包括但不限于使用Deezer开源的Spleeter模型，该模型可以将人声和背景声进行分离，敲击音频即为背景乐声，将二者分离后即可达到降噪的目的。如图5，图5为未进行人声过滤的袁术音频数据，通过人声分离之后，可以得到图6所示的音频数据，也即敲击音频数据。
进一步地，在得到敲击音频数据之后，基于敲击音频数据所对应的标签可以将敲击音频数据划分为成熟水果敲击音频数据和生水果敲击音频数据，本实施例中训练样本数据集包括正样本数据集以及负样本数据集，其中，负样本数据集为生水果敲击音频数据，正样本数据集为成熟水果敲击音频数据作为正样本数据集。本实施例中的标签包括水果的颜色、进食的甜度及口感等，标签为用户所设置的，标签的分类设置可以根据实际需求进行调整，本实施例中对此不加以限制。
步骤S403：根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型。
需要说明的是，本实施例中所采用的模型为机器学习模型，预设机器学习模型所采用的模型类型可以根据实际训练需求进行相应地设置，本实施例中对此不加以限制。进一步地，在得到训练样本数据之后，可以利用XGBoost算法来训练预设机器学习模型，XGBoost是一种基于决策树的可扩展监督学习算法，模型的训练包括参数调整，本实施例中可以结合K折交叉验证与网格搜索算法对预设机器学习模型的参数进行调整，模型的生成实际基于Flask框架、Gunicorn HTTP服务器以及Nginx负载均衡，需要强调的是，对于上述算法、框架以及服务器的选择可以根据实际模型构建需求进行相应地调整，本实施例中对此不加以限制。
进一步地，本实施例中为了更加准确地获取敲击音频数据的音频特征，可以按照如下方式实现。
需要说明的是，音频当中存在次数不等的敲击声，且敲击声所占整个音频的时长较短，在特征提取前需要对音频数据进行分割处理，准确区分出敲击时段，才能正确获取其音频特征。本实施例中按照预设时间窗口进行敲击音频片段的截取，如图3所示，通过预设时间窗口可以截取到生水果敲击音频片段和熟水果敲击音频片段，图3中A和B为预设时间窗口对应的两个时间点，然后基于所截取的生水果敲击音频片段和熟水果敲击音频片段可以得到生水果敲击时的敲击音频特征，以及成熟水果敲击时的敲击音频特征，最后构建生敲击音频特征向量和成熟水果敲击音频特征向量，并将生敲击音频特征向量和成熟水果敲击音频特征向量输入至预设机器模型中，以对预设机器模型的模型参数进行调整，调整后得到的目标模型参数作为预设机器学习模型的当前模型参数，即可得到成熟度检测模型。
本实施例通过获取所述水果种类对应的音频数据；根据所述音频数据构建训练样本数据集；根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型，通过对机器学习模型进行训练得到成熟度检测模型，降低了成熟度检测对于计算能力的需求，同时也提高了成熟度检测的准确性。
参考图7，图7为本申请一种水果成熟度检测方法第三实施例的流程示意图。
基于上述第一实施例或第二实施例，提出本申请一种水果成熟度检测方法的第三实施例。
以基于上述第一实施例为例进行说明，本实施例中所述步骤S20包括：
步骤S201：获取所述敲击音频数据对应的音频片段的起始时刻。
步骤S202：根据所述起始时刻和预设时间窗口确定目标结束时刻。
需要说明的是，敲击音频数据中包含有多个音频片段，如图3所示，每个音频片段都具有相应的起始时刻，例如图3中的A点，即为该音频片段的起始时刻，本实施例中为了更准确地获取到敲击音频特征，需要从音频片段中截取出相应的音频片段。容易理解的是，音频片段包括起始时刻和结束时刻，根据起始时刻和结束时刻可以确定该音频片段的时长，不同的时长影响音频特征的提取结果，本实施例中将预设时间窗口作为敲击音频片段对应的时长，将起始时刻加上预设时间窗口可以得到敲击音频片段对应的结束时刻，即目标结束时刻，其中，预设时间窗口可以根据实际需求进行相应地设置，本实施例中对此不加以限制。
步骤S203：按照所述起始时刻和所述目标结束时刻从所述音频片段中截取相应的敲击音频片段。
在具体实施中，如图3所示，以图3为例进行说明，根据A点对应的起始时刻，然后结合预设时间窗口可以确定目标结束时刻，也即B点对应的结束时刻，然后按照A点对应的起始时刻和B点对应的结束时刻可以截取出AB敲击音频片段。
进一步地，本实施例中所述步骤S50之后还包括：
步骤S60：根据检测的成熟度生成所述待检测水果的成熟度分值。
步骤S70：对所述成熟度分值进行展示。
在具体实施中，本实施例中在检测到待检测水果为成熟水果还是生水果之后，还能够根据待检测水果的成熟度确定相应的成熟度分值，为了使用户更加直观了解到水果的成熟情况以及成熟品质等，本实施例中通过图形化的方式将成熟度分值展示给用户，如图8所示。
本实施例通过获取所述敲击音频数据对应的音频片段的起始时刻，根据所述起始时刻和预设时间窗口确定目标结束时刻，按照所述起始时刻和所述目标结束时刻从所述音频片段中截取相应的敲击音频片段，能够基于预设时间窗口获取准确的敲击音频片段，提高成熟度检测的准确性，同时还能够通过评分直观展示出水果的成熟情况以及相应的成熟品质，提高用户体验。
此外，本申请实施例还提出一种存储介质，所述存储介质上存储有水果成熟度检测程序，所述水果成熟度检测程序被处理器执行时实现如上文所述的水果成熟度检测方法的步骤。
由于本存储介质采用了上述所有实施例的全部技术方案，因此至少具有上述实施例的技术方案所带来的所有有益效果，在此不再一一赘述。
参照图9，图9为本申请水果成熟度检测装置第一实施例的结构框图。
如图9所示，本申请实施例提出的水果成熟度检测装置包括：
获取模块10，用于获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类。
在本实施例中，本实施例的执行主体可以是水果成熟度检测装置，水果成熟度检测装置可以是个人电脑或服务器等电子设备，还可以为其他可实现相同或相似功能的控制器，本实施例对此不加以限制，在本实施例及下述各实施例中，以水果成熟度检测装置为例对本申请水果成熟度检测方法进行说明。
需要说明的是，待检测水果为需要进行成熟度检测的水果，待检测水果可以为西瓜或香瓜等可以通过敲击判断成熟度的水果，现有技术中是通过摄像装置获取待检测水果的图像信息，以及拾音设备获取待检测水果的敲击声，结合图像信息和敲击声对待检测水果进行成熟度检测，现有技术中需要将图像信息与敲击声进行结合，所采取的是需要较高计算能力的深度学习模型，并且主要以图像信息为主，通过图像信息中所包含的待检测水果的颜色、体积大小以及纹路等图像特征确定待检测水果的成熟度，计算要求较高，并且以图像信息为主的检测方式所得到的成熟度检测结果不够准确。本实施例中在对待检测水果进行成熟度检测时，无需获取待检测水果的图像信息，通过获取到的待检测水果对应的敲击声即可实现成熟度检测，通过敲击声进行的成熟度检测精确度较高，并且由于不涉及图像信息的处理，大大降低了对模型计算能力的需求。
在具体实施中，以水果成熟度检测设备为移动终端为例进行说明，移动终端包括手机或平板等设备，移动终端中安装有应用程序，用户通过操作该应用程序开启移动终端的录音功能，在用户或其他人对待检测水果进行敲击时，通过移动终端的录音功能获取相应的敲击音频数据，在获取到敲击音频数据之后关闭移动终端的录音功能，其中，录音功能的开启时长可以根据实际检测需求进行相应地设置，本实施例中对此不加以限制。进一步地，在获取敲击音频数据之后，根据用户所输入的水果信息确定待检测水果对应的水果种类，例如西瓜或香瓜等。此外，本实施例中还可以为了简化用户的操作，通过应用程序开启移动终端摄像头，利用摄像头拍摄待检测水果的图像信息，基于图像信息中所包含的待检测水果的纹路特征和颜色特征确定待检测水果的种类，具体方式可以根据检测需求进行相应地选择，本实施例中对此不加以限制。
需要说明的是，所采集的水果敲击声会受到多种因素的影响，即使对于同一个水果来说，如果敲击的力度、敲击的手法以及敲击的部位不同，所采集到的敲击声音都会不同。并且，不同移动终端之间的录音元件或系统设定有所不同，采集的音频参数也会有所区别，因此需要对所有采集的音频进行再采样，统一至相同的采样率。本实施例中为了筛选出同一采样频率的敲击声，提高检测准确性，可以按照如下方式实现。
在具体实现中，先采集若干个待检测水果的敲击音频数据，然后检测各个敲击音频数据对应的时刻并进行相应的时刻标记，从而得到多个时刻下待检测水果的参考敲击音频数据，该时刻为接收到音频数据的时刻，然后按照预设采样频率筛选出处于同一采样频率下的时刻，例如假设预设频率为T，获取到T 0时刻、T 1时刻、T 2时刻以及T 3时刻的参考敲击音频，又假设T 1-T 0=T，T 2-T 1小于T，T 3-T 2小于T，T 3-T 1=T，可以得到T 0时刻、T 1时刻以及T 3时刻处于同一采样频率，T 0时刻、T 1时刻以及T 3时刻对应的参考音频数据即可作为待检测水果的音频数据。最后将处于同一采样频率下的时刻所对应的参考敲击音频数据作为待检测水果的敲击音频数据，即可保证所获取到的敲击音频数据处于同一采样频率，其中，预设采样频率可以根据实际音频采样需求进行相应地设置，本实施例中对此不加以限制。
截取模块20，用于按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段。
需要说明的是，音频当中存在次数不等的敲击声，且敲击声所占整个音频的时长较短，在特征提取前需要对音频数据进行分割处理，准确区分出敲击时段，才能正确获取其音频特征。本实施例中按照预设时间窗口进行敲击音频片段的截取，如图3所示，通过预设时间窗口可以截取到生水果敲击音频片段和熟水果敲击音频片段，图3中A和B为预设时间窗口对应的两个时间点，其中，预设时间窗口可以根据实际需求进行相应地设置，本实施例中对此不加以限制。
提取模块30，用于从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征。
需要说明的是，水果敲击声相对与外界噪声具有短时间内变化快，响度高和周期性弱的特点，因此在时间域上其音频的短时能量、均方根能量、过零率和响度标准差等特征较为显著。而成熟水果因为其水分充沛，敲击声音相较生水果更为沉闷，其频率更低，在频域上其音频的频谱质心、声谱衰减和梅尔频率倒谱系数等特征较为明显，基于敲击音频片段可以提取出上述音频特征。
构建模块40，用于根据所述水果种类确定相应的成熟度检测模型。
需要说明的是，现有技术中采用深度学习算法建立模型，计算能力需求高，建模过程中存在黑箱问题，本实施例中利用机器学习算法替换现有技术中所采用的深度学习算法。成熟度检测模型是经过大量的样本数据进行训练得到的，不同种类的水果所得到的样本数据是不同的，因此所得到的成熟度检测模型也是不同的，基于水果种类与成熟度检测模型之间的对应关系可以获取到与水果种类匹配的成熟度检测模型，以提高检测的精度。本实施例中成熟度检测模型存储在服务器中，服务器与移动终端通过互联网形式建立通信连接，移动终端在进行成熟度检测时，可从服务器中获取到成熟度检测模型，本实施例中的服务器包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云服务器，其中，云服务器由基于云计算(Cloud Computing)的大量计算机或网络服务器构成。
检测模块50，用于根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度。
在具体实施中，在确定成熟度检测模型和敲击音频数据之后，将敲击音频数据输入至成熟度检测模型中，成熟度检测模型根据输入的敲击音频数据即可输出待检测水果所对应的成熟度。
本实施例通过获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；根据所述水果种类确定相应的成熟度检测模型；以及根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度，基于音频数据检测水果的成熟度无需借助较高的计算能力，并且基于时间窗口所提取到音频特征能够提高成熟度检测的精度。
应当理解的是，以上仅为举例说明，对本申请的技术方案并不构成任何限定，在具体应用中，本领域的技术人员可以根据需要进行设置，本申请对此不做限制。
需要说明的是，以上所描述的工作流程仅仅是示意性的，并不对本申请的保护范围构成限定，在实际应用中，本领域的技术人员可以根据实际的需要选择其中的部分或者全部来实现本实施例方案的目的，此处不做限制。
另外，未在本实施例中详尽描述的技术细节，可参见本申请任意实施例所提供的水果成熟度检测方法，此处不再赘述。
此外，需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质（如只读存储器（Read Only Memory，ROM）/RAM、磁碟、光盘）中，包括若干指令用以使得一台终端设备（可以是手机，计算机，服务器，或者网络设备等）执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种水果成熟度检测方法，其中，所述水果成熟度检测方法包括：
    获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；
    按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；
    从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；
    根据所述水果种类确定相应的成熟度检测模型；以及
    根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度。
  2. 如权利要求1所述的水果成熟度检测方法，其中，所述根据所述水果种类确定相应的成熟度检测模型之前，还包括：
    获取所述水果种类对应的音频数据；
    根据所述音频数据构建训练样本数据集；以及
    根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型。
  3. 如权利要求2所述的水果成熟度检测方法，其中，所述根据所述音频数据构建训练样本数据集，包括：
    对所述音频数据进行人声分离，以得到敲击音频数据；
    基于所述敲击音频数据对应的标签对所述敲击音频数据进行划分，以得到成熟水果敲击音频数据和生水果敲击音频数据；
    将所述生水果敲击音频数据作为负样本数据集，将所述成熟水果敲击音频数据作为正样本数据集；以及
    根据所述负样本数据集和所述正样本数据集构建训练样本数据集。
  4. 如权利要求3所述的水果成熟度检测方法，其中，所述根据所述训练样本数据集对预设机器学习模型进行训练，以得到成熟度检测模型，包括：
    按照预设时间窗口从所述负样本数据集中获取生水果敲击音频片段，以及从所述正样本数据集中获取熟水果敲击音频片段；
    从所述生水果敲击音频片段中提取生水果敲击音频特征，以及从所述熟水果敲击音频片段提取熟水果敲击音频特征；
    根据所述生水果敲击音频特征和所述熟水果敲击音频特征对预设机器学习模型的模型参数进行调整，以得到目标模型参数；以及
    将所述目标模型参数输入至所述预设机器学习模型，以得到成熟度检测模型。
  5. 如权利要求1所述的水果成熟度检测方法，其中，所述按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段，包括：
    获取所述敲击音频数据对应的音频片段的起始时刻；
    根据所述起始时刻和预设时间窗口确定目标结束时刻；以及
    按照所述起始时刻和所述目标结束时刻从所述音频片段中截取相应的敲击音频片段。
  6. 如权利要求1所述的水果成熟度检测方法，其中，所述获取待检测水果对应的敲击音频数据，包括：
    获取待检测水果在多个时刻的参考敲击音频数据；以及
    从多个所述时刻的参考敲击音频数据中筛选出符合预设采样频率的参考敲击音频数据，将所述符合预设采样频率的参考敲击音频数据作为所述待检测水果对应的敲击音频数据。
  7. 如权利要求1至6中任一项所述的水果成熟度检测方法，其中，所述根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度之后，还包括：
    根据检测的成熟度生成所述待检测水果的成熟度分值；以及
    对所述成熟度分值进行展示。
  8. 一种水果成熟度检测装置，其中，所述水果成熟度检测装置包括：
    获取模块，用于获取待检测水果对应的敲击音频数据，并获取所述待检测水果的水果种类；
    截取模块，用于按照预设时间窗口从所述敲击音频数据中截取相应的敲击音频片段；
    提取模块，用于从截取到的敲击音频片段中提取待检测水果对应的敲击音频特征；
    构建模块，用于根据所述水果种类确定相应的成熟度检测模型；以及
    检测模块，用于根据所述敲击音频特征通过所述成熟度检测模型检测所述待检测水果的成熟度。
  9. 一种水果成熟度检测设备，其中，所述水果成熟度检测设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的水果成熟度检测程序，所述水果成熟度检测程序配置为实现如权利要求1至7中任一项所述的水果成熟度检测方法的步骤。
  10. 一种存储介质，其中，所述存储介质上存储有水果成熟度检测程序，所述水果成熟度检测程序被处理器执行时实现如权利要求1至7任一项所述的水果成熟度检测方法的步骤。
PCT/CN2021/141480 2021-08-31 2021-12-27 水果成熟度检测方法、装置、设备及存储介质 WO2023029311A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111019373.XA CN115728382A (zh) 2021-08-31 2021-08-31 水果成熟度检测方法、装置、设备及存储介质
CN202111019373.X 2021-08-31

Publications (1)

Publication Number Publication Date
WO2023029311A1 true WO2023029311A1 (zh) 2023-03-09

Family

ID=85291987

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/141480 WO2023029311A1 (zh) 2021-08-31 2021-12-27 水果成熟度检测方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN115728382A (zh)
WO (1) WO2023029311A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117969670A (zh) * 2024-04-02 2024-05-03 湖南大学 一种基于声学特征的西瓜成熟度快速无损检测方法及系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07239320A (ja) * 1994-02-28 1995-09-12 Shizuoka Seiki Co Ltd 果実及び果菜類熟度判定装置
GB9704908D0 (en) * 1997-03-10 1997-04-30 Imperial College Method and apparatus for assessing the ripeness or firmness of fruit and vegetables
KR20040000363A (ko) * 2003-09-03 2004-01-03 주식회사 한성엔지니어링 수박의 숙성도 및 당도 검출장치
TWM439355U (en) * 2012-05-17 2012-10-21 Univ Ching Yun Identification device for texture of melon and fruit
CN104359978A (zh) * 2014-11-09 2015-02-18 兰州交通大学 一种西瓜测熟器
CN111141823A (zh) * 2020-01-13 2020-05-12 石河子大学 一种基于智能手机声信号的哈密瓜成熟度快速检测方法
CN111640451A (zh) * 2020-05-07 2020-09-08 Oppo广东移动通信有限公司 一种成熟度评估方法及装置、存储介质
CN112529152A (zh) * 2020-12-03 2021-03-19 开放智能机器（上海）有限公司 一种基于人工智能的西瓜成熟度检测的系统和方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07239320A (ja) * 1994-02-28 1995-09-12 Shizuoka Seiki Co Ltd 果実及び果菜類熟度判定装置
GB9704908D0 (en) * 1997-03-10 1997-04-30 Imperial College Method and apparatus for assessing the ripeness or firmness of fruit and vegetables
KR20040000363A (ko) * 2003-09-03 2004-01-03 주식회사 한성엔지니어링 수박의 숙성도 및 당도 검출장치
TWM439355U (en) * 2012-05-17 2012-10-21 Univ Ching Yun Identification device for texture of melon and fruit
CN104359978A (zh) * 2014-11-09 2015-02-18 兰州交通大学 一种西瓜测熟器
CN111141823A (zh) * 2020-01-13 2020-05-12 石河子大学 一种基于智能手机声信号的哈密瓜成熟度快速检测方法
CN111640451A (zh) * 2020-05-07 2020-09-08 Oppo广东移动通信有限公司 一种成熟度评估方法及装置、存储介质
CN112529152A (zh) * 2020-12-03 2021-03-19 开放智能机器（上海）有限公司 一种基于人工智能的西瓜成熟度检测的系统和方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117969670A (zh) * 2024-04-02 2024-05-03 湖南大学 一种基于声学特征的西瓜成熟度快速无损检测方法及系统

Also Published As

Publication number Publication date
CN115728382A (zh) 2023-03-03

Similar Documents

Publication Publication Date Title
WO2021208287A1 (zh) 用于情绪识别的语音端点检测方法、装置、电子设备及存储介质
WO2020173133A1 (zh) 情感识别模型的训练方法、情感识别方法、装置、设备及存储介质
WO2019127924A1 (zh) 样本权重分配方法、模型训练方法、电子设备及存储介质
WO2017101506A1 (zh) 信息处理方法及装置
WO2023035923A1 (zh) 一种视频审核方法、装置及电子设备
AU2017404565A1 (en) Electronic device, method and system of identity verification and computer readable storage medium
WO2022089170A1 (zh) 字幕区域识别方法、装置、设备及存储介质
WO2020233360A1 (zh) 一种产品测评模型的生成方法及设备
US11511200B2 (en) Game playing method and system based on a multimedia file
WO2023138549A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN109286848B (zh) 一种终端视频信息的交互方法、装置及存储介质
WO2023029311A1 (zh) 水果成熟度检测方法、装置、设备及存储介质
CN113033245A (zh) 一种功能调节方法、装置、存储介质及电子设备
CN110931019B (zh) 公安语音数据采集方法、装置、设备和计算机存储介质
CN109300484B (zh) 音频对齐方法、装置、计算机设备以及可读存储介质
CN114065720A (zh) 会议纪要生成方法、装置、存储介质及电子设备
CN112423019B (zh) 调整音频播放速度的方法、装置、电子设备及存储介质
Zhang et al. Songnet: Real-time music classification
CN110070891B (zh) 一种歌曲识别方法、装置以及存储介质
CN113035238B (zh) 音频评测方法、装置、电子设备和介质
US20230093165A1 (en) Information processing apparatus, information processing method, and program
CN112885380B (zh) 一种清浊音检测方法、装置、设备及介质
CN104281682A (zh) 文件分类系统及方法
CN112581937A (zh) 一种语音指令的获得方法及装置
EP3671735B1 (en) Method and system for determining speaker-user of voice-controllable device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21955839

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE