EP3677350A1 - Grayscale state sequence based feed sorting device - Google Patents
Grayscale state sequence based feed sorting device Download PDFInfo
- Publication number
- EP3677350A1 EP3677350A1 EP17926030.2A EP17926030A EP3677350A1 EP 3677350 A1 EP3677350 A1 EP 3677350A1 EP 17926030 A EP17926030 A EP 17926030A EP 3677350 A1 EP3677350 A1 EP 3677350A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- grayscale
- state sequence
- attitude data
- discharge
- preconfigured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/34—Sorting according to other particular properties
- B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/02—Measures preceding sorting, e.g. arranging articles in a stream orientating
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/36—Sorting apparatus characterised by the means used for distribution
- B07C5/361—Processing or control devices therefor, e.g. escort memory
- B07C5/362—Separating or distributor mechanisms
Definitions
- the present invention relates to the field of automatic production technologies, and in particular, to a grayscale state sequence based feed sorting device.
- Automatic feeding is the first section for implementing production automation, and efficient and reliable operation of a feed device is a necessary condition for successfully completing automatic production.
- An automatic production line requires that the feed device feeds materials to a loading station with a constant speed and a uniquely determined attitude.
- raw materials are usually unordered materials that may be mixed with defects or materials of another type.
- the feed device needs to screen raw material states to meet a requirement of the loading station; on the other hand, the feed device further needs to remove the defects or foreign materials in the raw materials.
- the core for achieving the objective of automatic feeding is a material sorting technology.
- the material sorting technology is a technology of sorting different products or products of different grades according to a certain standard or requirement to meet different needs. Material sorting can not only improve quality and utility of products, but also adapt to different needs of different users, thereby achieving rational, economic and effective use of resources, and reducing environmental pollution and ecological damage. In this era with rapid economic development and diversified subdivided needs, resources are gradually consumed and the environmental problem is becoming severer. Material sorting, as a main section during material circulation processing, is of great significance economically and ecologically.
- sorting methods are mainly classified into screening, magnetic separation, optical separation, electric separation, flotation separation and other sorting methods.
- the optical separation is a sorting method related to optical attributes of sorting objects, and materials to be sorted are identified according to optical attributes of the sorting objects, for example, light absorption, reflection, and scattering.
- the core of an optical sorting device is a photoelectric detection device and a corresponding data processing system.
- the photoelectric detection device in the optical sorting method is generally provided with an optical transmitting unit and an optical receiving unit.
- Main forms of the optical transmitting unit include a point light source, a line light source, or an area light source formed by a laser, an infrared ray, an ultraviolet ray, and other visible light of various wavelengths.
- the optical receiving unit mainly includes a point grayscale sensor, a linear array charge-coupled device, a grayscale or color image sensor, and the like.
- a Chinese patent application discloses a laser-transmission-based material sorting method and device.
- Materials to be sorted are irradiated by using a laser line source, the transmission light forms different intensity distribution at the back of the materials according to different transparent attributes and volume scattering of different materials, and a control system performs processing on the intensity distribution signal and compares the signal with reserved corresponding parameters of a material to be retained in the system, to determine whether the materials are foreign materials.
- the method is applicable to sorting of materials with very good transmission properties, such as raisins, vegetables, fruits, and marine products, but for non-transparent substances, it is relatively difficult to implement sorting.
- a Chinese patent application discloses a material sorting device and sorting method, and features of materials to be sorted are extracted by using dual infrared images of different wavelengths. In the method, different materials are distinguished according to infrared absorption differences between substances. Compared with an infrared image with a single wave band, more information is obtained from the dual infrared image in the method, thereby helping improve accuracy of material sorting. Although the method is applicable to most materials having infrared absorption differences, there are disadvantages of a low sorting speed and high system costs.
- a Chinese patent application (publication number: CN104390987A ) discloses a new optical fiber sensor and detection method for detecting a surface defect of a steel ball.
- a sensor probe is formed by using 19 fibers, and the surface defect of the steel ball is calculated by using photoelectric conversion signals from 6 channels.
- the method roughness and a displacement change defect on the surface of the steel ball may be accurately measured.
- a relatively large quantity of fibers are used, and operations of conversion and acquisition processing on signals from 6 channels need to be performed, it is difficult to apply the method to a material sorting system.
- a time sequence is a number sequence arranged in a time order.
- Time sequence analysis is a statistical method for dynamic data. The method is based on a random process theory and a mathematical statistical method, and resolve actual problems by studying statistical regularity of random data sequences.
- the time sequence analysis is a theory and method for establishing a mathematical model through curve fitting and parameter estimation according to time sequence data obtained through systematic observation.
- the time sequence analysis is usually applied to fields such as macro-control on the national economy, regional comprehensive development planning, enterprise operation and management, forecasting of market potentials, meteorological and hydrological forecasting, earthquake forecasting, forecasting of crop pest disasters, environmental pollution control, ecological balance, astronomy, and oceanography.
- the present invention provides a grayscale state sequence based feed sorting device, thereby overcoming defects in the prior art, and implementing material sorting with a high speed and a low error rate by detecting a grayscale state sequence of a material.
- each piece of the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data includes a plurality of grayscale states and a state transition sequence between grayscale states; and in a case that the grayscale state sequence of the material acquired by the grayscale sensor meets the state transition sequence in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data, it is determined that the grayscale state sequence of the material matches the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data.
- each grayscale state in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data includes a value range of a grayscale value and an occurrence frequency or duration of the grayscale value.
- the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data is obtained through learning by using the feed sorting device in advance, a material is enabled to move forward on the tray in a determined attitude, the grayscale sensor acquires a grayscale state sequence of the material within an entire detection area and sends the grayscale state sequence to the main controller, the main controller analyzes features of the grayscale state sequence to obtain grayscale states included in the material and a state transition sequence, that is, preconfigured standard attitude data, the preconfigured standard attitude data is stored in the main controller, and the standard attitude data is preconfigured as the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data by the user.
- the grayscale sensor is a photoelectric sensor and includes an optical transmitting unit and an optical receiving unit, where the optical transmitting unit transmits a modulated optical signal, and the optical receiving unit outputs a grayscale value of a material at a light spot detection position.
- a thru-beam-type photoelectric sensor is used as the photoelectric sensor; and if the material is non-transparent, a reflective photoelectric sensor is used as the photoelectric sensor.
- an emission light wavelength of the optical transmitting unit is adjusted according to a reflective attribute of the material.
- a plurality of pieces of standard discharge-allowed attitude data, a plurality of pieces of standard discharge-prohibited attitude data, and a plurality of treatment solutions for unmatched materials are preconfigured in the main controller, and a discharge attitude or a combination of discharge attitudes of a material is flexibly changed according to different requirements of the discharge station.
- grayscale state sequences of materials that are important or easy to be confused are pre-selected as feature templates, and operations of the rejection mechanism that respectively correspond to the feature templates are set, to help distinguish states with high resemblance; and for other grayscale state sequences that do not belong to the feature templates, treatment solutions are preconfigured, and corresponding operations of the rejection mechanism or a warning operation is performed according to the treatment solutions, thereby reducing a setting quantity and difficulty of the feature templates of materials whose states are easy to be distinguished.
- a grayscale state sequence based feed sorting device is designed. As shown in FIG. 1 , the device includes three parts: a grayscale sensor 3, a main controller 4, and a rejection mechanism 5, and in combination with a tray 1, a guide chute 2, and a discharge station 6, a programmable universal feed sorting devices is formed.
- a working process of the device is as follows: all materials are placed on the tray 1 and move forward along with the tray 1, the grayscale sensor 3 acquires a grayscale state sequence of a material and sends the grayscale state sequence to the main controller 4, and the main controller 4 determines whether the acquired grayscale state sequence is already identified.
- the main controller 4 matches the acquired grayscale state sequence of the material with preconfigured standard discharge attitude data; if the grayscale state sequence of the material matches the preconfigured standard discharge attitude data, it indicates that the material is in a target discharge attitude 7 and the material is allowed to pass through the rejection mechanism 5; and if the grayscale state sequence of the material does not match the preconfigured standard discharge attitude data, it indicates that the material is in a non-discharge attitude 8, and the main controller 4 controls the rejection mechanism 5 to return the material to an initial position of the tray 1; through falling of the material in the rejection mechanism 5 and vibration of the tray 1, the discharge attitude of the returned material is changed; and if the grayscale state sequence is not identified, the main controller 4 controls the tray 1 to stop operation, and simultaneously sends abnormal alarm information to a staff member. Along with operation of the feed sorting device, all materials finally reach the discharge station 6 in the target discharge attitude through the guide chute 2.
- the grayscale sensor 3 is a photoelectric sensor and includes an optical transmitting unit and an optical receiving unit, where the optical transmitting unit transmits a modulated optical signal, and the optical receiving unit outputs a grayscale value of a material at a light spot detection position. If the material is transparent or semitransparent, a thru-beam-type photoelectric sensor is used as the photoelectric sensor; and if the material is non-transparent, a reflective photoelectric sensor is used as the photoelectric sensor. In a case that a reflective photoelectric sensor is used as the photoelectric sensor, an emission light wavelength of the optical transmitting unit is adjusted according to a reflective attribute of the material, so that a reflective signal has better quality and a higher signal-to-noise ratio.
- the standard discharge attitude data is a state sequence including grayscale states corresponding to discharge attitudes of a material and a transition relationship between states.
- the state sequence is predesigned in the main controller, and transition conditions between all state transition sequences and states constitute a standard for determining a material type.
- the state sequence may be constituted by manually controlling the material to move with a suitable and constant speed in the target attitude in front of the grayscale sensor, and simultaneously controlling the main controller to perform sampling and learning. Therefore, the main controller needs to be provided with a learning control interface, for example, a learning key.
- the main controller After the learning key is pressed, the main controller enters into a learning state, the material in a correct attitude is enabled to pass through in front of the grayscale sensor with a constant speed, and after detecting that the material completely passes through the detection area, the main controller performs state transition on the acquired grayscale sequence. It is determined that data points of proximate grayscale values are a same grayscale state, thereby obtaining a state space including all grayscale states of attitudes of a target material.
- a target feature may be defined as a state sequence including a set of specified state transition sequences in the state space. S is used for representing a grayscale state, G is used for representing a grayscale value, and F is used for representing an occurrence frequency of a grayscale or duration of the grayscale.
- FIG. 2 is a diagram of transition of discharge attitude states of a single target material.
- a condition of entering an S 1 state is that: a grayscale value is between 0.7 and 0.9, and a continuous occurrence frequency of the grayscale value is equal to 4.
- a condition of entering an S2 state is that: a grayscale sequence in which the grayscale is in a range of 0.3 to 0.5 continuously occurs for three times before timeout.
- a condition of entering an S4 state is that the grayscale value is decreased to be less than 0.1 before timeout. An unconditional output result of S4 is true, and an unconditional output result of S3 is false.
- FIG. 3 shows four possible feed attitudes a, b, c, and d of the material A in FIG. 1 . From FIG. 1 , it can be seen that the discharge station requires a discharge attitude to be the attitude a in FIG. 3 .
- the main controller needs to calculate four grayscale sequences in FIG. 3 , separate the attitude a, and return materials of the attitudes b, c, and d to the tray through the rejection mechanism.
- Symbol G[x] is used for representing a grayscale sequence x. Through measurement, it is obtained that:
- a result obtained from state transition judgment performed on G[a] is S[a], and an output result is true.
- a grayscale sequence represented by G[b] does not meet the condition of entering S1, and the grayscale sequence thereof is empty, and therefore a result is empty.
- a grayscale sequence represented by G[c] may successfully enter the S1 state, but wait overtime in the S1 state, thereby entering S3, and a result is false.
- a judgment result of G[d] is the same as that of G[c].
- a material with an abnormal discharge attitude may be strictly screened out by using a state sequence and transition conditions between states.
- the main controller may process, in real time, grayscale data received by the grayscale sensor, without waiting until the entire material grayscale state sequence is acquired.
- Grayscale state sequence analysis is an almost real-time online data processing method. The entire process of grayscale sampling, feature identification, calculation and judgment, and responding has a small calculation burden, and may be completed before the material moves to the rejection device. Such a design has almost no impact on a discharge speed.
- learning may be performed again, to create a transition diagram of discharge attitude grayscale states of a new target material. This conversion operation can be completed by guiding a computing unit with a small amount of manual intervention.
- a grayscale state sequence based feed sorting device provided in the present invention provides a new idea for automatic feeding.
- Automatic feeding is a section of great significance during automatic production. It is difficult to universalize a conventional feed device because material objects are of various types and have greatly different features. As a result, it is expensive to customize a feed device. Therefore, it is of great economic value and practical significance to implement a programmable universal feed device with high efficiency and high reliability.
Landscapes
- Sorting Of Articles (AREA)
Abstract
Description
- The present invention relates to the field of automatic production technologies, and in particular, to a grayscale state sequence based feed sorting device.
- Automatic feeding is the first section for implementing production automation, and efficient and reliable operation of a feed device is a necessary condition for successfully completing automatic production. An automatic production line requires that the feed device feeds materials to a loading station with a constant speed and a uniquely determined attitude. However, raw materials are usually unordered materials that may be mixed with defects or materials of another type. On one hand, the feed device needs to screen raw material states to meet a requirement of the loading station; on the other hand, the feed device further needs to remove the defects or foreign materials in the raw materials. The core for achieving the objective of automatic feeding is a material sorting technology.
- The material sorting technology is a technology of sorting different products or products of different grades according to a certain standard or requirement to meet different needs. Material sorting can not only improve quality and utility of products, but also adapt to different needs of different users, thereby achieving rational, economic and effective use of resources, and reducing environmental pollution and ecological damage. In this era with rapid economic development and diversified subdivided needs, resources are gradually consumed and the environmental problem is becoming severer. Material sorting, as a main section during material circulation processing, is of great significance economically and ecologically.
- With the emergence of concepts such as Industry 4.0 and Made in China 2025, industrial automation has prospered, and the material sorting technology has also been gradually developed from inefficient manual sorting to automatic sorting. Compared with the conventional manual sorting, the automatic sorting technology has incomparable advantages in aspects such as efficiency and accuracy, and therefore is developing rapidly. Various physicochemical properties, such as classes and shapes, of materials to be sorted differ greatly. As a result, mechanisms of sorting processes vary enormously. Therefore, a specialized subject, namely, material sorting, has been developed.
- Based on the sorting mechanisms, sorting methods are mainly classified into screening, magnetic separation, optical separation, electric separation, flotation separation and other sorting methods. The optical separation is a sorting method related to optical attributes of sorting objects, and materials to be sorted are identified according to optical attributes of the sorting objects, for example, light absorption, reflection, and scattering. The core of an optical sorting device is a photoelectric detection device and a corresponding data processing system. The photoelectric detection device in the optical sorting method is generally provided with an optical transmitting unit and an optical receiving unit. Main forms of the optical transmitting unit include a point light source, a line light source, or an area light source formed by a laser, an infrared ray, an ultraviolet ray, and other visible light of various wavelengths. The optical receiving unit mainly includes a point grayscale sensor, a linear array charge-coupled device, a grayscale or color image sensor, and the like.
- A Chinese patent application (publication number:
CN102989693B ) discloses a laser-transmission-based material sorting method and device. Materials to be sorted are irradiated by using a laser line source, the transmission light forms different intensity distribution at the back of the materials according to different transparent attributes and volume scattering of different materials, and a control system performs processing on the intensity distribution signal and compares the signal with reserved corresponding parameters of a material to be retained in the system, to determine whether the materials are foreign materials. The method is applicable to sorting of materials with very good transmission properties, such as raisins, vegetables, fruits, and marine products, but for non-transparent substances, it is relatively difficult to implement sorting. A Chinese patent application (publication number:CN106269570A ) discloses a material sorting device and sorting method, and features of materials to be sorted are extracted by using dual infrared images of different wavelengths. In the method, different materials are distinguished according to infrared absorption differences between substances. Compared with an infrared image with a single wave band, more information is obtained from the dual infrared image in the method, thereby helping improve accuracy of material sorting. Although the method is applicable to most materials having infrared absorption differences, there are disadvantages of a low sorting speed and high system costs. A Chinese patent application (publication number:CN104390987A ) discloses a new optical fiber sensor and detection method for detecting a surface defect of a steel ball. In the optical fiber sensor, a sensor probe is formed by using 19 fibers, and the surface defect of the steel ball is calculated by using photoelectric conversion signals from 6 channels. In the method, roughness and a displacement change defect on the surface of the steel ball may be accurately measured. However, because a relatively large quantity of fibers are used, and operations of conversion and acquisition processing on signals from 6 channels need to be performed, it is difficult to apply the method to a material sorting system. - A time sequence is a number sequence arranged in a time order. Time sequence analysis is a statistical method for dynamic data. The method is based on a random process theory and a mathematical statistical method, and resolve actual problems by studying statistical regularity of random data sequences. The time sequence analysis is a theory and method for establishing a mathematical model through curve fitting and parameter estimation according to time sequence data obtained through systematic observation. The time sequence analysis is usually applied to fields such as macro-control on the national economy, regional comprehensive development planning, enterprise operation and management, forecasting of market potentials, meteorological and hydrological forecasting, earthquake forecasting, forecasting of crop pest disasters, environmental pollution control, ecological balance, astronomy, and oceanography.
- By summarizing the current sorting methods, it can be found that an image-based method is usually used to achieve relatively high sorting accuracy, but high-speed sorting cannot be implemented due to a limited image processing speed; and to implement high-speed sorting, a point photoelectric sensor is usually used to improve a sorting speed, but a sorting error rate is relatively high. Therefore, a material sorting method that can implement high-speed sorting and has a low error rate is required. Material grayscale state sequence analysis is a technical approach to achieve the objective.
- To resolve the technical problem put forward in the background technology above, the present invention provides a grayscale state sequence based feed sorting device, thereby overcoming defects in the prior art, and implementing material sorting with a high speed and a low error rate by detecting a grayscale state sequence of a material.
- In order to achieve the technical objective described above, a technical solution of the present invention is as follows:
- A grayscale state sequence based feed sorting device, including a grayscale sensor, a main controller, a rejection mechanism, a tray, and a guide chute, where all materials are placed on the tray and move forward along with the tray, the grayscale sensor acquires a grayscale state sequence of a material and sends the grayscale state sequence to the main controller, and the main controller performs identifies the grayscale state sequence;
- the main controller matches the grayscale state sequence of the material with preconfigured standard discharge-allowed attitude data and preconfigured standard discharge-prohibited attitude data; if the grayscale state sequence matches any one piece of the preconfigured standard discharge-allowed attitude data, it indicates that the material is in a target discharge attitude and the material is allowed to pass through the rejection mechanism; and if the grayscale state sequence matches the preconfigured standard discharge-prohibited attitude data, it indicates that the material is in a non-discharge attitude, and the main controller controls the rejection mechanism to return the material to an initial position of the tray; through a falling process in the rejection mechanism and vibration of the tray, the attitude of the returned material is changed and a system performs identification again when the material reaches the rejection mechanism again;
- if the grayscale state sequence does not match any piece of the standard discharge-allowed attitude data and the standard discharge-prohibited attitude data, the main controller performs a corresponding operation on the material in this condition according to a preconfigured treatment solution, where the treatment solution includes allowing passing through the rejection mechanism, prohibiting passing through the rejection mechanism, and shutdown warning; and
- along with operation of the feed sorting device, the material finally reaches a discharge station in the target discharge attitude through the guide chute, or an operation is completed according to a treatment solution preconfigured by a user.
- Further, each piece of the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data includes a plurality of grayscale states and a state transition sequence between grayscale states; and in a case that the grayscale state sequence of the material acquired by the grayscale sensor meets the state transition sequence in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data, it is determined that the grayscale state sequence of the material matches the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data.
- Further, each grayscale state in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data includes a value range of a grayscale value and an occurrence frequency or duration of the grayscale value.
- Further, the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data is obtained through learning by using the feed sorting device in advance, a material is enabled to move forward on the tray in a determined attitude, the grayscale sensor acquires a grayscale state sequence of the material within an entire detection area and sends the grayscale state sequence to the main controller, the main controller analyzes features of the grayscale state sequence to obtain grayscale states included in the material and a state transition sequence, that is, preconfigured standard attitude data, the preconfigured standard attitude data is stored in the main controller, and the standard attitude data is preconfigured as the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data by the user.
- Further, the grayscale sensor is a photoelectric sensor and includes an optical transmitting unit and an optical receiving unit, where the optical transmitting unit transmits a modulated optical signal, and the optical receiving unit outputs a grayscale value of a material at a light spot detection position.
- Further, if the material is transparent or semitransparent, a thru-beam-type photoelectric sensor is used as the photoelectric sensor; and if the material is non-transparent, a reflective photoelectric sensor is used as the photoelectric sensor.
- Further, in a case that a reflective photoelectric sensor is used as the photoelectric sensor, an emission light wavelength of the optical transmitting unit is adjusted according to a reflective attribute of the material.
- Further, a plurality of pieces of standard discharge-allowed attitude data, a plurality of pieces of standard discharge-prohibited attitude data, and a plurality of treatment solutions for unmatched materials are preconfigured in the main controller, and a discharge attitude or a combination of discharge attitudes of a material is flexibly changed according to different requirements of the discharge station.
- Further, grayscale state sequences of materials that are important or easy to be confused are pre-selected as feature templates, and operations of the rejection mechanism that respectively correspond to the feature templates are set, to help distinguish states with high resemblance; and for other grayscale state sequences that do not belong to the feature templates, treatment solutions are preconfigured, and corresponding operations of the rejection mechanism or a warning operation is performed according to the treatment solutions, thereby reducing a setting quantity and difficulty of the feature templates of materials whose states are easy to be distinguished.
- Beneficial effects achieved by using the technical solution described above:
- (1) In the present invention, a time sequence analysis method is applied to a material sorting system. When a material passes through a photoelectric detection area, a calculation unit performs discrete time sampling on a grayscale sensor to obtain a grayscale state sequence, and type information is obtained by analyzing and calculating a variation feature of the state sequence. Compared with a classification method based on a single grayscale value, the method can greatly improve the accuracy of material feature identification.
- (2) In the present invention, there is no need to stop in the photoelectric detection area during a feeding process, and compared with an image processing method, the grayscale state sequence feature identification method has a higher speed, thereby helping improve operation efficiency of a feed device.
- (3) The present invention is compatible to a thru-beam-type grayscale sensor and a reflective grayscale sensor, thereby meeting feed sorting requirements of transparent, semitransparent, and non-transparent materials.
-
-
FIG. 1 is a schematic structural diagram of the present invention; -
FIG. 2 is a diagram of grayscale state transition of material discharge according to an embodiment; and -
FIG. 3 is a schematic diagram of a grayscale sequence of different attitudes of a material A according to an embodiment. - Descriptions of reference numerals: 1. Tray; 2. Guide chute; 3. Grayscale sensor; 4. Main controller; 5. Rejection mechanism; 6. Discharge station; 7. Target discharge attitude; 8. Non-discharge attitude.
- Technical solutions of the present invention are described below in detail with reference to the accompanying drawings.
- In the present invention, a grayscale state sequence based feed sorting device is designed. As shown in
FIG. 1 , the device includes three parts: agrayscale sensor 3, amain controller 4, and a rejection mechanism 5, and in combination with atray 1, aguide chute 2, and a discharge station 6, a programmable universal feed sorting devices is formed. - A working process of the device is as follows: all materials are placed on the
tray 1 and move forward along with thetray 1, thegrayscale sensor 3 acquires a grayscale state sequence of a material and sends the grayscale state sequence to themain controller 4, and themain controller 4 determines whether the acquired grayscale state sequence is already identified. If the grayscale state sequence is already identified, themain controller 4 matches the acquired grayscale state sequence of the material with preconfigured standard discharge attitude data; if the grayscale state sequence of the material matches the preconfigured standard discharge attitude data, it indicates that the material is in atarget discharge attitude 7 and the material is allowed to pass through the rejection mechanism 5; and if the grayscale state sequence of the material does not match the preconfigured standard discharge attitude data, it indicates that the material is in anon-discharge attitude 8, and themain controller 4 controls the rejection mechanism 5 to return the material to an initial position of thetray 1; through falling of the material in the rejection mechanism 5 and vibration of thetray 1, the discharge attitude of the returned material is changed; and if the grayscale state sequence is not identified, themain controller 4 controls thetray 1 to stop operation, and simultaneously sends abnormal alarm information to a staff member. Along with operation of the feed sorting device, all materials finally reach the discharge station 6 in the target discharge attitude through theguide chute 2. - In this embodiment, the
grayscale sensor 3 is a photoelectric sensor and includes an optical transmitting unit and an optical receiving unit, where the optical transmitting unit transmits a modulated optical signal, and the optical receiving unit outputs a grayscale value of a material at a light spot detection position. If the material is transparent or semitransparent, a thru-beam-type photoelectric sensor is used as the photoelectric sensor; and if the material is non-transparent, a reflective photoelectric sensor is used as the photoelectric sensor. In a case that a reflective photoelectric sensor is used as the photoelectric sensor, an emission light wavelength of the optical transmitting unit is adjusted according to a reflective attribute of the material, so that a reflective signal has better quality and a higher signal-to-noise ratio. - To perform the grayscale state matching operation, standard discharge attitude data needs to be predefined. The standard discharge attitude data is a state sequence including grayscale states corresponding to discharge attitudes of a material and a transition relationship between states. The state sequence is predesigned in the main controller, and transition conditions between all state transition sequences and states constitute a standard for determining a material type. The state sequence may be constituted by manually controlling the material to move with a suitable and constant speed in the target attitude in front of the grayscale sensor, and simultaneously controlling the main controller to perform sampling and learning. Therefore, the main controller needs to be provided with a learning control interface, for example, a learning key. After the learning key is pressed, the main controller enters into a learning state, the material in a correct attitude is enabled to pass through in front of the grayscale sensor with a constant speed, and after detecting that the material completely passes through the detection area, the main controller performs state transition on the acquired grayscale sequence. It is determined that data points of proximate grayscale values are a same grayscale state, thereby obtaining a state space including all grayscale states of attitudes of a target material. A target feature may be defined as a state sequence including a set of specified state transition sequences in the state space. S is used for representing a grayscale state, G is used for representing a grayscale value, and F is used for representing an occurrence frequency of a grayscale or duration of the grayscale. Specific implementations are described by using a single piece of preconfigured target attitude data as an example herein. Extension to a plurality of pieces of preconfigured target attitude data may be implemented according to this example. Such extension without creative efforts still falls within the patent protection scope of the present invention.
-
FIG. 2 is a diagram of transition of discharge attitude states of a single target material. A condition of entering anS 1 state is that: a grayscale value is between 0.7 and 0.9, and a continuous occurrence frequency of the grayscale value is equal to 4. A condition of entering an S2 state is that: a grayscale sequence in which the grayscale is in a range of 0.3 to 0.5 continuously occurs for three times before timeout. A condition of entering an S4 state is that the grayscale value is decreased to be less than 0.1 before timeout. An unconditional output result of S4 is true, and an unconditional output result of S3 is false. From the figure, it can be seen that a grayscale state sequence of attitudes of the target material is M={S1->S2->S4}, and the result is true only for a material grayscale state sequence the same as this state sequence, and then it is determined that the attitude of the material is a discharge attitude; otherwise, the rejection mechanism performs an operation of returning the material to the tray. - A derivation process of a grayscale state sequence is described below by using four surfaces of a material A as an example.
FIG. 3 shows four possible feed attitudes a, b, c, and d of the material A inFIG. 1 . FromFIG. 1 , it can be seen that the discharge station requires a discharge attitude to be the attitude a inFIG. 3 . The main controller needs to calculate four grayscale sequences inFIG. 3 , separate the attitude a, and return materials of the attitudes b, c, and d to the tray through the rejection mechanism. Symbol G[x] is used for representing a grayscale sequence x. Through measurement, it is obtained that: - G[a]={0.05, 0.88, 0.87, 0.85, 0.86, 0.66, 0.41, 0.42, 0.39, 0.04}
- G[b]={0.03, 0.38, 0.41, 0.43, 0.40, 0.39, 0.61, 0.78, 0.86, 0.01}
- G[c]={0.02, 0.81, 0.83, 0.79, 0.85, 0.79, 0.81, 0.78, 0.77, 0.78, 0.87, 0.79, 0.81, 0.83, 0.82, 0.84, 0.81, 0.83, 0.78, 0.75, 0.81, 0.01}
- G[d]={0.04, 0.15, 0.43, 0.51, 0.81, 0.82, 0.79, 0.81, 0.81, 0.78, 0.79, 0.84, 0.79, 0.85, 0.81, 0.79, 0.83, 0.81, 0.35, 0.15, 0.01}
-
- A result obtained from state transition judgment performed on G[a] is S[a], and an output result is true. A grayscale sequence represented by G[b] does not meet the condition of entering S1, and the grayscale sequence thereof is empty, and therefore a result is empty. A grayscale sequence represented by G[c] may successfully enter the S1 state, but wait overtime in the S1 state, thereby entering S3, and a result is false. A judgment result of G[d] is the same as that of G[c].
- By means of judgment on the output results in the diagram of sequence state result transition, "true" represents being the same as the target discharge attitude, and "false" represents being different from the target discharge attitude. Therefore, it can be concluded that the state a belongs to the target discharge attitude, and the states b, c, and d do not belong to the target discharge attitude.
- From the foregoing judgment results, it can be seen that a material with an abnormal discharge attitude may be strictly screened out by using a state sequence and transition conditions between states. The main controller may process, in real time, grayscale data received by the grayscale sensor, without waiting until the entire material grayscale state sequence is acquired. Grayscale state sequence analysis is an almost real-time online data processing method. The entire process of grayscale sampling, feature identification, calculation and judgment, and responding has a small calculation burden, and may be completed before the material moves to the rejection device. Such a design has almost no impact on a discharge speed. When material type changes, learning may be performed again, to create a transition diagram of discharge attitude grayscale states of a new target material. This conversion operation can be completed by guiding a computing unit with a small amount of manual intervention.
- A grayscale state sequence based feed sorting device provided in the present invention provides a new idea for automatic feeding. Automatic feeding is a section of great significance during automatic production. It is difficult to universalize a conventional feed device because material objects are of various types and have greatly different features. As a result, it is expensive to customize a feed device. Therefore, it is of great economic value and practical significance to implement a programmable universal feed device with high efficiency and high reliability.
- The embodiment described above is merely for describing the technical concept of the present invention, but is not intended to limit the protection scope of the present invention. All technical concepts put forward according to the present invention and any modifications made based on the technical solution fall within the protection scope of the present invention.
Claims (9)
- A grayscale state sequence based feed sorting device, comprising a grayscale sensor, a main controller, a rejection mechanism, a tray, and a guide chute, wherein all materials are placed on the tray and move forward along with the tray, the grayscale sensor acquires a grayscale state sequence of a material and sends the grayscale state sequence to the main controller, and the main controller identifies the grayscale state sequence;
the main controller matches the grayscale state sequence with preconfigured standard discharge-allowed attitude data and preconfigured standard discharge-prohibited attitude data; if the grayscale state sequence matches any one piece of the preconfigured standard discharge-allowed attitude data, it indicates that the material is in a target discharge attitude and the material is allowed to pass through the rejection mechanism; and if the grayscale state sequence matches the preconfigured standard discharge-prohibited attitude data, it indicates that the material is in a non-discharge attitude, and the main controller controls the rejection mechanism to return the material to an initial position of the tray; through a falling process in the rejection mechanism and vibration of the tray, the attitude of the returned material is changed and a system performs identification again when the material reaches the rejection mechanism again;
if the grayscale state sequence does not match any piece of the standard discharge-allowed attitude data and the standard discharge-prohibited attitude data, the main controller performs a corresponding operation on the material in this condition according to a preconfigured treatment solution, wherein the treatment solution comprises allowing passing through the rejection mechanism, prohibiting passing through the rejection mechanism, and shutdown warning; and
along with operation of the feed sorting device, the material finally reaches a discharge station in the target discharge attitude through the guide chute, or an operation is completed according to a treatment solution preconfigured by a user. - The grayscale state sequence based feed sorting device according to claim 1, wherein each piece of the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data comprises a plurality of grayscale states and a state transition sequence between grayscale states; and in a case that the grayscale state sequence of the material acquired by the grayscale sensor meets the state transition sequence in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data, it is determined that the grayscale state sequence of the material matches the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data.
- The grayscale state sequence based feed sorting device according to claim 2, wherein each grayscale state in the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data comprises a value range of a grayscale value and an occurrence frequency or duration of the grayscale value.
- The grayscale state sequence based feed sorting device according to claim 2, wherein the preconfigured standard discharge-allowed attitude data or standard discharge-prohibited attitude data is obtained through learning by using the feed sorting device in advance, a material is enabled to move forward on the tray in a determined attitude, the grayscale sensor acquires a grayscale state sequence of the material within an entire detection area and sends the grayscale state sequence to the main controller, the main controller analyzes features of the grayscale state sequence to obtain grayscale states comprised in the material and a state transition sequence, that is, preconfigured standard attitude data, the preconfigured standard attitude data is stored in the main controller, and the standard attitude data is preconfigured as the standard discharge-allowed attitude data or the standard discharge-prohibited attitude data by the user.
- The grayscale state sequence based feed sorting device according to claim 1, wherein the grayscale sensor is a photoelectric sensor and comprises an optical transmitting unit and an optical receiving unit, the optical transmitting unit transmits a modulated optical signal, and the optical receiving unit outputs a grayscale value of the material at a light spot detection position.
- The grayscale state sequence based feed sorting device according to claim 5, wherein if the material is transparent or semitransparent, a thru-beam-type photoelectric sensor is used as the photoelectric sensor; and if the material is non-transparent, a reflective photoelectric sensor is used as the photoelectric sensor.
- The grayscale state sequence based feed sorting device according to claim 6, wherein in a case that a reflective photoelectric sensor is used as the photoelectric sensor, an emission light wavelength of the optical transmitting unit is adjusted according to a reflective attribute of the material.
- The grayscale state sequence based feed sorting device according to claim 1, wherein a plurality of pieces of standard discharge-allowed attitude data, a plurality of pieces of standard discharge-prohibited attitude data, and a plurality of treatment solutions for unmatched materials are preconfigured in the main controller, and a discharge attitude or a combination of discharge attitudes of a material is flexibly changed according to different requirements of the discharge station.
- The grayscale state sequence based feed sorting device according to claim 1, wherein grayscale state sequences of materials that are important or easy to be confused are pre-selected as feature templates, and operations of the rejection mechanism that respectively correspond to the feature templates are set; for other grayscale state sequences that do not belong to the feature templates, treatment solutions are preconfigured, and corresponding operations of the rejection mechanism or a warning operation is performed according to the treatment solutions.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710853450.9A CN107671011B (en) | 2017-09-20 | 2017-09-20 | A kind of feeding sorting unit based on grey states sequence |
| PCT/CN2017/103845 WO2019056402A1 (en) | 2017-09-20 | 2017-09-28 | CHARGE SORTING DEVICE BASED ON GRAY LEVEL STATE SEQUENCE |
Publications (4)
| Publication Number | Publication Date |
|---|---|
| EP3677350A1 true EP3677350A1 (en) | 2020-07-08 |
| EP3677350A4 EP3677350A4 (en) | 2020-11-25 |
| EP3677350B1 EP3677350B1 (en) | 2024-12-25 |
| EP3677350C0 EP3677350C0 (en) | 2024-12-25 |
Family
ID=61136629
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP17926030.2A Active EP3677350B1 (en) | 2017-09-20 | 2017-09-28 | Grayscale state sequence based feed sorting device |
Country Status (6)
| Country | Link |
|---|---|
| EP (1) | EP3677350B1 (en) |
| CN (1) | CN107671011B (en) |
| ES (1) | ES3009084T3 (en) |
| HU (1) | HUE070483T2 (en) |
| PL (1) | PL3677350T3 (en) |
| WO (1) | WO2019056402A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109761017B (en) * | 2019-01-29 | 2024-05-03 | 无锡先导智能装备股份有限公司 | Feed inlet detection device |
| CN114951047B (en) * | 2022-05-26 | 2023-08-22 | 河海大学 | Universal intelligent sorting method in vibration feeding based on optical fiber sensor |
| CN119456466B (en) * | 2024-10-09 | 2026-02-03 | 广东何氏模具有限公司 | A flexible feeding and sorting machine |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3319243B2 (en) * | 1995-08-29 | 2002-08-26 | 神鋼電機株式会社 | Parts feeder |
| KR101111502B1 (en) * | 2011-03-11 | 2012-02-22 | 주식회사코마스 | Improved Simultaneous Sorting Method for Grain Color Sorter |
| CN102663362B (en) * | 2012-04-09 | 2014-11-05 | 宁波中科集成电路设计中心有限公司 | Moving target detection method based on gray features |
| CN103008259B (en) * | 2012-12-28 | 2014-08-20 | 合肥美亚光电技术股份有限公司 | Material sorting device based on laser transmission |
| CN102989693B (en) * | 2012-12-28 | 2014-11-12 | 合肥美亚光电技术股份有限公司 | Laser transmission based method and devices for sorting materials |
| CN103101741B (en) * | 2013-02-04 | 2017-04-05 | 江苏中科百易矿业科技有限公司 | Implementation method of laser vision conveyer belt damage detection device |
| CN203484357U (en) * | 2013-10-08 | 2014-03-19 | 合肥美亚光电技术股份有限公司 | Dual infrared on-line plastic material sorting device |
| CN103896025B (en) * | 2014-04-02 | 2016-03-02 | 东南大学 | Based on flexible vibration transmission system and the method for work thereof of smart camera |
| CN203997897U (en) * | 2014-07-04 | 2014-12-10 | 佛山科学技术学院 | For mosaic particle order is arranged to the feeding device of output |
| CN104390987B (en) | 2014-12-16 | 2017-01-04 | 济南大学 | A kind of novel optical fiber sensor detecting steel ball surface defect and detection method |
| CN104773446B (en) * | 2015-03-26 | 2017-02-22 | 宁波科瑞特光电技术有限公司 | Vibrating disk feeding system for reinforcement steel sheets |
| CN106269570A (en) * | 2015-05-29 | 2017-01-04 | 合肥美亚光电技术股份有限公司 | The sorting unit of material and method for separating |
| CN205165215U (en) * | 2015-11-24 | 2016-04-20 | 河南理工大学 | Dry process lump coal sorting unit |
| CN105251707B (en) * | 2015-11-26 | 2017-07-28 | 长沙理工大学 | Bad part eject sorting equipment based on medical large transfusion visible foreign matters detecting system |
| CN106980816B (en) * | 2017-02-22 | 2020-08-25 | 贵州电网有限责任公司凯里供电局 | Automatic insulator string identification method based on optical image |
-
2017
- 2017-09-20 CN CN201710853450.9A patent/CN107671011B/en active Active
- 2017-09-28 HU HUE17926030A patent/HUE070483T2/en unknown
- 2017-09-28 EP EP17926030.2A patent/EP3677350B1/en active Active
- 2017-09-28 ES ES17926030T patent/ES3009084T3/en active Active
- 2017-09-28 WO PCT/CN2017/103845 patent/WO2019056402A1/en not_active Ceased
- 2017-09-28 PL PL17926030.2T patent/PL3677350T3/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| EP3677350B1 (en) | 2024-12-25 |
| HUE070483T2 (en) | 2025-06-28 |
| WO2019056402A1 (en) | 2019-03-28 |
| CN107671011B (en) | 2019-07-12 |
| EP3677350A4 (en) | 2020-11-25 |
| ES3009084T3 (en) | 2025-03-26 |
| EP3677350C0 (en) | 2024-12-25 |
| CN107671011A (en) | 2018-02-09 |
| PL3677350T3 (en) | 2025-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11756201B2 (en) | Method and system for performing characterization of one or more materials | |
| CN1515895B (en) | Apparatus for monitoring stripping material edge | |
| EP3677350B1 (en) | Grayscale state sequence based feed sorting device | |
| WO2022144722A1 (en) | Machine learning-based generation of rule-based classification recipes for inspection system | |
| EP2422182B1 (en) | Method and apparatus for multi-parameter data analysis | |
| CN109426225A (en) | A kind of method of quality control of tobacco cutting production | |
| CN104132945A (en) | On-line surface quality visual inspection device for bar based on optical fiber conduction | |
| CN106598622A (en) | Smart vision inspection platform | |
| CN119559564B (en) | Power transmission line icing analysis method and system based on image recognition | |
| CN104318249A (en) | Grinding surface roughness grade identification method based on multi-classification support vector machine | |
| De Gélis et al. | Benchmarking change detection in urban 3d point clouds | |
| Bagga et al. | An online tool life prediction system for CNC turning using computer vision techniques | |
| CN118586789A (en) | A wood-plastic board production management system based on the Internet of Things | |
| CN205785431U (en) | X-ray liquid level detection system | |
| CN106599902B (en) | A kind of sugar crystal Classification and Identification and crystalline quality control method based on image | |
| Yang et al. | Wood veneer defect detection system based on machine vision | |
| EP3835737A1 (en) | Method and apparatus for determining a vertical position of a horizontally extending interface between a first component and a second component | |
| CN117132174A (en) | Model training method and system applied to quality detection of industrial assembly line | |
| CN111359892A (en) | Device and method for detecting and grading appearance of Tianbao honey pomelos based on machine vision | |
| CN120293990A (en) | A glass product quality detection method and system for glass manufacturing equipment | |
| CN118014443B (en) | Medicine monitoring method and device | |
| CN117147570A (en) | Manufacturing control method, device, equipment and storage medium based on machine vision | |
| CN118968402B (en) | A method, system, electronic device and storage medium for counting the number of ampoule bottles | |
| US20250198746A1 (en) | System and method for inspecting thickness | |
| Díaz et al. | Implementation of an Automated Solanum Tuberosum Grading System using Image Processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20200331 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| AX | Request for extension of the european patent |
Extension state: BA ME |
|
| A4 | Supplementary search report drawn up and despatched |
Effective date: 20201026 |
|
| RIC1 | Information provided on ipc code assigned before grant |
Ipc: B07C 5/342 20060101AFI20201020BHEP Ipc: B07C 5/36 20060101ALI20201020BHEP |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20220713 |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
| INTG | Intention to grant announced |
Effective date: 20240920 |
|
| RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: YANG, HUAN Inventor name: ZHANG, XUPING Inventor name: ZHANG, YIXIN Inventor name: WANG, SHUN |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017087014 Country of ref document: DE |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
| U01 | Request for unitary effect filed |
Effective date: 20250121 |
|
| U07 | Unitary effect registered |
Designated state(s): AT BE BG DE DK EE FI FR IT LT LU LV MT NL PT RO SE SI Effective date: 20250127 |
|
| REG | Reference to a national code |
Ref country code: SK Ref legal event code: T3 Ref document number: E 45798 Country of ref document: SK |
|
| REG | Reference to a national code |
Ref country code: ES Ref legal event code: FG2A Ref document number: 3009084 Country of ref document: ES Kind code of ref document: T3 Effective date: 20250326 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20250325 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20250326 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20250325 |
|
| REG | Reference to a national code |
Ref country code: HU Ref legal event code: AG4A Ref document number: E070483 Country of ref document: HU |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241225 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20250425 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: U11 Free format text: ST27 STATUS EVENT CODE: U-0-0-U10-U11 (AS PROVIDED BY THE NATIONAL OFFICE) Effective date: 20251001 |
|
| U20 | Renewal fee for the european patent with unitary effect paid |
Year of fee payment: 9 Effective date: 20250826 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: PL Payment date: 20250820 Year of fee payment: 9 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: HU Payment date: 20250822 Year of fee payment: 9 Ref country code: GB Payment date: 20250919 Year of fee payment: 9 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CZ Payment date: 20250926 Year of fee payment: 9 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SK Payment date: 20250821 Year of fee payment: 9 |
|
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: L10 Free format text: ST27 STATUS EVENT CODE: U-0-0-L10-L00 (AS PROVIDED BY THE NATIONAL OFFICE) Effective date: 20251105 |
|
| 26N | No opposition filed |
Effective date: 20250926 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20251001 Year of fee payment: 9 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: ES Payment date: 20251015 Year of fee payment: 9 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: R18 Free format text: ST27 STATUS EVENT CODE: U-0-0-R10-R18 (AS PROVIDED BY THE NATIONAL OFFICE) Effective date: 20260331 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20241225 |