EP3426020A1 - Control system for automatic milking installation and method of controlling automatic milking installation - Google Patents
Control system for automatic milking installation and method of controlling automatic milking installationInfo
- Publication number
- EP3426020A1 EP3426020A1 EP17712260.3A EP17712260A EP3426020A1 EP 3426020 A1 EP3426020 A1 EP 3426020A1 EP 17712260 A EP17712260 A EP 17712260A EP 3426020 A1 EP3426020 A1 EP 3426020A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- transparent cover
- cover surface
- dirt
- amount
- graphical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000009434 installation Methods 0.000 title claims abstract description 25
- 238000003384 imaging method Methods 0.000 claims abstract description 40
- 238000001454 recorded image Methods 0.000 claims abstract description 16
- 241001465754 Metazoa Species 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims abstract description 9
- 230000008569 process Effects 0.000 claims abstract description 7
- 235000013365 dairy product Nutrition 0.000 claims abstract description 5
- 238000004590 computer program Methods 0.000 claims description 8
- 239000002245 particle Substances 0.000 claims description 7
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 230000003278 mimic effect Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 229910052729 chemical element Inorganic materials 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract 1
- 238000011109 contamination Methods 0.000 description 10
- 238000004140 cleaning Methods 0.000 description 8
- 238000012360 testing method Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007429 general method Methods 0.000 description 2
- 229940036310 program Drugs 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 241000283707 Capra Species 0.000 description 1
- 241000124008 Mammalia Species 0.000 description 1
- 241001494479 Pecora Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 210000000481 breast Anatomy 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000008267 milk Substances 0.000 description 1
- 235000013336 milk Nutrition 0.000 description 1
- 210000004080 milk Anatomy 0.000 description 1
- 210000002445 nipple Anatomy 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01J—MANUFACTURE OF DAIRY PRODUCTS
- A01J5/00—Milking machines or devices
- A01J5/017—Automatic attaching or detaching of clusters
- A01J5/0175—Attaching of clusters
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01J—MANUFACTURE OF DAIRY PRODUCTS
- A01J5/00—Milking machines or devices
- A01J5/007—Monitoring milking processes; Control or regulation of milking machines
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01J—MANUFACTURE OF DAIRY PRODUCTS
- A01J7/00—Accessories for milking machines or devices
- A01J7/02—Accessories for milking machines or devices for cleaning or sanitising milking machines or devices
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01J—MANUFACTURE OF DAIRY PRODUCTS
- A01J7/00—Accessories for milking machines or devices
- A01J7/04—Accessories for milking machines or devices for treatment of udders or teats, e.g. for cleaning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
Definitions
- the present invention relates generally to solutions for controlling automatic milking installations. More particularly the invention relates to a system according to the preamble of claim 1 and a corresponding method. The invention also relates to a computer program and a processor-readable medium.
- WO 2010/031632 shows a solution for testing a camera arranged in an automated milking system.
- a camera test arrangement is provided for determining the performance of a camera arranged for use in an automated milking system.
- the camera test arrangement includes a camera test device with means for providing a repeatable test environment for the camera.
- the camera test arrangement further includes image processing means for processing data obtained from the camera when arranged in the camera test device and for determining the performance of the camera.
- the invention provides reliable yet non- expensive means for ensuring proper functioning of the camera.
- the object of the present invention is therefore to offer an improved solution for prompting the operator of a milking installation to keep the vision system sufficiently clean to ensure the functionality of the system.
- the object is achieved by the initially described system, wherein the control unit is configured to process the recorded image data to determine a parameter indicating an amount of dirt on the transparent cover surface.
- the control unit is further configured to generate graphics da- ta for presentation to the user via the graphical user interface, which graphics data contain at least one graphical element reflecting the amount of dirt on the transparent cover surface.
- This system is advantageous because it renders it impossible for the operator to interact with the system without being remin- ded to clean the cover surface of the imaging unit should this be necessary.
- the at least one graphical element is configured to express the amount of dirt on the transparent cover surface proportionally, such that a relatively small amount of dirt is expressed as one or more graphical elements of comparatively small size, number and/or high transparency, and a relatively large amount of dirt is expressed as one or more graphical elements of comparatively large size, number and/or low transparency.
- the control unit determines that the amount of dirt on the transparent cover surface is below a first threshold level, the graphics data reflect that the transparent cover surface is clean. Namely, inevitably there will always be some dirt on the transparent cover surface. However, as long as this does not influence the quality of the image data, there is no need to inform the user about the dirt.
- the control unit determines that the amount of dirt on the transparent cover surface is above a second threshold level
- the graphics data reflect that the transparent cover surface is highly obstructed with dirt by the control unit being configured to cause the at least one graphical element to be presented in a cyclic manner alternating between first and second phases.
- the at least one graphical element blocks a relatively large proportion of information in at least one of the at least one display view; and in the second phase, the at least one graphical element blocks a relatively small proportion of the information in said at least one display view. Consequently, it becomes evident to the user that the transparent cover surface is heavily soiled; and still, it is possible to interact via the user interface.
- the at least one graphical element may be presented alternating- ly either in the first or second phase.
- the control unit is configured to generate the graphics data such that the at least one graphical element gradually transitions between the first and second phases, either in a stepwise manner or continuously.
- control unit is configured to generate the graphics data such that the at least one graphical element is superimposed on at least one first display view of the at least one display view.
- the at least one graphical element is arranged to mimic a number of dirt particles.
- control unit is configured to repeatedly update the determining of the parameter indicating the amount of dirt on the transparent cover surface; and repeatedly generate updatings of the at least one graphical element based on the updated determining of the parameter indicating the amount of dirt on the transparent cover surface.
- the at least one graphical element reflect this, e.g. by increasing in size and/or number and/or with decreasing transparency.
- the graphics data reflecting that the transparent cover surface is clean, e.g. by not presenting any graphical elements. This is advantageous because the user is provided with immediate feedback on the quality of any cleaning performed.
- the control unit is configured to generate the graphics data such that the at least one graphical element is presented in the graphical user interface in a manner associating the determined amount of dirt to a specific imaging unit of said imaging units on whose transparent cover surface has been determined. For example, if an imaging unit employed at a particular milking point has a dirty transparent cover surface, the at least one graphical element is presented in a graphical user interface associated with that milking point. Naturally, this facilitates identification of the imaging unit in need of cleaning.
- the system includes a user-manipulable input member configured to receive a user command, which, when received by the control unit, temporarily prevents the at least one graphical element from being presented via the graphical user interface.
- a user command which, when received by the control unit, temporarily prevents the at least one graphical element from being presented via the graphical user interface.
- the object is achieved by the method described initially, wherein the recorded image data are processed to determine a parameter indicating an amount of dirt on the transparent cover surface. Furthermore, the graphics data are generated for presentation to the user via the graphical user interface, which graphics data contain at least one graphical element reflecting the amount of dirt on the transparent cover surface.
- the object is achieved by a computer program loadable into the memory of at least one processor, and includes software adapted to implement the method proposed above when said program is run on at least one processor.
- the object is achieved by a processor-readable medium, having a program recorded thereon, where the program is to control at least one processor to perform the method proposed above when the prog- ram is loaded into the at least one processor.
- Figure 1 illustrates how an imaging unit records image data containing data representing a portion of an animal
- Figure 2 shows a user interacting with a graphical user interface of the proposed system
- Figures 3-7 illustrate the graphical user interface according to embodiments of the invention.
- Figure 8 illustrates, by means of a flow diagram, the general method according to the invention.
- Figure 1 illustrates how an imaging unit 1 10 records image data Dimg containing data representing a portion of an animal A.
- the recorded image data Dimg inevitably contain a relatively large amount of data in addition to said data representing the portion of the animal A.
- the recorded image data Dimg also include data reflecting the presence of any dirt particles located within a view field of the imaging unit 1 10.
- Figure 2 shows a user U interacting with a graphical user interface 135 of the proposed system for controlling an automatic mil- king installation.
- the graphical user interface 135 is preferably implemented by a display unit 130.
- the graphical user interface 135 is configured to enable the user U to interact with the automatic milking installation via at least one display view DV1 , DV2 and/or DV3.
- the system includes an imaging unit 1 10 and a control unit 120.
- the imaging unit 1 10 is configured to record image data Di mg containing data representing at least one portion of a dairy animal A, say the animal's A udder and its teats.
- the imaging unit 1 10 has a transparent cover surface 1 17 configured to protect an optics section 1 5 of the imaging unit 1 10.
- the transparent cover surface 1 17 is transparent with respect to a range of wavelengths within which the imaging unit 1 10 is operative to record the image data Dimg. For example, if the imaging unit 1 10 is configured to record the image data Di mg from a first wavelength (say in the infra- red part of the light spectrum) to a second wavelength (say in in the ultraviolet part of the light spectrum), the transparent cover surface 1 17 is transparent at least with respect to a spectrum of electromagnetic energy ranging between the first and second wavelengths.
- the control unit 120 is configured to receive the recorded image data Dimg, and based thereon, produce at least one control signal S-Ctrl arranged to control at least one function of the automatic milking installation.
- control unit 120 is configured to process the recorded image data Di mg to determine a parameter indicating an amount of dirt on the transparent cover surface 1 17. For example, the technique described in WO 2010/031632 (hereby incorporated by reference) can be used to determine the parameter in question. Additionally, the control unit 120 is configured to generate graphics data D gr for presentation to the user U via the graphical user interface 135, which graphics data D gr contain at least one graphical element reflecting the amount of dirt on the transparent cover surface 1 17.
- Figures 3-7 illustrate the graphical user interface 135 according to embodiments of the invention.
- Figure 3 shows one example of how the graphical user interface 135 can be implemented to show display views DV1 , DV2 and DV3 respectively, each of which represents a particular function of the automatic milking installation.
- Figure 3 illustrates a situation where no dirt has been determined to be lo- cated on the transparent cover surface 1 17, or at least an amount of dirt below or equal to a first threshold level has been determined. Namely, in such a case the graphics data D gr reflect that the transparent cover surface 1 17 is clean, e.g. by leaving a dis- play window DV1 associated with an imaging unit 1 10 completely unaffected 300. In other words, the display window DV1 shows no signs of the transparent cover surface 17 being dirty.
- the control unit 120 is configured to generate the graphics data D gr such that the graphics data D gr reflect that the transparent cover surface 1 17 is dirty.
- at least one graphical element in the graphics data D gr is configured to express the amount of dirt on the transparent cover surface 1 17 proportionally.
- a relatively small amount of dirt can be expressed as one or more graphical elements of comparatively small size, number, and/or high transparency, and a relatively large amount of dirt is expressed as one or more graphical elements of comparatively large size, number and/or low transparency.
- Figure 4 shows an example where a comparatively large number of graphical elements 400 are included in the graphics data D g r in a display window DV1 to reflect the fact that the transparent cover surface 1 17 is relatively dirty.
- control unit 120 is configured to generate the graphics data D gr , such that the graphical elements are presented in the graphical user interface 135 in a manner associating the determined amount of dirt to a specific imaging unit 1 10 of said imaging units 1 10 on whose transparent cover surface 1 17 has been determined.
- the graphical elements 400 presented in display view DV1 of the graphical user interface 135 may indicate that the transparent cover surface 1 17 of a first imaging unit has a medium degree of contamination, whereas the fact that display views DV2 and DV3 do not include any obstructing graphical elements at all may indicate that any associated ima- ging units have clean transparent cover surface 1 1 7, or at least that the degree of contamination is below the first threshold level.
- Fig ure 5 shows an example where somewhat less graphical elements 500 are included in the g raphics data D gr to reflect the fact that the transparent cover surface 1 1 7 is somewhat less dirty.
- the g raphical elements 500 are spread out over the entire graphical user interface 1 35 , inter alia covering display views DV1 , DV2 and DV3.
- the graphical elements 600 block a relatively large proportion of information in at least one display view, say DV1 ; and in the second phase, the graphical elements block a relatively small proportion of the information in the display view DV1 , for example as 400 in Figure 4.
- the graphics data D gr can reflect that the transparent cover surface 1 1 7 is highly obstructed with dirt, and at the same time, the user U can extract information from the display view DV1 in question .
- the control unit 120 may be configured to generate the graphics data Dg r such that the at least one graphical element is presented in either the first or the second phase in an alternating manner.
- the control unit 120 is configured to generate the g raphics data D gr such that the at least one graphical element gradually transitions between the first and second phases, either stepwise, or in a continuous manner.
- the system may include a user-manipulable input member 210 for inhibiting presentation of the at least one graphical element.
- the user-manipulable input member 210 is configured to forward a user command CMD (e.g. generated in response to activation of an on-screen button , a physical button or key on a keyboard) to the control unit 120.
- the control unit 120 is configured to temporarily prevent the at least one graphical element 400, 500 or 600 from being presented via the graphical user interface 135. Consequently, for example while the user U keeps an on-screen button activated, no graphical elements are shown via the graphical user interface 35.
- control unit 120 is configured to generate the graphics data D gr , such that the at least one graphical element is superimposed on at least one, e.g. DV1 , and the information therein is thereby at least partially blocked from being visually inspected by the user U.
- the at least one graphical element may either be completely opaque, or more or less transparent depending on the degree of visual obstruction desired.
- the graphical elements 400, 500 and 600 respectively are arranged to mimic a number of dirt particles. This number is typically not equal to the number of dirt particles on the transparent cover surface 1 17, however it is proportional thereto as discussed above.
- Figure 7 illustrates an alternative way of indicating the amount of dirt on the transparent cover surface 1 17 via the graphics data Dgr.
- the graphical user interface 135 includes an indicator bar 700, which designates how dirty the control unit 120 has determined the transparent cover surface 1 17 to be.
- a first field 710 of the indicator bar 700 may correspond to a relatively low degree of contamination
- a second field 720 of the indi- cator bar 700 may correspond to medium degree of contamination
- a third field 730 of the indicator bar 700 may correspond to a relatively high degree of contamination.
- an intensity of the field may express more detailed information about the degree of contamination.
- the fact that the second field 720 has a somewhat lower intensity than the first field 710 is preferably interpreted to indicate that transparent cover surface 1 17 has not yet reached the medium degree of contamination.
- control unit 120 determines that the amount of dirt on the transparent cover surface 1 17 is below the first threshold level, the control unit 120 is configured to generate the graphics data D gr to reflect that the transparent cover surface 1 17 is clean, for example as in Figure 3, i.e. where no obstructing graphical elements are included in the graphics data D gr . This is beneficial, since it provides the user U with feedback as to whether or not a cleaning of the transparent cover surface 1 17 has been successful.
- control unit 120 is preferably configured to repeatedly update the determining of the parameter indicating the amount of dirt on the transparent cover surface 1 17.
- control unit 120 is configured to repeatedly generate upda- tings of the graphical elements 400, 500, 600 and 700 in the graphics data D gr .
- the control unit 120 is configured to generate the graphics data D gr such that the graphics data D gr reflect that the transparent cover surface 1 17 is clean.
- control unit 120 is configured to effect the above-mentioned procedure in a fully automatic manner, for instance by an executing computer program. Therefore, the control unit 120 may be communicatively connected to a memory unit 125 storing a computer program product, which , in turn, contains software for making at least one processor in the control unit 120 execute the above-described actions when the computer program product is run on the control unit 120.
- image data are recorded by means of an imaging unit.
- the recorded image data contain data that represent at least one portion of a dairy animal. It is further presumed that the imaging unit has a transparent cover surface that is configured to protect an optics section of the imaging unit, and through which transparent cover surface the image data are recorded.
- a control signal is produced based on the recorded image data.
- the at least one control signal is arranged to control at least one function of the automatic milking installation, for example attaching and detaching teatcups.
- a user is enabled to interact with the automatic milking installation via at least one display view in a graphical user interface.
- steps 820 and 830 the procedure loops back to step 810 for updated recording of the image data.
- step 810 the recorded image data are processed to determine a parameter indi- eating an amount of dirt on the transparent cover surface.
- graphics data are generated for presentation to the user via the graphical user interface.
- the graphics data contain at least one graphical element reflecting the amount of dirt on the transparent cover surface, thus informing the user of any dirt on the transparent cover surface protecting the optics section of the imaging unit. Then, the procedure loops back to step 81 0.
- All of the process steps, as well as any sub-sequence of steps, described with reference to Figure 8 above may be controlled by means of a programmed processor.
- the embodiments of the invention described above with reference to the drawings comprise processor and processes performed in at least one processor, the invention thus also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice.
- the program may be in the form of source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other form suitable for use in the implementation of the pro- cess according to the invention .
- the prog ram may either be a part of an operating system, or be a separate application.
- the carrier may be any entity or device capable of carrying the program.
- the carrier may comprise a storage medium, such as a Flash memory, a ROM (Read Only Memory), for ex- ample a DVD (Digital Video/Versatile Disk), a CD (Compact Disc) or a semiconductor ROM , an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic recording medium, for example a floppy disc or hard disc.
- the carrier may be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable or by radio or by other means.
- the carrier may be constituted by such cable or device or means.
- the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing , or for use in the performance of, the relevant processes.
- the invention is advantageous in connection with cow milking, the invention is equally well adapted for implementation in milking machines for any other kind of mammals, such as goats, sheep or buffaloes.
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
SE1650333 | 2016-03-11 | ||
PCT/SE2017/050222 WO2017155457A1 (en) | 2016-03-11 | 2017-03-08 | Control system for automatic milking installation and method of controlling automatic milking installation |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3426020A1 true EP3426020A1 (en) | 2019-01-16 |
EP3426020B1 EP3426020B1 (en) | 2020-11-18 |
Family
ID=58361063
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17712260.3A Active EP3426020B1 (en) | 2016-03-11 | 2017-03-08 | Control system for an automatic milking installation and method of controlling an automatic milking installation |
Country Status (3)
Country | Link |
---|---|
US (1) | US20200267924A1 (en) |
EP (1) | EP3426020B1 (en) |
WO (1) | WO2017155457A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP4171205A1 (en) * | 2020-06-29 | 2023-05-03 | DeLaval Holding AB | System and computer-implemented method for image data quality assurance in an installation arranged to perform animal-related actions, computer program and non-volatile data carrier |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8208043B2 (en) * | 2008-05-09 | 2012-06-26 | Aptina Imaging Corporation | Lens cleaning warning system and method |
WO2010031632A1 (en) | 2008-09-19 | 2010-03-25 | Delaval Holding Ab | Camera test arrangement, device and method in an automated milking system |
US9848575B2 (en) * | 2011-03-17 | 2017-12-26 | Mirobot Ltd. | Human assisted milking robot and method |
US9253375B2 (en) * | 2013-04-02 | 2016-02-02 | Google Inc. | Camera obstruction detection |
EP3164831A4 (en) * | 2014-07-04 | 2018-02-14 | Light Labs Inc. | Methods and apparatus relating to detection and/or indicating a dirty lens condition |
-
2017
- 2017-03-08 WO PCT/SE2017/050222 patent/WO2017155457A1/en active Application Filing
- 2017-03-08 EP EP17712260.3A patent/EP3426020B1/en active Active
- 2017-03-08 US US16/083,752 patent/US20200267924A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
WO2017155457A1 (en) | 2017-09-14 |
US20200267924A1 (en) | 2020-08-27 |
EP3426020B1 (en) | 2020-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6205172B2 (en) | Near-field camera obstacle detection | |
KR102101916B1 (en) | Taking photos through visual obstructions | |
US20130120536A1 (en) | Optical Self-Diagnosis of a Stereoscopic Camera System | |
CN105554437B (en) | Method and apparatus for visualizing loitering objects | |
EP3375282B1 (en) | Method, information processing apparatus and program for determining activity quantities of animals | |
WO2010076882A1 (en) | Image classification standard update method, program, and image classification device | |
JP2018191087A (en) | Adhesive matter detection device and adhesive matter detection method | |
JP2006238802A (en) | Cell observation apparatus, cell observation method, microscope system, and cell observation program | |
US20160165129A1 (en) | Image Processing Method | |
CN108322637B (en) | Equipment protection method and device under strong light | |
JP2009034517A (en) | Abnormal tissue pattern detection apparatus, method and program | |
KR101941585B1 (en) | Embedded system for examination based on artificial intelligence thereof | |
CN103959332A (en) | Image processing | |
EP3426020B1 (en) | Control system for an automatic milking installation and method of controlling an automatic milking installation | |
AU2013240637A1 (en) | System and method for grooming-related farm decision support | |
US20150116543A1 (en) | Information processing apparatus, information processing method, and storage medium | |
JPWO2019244944A1 (en) | 3D reconstruction method and 3D reconstruction device | |
WO2021133090A3 (en) | Method for investigating optical element embedded in intraoral scanner, and system using same | |
US20170091585A1 (en) | Methods and systems for tank level monitoring and alerting | |
WO2020039606A1 (en) | Gas detection device, information processing device, and program | |
CA2860014A1 (en) | Video based indoor leak detection | |
WO2008093344A1 (en) | System for detecting particles in a dairy fluid such as milk | |
KR101553707B1 (en) | Image stain detection method for camera module defect determination | |
EP2417902B1 (en) | Pupil covering state detection device, and in-vehicle camera provided therewith | |
JP2017055262A (en) | Data missing pixel detector |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180911 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602017027728 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: A01J0005007000 Ipc: A01J0007020000 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: A01J 7/04 20060101ALI20200320BHEP Ipc: A01J 5/007 20060101ALI20200320BHEP Ipc: A01J 5/017 20060101ALI20200320BHEP Ipc: A01J 7/02 20060101AFI20200320BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20200630 |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: HALLSTROEM, ANDERS |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602017027728 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1334740 Country of ref document: AT Kind code of ref document: T Effective date: 20201215 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: FP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 1334740 Country of ref document: AT Kind code of ref document: T Effective date: 20201118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210218 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210318 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210219 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210218 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210318 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG9D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602017027728 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20210819 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20210331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210331 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210331 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210308 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210308 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20210318 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NL Payment date: 20220215 Year of fee payment: 6 Ref country code: FR Payment date: 20220221 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210331 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230202 Year of fee payment: 7 Ref country code: DE Payment date: 20230131 Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20201118 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20170308 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MM Effective date: 20230401 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230401 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20230331 |