US20220337785A1 - Method for observing working processes of an agricultural machine, digital video system and agricultural machine - Google Patents
Method for observing working processes of an agricultural machine, digital video system and agricultural machine Download PDFInfo
- Publication number
- US20220337785A1 US20220337785A1 US17/622,016 US202017622016A US2022337785A1 US 20220337785 A1 US20220337785 A1 US 20220337785A1 US 202017622016 A US202017622016 A US 202017622016A US 2022337785 A1 US2022337785 A1 US 2022337785A1
- Authority
- US
- United States
- Prior art keywords
- image reproduction
- machine
- image
- images
- image acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000008569 process Effects 0.000 title claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000008878 coupling Effects 0.000 claims abstract description 8
- 238000010168 coupling process Methods 0.000 claims abstract description 8
- 238000005859 coupling reaction Methods 0.000 claims abstract description 8
- 230000015654 memory Effects 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 4
- 238000003306 harvesting Methods 0.000 description 22
- 239000000463 material Substances 0.000 description 5
- 235000002595 Solanum tuberosum Nutrition 0.000 description 4
- 244000061456 Solanum tuberosum Species 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 241000273930 Brevoortia tyrannus Species 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 239000002689 soil Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 235000016068 Berberis vulgaris Nutrition 0.000 description 1
- 241000335053 Beta vulgaris Species 0.000 description 1
- 241001124569 Lycaenidae Species 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D34/00—Mowers; Mowing apparatus of harvesters
- A01D34/001—Accessories not otherwise provided for
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D13/00—Diggers, e.g. potato ploughs
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D33/00—Accessories for digging harvesters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/005—Reproducing at a different information rate from the information rate of recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Definitions
- the present invention relates to a method for observing working processes of an agricultural machine, a digital video system for observing working processes of an agricultural machine, and to an agricultural working machine.
- a feature of these video systems is that always precisely the video image that the camera assigned to the monitor is recording is displayed on this monitor. Depending on how the camera has been positioned, parts of the machine that are not relevant to the driver are also shown. If the user wants a section of the image to be shown larger, the camera must be laboriously repositioned.
- a vision system for a vehicle is moreover known from EP 3 437 929 A1, in which a display of a field of view or view region of a relevant region closely sur-rounding a trailer is superimposed on the monitor of an operator, depending on the driving situation.
- a method for observing the crop processing and/or handling working processes of an agricultural machine comprising a digital video system with a plurality of digital image acquisition units, with at least one display unit, and with a network coupling element preferably designed as a switch, via which the image acquisition units are connected to the display unit, the image acquisition units record image data for images, in particular of a video stream.
- At least one of the images, in particular of the video stream is reproduced at least partially in one of in particular a plurality of configurable shapes comprising one or multiple image reproduction regions on the display unit, which is in particular designed as a touchscreen, of an EDP unit that is preferably designed as a Panel PC.
- the image data are transmitted from the respective image acquisition unit via the switch, in particular via an Ethernet switch, to the EDP unit.
- the respective elements of the digital video system are in particular connected via network cables (Ethernet cables).
- the physical connection of the image acquisition units to the at least one display unit is thus made indirectly via the EDP unit to which the display unit is connected, or which comprises the display unit.
- the display unit thus comprises at least one display and the electronics belonging to it.
- the image acquisition units are directed here at crop processing and/or handling working processes of the machine, and thus in particular at machine elements, for example at conveyor belts that are conveying the crop, drop steps or the like.
- Individual cameras can likewise, for example, be directed at separators for separating crop from associated material such as stones or soil, or at drop steps.
- Digital image acquisition units can likewise, for example, be directed at crop collecting zones (bunkers), crop transfer zones or crop pickup zones.
- the image acquisition units are, in particular, digital video cameras that output individual images in, for example, RAW format, which also transmit complete, ready-compressed video streams in MPEG or other video formats into the system.
- they are high-resolution cameras with a sensor resolution of at least 1280 ⁇ 720 points, preferably of at least 1280 ⁇ 800 points, in particular at least a 1.3 megapixel sensor with, for example, at least 1280 ⁇ 1024 points (pixels).
- the data for the images of a video stream can also be present merely as deviations from a previously transmitted image. They are forwarded, as described, in particular via cables, to the network coupling element preferably designed as an Ethernet switch that forwards the data of the image acquisition units via one or multiple network coupling elements to a data-processing EDP unit, preferably a Panel PC.
- the data are made available there for the display unit in accordance with a configuration, wherein the configuration can be preset by the operator or also standardized by the program.
- the image data for the images of a video stream are prepared in the EDP unit in such a way that they can be reproduced on the display unit, in particular designed as a touchscreen, of the EDP unit, in particular designed as a Panel PC, and that this is done in the respective image reproduction zone of a defined shape.
- the arrangement of one or multiple image reproduction regions on the available screen surface of the display unit is referred to as a shape.
- the connecting cables are in particular Ethernet cables, for example in the form of 100 BASE-TX or 1000 BASE-TX cables, via which sufficient quantities of data are transferred.
- Individual components of the video system, in particular of the Ethernet switch and of the EDP unit, are preferably provided with a gigabit Ethernet interface in order to transfer the quantity of data to be processed as effi-ciently as possible.
- the reproduction regions of a shape can be configured in such a way that they reproduce identical images, i.e. the same video stream.
- a total of up to eight image reproduction regions can be displayed in one shape, meaning that up to eight video streams can be reproduced simultaneously.
- the EDP unit is in particular equipped for this purpose with appropriately powerful graphics processors and/or microprocessors.
- the EDP unit moreover comprises the usual means for storing and processing data, in particular working and data memory, controllers, network interfaces and, if relevant, communication interfaces. Further possible details of the EDP unit are moreover described below in connection with the respective developments according to the invention.
- the configuration of the shapes and of the image reproduction regions can be stored in the EDP unit in a form that can be stored and appropriately recalled, so that the visible region of the display unit, which preferably is at least 12 inches in size, can be configured personally for a machine operator, a service technician or other operating personnel.
- information relating to the machine type can be transmitted via an interface for machine control which can likewise be connected to the switch and, moreover, to the EDP unit.
- This then leads to an initial selection or provision of corresponding shapes that can be stored in a pre-configured manner.
- a standardized shape can likewise be made available for un-known machine types that are not stored in the EDP unit. It is likewise possible to read individual shape configurations in through an interface present in the EDP unit and by means of a portable storage medium, or to export them onto this storage medium.
- the operating personnel now have the possibility, according to the invention, of being shown precisely the desired video images out of the large number of possible window arrangements, in particular with portrait and landscape format displays, on the display unit or its screen.
- a machine operator can thus set different emphases in the observation of the working processes, in particular for different ground conditions. For example, under heavy, wet soil conditions, parts of a harvesting machine that are liable to become clogged can be specifically moni-tored, whereas in the presence of finer, drier and lighter ground conditions, the separation of the crop into different batch sizes can, for example, receive closer monitoring.
- the operating personnel are able to recognize critical machine states more quickly and to then rectify them using the machine controller which, in particular, is not connected to the digital video system.
- the digital video system can as a result be retrofitted more easily into existing machines.
- the processes observed by the digital video system are, in particular, processes taking place “inside” the machine; there are thus not only image acquisition units directed at the exterior surroundings of the machine, but there are also image acquisition units directed at machine elements handling and processing the crop. Handling the crop also refers to its storage in a bunker of the machine.
- the images of one of the image acquisition units are preferably in each case at least partially reproduced simultaneously in multiple image reproduction regions of a shape, wherein different regions of interest of the image acquisition unit are defined in advance, and can each be reproduced in a dedicated reproduction region of this shape.
- one video stream could only ever be reproduced in one reproduction region of the screen.
- the video streams can here be subjected to further computation in another high-performance image processing unit.
- the definition of the regions of a video stream that are to be reproduced can take place according to the invention via the touchscreen during operation of the system, or in a configuration template on the basis of a still image.
- image regions can here be selected as a region of interest and then appropriately displayed, in particular in their own image reproduction region.
- the system is technically designed so that the selected image section (region of interest) is displayed in magnified form.
- the images of the image acquisition unit (or their display) can thus be zoomed within the respective image reproduction region of a shape.
- the selection of the region of interest is thus associated with zooming in to the respective image segments, in particular through a two-finger input.
- the different video streams can thus be specifically displayed, in segments if relevant, in enlarged form, so that an even better observation of particularly critical working regions is possible.
- the images or image segments selected within the respective image reproduction region can preferably be stored in a memory unit.
- This can be present locally in the EDP unit, for example in the form of a volatile permanent memory; it is also possible for a portable storage medium that can be connected via a suitable interface of the EDP unit to be used.
- the system is furthermore preferably configured in such a way that a selection menu is displayed as a result of a touchscreen input on the corresponding image reproduction region, by means of which storage of the video stream displayed at the time (in particular including regions that are zoomed and conse-quently partially displayed) can be initiated.
- this pro-cess can similarly be initiated through a general settings menu which may have submenus.
- the system is, furthermore, preferably configured such that two or more image reproduction regions, in particular all the image reproduction regions, and the video streams shown simultaneously in them, also preferably in the arrangement that is visible on the screen, in particular on a portable medium, are stored so that working processes taking place simultaneously at different locations on the machine can also be observed simultaneously, including subsequently, without it being necessary for this purpose to attend to separate video streams displayed simultaneously next to one another.
- the EDP unit thus offers the conven-ient creation of a common video stream on the basis of the contents shown on the display unit.
- “Portable medium” refers to storage media that can be connected temporarily to the EDP unit via a hardware interface, not being necessary for the operation of the EDP unit, and in particular only used for data transport. This can, for example, be a USB stick. It can, however, also refer to a cloud memory that is preferably provided by the machine manufacturer and is available via interfaces of the machine and/or of the tractor.
- the observation and supervision of the working processes is in particular improved if images of the video stream of an image acquisition unit, temporarily stored in a preferably volatile or semi-permanent buffer memory, are reproduced in a slowed down manner in at least one image reproduction region, in particular wherein the same video stream is not reproduced, in full or zoomed, in a further image reproduction region of the same shape, or is slowed down differently.
- the incoming video stream is here slowed down in particular to 25% of real-time, wherein, after a time that is in particular predefinable, for example after 4 sec-onds, a jump is made in the reproduction to what is then the current image, after which the subsequent video stream is correspondingly again reproduced in a slowed down manner. This permits very good monitoring of machine parts that in particular are moving quickly, in particular rotating.
- the system is, in particular, designed in such a way that slowed-down video streams are reproduced in at least two image reproduction regions of a shape.
- the images, in particular configured as a video stream, of one or multiple image reproduction regions and/or image acquisition units are transmitted wirelessly by means of a communication unit that is in particular connected via an additional switch, for example an Ethernet switch, to a remote and in particular mobile terminal device or user terminal device.
- a communication unit that is in particular connected via an additional switch, for example an Ethernet switch
- a remote and in particular mobile terminal device or user terminal device In addition to the transmission of the image to the monitor or the display unit in the cab of the tractor or self-propelled vehicle, it is thus possible to transmit the video image to a mobile terminal device.
- the user of the terminal device connects his mobile terminal device for this purpose preferably to a WLAN that is provided by a communication unit of the machine.
- the terminal device user can display individual shapes adapted to the terminal device with at least one video stream of one of the plurality of image acquisition units.
- he can access a web page of the digital video system, preferably hosted on the EDP unit, by way of which an in particular configurable shape is made available for a remote, in particular mobile, user terminal device, wherein preferably the image reproduction regions are likewise predefined on the server side and/or are configurable.
- the images of one or multiple image reproduction regions of a shape, in particular all image reproduction regions, are advantageously stored in such a way that respectively sequential clips of a preferably adjustable length are created or stored. Due to the limitation on the size of the resulting clips associated with this, said clips preferably being automatically provided with a timestamp in the name, the subsequent editing and viewing of individual clips that were created at specific, for instance critical, time points is possible more easily.
- one of up to at least 24 image acquisition units, in particular digital cameras, preferably up to a maximum of 48 image acquisition units, connected to one another via multiple switches connected in series, is selected in a configuration step for an image reproduction region. Due to cascading the switches, the system can be extended, in particular also easily subsequently extended, if corresponding ports for at least one further switch are kept free on each switch. To avoid data bottlenecks on the transmission cable, the bandwidth required in the network of the digital video system is preferably reduced, in that only the cameras displayed in the currently active shape are streamed.
- a corresponding screen signal is displayed overlaying the previous image reproduction regions and/or reproducing at least one associated machine element in an image reproduction region.
- a shape can be changed automatically for this purpose.
- An interface to the machine controller is again necessary for this method, again provided via the Ethernet switch, a further Ethernet switch, or by a direct connection to the EDP unit.
- the respective screen signal can in particular be reproduced in a shape configured only for this.
- the operating terminal is typically located in the driver's cab as well as, for example, at a harvesting table of a potato harvester.
- an automated changeover from one or multiple video streams to a shape configured for this purpose thus takes place.
- One or multiple image segments, preferably in the form of regions of interest of one or multiple video streams, is then displayed in this, so that the operating personnel are purposefully informed about the poten-tially critical situation, and can introduce countermeasures or checks on this basis.
- a color-based emphasis is preferably applied at the same time, in particular when there is a deviation from normal operation or in critical operating states, for example when individual regions are under excessive load.
- a slowed-down reproduction of the relevant video stream can also take place in such a shape, in particular for fast moving or rotating machine parts. It is, similarly, advantageous according to the invention if such an overlay or display of a shape can be hidden again by a screen input, in particular in the form of a swipe movement, in order to return to the previous view or the previous shape.
- the video streams must be displayed with the lowest possible latency time (delay time) between the camera lens and the display unit. Delay times ⁇ 100 ms are desirable.
- a time is advantageously assigned to individual images for this purpose. For this purpose, at least a plurality of the images can be provided with a timestamp at one component, and the time at a further component of the (digital video) system can be taken and/or the time on receipt of a recognizable image is recorded at various components of the system, so that if the time delay is too long the associated image reproduction region can be marked, for example made black and/or given a red exclamation mark.
- an animated graphic is shown (a rotating cogwheel, for example) in order to make it clear that the system is working properly and has not stopped (frozen).
- a pop-up dialogue is in an advantageous manner first opened for the configuration of one of the image reproduction regions, representing the active image reproduction region at a reduced size, preferably before settings can be made. This provides reassurance and checking of the desired changes that are to be carried out, so that a different image reproduction region is not configured accidentally. The vibrations that often occur in field work thus do not lead to, or lead to fewer, unwanted inputs on a display unit designed as a touchscreen.
- the shapes comprise at least one and preferably up to eight in particular rectangular image reproduction regions, wherein preferably an arrangement of the image reproduction regions is proposed depending on the size and shape of the selected regions of interest and/or the position of the image acquisition units in the material flow of the harvested material by means of an algorithm available on the EDP unit.
- the shapes are switched automatically, in particular at predefinable time intervals, in order to provide a sequential view into various machine regions.
- Predefined shapes are advantageously furthermore used depending on signals provided to the machine controller for the operating states of road travel, field work and reversing.
- a shape that provides a view of any machine parts that should be folded in or raised is displayed, whereas for field work, the desired working processes that vary the flow of material are correspondingly observed. Similar considerations apply to the shape views when reversing.
- the images of at least one image acquisition unit are also advantageously displayed as a web video stream via a remote connection, in particular wherein the resolution of the shape view or the repetition frequency is adapted to the bandwidth of the connection that is, for example, reported by a mobile terminal device.
- This system can also be realized on the EDP unit, entirely independently of the display on the display unit, in order to avoid disturbing the operating personnel.
- the system is in particular not designed for machine control, meaning that it does not manipulate the controller of the machine, but serves solely for monitoring the processes.
- a digital video system for observing the crop processing and/or handling working processes of an agricultural machine, in particular a harvesting machine, comprising a plurality of digital image acquisition units and at least one display unit designed as a touchscreen of an EDP unit designed in particular as a Panel PC, and which is provided with a network coupling element designed preferably as a switch, in particular an Ethernet switch, via which the image acquisition units are connected to the display unit.
- the system here is designed for carrying out a method as cited above, by means, as described above, of one or multiple computer programs.
- the system is preferably provided with a plurality of network coupling elements in the form of (Ethernet) switches, wherein at least one image acquisition unit is connected to the EDP unit via the switches networked in series.
- Ethernet network coupling elements in the form of (Ethernet) switches
- this is automatically recognized, configured and returned to operation by the system, in particular when the ex-change is of an original part.
- the transmission of the desired signals to a machine controller can in particular be realized in an improved way if the switch that is directly connected to the EDP unit is connected to an interface for a CAN bus and/or a communication unit of the machine.
- a CAN bus or a communication unit can also be connected directly to the EDP unit if this is provided with suitable interfaces.
- the object presented at the beginning is likewise achieved through an agricultural working machine, in particular a harvesting machine, comprising a video system as described above or below.
- FIG. 1 shows an object according to the invention in the form of a drawn harvesting machine.
- FIG. 2 shows components of the object according to the invention of FIG. 1 .
- FIG. 3 shows a further object according to the invention.
- FIG. 4 shows nine variations of shapes.
- FIG. 5 shows program levels of a computer program that enables the method according to the invention.
- FIG. 6 shows a further shape
- FIG. 7 shows an associated configuration view for the shape of FIG. 6 .
- An apparatus according to the invention is designed according to FIG. 1 as a harvesting machine 2 drawn by a tractor 1 and designed as a potato harvester.
- An EDP unit belonging to the harvesting machine 2 according to the invention is arranged with a display unit 3 designed as a touchscreen in a driver's cab 5 of the tractor 1 . This is connected to a switch on the harvesting machine side via an Ethernet cable, not shown.
- the image data recorded by digital image acquisition units 4 which, in the present case, are assigned schematically to individual handling stages for the harvested material, are reproduced on the display unit 3 .
- the image acquisition units 4 are digital cameras which generally in systems according to the invention transmit the data over Ethernet cables 6 to a switch 7 that is designed as an Ethernet switch. From there, the data are then transmitted to an EDP unit designed as a Panel PC 8 , again preferably via Ethernet cable 6 .
- a further Panel PC 8 can optionally be connected, on whose display unit a range of configurable shapes can also be displayed.
- a main Panel PC master
- An interface for machine control in the form of a CAN to Ethernet gateway 9 is also present. Access to WLAN and to a machine control unit 12 connected via a CAN bus line 11 is provided through this. It is also, for example, possible to access a corresponding bus system in parallel via an ISO bus line 13 (shown dashed). In this case, the gateway 9 also comprises a corresponding ISO bus interface.
- the digital image acquisition units 4 are only connected to one Ethernet switch 7 , which in the case of a drawn harvesting machine 2 is preferably arranged on the harvesting machine side
- the variant embodiment according to FIG. 3 comprises a total of four switches 7 , of which the upper one in the figure is assigned to the tractor 1 , while the switches 7 that are directly connected to the digital image acquisition units 4 are arranged on the side of the harvesting machine 2 .
- a dashed line 14 indicates the separation of the tractor 1 and the harvesting machine 2 , illustrated as boxes in the figure.
- the optical image acquisition units 4 arranged in FIG. 4 at the right-hand switch 7 on the harvesting machine side are thus then connected via a total of four switches 7 to a master Panel PC 8 on the left in the figure and to a further Panel PC (slave) 8 located adjacently on the right in the figure.
- An ISO bus interface 15 is also present in this exemplary embodiment. Machine signals or control signals are received via this, and they, depending on the type of the signal, lead to automatic activation of a shape attracting the attention of the machine driver, via which a video stream of an optical image acquisition unit 4 assigned to the control signals is displayed on the display unit.
- a CAN to Ethernet gateway 10 This is placed in the settings of the program running on the EDP unit.
- a router that can be connected via the gateway 10 provides a WLAN connection 16 to a remote and mobile user terminal device 17 , on which shapes can also be configured or retrieved by a web server running on the master Panel PC.
- a diagnostic interface 18 and one or multiple further devices 20 that convey control signals can additionally be connected to the tractor-side Ethernet switch 7 .
- the nine shapes represented in FIG. 4 can, for example, be configured. Between one and eight image reproduction regions 19 can thus be furnished with video streams of the optical image processing units 4 .
- levels of an associated computer program are illustrated for the illustrated exemplary embodiment.
- a working view 21 for the respective shapes is located in the upper left, from where individual image reproduction regions 19 can be edited according to box 22 .
- An input facility on the touchscreen is assigned, moreover, to the working view 21 , via which the operating personnel reach a settings level 23 from where a shape management 24 , a camera management 25 , a sequence management 26 , a web server management 27 and user settings 28 can be reached.
- the shapes are configured in the shape management 24 .
- Settings, if present, of the image acquisition units 4 for example the color configuration, com-pression or resolution, can be made in the camera management 25 .
- the sequence management 26 allows the saving of individual image reproduction segments 19 , or the management of the individual video streams.
- a web page repre-sentation of individual shapes is correspondingly configured in the web server management 27 ; an access management can, in particular, be configured here, with which third parties can only log on to the web server of the EDP unit with a password from a user terminal device that is preferably mobile, or is at least remote from the harvesting machine.
- the user profile can be stored or opened in the user management 28 . It is also possible to set whether the system always starts up with the most recently used user profile, or with a standard configuration.
- a shape is designed for the reproduction of five video streams on the basis of the five image reproduction regions 19 .
- the associated configuration interface of the shape can be opened directly by means of a correspondingly assigned icon 29 .
- individual image reproduction regions 19 are again first formed, and, while these are reduced in size, they correspond otherwise to those of FIG. 6 .
- the top-most image reproduction region 19 shows the video stream of the upper left image reproduction region in FIG. 6 at a reduced size
- the image reproduction region 19 located in FIG. 7 below the image reproduction region in the second row shows the second video stream in the upper row from the left according to FIG. 6 , and so on.
- the occupancy of the image reproduction regions 4 with the optical image acquisition units 4 that are available is made by the finger-operated field 31 . Tapping this field, or the further fields 31 , displays a drop-down list where the cameras known to the system can then be selected.
- the regions of the touchscreen located on the lower left in FIG. 7 are a button for returning to the previous dialogue (button 32 ), a help button 33 , and an information button 34 .
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Harvesting Machines For Root Crops (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
Description
- This application claims priority to PCT Application No. PCT/EP2020/067760, filed Jun. 24, 2020, which itself claims priority to German Application No. 10 2019 116961.5, filed Jun. 24, 2019, the entirety of both of which are hereby incorporated by reference.
- The present invention relates to a method for observing working processes of an agricultural machine, a digital video system for observing working processes of an agricultural machine, and to an agricultural working machine.
- Analog video systems for observing the crop processing and/or handling working processes of an agricultural machine, in particular of a drawn or self-powered harvester, preferably for potato or beet clearing, have been used for years as assistance systems for the machine driver. A feature of these video systems is that always precisely the video image that the camera assigned to the monitor is recording is displayed on this monitor. Depending on how the camera has been positioned, parts of the machine that are not relevant to the driver are also shown. If the user wants a section of the image to be shown larger, the camera must be laboriously repositioned. Various manufacturers, furthermore, now offer video systems in which a maximum of four cameras can be connected to each monitor, and can then also be displayed simultaneously. Even with these cameras, a more precise observation of the respective parts of the machine is associated with a repositioning of the respective camera.
- A vision system for a vehicle is moreover known from
EP 3 437 929 A1, in which a display of a field of view or view region of a relevant region closely sur-rounding a trailer is superimposed on the monitor of an operator, depending on the driving situation. - Against this background, it is the object of the present invention to improve the monitoring of the working processes of a harvester.
- In a method according to the invention for observing the crop processing and/or handling working processes of an agricultural machine, in particular a harvesting machine, comprising a digital video system with a plurality of digital image acquisition units, with at least one display unit, and with a network coupling element preferably designed as a switch, via which the image acquisition units are connected to the display unit, the image acquisition units record image data for images, in particular of a video stream. At least one of the images, in particular of the video stream, is reproduced at least partially in one of in particular a plurality of configurable shapes comprising one or multiple image reproduction regions on the display unit, which is in particular designed as a touchscreen, of an EDP unit that is preferably designed as a Panel PC. The image data are transmitted from the respective image acquisition unit via the switch, in particular via an Ethernet switch, to the EDP unit. The respective elements of the digital video system are in particular connected via network cables (Ethernet cables). The physical connection of the image acquisition units to the at least one display unit is thus made indirectly via the EDP unit to which the display unit is connected, or which comprises the display unit. The display unit thus comprises at least one display and the electronics belonging to it.
- The image acquisition units are directed here at crop processing and/or handling working processes of the machine, and thus in particular at machine elements, for example at conveyor belts that are conveying the crop, drop steps or the like. Individual cameras can likewise, for example, be directed at separators for separating crop from associated material such as stones or soil, or at drop steps. Digital image acquisition units can likewise, for example, be directed at crop collecting zones (bunkers), crop transfer zones or crop pickup zones. The image acquisition units are, in particular, digital video cameras that output individual images in, for example, RAW format, which also transmit complete, ready-compressed video streams in MPEG or other video formats into the system. Preferably they are high-resolution cameras with a sensor resolution of at least 1280×720 points, preferably of at least 1280×800 points, in particular at least a 1.3 megapixel sensor with, for example, at least 1280×1024 points (pixels).
- The data for the images of a video stream can also be present merely as deviations from a previously transmitted image. They are forwarded, as described, in particular via cables, to the network coupling element preferably designed as an Ethernet switch that forwards the data of the image acquisition units via one or multiple network coupling elements to a data-processing EDP unit, preferably a Panel PC. The data are made available there for the display unit in accordance with a configuration, wherein the configuration can be preset by the operator or also standardized by the program. The image data for the images of a video stream are prepared in the EDP unit in such a way that they can be reproduced on the display unit, in particular designed as a touchscreen, of the EDP unit, in particular designed as a Panel PC, and that this is done in the respective image reproduction zone of a defined shape. The arrangement of one or multiple image reproduction regions on the available screen surface of the display unit is referred to as a shape.
- The connecting cables are in particular Ethernet cables, for example in the form of 100 BASE-TX or 1000 BASE-TX cables, via which sufficient quantities of data are transferred. Individual components of the video system, in particular of the Ethernet switch and of the EDP unit, are preferably provided with a gigabit Ethernet interface in order to transfer the quantity of data to be processed as effi-ciently as possible.
- The reproduction regions of a shape can be configured in such a way that they reproduce identical images, i.e. the same video stream. Preferably a total of up to eight image reproduction regions can be displayed in one shape, meaning that up to eight video streams can be reproduced simultaneously. The EDP unit is in particular equipped for this purpose with appropriately powerful graphics processors and/or microprocessors. The EDP unit moreover comprises the usual means for storing and processing data, in particular working and data memory, controllers, network interfaces and, if relevant, communication interfaces. Further possible details of the EDP unit are moreover described below in connection with the respective developments according to the invention.
- The configuration of the shapes and of the image reproduction regions can be stored in the EDP unit in a form that can be stored and appropriately recalled, so that the visible region of the display unit, which preferably is at least 12 inches in size, can be configured personally for a machine operator, a service technician or other operating personnel.
- In one development of the invention, information relating to the machine type can be transmitted via an interface for machine control which can likewise be connected to the switch and, moreover, to the EDP unit. This then leads to an initial selection or provision of corresponding shapes that can be stored in a pre-configured manner. A standardized shape can likewise be made available for un-known machine types that are not stored in the EDP unit. It is likewise possible to read individual shape configurations in through an interface present in the EDP unit and by means of a portable storage medium, or to export them onto this storage medium.
- The operating personnel now have the possibility, according to the invention, of being shown precisely the desired video images out of the large number of possible window arrangements, in particular with portrait and landscape format displays, on the display unit or its screen. A machine operator can thus set different emphases in the observation of the working processes, in particular for different ground conditions. For example, under heavy, wet soil conditions, parts of a harvesting machine that are liable to become clogged can be specifically moni-tored, whereas in the presence of finer, drier and lighter ground conditions, the separation of the crop into different batch sizes can, for example, receive closer monitoring.
- As a result of setting this emphasis, the operating personnel are able to recognize critical machine states more quickly and to then rectify them using the machine controller which, in particular, is not connected to the digital video system. The digital video system can as a result be retrofitted more easily into existing machines.
- The processes observed by the digital video system are, in particular, processes taking place “inside” the machine; there are thus not only image acquisition units directed at the exterior surroundings of the machine, but there are also image acquisition units directed at machine elements handling and processing the crop. Handling the crop also refers to its storage in a bunker of the machine.
- The images of one of the image acquisition units are preferably in each case at least partially reproduced simultaneously in multiple image reproduction regions of a shape, wherein different regions of interest of the image acquisition unit are defined in advance, and can each be reproduced in a dedicated reproduction region of this shape. Until now, one video stream could only ever be reproduced in one reproduction region of the screen. By transmitting data packets simultaneously to multiple participants (multicast) in the EDP unit, the data are now provided for at least two reproduction regions. The video streams can here be subjected to further computation in another high-performance image processing unit.
- The definition of the regions of a video stream that are to be reproduced can take place according to the invention via the touchscreen during operation of the system, or in a configuration template on the basis of a still image. Using, for example, an image region for one-finger or two-finger gestures similar to those known from mobile telephones, image regions can here be selected as a region of interest and then appropriately displayed, in particular in their own image reproduction region.
- If a corresponding segment is to be displayed in the identical image reproduction region, the system is technically designed so that the selected image section (region of interest) is displayed in magnified form. The images of the image acquisition unit (or their display) can thus be zoomed within the respective image reproduction region of a shape. The selection of the region of interest is thus associated with zooming in to the respective image segments, in particular through a two-finger input. The different video streams can thus be specifically displayed, in segments if relevant, in enlarged form, so that an even better observation of particularly critical working regions is possible. Thus, for example, in the case of image acquisition directed as a whole at a separating stage, it is possible to deliberately zoom in to an input region and an output region, and for a total of three image reproduction regions (overall view, input region, output region) then to be displayed in each case in enlarged form in the same shape.
- The images or image segments selected within the respective image reproduction region can preferably be stored in a memory unit. This can be present locally in the EDP unit, for example in the form of a volatile permanent memory; it is also possible for a portable storage medium that can be connected via a suitable interface of the EDP unit to be used.
- The system is furthermore preferably configured in such a way that a selection menu is displayed as a result of a touchscreen input on the corresponding image reproduction region, by means of which storage of the video stream displayed at the time (in particular including regions that are zoomed and conse-quently partially displayed) can be initiated. Alternatively or in addition, this pro-cess can similarly be initiated through a general settings menu which may have submenus.
- The system is, furthermore, preferably configured such that two or more image reproduction regions, in particular all the image reproduction regions, and the video streams shown simultaneously in them, also preferably in the arrangement that is visible on the screen, in particular on a portable medium, are stored so that working processes taking place simultaneously at different locations on the machine can also be observed simultaneously, including subsequently, without it being necessary for this purpose to attend to separate video streams displayed simultaneously next to one another. The EDP unit thus offers the conven-ient creation of a common video stream on the basis of the contents shown on the display unit. “Portable medium” refers to storage media that can be connected temporarily to the EDP unit via a hardware interface, not being necessary for the operation of the EDP unit, and in particular only used for data transport. This can, for example, be a USB stick. It can, however, also refer to a cloud memory that is preferably provided by the machine manufacturer and is available via interfaces of the machine and/or of the tractor.
- The observation and supervision of the working processes is in particular improved if images of the video stream of an image acquisition unit, temporarily stored in a preferably volatile or semi-permanent buffer memory, are reproduced in a slowed down manner in at least one image reproduction region, in particular wherein the same video stream is not reproduced, in full or zoomed, in a further image reproduction region of the same shape, or is slowed down differently. The incoming video stream is here slowed down in particular to 25% of real-time, wherein, after a time that is in particular predefinable, for example after 4 sec-onds, a jump is made in the reproduction to what is then the current image, after which the subsequent video stream is correspondingly again reproduced in a slowed down manner. This permits very good monitoring of machine parts that in particular are moving quickly, in particular rotating.
- The system is, in particular, designed in such a way that slowed-down video streams are reproduced in at least two image reproduction regions of a shape.
- Preferably the images, in particular configured as a video stream, of one or multiple image reproduction regions and/or image acquisition units, are transmitted wirelessly by means of a communication unit that is in particular connected via an additional switch, for example an Ethernet switch, to a remote and in particular mobile terminal device or user terminal device. In addition to the transmission of the image to the monitor or the display unit in the cab of the tractor or self-propelled vehicle, it is thus possible to transmit the video image to a mobile terminal device. The user of the terminal device connects his mobile terminal device for this purpose preferably to a WLAN that is provided by a communication unit of the machine. Then, in a program launched on the mobile terminal device, the terminal device user can display individual shapes adapted to the terminal device with at least one video stream of one of the plurality of image acquisition units. Alternatively he can access a web page of the digital video system, preferably hosted on the EDP unit, by way of which an in particular configurable shape is made available for a remote, in particular mobile, user terminal device, wherein preferably the image reproduction regions are likewise predefined on the server side and/or are configurable.
- It is thus easy, for example for drivers of collection vehicles, to display an image reproduction segment that shows the video stream of a bunker filling in a shape. The working safety and the convenience of potato harvesters can also be improved in this way for harvesting personnel who work at a harvesting table during harvesting, in that the harvesting personnel on the harvester are shown the video image of the separators, which cannot be seen from the harvesting table. Any settings can then be made directly by the operating personnel from the harvesting table operating terminal, wherein the result of the adjustment can be seen directly on the mobile user terminal device. The driver is thus relieved of supervision and control tasks, and can thereby concentrate on driving the harvester. Managers of agricultural businesses can similarly check the way the machine is operating during the harvesting, for example, without having to mount the machine. The reproduction on a remote, in particular mobile, user terminal device takes place in particular without interfering with the reproduction on the display unit in the driver's cab.
- The images of one or multiple image reproduction regions of a shape, in particular all image reproduction regions, are advantageously stored in such a way that respectively sequential clips of a preferably adjustable length are created or stored. Due to the limitation on the size of the resulting clips associated with this, said clips preferably being automatically provided with a timestamp in the name, the subsequent editing and viewing of individual clips that were created at specific, for instance critical, time points is possible more easily.
- It should be clear that method steps described above and below can be realized by means of a computer program which runs in a working memory of the EDP unit. It is also possible that there are multiple interacting programs that run in the same EDP unit or in the same memory or in different memories of the EDP unit, for example a CPU and a GPU.
- Preferably, one of up to at least 24 image acquisition units, in particular digital cameras, preferably up to a maximum of 48 image acquisition units, connected to one another via multiple switches connected in series, is selected in a configuration step for an image reproduction region. Due to cascading the switches, the system can be extended, in particular also easily subsequently extended, if corresponding ports for at least one further switch are kept free on each switch. To avoid data bottlenecks on the transmission cable, the bandwidth required in the network of the digital video system is preferably reduced, in that only the cameras displayed in the currently active shape are streamed.
- Preferably, depending on a deviation from normal operation detected by a machine controller, on the activation of machine functions, and/or on changes to the machine settings made via an operating terminal, a corresponding screen signal is displayed overlaying the previous image reproduction regions and/or reproducing at least one associated machine element in an image reproduction region. A shape can be changed automatically for this purpose. An interface to the machine controller is again necessary for this method, again provided via the Ethernet switch, a further Ethernet switch, or by a direct connection to the EDP unit. The respective screen signal can in particular be reproduced in a shape configured only for this. The operating terminal is typically located in the driver's cab as well as, for example, at a harvesting table of a potato harvester.
- Preferably, in the event of a deviation from normal operation, the activation of machine functions, and/or changes to the machine settings, an automated changeover from one or multiple video streams to a shape configured for this purpose thus takes place. One or multiple image segments, preferably in the form of regions of interest of one or multiple video streams, is then displayed in this, so that the operating personnel are purposefully informed about the poten-tially critical situation, and can introduce countermeasures or checks on this basis. A color-based emphasis is preferably applied at the same time, in particular when there is a deviation from normal operation or in critical operating states, for example when individual regions are under excessive load. A slowed-down reproduction of the relevant video stream can also take place in such a shape, in particular for fast moving or rotating machine parts. It is, similarly, advantageous according to the invention if such an overlay or display of a shape can be hidden again by a screen input, in particular in the form of a swipe movement, in order to return to the previous view or the previous shape.
- The video streams must be displayed with the lowest possible latency time (delay time) between the camera lens and the display unit. Delay times<100 ms are desirable. A time is advantageously assigned to individual images for this purpose. For this purpose, at least a plurality of the images can be provided with a timestamp at one component, and the time at a further component of the (digital video) system can be taken and/or the time on receipt of a recognizable image is recorded at various components of the system, so that if the time delay is too long the associated image reproduction region can be marked, for example made black and/or given a red exclamation mark.
- Preferably, at the display unit, which also simultaneously represents the operating unit, an animated graphic is shown (a rotating cogwheel, for example) in order to make it clear that the system is working properly and has not stopped (frozen).
- Similarly, a pop-up dialogue is in an advantageous manner first opened for the configuration of one of the image reproduction regions, representing the active image reproduction region at a reduced size, preferably before settings can be made. This provides reassurance and checking of the desired changes that are to be carried out, so that a different image reproduction region is not configured accidentally. The vibrations that often occur in field work thus do not lead to, or lead to fewer, unwanted inputs on a display unit designed as a touchscreen.
- According to a further advantageous design of the invention, the shapes comprise at least one and preferably up to eight in particular rectangular image reproduction regions, wherein preferably an arrangement of the image reproduction regions is proposed depending on the size and shape of the selected regions of interest and/or the position of the image acquisition units in the material flow of the harvested material by means of an algorithm available on the EDP unit. This is a simple optimization problem for the optimum exploitation and arrangement of the available surface area, if the operating personnel want to leave this to the system.
- In the case of a plurality of shapes, it is advantageous in a further exemplary embodiment of the invention if the shapes are switched automatically, in particular at predefinable time intervals, in order to provide a sequential view into various machine regions.
- Predefined shapes are advantageously furthermore used depending on signals provided to the machine controller for the operating states of road travel, field work and reversing. Thus, for example, when traveling on the road, a shape that provides a view of any machine parts that should be folded in or raised is displayed, whereas for field work, the desired working processes that vary the flow of material are correspondingly observed. Similar considerations apply to the shape views when reversing.
- The images of at least one image acquisition unit are also advantageously displayed as a web video stream via a remote connection, in particular wherein the resolution of the shape view or the repetition frequency is adapted to the bandwidth of the connection that is, for example, reported by a mobile terminal device. This system can also be realized on the EDP unit, entirely independently of the display on the display unit, in order to avoid disturbing the operating personnel.
- The system is in particular not designed for machine control, meaning that it does not manipulate the controller of the machine, but serves solely for monitoring the processes.
- The object presented at the beginning is also achieved by a digital video system for observing the crop processing and/or handling working processes of an agricultural machine, in particular a harvesting machine, comprising a plurality of digital image acquisition units and at least one display unit designed as a touchscreen of an EDP unit designed in particular as a Panel PC, and which is provided with a network coupling element designed preferably as a switch, in particular an Ethernet switch, via which the image acquisition units are connected to the display unit. The system here is designed for carrying out a method as cited above, by means, as described above, of one or multiple computer programs.
- For the sake of scalability of the system it is preferably provided with a plurality of network coupling elements in the form of (Ethernet) switches, wherein at least one image acquisition unit is connected to the EDP unit via the switches networked in series.
- Preferably, when a camera is exchanged, this is automatically recognized, configured and returned to operation by the system, in particular when the ex-change is of an original part.
- The transmission of the desired signals to a machine controller can in particular be realized in an improved way if the switch that is directly connected to the EDP unit is connected to an interface for a CAN bus and/or a communication unit of the machine. Alternatively, a CAN bus or a communication unit can also be connected directly to the EDP unit if this is provided with suitable interfaces.
- The object presented at the beginning is likewise achieved through an agricultural working machine, in particular a harvesting machine, comprising a video system as described above or below.
- Reference is now made more particularly to the drawings, which illustrate the best presently known mode of carrying out the invention and wherein similar reference characters indicate the same parts throughout the views.
-
FIG. 1 shows an object according to the invention in the form of a drawn harvesting machine. -
FIG. 2 shows components of the object according to the invention ofFIG. 1 . -
FIG. 3 shows a further object according to the invention. -
FIG. 4 shows nine variations of shapes. -
FIG. 5 shows program levels of a computer program that enables the method according to the invention. -
FIG. 6 shows a further shape. -
FIG. 7 shows an associated configuration view for the shape ofFIG. 6 . - Individual technical features of the exemplary embodiments described below can also be used in combination with exemplary embodiments described above as well as the features of the independent claims and any further claims related to objects of the invention. To the extent it may be helpful, elements with the same functional effect are given identical reference signs.
- An apparatus according to the invention is designed according to
FIG. 1 as a harvestingmachine 2 drawn by a tractor 1 and designed as a potato harvester. An EDP unit belonging to the harvestingmachine 2 according to the invention is arranged with adisplay unit 3 designed as a touchscreen in a driver's cab 5 of the tractor 1. This is connected to a switch on the harvesting machine side via an Ethernet cable, not shown. The image data recorded by digitalimage acquisition units 4 which, in the present case, are assigned schematically to individual handling stages for the harvested material, are reproduced on thedisplay unit 3. - The
image acquisition units 4 are digital cameras which generally in systems according to the invention transmit the data overEthernet cables 6 to aswitch 7 that is designed as an Ethernet switch. From there, the data are then transmitted to an EDP unit designed as aPanel PC 8, again preferably viaEthernet cable 6. Afurther Panel PC 8 can optionally be connected, on whose display unit a range of configurable shapes can also be displayed. In an associated configuration program of a main Panel PC (master) it is then possible to select which of the cameras are displayed on this PC and which on the subsidiary, optional Panel PC (the slave, connected via a data line in the form of a dashed Ethernet cable 6). - An interface for machine control in the form of a CAN to
Ethernet gateway 9 is also present. Access to WLAN and to amachine control unit 12 connected via aCAN bus line 11 is provided through this. It is also, for example, possible to access a corresponding bus system in parallel via an ISO bus line 13 (shown dashed). In this case, thegateway 9 also comprises a corresponding ISO bus interface. - Whereas according to the exemplary embodiment of
FIG. 2 the digitalimage acquisition units 4 are only connected to oneEthernet switch 7, which in the case of a drawnharvesting machine 2 is preferably arranged on the harvesting machine side, the variant embodiment according toFIG. 3 comprises a total of fourswitches 7, of which the upper one in the figure is assigned to the tractor 1, while theswitches 7 that are directly connected to the digitalimage acquisition units 4 are arranged on the side of the harvestingmachine 2. A dashedline 14 indicates the separation of the tractor 1 and the harvestingmachine 2, illustrated as boxes in the figure. - The optical
image acquisition units 4 arranged inFIG. 4 at the right-hand switch 7 on the harvesting machine side are thus then connected via a total of fourswitches 7 to amaster Panel PC 8 on the left in the figure and to a further Panel PC (slave) 8 located adjacently on the right in the figure. An ISO bus interface 15 is also present in this exemplary embodiment. Machine signals or control signals are received via this, and they, depending on the type of the signal, lead to automatic activation of a shape attracting the attention of the machine driver, via which a video stream of an opticalimage acquisition unit 4 assigned to the control signals is displayed on the display unit. The same applies for a CAN to Ethernet gateway 10. This is placed in the settings of the program running on the EDP unit. - A router that can be connected via the gateway 10 provides a
WLAN connection 16 to a remote and mobileuser terminal device 17, on which shapes can also be configured or retrieved by a web server running on the master Panel PC. Adiagnostic interface 18 and one or multiplefurther devices 20 that convey control signals can additionally be connected to the tractor-side Ethernet switch 7. - In a configuration program segment, i.e. the settings of a software program running on the
master Panel PC 8, the nine shapes represented inFIG. 4 can, for example, be configured. Between one and eightimage reproduction regions 19 can thus be furnished with video streams of the opticalimage processing units 4. - In the exemplary embodiment of
FIG. 5 , levels of an associated computer program are illustrated for the illustrated exemplary embodiment. A workingview 21 for the respective shapes is located in the upper left, from where individualimage reproduction regions 19 can be edited according tobox 22. An input facility on the touchscreen is assigned, moreover, to the workingview 21, via which the operating personnel reach asettings level 23 from where ashape management 24, acamera management 25, asequence management 26, aweb server management 27 anduser settings 28 can be reached. - The shapes are configured in the
shape management 24. Settings, if present, of theimage acquisition units 4, for example the color configuration, com-pression or resolution, can be made in thecamera management 25. Thesequence management 26 allows the saving of individualimage reproduction segments 19, or the management of the individual video streams. A web page repre-sentation of individual shapes is correspondingly configured in theweb server management 27; an access management can, in particular, be configured here, with which third parties can only log on to the web server of the EDP unit with a password from a user terminal device that is preferably mobile, or is at least remote from the harvesting machine. Finally, the user profile can be stored or opened in theuser management 28. It is also possible to set whether the system always starts up with the most recently used user profile, or with a standard configuration. - According to the exemplary embodiment of
FIG. 6 , a shape is designed for the reproduction of five video streams on the basis of the fiveimage reproduction regions 19. The associated configuration interface of the shape can be opened directly by means of a correspondingly assignedicon 29. Here, according toFIG. 7 , individualimage reproduction regions 19 are again first formed, and, while these are reduced in size, they correspond otherwise to those ofFIG. 6 . The top-mostimage reproduction region 19, for example, shows the video stream of the upper left image reproduction region inFIG. 6 at a reduced size, theimage reproduction region 19 located inFIG. 7 below the image reproduction region in the second row shows the second video stream in the upper row from the left according toFIG. 6 , and so on. The occupancy of theimage reproduction regions 4 with the opticalimage acquisition units 4 that are available is made by the finger-operatedfield 31. Tapping this field, or thefurther fields 31, displays a drop-down list where the cameras known to the system can then be selected. The regions of the touchscreen located on the lower left inFIG. 7 are a button for returning to the previous dialogue (button 32), a help button 33, and an information button 34.
Claims (23)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102019116961.5A DE102019116961A1 (en) | 2019-06-24 | 2019-06-24 | Method for observing work processes of an agricultural machine, digital video system and agricultural machine |
DE102019116961.5 | 2019-06-24 | ||
PCT/EP2020/067760 WO2020260427A1 (en) | 2019-06-24 | 2020-06-24 | Method for monitoring working processes of an agricultural machine, digital video system, and agricultural machine |
Publications (2)
Publication Number | Publication Date |
---|---|
US20220337785A1 true US20220337785A1 (en) | 2022-10-20 |
US11963477B2 US11963477B2 (en) | 2024-04-23 |
Family
ID=71661808
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/622,016 Active 2040-07-08 US11963477B2 (en) | 2019-06-24 | 2020-06-24 | Method for observing working processes of an agricultural machine, digital video system and agricultural machine |
Country Status (9)
Country | Link |
---|---|
US (1) | US11963477B2 (en) |
EP (1) | EP3986112A1 (en) |
JP (2) | JP2022538241A (en) |
CN (1) | CN114096146A (en) |
AU (1) | AU2020303258A1 (en) |
CA (1) | CA3145156A1 (en) |
DE (1) | DE102019116961A1 (en) |
IL (1) | IL289361A (en) |
WO (1) | WO2020260427A1 (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090147131A1 (en) * | 2007-12-07 | 2009-06-11 | Canon Kabushiki Kaisha | Image editing apparatus, image editing method, and storage medium storing image editing program |
US20110119716A1 (en) * | 2009-03-12 | 2011-05-19 | Mist Technology Holdings, Inc. | System and Method for Video Distribution Management with Mobile Services |
US20170112056A1 (en) * | 2015-10-23 | 2017-04-27 | Carnegie Mellon University | System for evaluating agricultural material |
US20180084719A1 (en) * | 2016-09-27 | 2018-03-29 | Claas Selbstfahrende Erntemaschinen Gmbh | Crop flow monitoring of a crop gathering device |
US20190116717A1 (en) * | 2016-06-03 | 2019-04-25 | Kverneland Group Kerteminde As | Method and device for operating an agricultural machine |
US20200128721A1 (en) * | 2018-10-31 | 2020-04-30 | The Climate Corporation | Automated sample collection and tracking system |
Family Cites Families (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3162898B2 (en) * | 1992-12-28 | 2001-05-08 | 三洋電機株式会社 | Multi-screen display device |
JP2005175701A (en) * | 2003-12-09 | 2005-06-30 | Hitachi Building Systems Co Ltd | Remote video image monitoring system |
JP4514778B2 (en) * | 2004-01-22 | 2010-07-28 | 株式会社日立国際電気 | Video distribution device |
JP4349412B2 (en) * | 2006-12-12 | 2009-10-21 | ソニー株式会社 | Monitoring device and monitoring method |
US7677169B2 (en) * | 2007-09-06 | 2010-03-16 | Cnh America Llc | Apparatus and method using imaging in building a cotton module |
DE202008009834U1 (en) * | 2008-07-22 | 2009-12-10 | Liebherr-Elektronik Gmbh | Camera for use in a digital network |
JP2010278757A (en) * | 2009-05-28 | 2010-12-09 | Canon Inc | Broadcast receiving device, method of controlling the same, and computer program of the same |
JP5465488B2 (en) * | 2009-08-20 | 2014-04-09 | ヤンマー株式会社 | Combine |
US9345194B2 (en) | 2009-09-30 | 2016-05-24 | Cnh Industrial America Llc | Automatic display of remote camera image |
JP5155279B2 (en) | 2009-10-29 | 2013-03-06 | 株式会社日立製作所 | Centralized monitoring system and centralized monitoring method using multiple surveillance cameras |
US8451139B2 (en) * | 2010-02-22 | 2013-05-28 | Cnh America Llc | System and method for coordinating harvester and transport vehicle unloading operations |
JP5521690B2 (en) * | 2010-03-26 | 2014-06-18 | 日本電気株式会社 | Screen data distribution system, server, screen data distribution method, and program |
JP5829824B2 (en) * | 2011-03-29 | 2015-12-09 | セコム株式会社 | Monitoring device and program |
JP5920702B2 (en) * | 2012-02-07 | 2016-05-18 | 株式会社熊平製作所 | Digital recording device |
DE102012107550A1 (en) | 2012-08-17 | 2014-05-15 | Claas Selbstfahrende Erntemaschinen Gmbh | Display device for agricultural machinery |
WO2014050180A1 (en) * | 2012-09-26 | 2014-04-03 | 株式会社日立国際電気 | Image display system, operation terminal, operation method and program |
JP2014117243A (en) * | 2012-12-18 | 2014-06-30 | Yanmar Co Ltd | Combine harvester |
US20140240506A1 (en) * | 2013-02-22 | 2014-08-28 | Caterpillar Inc. | Display System Layout for Remote Monitoring of Machines |
EP2798939B1 (en) * | 2013-04-29 | 2018-06-20 | CLAAS E-Systems KGaA mbH & Co KG | Operating system for and method of operating a controllable transfer device for harvested goods |
WO2014183042A1 (en) * | 2013-05-10 | 2014-11-13 | Cnh Industrial America Llc | Control architecture for multi-robot system |
JP2015100075A (en) * | 2013-11-20 | 2015-05-28 | 株式会社東芝 | Image processing device, system and method |
JP6364838B2 (en) * | 2014-03-14 | 2018-08-01 | 日本電気株式会社 | Video processing apparatus and video processing method |
JP6134301B2 (en) * | 2014-09-29 | 2017-05-24 | 日立建機株式会社 | Video transmission system for remote control of construction machinery |
RU2568695C1 (en) | 2014-11-12 | 2015-11-20 | Общество с ограниченной ответственностью "Научно-производственное предприятие "Резонанс" | Working machine indicator |
DE102015115883A1 (en) * | 2015-09-21 | 2017-03-23 | Claas E-Systems Kgaa Mbh & Co Kg | Agricultural working machine |
CN106888370A (en) * | 2017-03-31 | 2017-06-23 | 江苏大学 | It is a kind of based on embedded with technology of Internet of things united reaper video monitoring system |
JP7039938B2 (en) * | 2017-06-27 | 2022-03-23 | ソニーグループ株式会社 | Information processing equipment and methods, as well as information processing systems |
JP6914119B2 (en) * | 2017-06-28 | 2021-08-04 | 三菱マヒンドラ農機株式会社 | combine |
DE102017117287A1 (en) * | 2017-07-31 | 2019-01-31 | Mekra Lang Gmbh & Co. Kg | VISIBILITY SYSTEM WITH DRIVING SITUATION-ORIENTED VIEW FIELD / VIEW AREA INSIGHT |
JP7077557B2 (en) * | 2017-09-26 | 2022-05-31 | ソニーグループ株式会社 | Display control device and display control method |
US10412889B2 (en) * | 2017-12-05 | 2019-09-17 | Deere & Company | Combine harvester control information for a remote user with visual feed |
-
2019
- 2019-06-24 DE DE102019116961.5A patent/DE102019116961A1/en active Pending
-
2020
- 2020-06-24 US US17/622,016 patent/US11963477B2/en active Active
- 2020-06-24 WO PCT/EP2020/067760 patent/WO2020260427A1/en unknown
- 2020-06-24 CA CA3145156A patent/CA3145156A1/en active Pending
- 2020-06-24 AU AU2020303258A patent/AU2020303258A1/en active Pending
- 2020-06-24 EP EP20742171.0A patent/EP3986112A1/en active Pending
- 2020-06-24 CN CN202080046757.8A patent/CN114096146A/en active Pending
- 2020-06-24 JP JP2021576700A patent/JP2022538241A/en active Pending
-
2021
- 2021-12-23 IL IL289361A patent/IL289361A/en unknown
-
2024
- 2024-02-02 JP JP2024014851A patent/JP2024050768A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090147131A1 (en) * | 2007-12-07 | 2009-06-11 | Canon Kabushiki Kaisha | Image editing apparatus, image editing method, and storage medium storing image editing program |
US20110119716A1 (en) * | 2009-03-12 | 2011-05-19 | Mist Technology Holdings, Inc. | System and Method for Video Distribution Management with Mobile Services |
US20170112056A1 (en) * | 2015-10-23 | 2017-04-27 | Carnegie Mellon University | System for evaluating agricultural material |
US20190116717A1 (en) * | 2016-06-03 | 2019-04-25 | Kverneland Group Kerteminde As | Method and device for operating an agricultural machine |
US20180084719A1 (en) * | 2016-09-27 | 2018-03-29 | Claas Selbstfahrende Erntemaschinen Gmbh | Crop flow monitoring of a crop gathering device |
US20200128721A1 (en) * | 2018-10-31 | 2020-04-30 | The Climate Corporation | Automated sample collection and tracking system |
Also Published As
Publication number | Publication date |
---|---|
IL289361A (en) | 2022-02-01 |
CN114096146A (en) | 2022-02-25 |
AU2020303258A1 (en) | 2022-01-27 |
US11963477B2 (en) | 2024-04-23 |
JP2024050768A (en) | 2024-04-10 |
JP2022538241A (en) | 2022-09-01 |
EP3986112A1 (en) | 2022-04-27 |
WO2020260427A1 (en) | 2020-12-30 |
DE102019116961A1 (en) | 2020-12-24 |
CA3145156A1 (en) | 2020-12-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11254263B2 (en) | Vehicular vision system | |
US20030222982A1 (en) | Integrated video/data information system and method for application to commercial vehicles to enhance driver awareness | |
CN101321271B (en) | Information processing apparatus and information processing method | |
EP2798939B1 (en) | Operating system for and method of operating a controllable transfer device for harvested goods | |
WO2015145736A1 (en) | Monitoring system and monitoring method | |
EP2892228A1 (en) | Selective capture and presentation of native image portions | |
US11240542B2 (en) | System and method for multiple video playback | |
CN104349124A (en) | Structure and method for expanding multi-screen display on video recorder | |
US9148613B1 (en) | Systems, methods, and software for merging video viewing cells | |
CN106412463A (en) | Playing instruction method of dynamically switching video and apparatus thereof | |
CN205864589U (en) | Video cascade instructor in broadcasting's equipment | |
CN113852795B (en) | Video picture adjusting method and device, electronic equipment and storage medium | |
US10889958B2 (en) | Display system for machine | |
US11963477B2 (en) | Method for observing working processes of an agricultural machine, digital video system and agricultural machine | |
WO2006040218A1 (en) | System for the detection and reproduction of images | |
EP3001162A1 (en) | Sound source visualisation system and conversion unit | |
US11863712B1 (en) | Daisy chaining dash cams | |
CN208021321U (en) | A kind of vehicle-mounted viewing system of SD input high definition output | |
RU2803203C2 (en) | Method for tracking working processes of agricultural machine, digital video system and agricultural machine | |
DE102019129865A1 (en) | DEVICE FOR DETERMINING THE DRIVING SITUATION OF A VEHICLE AND CONTROL PROCEDURE FOR IT | |
CN111078085B (en) | Information processing apparatus, control method of information processing apparatus, and storage medium | |
CN110855886A (en) | Unmanned platform remote control operation visual sound system | |
CN202374389U (en) | Camera monitoring equipment | |
CN108924602B (en) | Flat-level multi-channel large-screen cloud interaction system and interaction method | |
JP2654697B2 (en) | Measurement control system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: GRIMME LANDMASCHINENFABRIK GMBH & CO. KG, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WITTE, MARKUS;STROTHMANN, WOLFRAM;SCHOMAKER, AXEL;SIGNING DATES FROM 20220510 TO 20220511;REEL/FRAME:060399/0851 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
ZAAB | Notice of allowance mailed |
Free format text: ORIGINAL CODE: MN/=. |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
CC | Certificate of correction |