EP3117261A1 - Assistance system for a piece of agricultural machinery, and method for assisting an operator - Google Patents
Assistance system for a piece of agricultural machinery, and method for assisting an operatorInfo
- Publication number
- EP3117261A1 EP3117261A1 EP15707627.4A EP15707627A EP3117261A1 EP 3117261 A1 EP3117261 A1 EP 3117261A1 EP 15707627 A EP15707627 A EP 15707627A EP 3117261 A1 EP3117261 A1 EP 3117261A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- assistance system
- operator
- information
- agricultural
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
- G02B2027/0165—Head-up displays characterised by mechanical features associated with a head-down display
Definitions
- the invention relates to an assistance system for an agricultural
- Such an assistance system is known, for example, from EP 1 674 324 A1.
- a head-up display is shown in order to show the driver information such as crop parameters or operating parameters directly into the field of view. This makes it possible for the operator to read these parameters from the head-up display without averting the view from the field to be processed.
- the object of the present invention is therefore an assistance system
- the assistance system comprises a mobile display unit adapted to display to the operator information superimposed on the agricultural working environment in the sense of an augmented reality. Because the assistance system comprises a mobile display unit, it is basically possible to display different information depending on the position of the display unit. Thus, the operator can hold the mobile display unit in a particular direction to the work environment to properly display the information desired thereto. As a result, it is no longer necessary to display all the information on the display unit at the same time or to switch it over using the keyboard.
- the agricultural working machine can be a tractor, like
- a tractor for example, a tractor, or a combine harvester.
- a tillage implement On the working machine a tillage implement, a seeder a fertilizer spreader, a sprayer or the like may be arranged.
- the work machine may include a cab for the operator.
- the agricultural work environment may include a field and / or the cab of the work machine. It is also conceivable that the working environment includes an interior and / or exterior of the work machine, such as
- the agricultural working environment comprises a stable, a supply of feed and / or an animal.
- the agricultural work environment may include anything, including the operator in the
- the assistance system can be a data processing unit with a
- the data processing unit can be designed to be mobile Display unit and / or the content displayed on it (images of the working environment, the information and their positions, graphics, text, labels, notes, etc.) to calculate.
- the mobile display unit may include the data conditioning unit. It is also conceivable that the mobile
- Display unit is connected via a wired or wireless data interface with the data processing unit.
- the wireless data interface can be used as a WLAN, Bluetooth or infrared interface, as well as CAN, ISOBUS or NFC or similar. be educated.
- Mobile display unit may mean here that the display unit is movable relative to the work machine. Mobile display unit may also mean that the display unit has no fixed connection to the work machine and optionally via a cable or via a wireless data interface with the
- the information may be planning and / or management data, information on machine operation, help information for the operator (help desk) and / or training information.
- Planning and management data can mean here that it is
- Environmental information such as dew formation, weather data, geographical data or the stock temperature oa. is.
- Information about machine operation can be operating parameters (for example driving speed, direction, etc.), operating information and / or operating functions.
- Auxiliary information may be information about the virtual entity
- Operator support act For example, it can be virtual
- Customer service information includes, for example, assistance with the
- the assistance system provides information about the training. For example, a teacher can selectively display information on the mobile display unit for the operator to convey the operation of the work machine. Enriched reality can mean here that the information displayed correlates to the agricultural working environment. Likewise,
- Enriched reality means that a camera image and / or 3D data correlated to the agricultural work environment along with it
- Information is displayed.
- the information can with
- Rotationally invariant features can be pattern patterns
- Color structures that are working width, aspect ratios, etc. in the working environment.
- the mobile display unit may be a smart phone. As a result, the mobile display unit is always in the field of vision of the operator and is particularly easy to carry.
- the data glasses can be designed as a head-mounted display (HMD).
- the data glasses can be designed with a display for one eye or with two displays for both eyes of the operator. It is also conceivable that the data glasses for a single eye have multiple displays.
- the displays may be LCD, DMD or LCoS displays, for example. It is also conceivable that the data glasses have a projection unit, optionally a laser projector, with which the information is projected directly onto the retina of the operator.
- the data glasses can have a projection unit and at least one partially transparent region, by means of which the working environment is directly visible and via which the information can be faded in with the projection unit.
- the operator can directly record the working environment. At the same time the information by means of the projection unit in the partially transparent area with the
- the mobile display unit may be a tablet computer or a mobile phone. Since tablet computers or mobile phones are produced in large quantities, they are available at very low cost.
- Mobile phone may be configured with a touch-sensitive screen for input.
- the assistance system can be designed with a camera and / or with a 3D sensor for detecting the agricultural working environment.
- the working environment can be detected and displayed as part of the enriched reality on the mobile display unit.
- the camera can be designed, for example, as a CCD or CMOS camera.
- the assistance system may be adapted to the characteristics of the
- the recognition of the features can be done with an image and / or 3D data processing unit.
- the image and / or 3D data processing unit may comprise a digital filter.
- the image and / or 3D data processing unit can be integrated in the data processing unit described above.
- the assistance system can be designed with a position detection system for determining the position of the mobile display unit in relation to the agricultural working environment.
- the location detection system can be a camera, a
- Position sensor and / or a location system include.
- Assistance system comprises a location detection system, it can be particularly easily determined the orientation of the assistance system relative to the agricultural work environment. Due to the known location, the
- the position detection system is particularly simple.
- Position detection system the same camera can be used as the previously described camera for detecting the agricultural work environment. If the position detection system comprises a position sensor or a location system, the position of the assistance system can also be determined if the field of view of the camera is covered.
- the assistance system can be a microphone, a headphone and / or a
- Speaker include. This makes it possible that the operator with another person, such as a trainer or a
- the assistance system can have at least one data interface with a database, a planning and / or management system, a
- Control unit of the agricultural machine a help desk and / or a training system connectable. That's it
- Data interface can be designed as a radio interface and optionally include a GSM, WLAN, UMTS or LTE or other mobile data interface.
- a connection via a data network such as the Internet can be built.
- a data network such as the Internet
- Work environment are displayed in the context of an enriched reality, the information of the operator is displayed particularly clearly and linked to the working environment. Furthermore, the operator can direct the mobile display unit to the part of the work environment to which the
- the method may have the features previously described in relation to claims 1-11, individually or in any combination.
- the working environment with a picture and / or
- Signal processing unit are recognized and used to make positions for the
- Information to be calculated is then displayed in the mobile display unit so that it appears to the operator at the calculated positions.
- the method can be designed so that the information with the direct field of view of the operator or with a
- FIG. 1 shows a first embodiment of the assistance system with a data glasses in a perspective view.
- FIG. 2 shows a second embodiment of the assistance system with a
- FIG. 1 shows a first embodiment of the assistance system 1 with a data glasses 3 in a perspective view. You can see the field of view of an operator through a mobile display unit, here as
- Data glasses 3 is formed.
- the data glasses 3 is transparent in the areas 33 and gives a direct view of the scene 2 with the agricultural
- Work environment 21 free. It can be seen that the operator is currently working the field 21 with an agricultural machine and a plow (not shown here), wherein the plow furrows 22 with the working width A to each other. Furthermore, it can be seen that the data glasses 3 are connected via the data interfaces 37, 71 to a control unit 7 of the working machine. This allows, for example, operating parameters of
- the data glasses 3 have the projection unit 31 and the partially transparent area 32. This is arranged here in the region of the right eye of the operator, but can also run in front of the left eye or both eyes. In region 32, the operator can see directly on the field 21 as through the transparent regions 33. At the same time, information 62-65 can be displayed in area 32 by means of production unit 31.
- the data glasses 3 To capture the agricultural working environment 21, the data glasses 3 have a camera 34. In this way, the already picked plow furrows 22 can be detected and used, for example, the working width A can be determined automatically. In addition, the position of the data glasses 3 with respect to the working environment 21 is determined with the camera 34 and the position sensor 36. Together, they form the situation detection system.
- the position sensor 36 is here with
- Acceleration sensors formed and thus detects movement of the
- Data glasses 3 around all three spatial axes. This makes it possible to determine the position of the data glasses 3 with respect to the working environment 21 and to hide the information 62, 65 associated therewith in the partially transparent area 32 at the intended position.
- the data glasses 3 are formed with a microphone 38 and a headphone 35. This makes it possible for the operator to contact another person and, for example, instructions for
- Processing of the field 21 receives. Furthermore, the operator at a
- Headphone 35 corresponding instructions for editing can notify.
- the data glasses 3 are equipped with a Bluetooth data interface 37. This is by radio with a corresponding Bluetooth data interface 71 of
- Control unit 7 connected to the work machine. This makes it possible that digital data between the data glasses 3 and the control unit 7 are exchanged, such as operating parameters of the working machine. Furthermore, the control unit 7 is connected to the Internet via a UMTS mobile radio interface 72. This makes it possible for the assistance system 1 to communicate with a database, a server application and / or another person via the Internet.
- the data glasses 3 have a data processing unit 39, in which all data for the operation of the data glasses 3 are processed.
- the data include camera images of the camera 34, audio data of the microphone 38 and the headphone 35, position data of the position sensor 36, data streams from / to the
- Data processing unit 39 is formed with a microprocessor and a memory.
- the assistance system 1 is used as follows: The operator puts on the data glasses 3 similar to a normal glasses and begins with the fieldwork. The plow furrows 22 are generated at a distance A on the field 21. The operator is displayed via the data glasses 3 (mobile display unit) the information 62-65 superimposed in the sense of an enriched reality with a direct view of the working environment 21.
- the arrow 64 indicates the speed on the scale 63.
- the target speed is reached when the arrow 64 is in the center of the scale 63.
- the operator receives a feedback, if the
- the arrow 62 along the virtual Pflugfurche 65 indicates the still to be plowed furrow in advance. If the operator follows the virtual furrow 65, then the arrow 62 is located exactly on the virtual furrow 65. If the predicted travel path of the working machine deviates from this, the arrow 62 becomes corresponding to the virtual furrow 65 twisted so that the
- FIG. 2 shows a second embodiment of the assistance system 1 with a tablet computer 5. You can see the working machine 4 with the
- the cab 41 is here the agricultural
- the driver's cab 41 has the dashboard 42 with the screen 43, the display elements 44 and the switches 45. Furthermore, the steering wheel 46 can be seen, with which the operator can perform corresponding control movement of the working machine 4.
- the operator is holding the tablet computer 5 with his or her hand 1 1 in order to receive instructions for operating the driver's cab 41 from a support employee.
- the tablet computer 5 is a touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive touch-sensitive
- Screen 51 has.
- the camera image 61 of a detail of the dashboard 52 is shown on it. Furthermore, it can be seen that the
- Operation information 66-68 with the camera image 61 are displayed overlaid.
- the tablet computer has for this purpose on the back of a camera 53, which is adapted to capture a portion of the work environment 41 behind the tablet computer 5 as a camera image 61.
- the tablet computer 5 comprises a computing unit with a computer program product executed thereon, with which the image information of the
- the tablet computer 5 has a button 52 to one
- the operator can contact a training manager or service technician via
- Tablet computer 5 particularly flexible use.
- the assistance system 1 is used as follows: The tablet computer 5 is directed with the camera 53 onto the dashboard 52.
- the assistance system 1 now recognizes by an automatically running image processing in the camera image 61, the display elements 44 and the switch 45.
- This information is now superimposed in terms of an enriched reality with the cab 41 displayed on the screen 51.
- corresponding instructions for operating the work machine 4 are displayed in the information text 68, the arrows 66 and 67 correlated with the display elements 44 and the switch 45 are displayed.
- even complex information can be displayed ergonomically and particularly easily readable by the assistance system 1.
- the data glasses 3 in FIG. 1 can also be used for an application in the driver's cab 41 of the working machine 4 as described in FIG. 2. It is also conceivable that the tablet computer 5 is directed from the Fig. 2 on the scenery 2 of Fig. 1 and the operator information for editing the field 21 are displayed. Overall, the assistance system 1 according to the first and second embodiments are widely used. Conceivable here are planning and / or management tasks, provision of operating parameters,
- Control and operating tasks for the working machine 4 inquiries to a help desk and / or training.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102014103195.4A DE102014103195A1 (en) | 2014-03-11 | 2014-03-11 | Assistance system for an agricultural work machine and method for assisting an operator |
PCT/EP2015/054336 WO2015135786A1 (en) | 2014-03-11 | 2015-03-03 | Assistance system for a piece of agricultural machinery, and method for assisting an operator |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3117261A1 true EP3117261A1 (en) | 2017-01-18 |
Family
ID=52598755
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15707627.4A Pending EP3117261A1 (en) | 2014-03-11 | 2015-03-03 | Assistance system for a piece of agricultural machinery, and method for assisting an operator |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP3117261A1 (en) |
DE (1) | DE102014103195A1 (en) |
WO (1) | WO2015135786A1 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE202015002701U1 (en) | 2015-04-10 | 2016-07-13 | Alois Pöttinger Maschinenfabrik Ges.m.b.H. | Agricultural working tool |
DE102018203910B3 (en) * | 2018-03-14 | 2019-06-13 | Audi Ag | Driver assistance system and method for a motor vehicle to display an augmented reality |
WO2020178694A1 (en) | 2019-03-07 | 2020-09-10 | 3D2Cut Sa | System and method for assisting with the pruning of plants |
DE102019125348A1 (en) | 2019-09-20 | 2021-03-25 | 365FarmNet Group GmbH & Co. KG | Method for supporting a user in an agricultural activity |
DE102021203631A1 (en) | 2021-04-13 | 2022-10-13 | Gixel GmbH | Spectacles display system for displaying a virtual image in a user's field of view by means of a linear display unit |
DE102021109330A1 (en) * | 2021-04-14 | 2022-10-20 | Deere & Company | Augmented reality perception method for an agricultural utility vehicle |
DE102021127668A1 (en) | 2021-10-25 | 2023-04-27 | Horsch Leeb Application Systems Gmbh | Portable device and computer-implemented method for displaying information on a machine component of an agricultural machine from a reference work |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1674324A1 (en) * | 2004-12-22 | 2006-06-28 | CLAAS Selbstfahrende Erntemaschinen GmbH | Agricultural working machine |
US20130335303A1 (en) * | 2012-06-14 | 2013-12-19 | Qualcomm Incorporated | User interface interaction for transparent head-mounted displays |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4555725A (en) * | 1983-08-24 | 1985-11-26 | Deutz-Allis Corporation | Agricultural implement steering guidance system and method |
US5751576A (en) * | 1995-12-18 | 1998-05-12 | Ag-Chem Equipment Co., Inc. | Animated map display method for computer-controlled agricultural product application equipment |
US9440591B2 (en) * | 2009-05-13 | 2016-09-13 | Deere & Company | Enhanced visibility system |
DE102013106128A1 (en) * | 2012-07-16 | 2014-06-12 | Claas Selbstfahrende Erntemaschinen Gmbh | Agricultural work machine with at least one control device |
-
2014
- 2014-03-11 DE DE102014103195.4A patent/DE102014103195A1/en not_active Withdrawn
-
2015
- 2015-03-03 WO PCT/EP2015/054336 patent/WO2015135786A1/en active Application Filing
- 2015-03-03 EP EP15707627.4A patent/EP3117261A1/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1674324A1 (en) * | 2004-12-22 | 2006-06-28 | CLAAS Selbstfahrende Erntemaschinen GmbH | Agricultural working machine |
US20130335303A1 (en) * | 2012-06-14 | 2013-12-19 | Qualcomm Incorporated | User interface interaction for transparent head-mounted displays |
Non-Patent Citations (1)
Title |
---|
See also references of WO2015135786A1 * |
Also Published As
Publication number | Publication date |
---|---|
WO2015135786A1 (en) | 2015-09-17 |
DE102014103195A1 (en) | 2015-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2015135786A1 (en) | Assistance system for a piece of agricultural machinery, and method for assisting an operator | |
DE112006002954B4 (en) | Virtual interface system | |
EP3394708B1 (en) | Method for operating a virtual reality system, and virtual reality system | |
DE102016215199A1 (en) | Virtual head-up display application for a work machine | |
DE102014009697A1 (en) | Method for operating a mobile virtual reality system in a motor vehicle and mobile virtual reality system | |
EP3298474A1 (en) | Method for operating a head-mounted display in a motor vehicle, and system comprising a head-mounted display | |
EP2171640A1 (en) | Information device, method for informing and/or navigating a person, and computer program | |
DE102005011126A1 (en) | Generation of virtual worlds based on a real environment | |
DE102017116558A1 (en) | Method for guiding movement sequences and training device for guiding movement sequences | |
DE112013006544T5 (en) | Image processing apparatus and image processing method | |
DE102014213285B4 (en) | Head direction-dependent display of content on data glasses | |
DE102014009299A1 (en) | Method for operating a virtual reality glasses and system with a virtual reality glasses | |
DE102013100569A1 (en) | Method for displaying surrounding of vehicle of vehicle assembly and training system, involves detecting three-dimensional image data of surrounding by detection device arranged at vehicle | |
EP3434087B1 (en) | Method for supporting a machine operator when adjusting an agricultural soil working machine | |
DE102014018056A1 (en) | Method of operating a virtual reality glasses and virtual reality glasses | |
WO2016184972A1 (en) | Method for operating data glasses in a motor vehicle and system having data glasses | |
DE102019103360A1 (en) | Method and device for operating a display system with data glasses | |
EP3418839A1 (en) | Method for controlling an automation assembly | |
EP3967538A1 (en) | Method for depicting a virtual element | |
EP3568737B1 (en) | Method for operating a display system with augmented reality glasses in a motor vehicle | |
DE102019112189A1 (en) | Method and device for displaying an information object in augmented reality data glasses | |
EP3649538A1 (en) | Assembly and method for communicating by means of two visual output devices | |
AT518667B1 (en) | Driving Safety Training arrangement | |
DE102019119532A1 (en) | Method and device for operating a display system with data glasses in a motor vehicle | |
DE102016225267B4 (en) | Data glasses and method for operating data glasses |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20160111 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20200602 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
RAP3 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: AMAZONEN-WERK H. DREYER SE & CO. KG |
|
RAP3 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: AMAZONEN-WERKE H. DREYER SE & CO. KG |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230524 |