CN112883753B - Auxiliary operation method of forklift and electronic equipment - Google Patents
Auxiliary operation method of forklift and electronic equipment Download PDFInfo
- Publication number
- CN112883753B CN112883753B CN201911195682.5A CN201911195682A CN112883753B CN 112883753 B CN112883753 B CN 112883753B CN 201911195682 A CN201911195682 A CN 201911195682A CN 112883753 B CN112883753 B CN 112883753B
- Authority
- CN
- China
- Prior art keywords
- user interface
- electronic device
- forklift
- lock
- interface element
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003780 insertion Methods 0.000 claims description 22
- 230000037431 insertion Effects 0.000 claims description 22
- 238000003860 storage Methods 0.000 claims description 21
- 230000004044 response Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 239000007787 solid Substances 0.000 claims description 10
- 238000012545 processing Methods 0.000 description 20
- 238000010586 diagram Methods 0.000 description 19
- 230000006870 function Effects 0.000 description 10
- 230000005540 biological transmission Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 239000012790 adhesive layer Substances 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 238000004148 unit process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66F—HOISTING, LIFTING, HAULING OR PUSHING, NOT OTHERWISE PROVIDED FOR, e.g. DEVICES WHICH APPLY A LIFTING OR PUSHING FORCE DIRECTLY TO THE SURFACE OF A LOAD
- B66F9/00—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes
- B66F9/06—Devices for lifting or lowering bulky or heavy goods for loading or unloading purposes movable, with their loads, on wheels or the like, e.g. fork-lift trucks
- B66F9/075—Constructional features or details
- B66F9/0755—Position control; Position detectors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Structural Engineering (AREA)
- Transportation (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Civil Engineering (AREA)
- Life Sciences & Earth Sciences (AREA)
- Geology (AREA)
- Mechanical Engineering (AREA)
- Forklifts And Lifting Vehicles (AREA)
Abstract
The present disclosure relates to a forklift auxiliary operation method and an electronic device. A method of forklift assisted operation performed by an electronic device, comprising: receiving an image captured by a camera from a camera mounted at a center of a fork truck boom tip of a fork truck and configured to capture an image in front of the fork truck boom tip; and in the display mode, displaying an image from the camera in real time. The auxiliary operation method of the forklift can help operators to accurately position the forklift arm, and damage to the forklift arm or a forklift object is avoided.
Description
Technical Field
The present disclosure relates to forklifts, and more particularly, to forklifts having an auxiliary operating system, an auxiliary operating system for forklifts, and related methods of operation.
Background
When operating the forklift, it is difficult for the operator to accurately operate the forklift arm into a small insertion opening, such as a small gap, and damage to the forklift arm or the object to be forklift or the like is easily caused by inaccuracy of operation.
Disclosure of Invention
According to some embodiments of the present disclosure, a forklift auxiliary operation method performed by an electronic device includes: receiving an image captured by a camera from a camera mounted at a center of a fork truck boom tip of a fork truck and configured to capture an image in front of the fork truck boom tip; and in the display mode, displaying an image from the camera in real time.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving a first user input entering a positioning mode via a user interface on the electronic device; in response to receiving a first user input to enter a locate mode: displaying a reference center point on the user interface, the reference center point being a geometric center point of a display area for display of an image of one camera; a lock user interface element is displayed on the user interface, the lock user interface element having a lock region center point displayed.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving, via a user interface on the electronic device, a first user operation input by a user operating the lock user interface element; wherein the first user operational input comprises one or more of: moving; amplifying; and shrinking.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving, via a user interface on the electronic device, a second user operation input for a user to operate the lock user interface element, the second user operation input for locking image content covered by the lock user interface element; in response to receiving the second user operation input, the lock user interface element is caused to move along with the image content to which it is locked when the image content of the displayed camera changes.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving a third user input completing the positioning via a user interface on the electronic device; in response to receiving a third user input, a state of the lock user interface element is changed to indicate that positioning is complete.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving a fourth user input entering an auxiliary mode via a user interface on the electronic device; in response to receiving the fourth user input, entering an assist mode, wherein in the assist mode, if a lock region center point of the lock user interface element is mismatched from the reference center point, a state of the lock user interface element is changed to indicate the mismatch.
According to some embodiments of the present disclosure, before receiving a third user input completing positioning via a user interface on the electronic device, the forklift auxiliary operation method further comprises: and outputting a prompt of successful positioning through a user interface when the center point of the locking area of the locking user interface element is matched with the reference center point.
According to some embodiments of the present disclosure, the lock user interface element is a rectangular box, and the center point of the lock area is the geometric center point of the rectangular box.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: in the positioning mode, changing the state of the lock user interface element to indicate positioning completion by changing a rectangular box indicated by a dotted line to a rectangular box indicated by a solid line; and in the assist mode, changing the state of the lock user interface element to indicate the mismatch by changing the rectangular box represented by the solid line to the rectangular box represented by the dashed line.
According to some embodiments of the present disclosure, the reference center point is represented by a dashed circle and the lock region center point of the lock user interface element is represented by a solid circle.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: receiving a user selection of a connection mode via a user interface on the electronic device; and connecting the electronic device and the camera in a selected connection mode.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: in the case of the forklift having two forklift arms, images captured by respective cameras on the two forklift arms are displayed in the two display areas, respectively.
According to some embodiments of the present disclosure, the forklift auxiliary operation method further includes: in the case of the forklift having two forklift arms, corresponding lock user interface elements are set for images captured by two cameras respectively displayed via a user interface.
According to some embodiments of the present disclosure, the camera is mounted in a cavity provided inside the tip of the forklift arm for accommodating the camera and photographing is performed via an opening of the cavity on an end face of the tip.
According to some embodiments of the present disclosure, there is provided an electronic device including: a memory having stored thereon computer program instructions; an electronic device; and a processor coupled with the memory and the electronic device, wherein the processor, when executing the computer program instructions stored on the memory, causes the electronic device to perform the fork lift truck assisted method as described above.
According to some embodiments of the present disclosure, there is provided a computer instruction storage medium having stored thereon computer program instructions which, when executed by a processor of an electronic device, cause the electronic device to perform a fork lift truck assisted operation method as described above.
According to some embodiments of the present disclosure, there is provided a forklift auxiliary operation device including means for performing the operations of the forklift auxiliary operation method as described above.
Drawings
Fig. 1A is a schematic diagram of a forklift with an auxiliary operating system according to an embodiment of the present disclosure.
Fig. 1B shows a schematic view of a camera-mounted forklift arm according to an embodiment of the present disclosure.
Fig. 2A is a schematic diagram of a forklift arm for mounting a camera according to an embodiment of the present disclosure.
Fig. 2B is a schematic diagram of a camera-mounted forklift arm according to an embodiment of the present disclosure.
Fig. 3 illustrates an exemplary flowchart of a method of fork truck assisted operation performed by an electronic device, according to an embodiment of the present disclosure.
Fig. 4A-4H illustrate exemplary graphical user interfaces related to a fork-lift assisted method of operation performed by an electronic device, in accordance with embodiments of the present disclosure.
Fig. 5 is a schematic diagram illustrating a general hardware environment in which an electronic device according to an embodiment of the present disclosure may be implemented.
Detailed Description
The following description is presented to enable one of ordinary skill in the art to make and use the embodiments and is provided in the context of a particular application and its requirements. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments and applications without departing from the spirit or scope of the embodiments. Thus, the embodiments are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.
In this disclosure, the user interface may be an example of one or more graphical User Interfaces (UIs) used to implement various features, processes, or workflows. These GUIs can be presented on a variety of electronic devices including, but not limited to, laptop computers, desktop computers, computer terminals, television systems, tablet computers, electronic book readers, and smart phones. One or more of these electronic devices may incorporate a touch-sensitive surface. The touch sensitive surface is capable of processing multiple simultaneous input points, including processing data regarding pressure, degree, or location of each input point. The process can facilitate gestures using multiple fingers, including gathering and sweeping.
When referring to a user interface element in a "select" GUI in this disclosure, these terms are understood to include clicking or "hovering" over the user interface element using a mouse or other input device, or touching, tapping or gesturing over the user interface element using one or more fingers or a stylus. The user interface elements may be virtual buttons, menus, selectors, switches, sliders, erasers, knobs, thumbnails, links, icons, radial buttons, check boxes, and any other mechanism for receiving input from a user or providing feedback to a user.
SUMMARY
The present disclosure relates to an auxiliary operating system for a forklift, the auxiliary operating system including a camera mounted at a center of a forklift arm end of the forklift and an electronic device coupled to the camera, the camera capturing an image of a front of the forklift arm end, the electronic device capturing and displaying the captured image of the camera, helping an operator of the forklift to observe a positioning of the forklift arm, and operating the forklift arm accordingly to accurately position the forklift arm to a position aligned with an insertion opening.
The electronic device has a user interface via which an operator may interact with the electronic device to better operate the forklift arm with the auxiliary operating system provided by the present disclosure.
Fig. 1A is a schematic diagram of a forklift 100 with an auxiliary operating system according to an embodiment of the present disclosure.
As shown in fig. 1A, the forklift 100 includes an operator cabin 102 and a forklift arm 104. The truck 100 also includes an auxiliary operating system including a camera 106 mounted at the end of the truck arm 104 and electronics 108 mounted within the operator's compartment 102, for example, or generally used within the compartment by an operator. The camera 106 may capture images in front of the distal end of the forklift arm and the electronic device 108 may display the images captured by the camera 106 in real time.
An operator may better operate the forklift arm to work based on images/pictures in front of the distal end of the forklift arm 104 that are displayed in real time by the electronic device, reducing the risk of damaging the object being forked.
The camera 106 may be any camera device that can be used to take a photograph and is suitable for mounting at the end of the forklift arm 104. The camera 106 may be equipped with a WiFi or bluetooth module, for example, for wireless connection with an electronic device.
The number of the forklift arms on the forklift can be one or more, and each forklift arm can be provided with a camera. In this embodiment, for convenience of description, a forklift having two side-by-side forklift arms and one camera mounted on each forklift arm will be described as an example. Fig. 1B shows a schematic view of cameras 106 1 and 106 2 mounted on the left and right forklift arms of a forklift.
In some embodiments, the electronic device 108 may be a fork-lift truck-internal integrated electronic device. In other embodiments, the electronic device 108 may be an electronic device mounted in place within a forklift operator's compartment. The electronic device 108 may be a stand-alone electronic device or an electronic device that is typically used by an operator within an operator compartment. The electronic device 108 may have a display and one or more processors. The electronic device 108 may be any electronic device that has an associated secondary operating application installed.
The electronic device 108 and the camera 106 may be connected by wire or wirelessly. One or more of the electronic device 108 and the camera 106 may include an associated signal processing unit (not shown) or be associated with a separate signal processing unit. The signal processing unit processes, for example, the output signal of the camera 106 into a display signal that is displayed by the electronic device 108. The signal processing unit may also be a processing unit integrated in a forklift. The signal processing unit may be, for example, a device such as a processor having data processing capabilities.
Fig. 2A is a schematic diagram of a forklift arm for mounting a camera according to an embodiment of the present disclosure. Fig. 2A illustrates a forklift arm 104. As shown in fig. 2A, a cavity 202 is provided inside the distal end of the forklift arm 104 for receiving/mounting a camera. The cavity 202 has an opening 201 in the center of the end face of the distal end of the forklift arm 104. The opening 201 is provided in the center of the end face so that the center point of the image photographed by the camera is on the center line of travel of the fork truck arm straight ahead. Here, it is assumed that the cross section of the forklift arm is a uniform rectangle, but in actual cases, the cross section of the forklift arm may not be a uniform rectangle, and those skilled in the art will understand that it is sufficient to make the shooting center of the mounted camera as on the center line of travel of the forklift arm straight ahead as possible. This enables a good operating result to be achieved by a simple structural design.
In fig. 2A, the shape of the opening 201 is shown as a rectangle. Those skilled in the art will appreciate that the shape of the opening 201 may also be circular or any other suitable shape.
Fig. 2B is a schematic diagram of another example of mounting a camera on a forklift arm according to an embodiment of the present disclosure.
As shown in fig. 2B, the camera 106 may also be mounted on the end face of the forklift arm tip. For example, the camera 106 is mounted on the truck boom tip at the center of the end face of the truck boom tip. The camera 106 may have a protective member (not shown) that does not have an edge beyond the edge of the end face of the truck arm tip to avoid damage to the camera when the truck arm 104 is in operation.
Those skilled in the art will appreciate that the camera 106 may be mounted/secured within the cavity 202 or on the end face of the distal end of the forklift arm 104 in a variety of ways known in the art, such as with an adhesive layer, screw fastening members, snap-fit structures, and the like. And will not be described in detail herein.
Those skilled in the art will appreciate that regardless of the manner in which the camera is mounted and the particular mounting location, the camera's center of capture is as consistent as possible with the center line of travel of the truck arm directly in front.
Fig. 3 illustrates an exemplary flowchart of a method 300 of fork truck assisted operation performed by an electronic device, according to an embodiment of the present disclosure. Fig. 4A-4H illustrate associated graphical user interfaces.
As shown, the method 300 includes step 301: in the display mode, images from the camera are received and displayed in real time.
Fig. 4A shows a graphical user interface in a display mode. When the user enters the display mode, for example, by clicking on "display mode", the screen captured by the camera is displayed in real time on the user interface of the electronic device. And displaying images, which are shot by cameras on the left fork truck arm and the right fork truck arm, in front of the fork truck arms on the picture.
In this mode, the user can determine whether the fork arm is aligned with the insertion opening by means of the displayed image. Since the camera is disposed in the center of the distal end of the fork arm, if the insertion port is in the center in the captured image, it can be determined that the fork arm is aligned with the insertion port.
In practice, the operator typically moves the forklift arm near the insertion opening, but it is not necessarily precisely aligned with the insertion opening. For example, in a captured image, the insertion opening is not located at the center of the screen, but is located at a position above, below, and to the left and right of the screen. At this point, the operator may perform a more accurate forklift arm positioning by clicking on the "positioning mode".
As shown in fig. 3, the method 300 further comprises step 303: receiving a first user input to enter a positioning mode via a user interface on the electronic device, and in response to receiving the first user input to enter the positioning mode: the reference center point and the lock user interface element are displayed on the user interface. Wherein the reference center point is a geometric center point of a display area for display of an image of one camera. The lock user interface element has a lock region center point displayed.
FIG. 4B illustrates an exemplary graphical user interface after entering a locate mode. The user may enter the "locate mode" by, for example, clicking on the "locate mode" to enter a first user input via the user interface. After entering the positioning mode, in each of the left and right image display areas for the two cameras, an operable lock user interface element indicated by a rectangular white dotted line box and a lock area center point of the lock user interface element indicated by a white solid line circle are displayed. The lock region center point may be a geometric center point of the lock user interface element. At the same time, a reference center point of the entire image display area is also displayed, which is indicated by a white dotted circle.
The method 300 further comprises step 305: a first user operation input is received via a user interface on the electronic device for a user to operate the lock user interface element. The first user operational input may be moved, enlarged and/or reduced.
FIG. 4C illustrates an exemplary graphical user interface after receiving a first user operation input. The user may drag and lock the user interface element, i.e., the rectangular white dashed box, to move it, and/or zoom the rectangular white dashed box so that the rectangular white dashed box just overlaps the insertion opening, which is sized to just cover the insertion opening. During this operation, the lock area center point of the lock user interface element follows the movement and scaling of the rectangular white dashed box, always being in the center of the lock user interface element.
As shown, the white solid circle (lock area center point) does not coincide (or align/match) with the white dashed circle (reference center point), i.e., at this point the fork truck arm is not precisely aligned with the insertion port.
The method 300 further comprises step 307: and receiving a second user operation input for operating the locking user interface element by a user through a user interface on the electronic equipment, wherein the second user operation input is used for locking the image content covered by the locking user interface element, and after the second user operation input is received, when the image content of the displayed camera is changed, the locking user interface element moves along with the locked image content.
In some embodiments, a user may lock image content, such as an insertion port image, encompassed by the lock user interface element, for example, by double clicking or otherwise manipulating the lock user interface element. This means that when the operator then moves (e.g. moves to the right) the fork arm causing the image of the slot to move correspondingly (e.g. also to the right) in the screen, the lock user interface element (e.g. rectangular white dashed box) locking the slot image will move with it and the centre point (e.g. white solid circle) of the lock area in the lock user interface element will move simultaneously. This intuitively prompts the operator if his or her operation on the forklift arm is further aligned with the insertion opening.
It will be appreciated by those skilled in the art that the user's confirmation of the lock user interface element is not limited to a double click, and that a long press of the lock user interface element may, for example, cause the user interface to display a prompt of whether or not to lock, and the confirmation may be performed by selecting for that prompt. In other embodiments, the confirmation operation may be performed by pressing for longer than, for example, 3 seconds.
The method 300 further comprises step 309: a third user input is received via a user interface on the electronic device to complete the positioning, and in response to receiving the third user input, a state of the lock user interface element is changed to indicate that the positioning is complete.
When the operator moves the forklift arm, the screen shot by the camera is updated in real time, and the position of the insertion opening in the screen displayed on the electronic device is also moved, and accordingly, the lock user interface element (for example, a rectangular white dotted frame) moves along with the movement of the image of the insertion opening. When the lock user interface element is moved to a position such that the lock region center point (e.g., a white solid circle) coincides with the reference center point (e.g., a white dashed circle). It will be appreciated that at this point the fork arms are accurately aligned with the insertion opening. Fig. 4D shows the graphical user interface at this time. As shown, the white solid line circle coincides with the white dotted line circle.
In some embodiments, a prompt may also be displayed on the user interface at this time to alert the operator that the forklift arm has been properly aligned with the insertion opening.
The operator may then enter a third user input via a user interface on the electronic device to complete the positioning, for example by clicking the "positioning mode" again. In response to receiving the third user input, the electronic device may change a state of the lock user interface element to indicate that positioning is complete. Fig. 4E shows an exemplary graphical user interface at this point. As shown in fig. 4E, the rectangular frame indicated by the broken line is changed to the rectangular frame indicated by the solid line. Those skilled in the art will appreciate that there are many implementations of changing the state of the lock user interface element so long as the change in state can be embodied to alert the operator that positioning is complete. For example, the rectangular frame indicated by the broken line may be changed to a rectangular frame indicated by the broken line. The change in color of the rectangular box can also be used to indicate that positioning is complete.
The method 300 further comprises step 311: receiving a fourth user input entering an auxiliary mode via a user interface on the electronic device; in response to receiving the fourth user input, entering an assist mode, wherein in the assist mode, if a lock region center point of the lock user interface element is mismatched from the reference center point, a state of the lock user interface element is changed to indicate the mismatch, such that repositioning is required, for example.
After the positioning is completed, the operator can operate the fork arm to insert the insertion slot. At this time, the operator may enter the assist mode by clicking the "assist mode", for example. During the operator's operation of the fork truck arm to insert the insertion slot, the screen captured by the camera is still updated in real time, the lock user interface element (e.g., rectangular solid white line box) moves following the movement of the image of the insertion slot, and when the lock user interface element moves to a position such that the lock area center point (e.g., solid white line circle) deviates from the reference center point (e.g., solid white line circle) due to an undesirable operation, the deviation is indicated by changing the rectangular solid white line box to a rectangular solid white line box dashed line box, as shown in fig. 4F, for example. In some embodiments, a prompt may also be displayed on the user interface to alert the operator that the forklift arm has deviated from the insertion opening.
In some embodiments, the secondary operations application may also provide functionality for the user to select which wireless connection is used to connect the electronic device to the camera. For example, as shown in fig. 4G and 4H, a "connect" button is provided on the user interface along with icons representing WiFi connections (fig. 4G) or bluetooth connections (fig. 4H). The operator may select the corresponding wireless connection mode by clicking "connect" in the graphical user interface representing the corresponding connection mode.
Fig. 5 is a schematic diagram illustrating a general hardware environment in which an electronic device 700 according to an embodiment of the present disclosure may be implemented.
Referring now to fig. 5, a schematic diagram of an example of an electronic device 700 is shown. Electronic device 700 is only one example of a suitable electronic device and is not intended to suggest any limitation as to the scope of use or functionality of the embodiments described herein. In any event, electronic device 700 is capable of implementing and/or performing any of the functions set forth above.
In the electronic device 700, there is a computer system/server 7012 that can operate in conjunction with a number of other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with computer system/server 7012 include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems or devices, and the like.
The computer system/server 7012 can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server 7012 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
As shown in fig. 5, the computer system/server 7012 in the electronic device 700 is shown in the form of a general purpose computing device. Components of computer system/server 7012 may include, but are not limited to: one or more processors or processing units 7016, a system memory 7028, a bus 7018 that couples various system components including the system memory 7028 to the processing unit 7016.
Bus 7018 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include, but are not limited to, industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, peripheral Component Interconnect (PCI) bus, peripheral component interconnect express (PCIe), and Advanced Microcontroller Bus Architecture (AMBA).
Computer system/server 7012 typically includes a variety of computer system readable media. Such media can be any available media that is accessed by computer system/server 7012 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 7028 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 7032. The computer system/server 7012 can also include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, a storage system 7034 may be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown, and commonly referred to as a "hard disk drive"). Although not shown, a magnetic disk drive may be provided for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive may be provided for reading from and writing to a removable, nonvolatile optical disk such as a CD-ROM, DVD-ROM, or other optical media. In such cases, each may be connected to bus 7018 through one or more data medium interfaces. As will be further depicted and described below, the memory 7028 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of embodiments of the present disclosure.
By way of example, and not limitation, program/utility 7040 having a set (at least one) of program modules 7042, as well as an operating system, one or more application programs, other program modules, and program data, can be stored in memory 7028. An operating system, one or more application programs, other program modules, and program data, or some combination thereof, may each include an implementation of a network environment. Program modules 7042 generally perform the functions and/or methods in the embodiments as described herein.
The computer system/server 7012 can also communicate with one or more external devices 7014 (such as a keyboard, a pointing device, an electronic device 7024, etc.), one or more devices that enable a user to interact with the computer system/server 7012, and/or any device (e.g., a network card, a modem, etc.) that enables the computer system/server 7012 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 22. Also, the computer system/server 7012 can communicate with one or more networks, such as a Local Area Network (LAN), a general Wide Area Network (WAN), and/or a public network (e.g., the internet), via the network adapter 20. As depicted, the network adapter 20 communicates with other components of the computer system/server 7012 via bus 7018. It should be appreciated that although not shown, other hardware and/or software components can be utilized in conjunction with the computer system/server 7012. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems, among others.
The present disclosure may be embodied as systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium(s) having computer-readable program instructions thereon for causing a processor to perform aspects of the present disclosure.
A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices such as punch cards or in-groove bump structures having instructions stored thereon, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, is not to be construed as a transitory signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a pulse of light through a fiber optic cable), or an electrical signal transmitted through an electrical wire.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to individual computing/processing devices or over a network (e.g., the internet, a local area network, a wide area network, and/or a wireless network) to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards these computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer readable program instructions for performing the operations of the present disclosure may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as SMALLTALK, C ++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., through the internet using an internet service provider). In some embodiments, the custom electronic circuit, including, for example, a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), may be personalized by utilizing state information of computer readable program instructions that may be executed in order to perform aspects of the disclosure.
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a process for electronic device execution, such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will also appreciate that the various operations illustrated in the order of execution in the embodiments of the disclosure are not necessarily performed in the order illustrated. The order of operations may be adjusted as desired by those skilled in the art. Those skilled in the art may add more operations or omit some of them as desired.
The description of the various embodiments of the present disclosure has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims (14)
1. A method of forklift assisted operation performed by an electronic device, comprising:
receiving an image captured by a camera from a camera mounted at a center of a fork truck boom tip of a fork truck and configured to capture an image in front of the fork truck boom tip; and
In the display mode, displaying an image from the camera in real time;
Receiving a first user input entering a positioning mode via a user interface on the electronic device;
In response to receiving a first user input to enter a locate mode:
Displaying a reference center point on the user interface, the reference center point being a geometric center point of a display area for display of an image of one camera,
Displaying a lock user interface element on the user interface, the lock user interface element having a displayed lock region center point;
Receiving, via a user interface on an electronic device, a first user operation input by a user operating the lock user interface element, wherein the first user operation input includes one or more of: moving, amplifying and shrinking;
receiving, via a user interface on the electronic device, a second user operation input for a user to operate the lock user interface element, the second user operation input for locking image content covered by the lock user interface element; and
In response to receiving the second user operation input, when the image content of the displayed camera changes, causing the lock user interface element to move along with the image content it locks,
Wherein the fork truck arm is aligned with the insertion opening when the lockout user interface element is manipulated by a user such that the lockout area center point coincides with the reference center point.
2. The forklift auxiliary operation method performed by an electronic device according to claim 1, further comprising:
Receiving a third user input completing the positioning via a user interface on the electronic device;
In response to receiving a third user input, a state of the lock user interface element is changed to indicate that positioning is complete.
3. The forklift auxiliary operation method performed by an electronic device according to claim 2, further comprising:
Receiving a fourth user input entering an auxiliary mode via a user interface on the electronic device;
In response to receiving the fourth user input, entering an auxiliary mode,
Wherein in the assist mode, if the lock region center point of the lock user interface element is mismatched from the reference center point, the state of the lock user interface element is changed to indicate the mismatch.
4. The method of forklift assistance operation performed by an electronic device of claim 2, further comprising, prior to receiving a third user input completing positioning via a user interface on the electronic device:
And outputting a prompt of successful positioning through a user interface when the center point of the locking area of the locking user interface element is matched with the reference center point.
5. The method of claim 1, wherein the lock user interface element is a rectangular frame with a lock area center point being a geometric center point of the rectangular frame.
6. The forklift-assisted operation method performed by an electronic device of claim 3, further comprising:
In the positioning mode, changing the state of the lock user interface element to indicate positioning completion by changing a rectangular box indicated by a dotted line to a rectangular box indicated by a solid line; and
In the assist mode, the state of the lock user interface element is changed to indicate the mismatch by changing the rectangular box represented by the solid line to the rectangular box represented by the dashed line.
7. The fork truck assisted operation method performed by an electronic device of claim 1, wherein the reference center point is represented by a dashed circle and the lock area center point of the lock user interface element is represented by a solid circle.
8. The forklift auxiliary operation method performed by an electronic device according to any one of claims 1 to 7, further comprising:
receiving a user selection of a connection mode via a user interface on the electronic device; and
And connecting the electronic equipment and the camera in a selected connection mode.
9. The forklift auxiliary operation method performed by an electronic device according to any one of claims 1 to 7, further comprising:
In the case of the forklift having two forklift arms, images captured by respective cameras on the two forklift arms are displayed in the two display areas, respectively.
10. The forklift auxiliary operation method performed by an electronic device according to any one of claims 1 to 7, further comprising:
in the case of the forklift having two forklift arms, corresponding lock user interface elements are set for images captured by two cameras respectively displayed via a user interface.
11. The forklift auxiliary operation method performed by an electronic device according to any one of claims 1 to 7, wherein the camera is mounted in a cavity provided inside a tip of the forklift arm for accommodating the camera and photographing is performed via an opening of the cavity on an end face of the tip.
12. An electronic device, comprising:
A memory having stored thereon computer program instructions;
An electronic device; and
A processor coupled with the memory and the electronic device, wherein the computer program instructions stored on the memory, when executed by the processor, cause the electronic device to perform the fork lift truck assisted method of any of claims 1-11.
13. A computer instruction storage medium having stored thereon computer program instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the fork lift truck assisted operation method of any of claims 1-11.
14. A forklift auxiliary operation device comprising means for performing the operation of the forklift auxiliary operation method according to any one of claims 1 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911195682.5A CN112883753B (en) | 2019-11-29 | 2019-11-29 | Auxiliary operation method of forklift and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911195682.5A CN112883753B (en) | 2019-11-29 | 2019-11-29 | Auxiliary operation method of forklift and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112883753A CN112883753A (en) | 2021-06-01 |
| CN112883753B true CN112883753B (en) | 2024-06-18 |
Family
ID=76038533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911195682.5A Active CN112883753B (en) | 2019-11-29 | 2019-11-29 | Auxiliary operation method of forklift and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112883753B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN211493853U (en) * | 2019-11-29 | 2020-09-15 | 华晨宝马汽车有限公司 | Forklift with auxiliary operating system |
| CN112883754A (en) * | 2019-11-29 | 2021-06-01 | 华晨宝马汽车有限公司 | Auxiliary operation system of forklift |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4890918A (en) * | 1988-09-15 | 1990-01-02 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Docking alignment system |
| US7219769B2 (en) * | 2001-07-17 | 2007-05-22 | Kabushiki Kaisha Toyota Jidoshokki | Industrial vehicle equipped with load handling operation control apparatus |
| US8423182B2 (en) * | 2009-03-09 | 2013-04-16 | Intuitive Surgical Operations, Inc. | Adaptable integrated energy control system for electrosurgical tools in robotic surgical systems |
| CN104954660A (en) * | 2014-03-31 | 2015-09-30 | 国际商业机器公司 | Mobile equipment, method of image information processing in mobile equipment, and device thereof |
| CN110136190A (en) * | 2019-03-26 | 2019-08-16 | 华为技术有限公司 | A kind of distance measuring method and electronic equipment |
-
2019
- 2019-11-29 CN CN201911195682.5A patent/CN112883753B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN211493853U (en) * | 2019-11-29 | 2020-09-15 | 华晨宝马汽车有限公司 | Forklift with auxiliary operating system |
| CN112883754A (en) * | 2019-11-29 | 2021-06-01 | 华晨宝马汽车有限公司 | Auxiliary operation system of forklift |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112883753A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3661187A1 (en) | Photography method and mobile terminal | |
| CN211493853U (en) | Forklift with auxiliary operating system | |
| WO2018192390A1 (en) | Photographing method of mobile terminal, and mobile terminal | |
| US20130222663A1 (en) | User interface for a digital camera | |
| US11782514B2 (en) | Wearable device and control method thereof, gesture recognition method, and control system | |
| EP2854002A1 (en) | Method and device for switching between double cameras | |
| US11334210B2 (en) | Method for window adjustment, mobile terminal, and computer-readable storage medium | |
| US8631317B2 (en) | Manipulating display of document pages on a touchscreen computing device | |
| CN105659221A (en) | Graphical user interface having enhanced tool for connecting components | |
| KR102089624B1 (en) | Method for object composing a image and an electronic device thereof | |
| CN112883754B (en) | Auxiliary operation system of forklift | |
| US10754524B2 (en) | Resizing of images with respect to a single point of convergence or divergence during zooming operations in a user interface | |
| CN112883753B (en) | Auxiliary operation method of forklift and electronic equipment | |
| WO2019071854A1 (en) | Display method for mobile apparatus, and mobile apparatus | |
| CN111046447B (en) | Authority control method and authority control system | |
| US20150015501A1 (en) | Information display apparatus | |
| WO2018058312A1 (en) | Control method, control device, and electronic device | |
| CN112875588B (en) | Forklift auxiliary operation method and electronic equipment | |
| WO2020087504A1 (en) | Screenshot interaction method, electronic device, and computer-readable storage medium | |
| CN112875601B (en) | Fork truck auxiliary operation system | |
| EP2631756B1 (en) | User interface for a digital camera | |
| CN105159874A (en) | Method and apparatus for modifying character | |
| CN115576457A (en) | Display control method and device, augmented reality head-mounted device and medium | |
| CN111221482B (en) | Interface layout adjusting method based on command console | |
| CN115097976A (en) | Method, apparatus, device and storage medium for image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |