WO2018104582A1 - Optimizing data stream transmissions from marine vessel - Google Patents

Optimizing data stream transmissions from marine vessel Download PDF

Info

Publication number
WO2018104582A1
WO2018104582A1 PCT/FI2017/050862 FI2017050862W WO2018104582A1 WO 2018104582 A1 WO2018104582 A1 WO 2018104582A1 FI 2017050862 W FI2017050862 W FI 2017050862W WO 2018104582 A1 WO2018104582 A1 WO 2018104582A1
Authority
WO
WIPO (PCT)
Prior art keywords
marine vessel
feed
prioritized
viewing direction
remote operator
Prior art date
Application number
PCT/FI2017/050862
Other languages
French (fr)
Inventor
Oskar Levander
Jann Peter Strand
Original Assignee
Rolls-Royce Oy Ab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rolls-Royce Oy Ab filed Critical Rolls-Royce Oy Ab
Publication of WO2018104582A1 publication Critical patent/WO2018104582A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Definitions

  • the invention relates to remotely operating a marine vessel.
  • the invention relates to optimizing data stream transmissions from the marine vessel to a remote operator system.
  • the marine vessel may, for example, transmit a live camera feed or feeds from the marine vessel bridge to a remote operator system, and a remote operator at the remote operator system operating the marine vessel is able to see the live camera feed or feeds from the marine vessel bridge when operating the marine vessel.
  • data transmission capacity between a marine vessel and a remote operator system may vary and may not always be sufficient for providing an extensive or a 360 degree view at high resolution from the marine vessel bridge to the remote operator system.
  • a remote operator a man or a woman
  • transmission capacity from the marine vessel to the remote operator system can be optimized.
  • a method of optimizing data transmissions from a marine vessel comprises determining a focusing point in displayed camera feed data provided from the marine vessel, causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, receiving a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and causing display of the received prioritized feed data in the viewing direction.
  • determining a focusing point in camera feed data provided from the marine vessel comprises determining a primary viewing direction of a remote operator of the marine vessel; and causing display of the received prioritized feed data comprises causing display of the received prioritized feed data in the primary viewing direction of the remote operator.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
  • the method further comprises receiving at least one secondary marine vessel camera feed from the marine vessel, and causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
  • receiving at least one secondary marine vessel camera feed from the marine vessel comprises receiving the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
  • determining the primary viewing direction of the remote operator of the marine vessel comprises determining the primary viewing direction of the remote operator based on eye tracking or head tracking .
  • determining the primary viewing direction of the remote operator of the marine vessel comprises determining the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
  • the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized.
  • the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
  • the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
  • a method of optimizing data transmissions from a marine vessel comprises receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and causing transmission of the marine vessel camera feed to the remote operator system.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
  • the method further comprises causing transmission of at least one secondary marine vessel camera feed from the marine vessel at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
  • the method further comprises combining multiple marine vessel camera feeds into a combined marine vessel camera feed, wherein causing transmission of the marine vessel camera feed to the remote operator system comprises causing transmission of the combined marine vessel camera feed, where the part of the combined camera feed relating to the viewing direction has been prioritized.
  • the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the viewing direction has been prioritized .
  • the method further comprises obtaining transmission capacity information, and determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
  • the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
  • a remote operator system of optimizing data transmissions from a marine vessel comprises means for determining a focusing point in displayed camera feed data provided from the marine vessel, means for causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, means for receiving a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing display of the received prioritized feed data in the viewing direction .
  • the means for determining a focusing point in camera feed data provided from the marine vessel are configured to determine a primary viewing direction of a remote operator of the marine vessel, and the means for causing display of the received prioritized feed data are configured to cause display of the received prioritized feed data in the primary viewing direction of the remote operator.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
  • the remote operator system further comprises means for receiving at least one secondary marine vessel camera feed from the marine vessel, and means for causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
  • the means for receiving at least one secondary marine vessel camera feed from the marine vessel are configured to receive the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed .
  • the means for determining the primary viewing direction of the remote operator of the marine vessel are configured to determine the primary viewing direction of the remote operator based on eye tracking or head tracking.
  • the means for determining the primary viewing direction of the remote operator of the marine vessel are configured to determine the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
  • the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized.
  • the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
  • the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
  • a marine vessel control system of optimizing data transmissions from a marine vessel.
  • the marine vessel control system comprises means for receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, means for determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing transmission of the marine vessel camera feed to the remote operator system.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
  • the marine vessel control system further comprises means for causing transmission of at least one secondary marine vessel camera feed from the marine vessel to the remote operator system at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
  • the marine vessel control system further comprises means for combining multiple marine vessel camera feeds into a combined marine vessel camera feed, wherein the means for causing transmission are configured to cause transmission of the combined marine vessel camera feed to the remote operator system, where the part of the combined camera feed relating to the viewing direction has been prioritized.
  • the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
  • the marine vessel control system further comprises means for obtaining transmission capacity information, and means for determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
  • a computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to determine a focusing point in displayed camera feed data provided from the marine vessel, cause transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, receive a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause display of the received prioritized feed data in the viewing direction.
  • the computer program is embodied on a computer-readable medium.
  • a computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to receive a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determine a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause transmission of the marine vessel camera feed to the remote operator system.
  • the computer program is embodied on a computer-readable medium.
  • a control system comprising at least one processing unit and at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control system to determine a focusing point in displayed camera feed data provided from the marine vessel, cause transmission of a request to the marine vessel, the request identifying the viewing direction relating to the focusing and requesting prioritized feed data from the marine vessel relating to the viewing direction, receive a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause display of the received prioritized feed data in the viewing direction.
  • a control system comprising at least one processing unit and at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control system to receive a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determine a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause transmission of the marine vessel camera feed to the remote operator system.
  • FIG. 1 illustrates an operating environment of a remote operator of a marine vessel in accordance with an example embodiment .
  • FIG. 2 is an example block diagram of a remote operator system in accordance with an example embodiment .
  • FIG. 3 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment.
  • FIG. 4 is an example block diagram of a marine vessel control system in accordance with an example embodiment .
  • FIG. 5 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment
  • FIG. 6 illustrates a marine vessel in accordance with an example embodiment .
  • FIG. 7 is an example block diagram of a remote operator system in accordance with an example embodiment .
  • FIG. 8 is an example block diagram of a marine vessel control system in accordance with an example embodiment .
  • FIG. 1 illustrates an example operating environment 100 of a remote operator 102 of a marine vessel in accordance with an example embodiment.
  • the remote operator 102 handles various functions relating to the marine vessel remotely.
  • the remote operator 102 may have a complete steering control of the marine vessel.
  • a plurality of display screens 104A-104H may be provided that show camera feeds from the marine vessel.
  • the display screens 104A-104H provide a full 360 degree view from the bridge of the marine vessel. In another embodiment, for example, only a 180 degree view from the bridge of the marine vessel is provided.
  • the display screens may be configured to turn together with the remote operator 102. In other words, when the remote operator 102 turns, the display screens may also turn and the view on the display screens may also change .
  • FIG. 2 is an example block diagram of a remote operator system 200 in accordance with an example embodiment.
  • the remote operator system 200 comprises a control unit 202 configured to provide on overall control of a marine vessel for a remote operator.
  • the control unit 202 may comprise at least one processing unit and at least one memory, where the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control unit 202 to enable remote operation of the marine vessel.
  • the remote operator system 200 may also comprise a tracking system 206 and a display system 204.
  • the tracking system 206 may be configured to track the viewing direction of the remote operator 102.
  • the display system 204 is configured to display a marine vessel camera feed or feeds received from the marine vessel.
  • the display system 204 may be implemented using any appropriate technology.
  • the marine vessel camera feed or feeds may comprise a normal camera feed, an infrared camera feed or a night vision camera feed or any combination of these.
  • At least one of the control unit 202, the display system 204, the tracking system 206 and the data interface 206 constitute means for determining a focusing point in displayed camera feed data provided from the marine vessel, means for causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting a prioritized marine vessel camera feed data from the marine vessel relating to the viewing direction, means for receiving the a prioritized marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing display of the received prioritized marine vessel camera feed data in the viewing direction.
  • FIG. 3 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment. In the following, FIG. 2 and FIG. 3 will be discussed in more detail together.
  • a focusing point in displayed camera feed data provided from the marine vessel.
  • the focusing point may be determined, for example, by an intelligent decision support system.
  • the focusing point may also relate to a point in the camera feed data based on which zoom in/out operation is to be performed.
  • the focusing point may be determined based on a primary viewing direction of a remote operator of the marine vessel.
  • the control unit 202 may be configured to control the tracking system 206 to perform measurements based on which the primary viewing direction of the remote operator 102 can be determined.
  • the term "primary viewing direction" refers generally to the direction to which the remote operator 102 focuses his/her gaze.
  • the tracking system 206 may also comprise means for tracking the eyes or the head of the remote operator 102.
  • the means for tracking are implemented, for example, using a camera system that detects the face of the remote operator 102 and is able to track the remote operator's 102 eye or head.
  • the tracking system 206 may comprise a head-mounted unit worn by the remote operator 102.
  • the control unit 202 is able determine the primary viewing direction.
  • a specific viewing direction may be set as a zero direction.
  • the zero direction may be linked to a marine vessel camera feed that shows camera feed directly from ahead from the marine vessel.
  • the marine vessel may provide to the remote operator system 200 multiple camera feeds together providing a 360 degree view from the bridge of the marine vessel.
  • a request is caused to be transmitted to the marine vessel control system 208 at the marine vessel.
  • the control unit 202 may cause the transmission via a data interface 206 of the remote operator system 200.
  • the request identifies the viewing direction relating to the focusing point and requests prioritized feed data from the marine vessel relating to the viewing direction.
  • the viewing direction may be identified using a numeral value indicating the viewing direction of the remote operator 102 with respect to a default zero direction.
  • other viewing direction identification solutions may be used .
  • the remote operator system 200 receives a marine vessel camera feed from the marine vessel.
  • the marine vessel camera feed comprises the prioritized feed data relating to the viewing direction.
  • marine vessel camera feed comprises the prioritized feed data relating to the primary viewing direction of the remote operator.
  • the term "prioritized feed data" refers to a camera feed or camera feeds or part of the camera feed that is somehow prioritized over other camera feeds relating to other viewing directions or the remaining part of the camera feed.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than other camera feeds or parts of the camera feed.
  • the marine vessel control system 208 may also first transmit the marine vessel camera feed relating to the primary viewing direction first and only after that the other camera feeds.
  • at least one secondary marine vessel camera feed is received at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
  • the marine vessel camera feed received from the marine vessel control system 208 comprises a combined marine vessel camera feed comprising multiple marine vessel camera feeds.
  • a part or a feed of the combined marine vessel camera feed relating to the primary viewing direction of the remote operator may have been prioritized.
  • the marine vessel camera feed may comprise a single feed comprising a 360 degree view.
  • a part of the single feed relating to the primary viewing direction of the remote operator may have been prioritized.
  • the prioritization may mean, for example, that the part or feed relating to the primary viewing direction of the remote operator it has a higher resolution and/or a higher frame rate than other feeds or parts.
  • the other part of feeds may have been compressed to a lower resolution by the marine vessel control system 208 before transmission to the remote operator system 200.
  • the received prioritized feed data is caused to be displayed in the viewing direction by the display system 204.
  • the received prioritized feed data is caused to be displayed in the primary viewing direction of the remote operator by the display system 204. If the marine vessel transmits to the remote operator system 200 at least one secondary marine vessel camera feed, this feed or feeds may be caused to be displayed by the display system 204 in viewing directions other than the viewing direction.
  • the process of determining the primary viewing direction may be performed again as the new viewing direction may relate to a different marine vessel camera feed.
  • more than one marine vessel camera feed may be received from the marine vessel control system as prioritized marine vessel camera feeds. This may happen, for example, when the focusing point or the primary viewing direction of the remote operator 102 is close to a border area of two or more marine vessel camera feeds. If only one camera feed is prioritized (e.g. by having a higher resolution and/or frame rate than non-prioritized feed data) , the remote operator 102 may perceive a notable quality difference between two adjacent displays of the display system 204.
  • communication capacity is used effectively as the communication capacity between the marine vessel and the remote operator system is not always sufficient to stream live camera feeds, for example, at high resolution and/or high frame rate for large angles (for example, for a complete 360 degree view) .
  • the prioritization ensures that the primary field of view to which the remote operator 102 is able to focus is of higher quality.
  • Other live camera feeds can be of lower quality as the remote operator 102 cannot adequately focus his/gaze to them while viewing towards the primary viewing direction.
  • FIG. 4 is an example block diagram of a marine vessel control system 208 in accordance with an example embodiment .
  • the marine vessel control system 208 comprises a control unit 400 configured to control the marine vessel.
  • the control unit 400 may comprise at least one processing unit and at least one memory, where the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control unit to control operations of the marine vessel.
  • the marine vessel control system 208 comprises also a camera system 402.
  • the camera system 402 may be configured to provide multiple marine vessel camera feeds using multiple cameras.
  • the cameras may be configured to record live outside views from the bridge of the marine vessel so that altogether a complete 360 degree live view is provided by the cameras.
  • the camera system 402 may also comprises a single camera that is able to provide a 360 degree view.
  • the camera system 402 may comprise a normal camera, an infrared camera and/or a night vision camera or any combination of these.
  • control unit 400 is configured to combine multiple camera feeds into a combined marine vessel camera feed.
  • the control unit 400 may determine a part or a feed of the combined marine vessel camera feed that is prioritized over other parts of feeds of the combined marine vessel camera feed.
  • the marine vessel camera feed may comprise a single feed comprising a 360 degree view. A part of the single feed relating to a viewing direction relating to a focusing point in camera feed data provided from the marine vessel may have been prioritized.
  • the prioritized part or feed has, for example, a higher resolution and/or higher frame rate than other parts of feeds of the marine vessel camera feed.
  • the other part of feeds may be compressed to a lower resolution.
  • the marine vessel control system 208 may obtain transmission capacity information 408 relating to a communication system that is used to wirelessly transmit data from the marine vessel, for example, to the remote operator system 200. Based on the obtained information the marine vessel control system 208 determines transmission characteristics for the marine vessel camera feed.
  • the transmission characteristics refer, for example, resolution and/or frame rate that can be used for the prioritized feed data and the remaining non-prioritized parts of the marine vessel camera feed .
  • At least one of the control unit 400, the camera system 402 and the data interface 406 constitute means for receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting a prioritized marine vessel camera feed data from the marine vessel relating to the viewing direction, means for determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing transmission of the marine vessel camera feed relating to the viewing direction of the remote operator as a prioritized marine vessel camera feed to the remote operator system.
  • FIG. 5 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment. In the following, FIG. 4 and FIG. 5 will be discussed in more detail together.
  • a request is received from the remote operator system 200 via a data interface 406.
  • the request identifies a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requests prioritized feed data from the marine vessel relating to the viewing direction.
  • the request identifies a primary viewing direction of the remote operator 102 of the marine vessel and requests prioritized feed data from the marine vessel relating to the primary viewing direction of the remote operator 102.
  • the viewing direction may be identified using a numeral value indicating the viewing direction with respect to a zero direction. However, it is evident that also other primary viewing direction identification solutions may be used.
  • a marine vessel camera feed comprising the prioritized camera feed relating to the viewing direction is determined.
  • a marine vessel camera feed comprising the prioritized camera feed relating to the primary viewing direction of the remote operator 102 is determined. The determination may be based on the identification of the primary viewing direction in the request.
  • the marine vessel control system 208 may store correspondence data between viewing directions and marine vessel cameras. In other words, a specific viewing direction value may map to a specific camera. It is evident that this is only one possible embodiment for determining the marine vessel camera feed relating to the primary viewing direction of the remote operator 102.
  • the term "prioritized feed data" refers to a camera feed or part of the camera feed that is somehow prioritized over other camera feeds relating to other viewing directions or the remaining part of the camera feed.
  • the prioritized feed data has a higher resolution and/or a higher frame rate than other camera feeds or part of the camera feed.
  • the marine vessel control system 208 may also transmit the marine vessel camera feed relating to the primary viewing direction first and only after that the other camera feeds.
  • the marine vessel control system 208 transmits to the remote operator system 200 at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the marine vessel camera feed.
  • FIG. 6 illustrates a marine vessel 600 in accordance with an example embodiment.
  • a marine vessel refers to any craft designed for transportation on water. Examples of a marine vessel include but are not limited to a cargo ship and a ferry .
  • the marine vessel 600 comprises the marine vessel control system 208 configured to control the marine vessel 600 via remote operation instructions originating from the remote operator system 200.
  • the marine vessel control system 208 comprises the camera system 402 that is able to provide a 360 degree view 602 from the bridge of the marine vessel 600.
  • FIG. 7 is a more detailed schematic block diagram of the remote operator system 200. It should also be appreciated that at least some of the components described below may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of FIG. 7. As such, among other examples, the remote operator system 200 could be any computer device, such as any suitable servers, workstations, personal computers, laptop computers or a system comprising several separate subsystems .
  • the illustrated remote operator system 200 includes a controller or a processor 700 (i.e. a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions.
  • An operating system 720 controls the allocation and usage of the components of the remote operator system 200 and supports for one or more application programs 722.
  • the application programs 722 can include vessel control and operation related applications, or any other application.
  • the illustrated remote operator system 200 includes one or more memory components, for example, a non-removable memory 702 and/or removable memory 704.
  • the non-removable memory 702 may include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies.
  • the removable memory 704 may include flash memory (such as one or more removable flash drives) or smart cards.
  • the one or more memory components may be used for storing data and/or code for running the operating system 720 and the applications 722.
  • Example of data may include text, images, sound files, image data, video data, or other data sets to be sent to and/or received from one or more network servers, other devices or marine vessels via one or more wired or wireless networks.
  • the remote operator system 200 can support one or more input devices 708 and one or more output devices 716.
  • the input devices 708 may include, but are not limited to, a touchscreen 710 (i.e., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone 712 (i.e., capable of capturing voice input), and a physical keyboard 714.
  • Examples of the output devices 716 may include, but are not limited to a speaker 718 and a display system 204.
  • the display system 204 is configured to display marine vessel camera feed or feeds received from a marine vessel.
  • Other possible output devices can include piezoelectric or other haptic output devices. Some devices can serve more than one input /output function.
  • the remote operator system 200 can further include one or more input/output interfaces 706.
  • the illustrated components are not required or all- inclusive, as any of the components shown can be deleted and other components can be added.
  • the remote operator system 200 can further include or have a connection to a tracking system 206.
  • the tracking system 206 is configured to track the viewing direction of the remote operator 102.
  • the tracking system 206 may comprise means for tracking the eyes or head of the remote operator 102.
  • the means for tracking are implemented, for example, using a camera system that detects the face of the remote operator 102 and is able to track the remote operator's 102 eye or head.
  • the tracking system 206 may comprise a head-mounted unit worn by the remote operator 102.
  • FIG. 8 is a more detailed schematic block diagram of the marine vessel control system 208. It should also be appreciated that at least some of the components described below may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of FIG. 8. As such, among other examples, the marine vessel control system 208 could be any computer device, such as any suitable servers, workstations, personal computers, laptop computers or a system comprising several separate subsystems .
  • the illustrated marine vessel control system 208 includes a controller or a processor 800 (i.e. a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions.
  • An operating system 816 controls the allocation and usage of the components of the marine vessel control system 208 and supports for one or more application programs 818.
  • the application programs 818 can include vessel control and operation related applications, or any other application.
  • the illustrated marine vessel control system 208 includes one or more memory components, for example, a non-removable memory 802 and/or removable memory 804.
  • the non-removable memory 802 may include RAM, ROM, flash memory, a hard disk, or other well- known memory storage technologies.
  • the removable memory 804 may include flash memory (such as one or more removable flash drives) or smart cards.
  • the one or more memory components may be used for storing data and/or code for running the operating system 816 and the applications 818.
  • Example of data may include text, images, sound files, image data, video data, or other data sets to be sent to and/or received from one or more network servers, other devices, marine vessels or the remote operator system 200 via one or more wired or wireless networks.
  • the marine vessel control system 208 can support one or more input devices 808 and one or more output devices 816.
  • the input devices 808 may include, but are not limited to, a touchscreen 810 (i.e., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone 812 (i.e., capable of capturing voice input), and a physical keyboard 814.
  • Examples of the output devices 816 may include, but are not limited to a speaker 818 and a display or displays 820.
  • the marine vessel control system 208 can further include one or more input/output interfaces 806.
  • the illustrated components are not required or all-inclusive, as any of the components shown can be deleted and other components can be added.
  • the marine vessel control system 208 can further include or have a connection to a camera system 402.
  • the camera system 402 is configured to provide multiple marine vessel camera feeds using multiple cameras or a single camera providing a 360 degree view.
  • the cameras may be configured to record live outside views from the bridge of the marine vessel 600 so that altogether a complete 360 degree live view is provided by the cameras.
  • a camera of cameras may alternatively or additionally be arranged also in other locations, for example, the mast, the roof above the bridge, the bow, the stern etc .
  • the exemplary embodiments can include, for example, any suitable servers, workstations, personal computers, laptop computers, other devices, and the like, capable of performing the processes of the exemplary embodiments.
  • the devices and subsystems of the exemplary embodiments can communicate with each other using any suitable protocol and can be implemented using one or more programmed computer systems or devices.
  • One or more interface mechanisms can be used with the exemplary embodiments, including, for example, Internet access, telecommunications in any suitable form (e.g., voice, modem, and the like), wireless communications media, and the like.
  • employed communications networks or links can include one or more satellite communications networks, wireless communications networks, cellular communications networks, 3G communications networks, 4G communications networks, Public Switched Telephone Network (PSTNs), Packet Data Networks (PDNs), the Internet, intranets, a combination thereof, and the like.
  • PSTNs Public Switched Telephone Network
  • PDNs Packet Data Networks
  • the exemplary embodiments are for exemplary purposes, as many variations of the specific hardware used to implement the exemplary embodiments are possible, as will be appreciated by those skilled in the hardware and/or software art(s) .
  • the functionality of one or more of the components of the exemplary embodiments can be implemented via one or more hardware and/or software devices.
  • the exemplary embodiments can store information relating to various processes described herein. This information can be stored in one or more memories, such as a hard disk, optical disk, magneto- optical disk, RAM, and the like.
  • One or more databases can store the information used to implement the exemplary embodiments of the present inventions.
  • the databases can be organized using data structures (e.g., records, tables, arrays, fields, graphs, trees, lists, and the like) included in one or more memories or storage devices listed herein.
  • the processes described with respect to the exemplary embodiments can include appropriate data structures for storing data collected and/or generated by the processes of the devices and subsystems of the exemplary embodiments in one or more databases.
  • All or a portion of the exemplary embodiments can be conveniently implemented using one or more general purpose processors, microprocessors, digital signal processors, micro-controllers, and the like, programmed according to the teachings of the exemplary embodiments of the present inventions, as will be appreciated by those skilled in the computer and/or software art(s) .
  • Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the exemplary embodiments, as will be appreciated by those skilled in the software art.
  • the exemplary embodiments can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be appreciated by those skilled in the electrical art(s) .
  • the exemplary embodiments are not limited to any specific combination of hardware and/or software.
  • the exemplary embodiments of the present inventions can include software for controlling the components of the exemplary embodiments, for driving the components of the exemplary embodiments, for enabling the components of the exemplary embodiments to interact with a human user, and the like.
  • software can include, but is not limited to, device drivers, firmware, operating systems, development tools, applications software, and the like.
  • Such computer readable media further can include the computer program product of an embodiment of the present inventions for performing all or a portion (if processing is distributed) of the processing performed in implementing the inventions.
  • Computer code devices of the exemplary embodiments of the present inventions can include any suitable interpretable or executable code mechanism.
  • the components of the exemplary embodiments can include computer readable medium or memories for holding instructions programmed according to the teachings of the present inventions and for holding data structures, tables, records, and/or other data described herein.
  • Computer readable medium can include any suitable medium that participates in providing instructions to a processor for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, and the like.
  • Non-volatile media can include, for example, optical or magnetic disks, magneto-optical disks, and the like.
  • Volatile media can include dynamic memories, and the like.
  • Computer-readable media can include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CD ⁇ R, CD ⁇ RW, DVD, DVD-RAM, DVD ⁇ RW, DVD ⁇ R, HD DVD, HD DVD-R, HD DVD-RW, HD DVD-RAM, Blu-ray Disc, any other suitable optical medium, punch cards, paper tape, optical mark sheets, any other suitable physical medium with patterns of holes or other optically recognizable indicia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge, a carrier wave or any other suitable medium from which a computer can read.
  • a floppy disk CD ⁇ R, CD ⁇ RW, DVD, DVD-RAM, DVD ⁇ RW, DVD ⁇ R, HD DVD, HD DVD-RW, HD DVD-RAM, Blu-ray Disc
  • any other suitable optical medium punch cards, paper tape, optical mark sheets, any other suitable physical medium with patterns of holes or other optically recognizable

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Studio Devices (AREA)

Abstract

According to an aspect, there is provided method of optimizing data transmissions from a marine vessel. The method comprises determining a focusing point in displayed camera feed data provided from the marine vessel, causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, receiving a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and causing display of the received prioritized feed data in the viewing direction.

Description

OPTIMIZING DATA STREAM TRANSMISSIONS FROM A MARINE VESSEL
BACKGROUND
Field:
The invention relates to remotely operating a marine vessel. In particular, the invention relates to optimizing data stream transmissions from the marine vessel to a remote operator system. Description of the Related Art:
There exist various solutions for operating a marine vessel remotely. The marine vessel may, for example, transmit a live camera feed or feeds from the marine vessel bridge to a remote operator system, and a remote operator at the remote operator system operating the marine vessel is able to see the live camera feed or feeds from the marine vessel bridge when operating the marine vessel. SUMMARY
When marine vessels are operated remotely, data transmission capacity between a marine vessel and a remote operator system may vary and may not always be sufficient for providing an extensive or a 360 degree view at high resolution from the marine vessel bridge to the remote operator system. In an embodiment, as a remote operator (a man or a woman) at the remote operator system is able to focus eyes only to a small portion of the whole 360 degree view, there is actually no need to provide the whole 360 degree view at high resolution and/or frame rate from the marine vessel bridge to the remote operator system. It is sufficient that a better quality view is provided, for example, only relating to the viewing direction towards which the remote operator is currently looking. Thus, transmission capacity from the marine vessel to the remote operator system can be optimized.
According to an aspect, there is provided a method of optimizing data transmissions from a marine vessel. The method comprises determining a focusing point in displayed camera feed data provided from the marine vessel, causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, receiving a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and causing display of the received prioritized feed data in the viewing direction.
In an embodiment, determining a focusing point in camera feed data provided from the marine vessel comprises determining a primary viewing direction of a remote operator of the marine vessel; and causing display of the received prioritized feed data comprises causing display of the received prioritized feed data in the primary viewing direction of the remote operator.
In an embodiment, the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
In an embodiment, alternatively or in addition to the above described embodiments, the method further comprises receiving at least one secondary marine vessel camera feed from the marine vessel, and causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
In an embodiment, alternatively or in addition to the above described embodiments, receiving at least one secondary marine vessel camera feed from the marine vessel comprises receiving the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
In an embodiment, alternatively or in addition to the above described embodiments, determining the primary viewing direction of the remote operator of the marine vessel comprises determining the primary viewing direction of the remote operator based on eye tracking or head tracking .
In an embodiment, alternatively or in addition to the above described embodiments, determining the primary viewing direction of the remote operator of the marine vessel comprises determining the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
According to another aspect, there is provided a method of optimizing data transmissions from a marine vessel. The method comprises receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and causing transmission of the marine vessel camera feed to the remote operator system.
In an embodiment, wherein the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
In an embodiment, alternatively or in addition to the above described embodiments, the method further comprises causing transmission of at least one secondary marine vessel camera feed from the marine vessel at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
In an embodiment, alternatively or in addition to the above described embodiments, the method further comprises combining multiple marine vessel camera feeds into a combined marine vessel camera feed, wherein causing transmission of the marine vessel camera feed to the remote operator system comprises causing transmission of the combined marine vessel camera feed, where the part of the combined camera feed relating to the viewing direction has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, wherein the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the viewing direction has been prioritized . In an embodiment, alternatively or in addition to the above described embodiments, the method further comprises obtaining transmission capacity information, and determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
According to another aspect, there is provided a remote operator system of optimizing data transmissions from a marine vessel. The remote operator system comprises means for determining a focusing point in displayed camera feed data provided from the marine vessel, means for causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, means for receiving a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing display of the received prioritized feed data in the viewing direction .
In an embodiment, the means for determining a focusing point in camera feed data provided from the marine vessel are configured to determine a primary viewing direction of a remote operator of the marine vessel, and the means for causing display of the received prioritized feed data are configured to cause display of the received prioritized feed data in the primary viewing direction of the remote operator.
In an embodiment, the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data. In an embodiment, alternatively or in addition to the above described embodiments, the remote operator system further comprises means for receiving at least one secondary marine vessel camera feed from the marine vessel, and means for causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
In an embodiment, alternatively or in addition to the above described embodiments, the means for receiving at least one secondary marine vessel camera feed from the marine vessel are configured to receive the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed .
In an embodiment, alternatively or in addition to the above described embodiments, the means for determining the primary viewing direction of the remote operator of the marine vessel are configured to determine the primary viewing direction of the remote operator based on eye tracking or head tracking.
In an embodiment, alternatively or in addition to the above described embodiments, the means for determining the primary viewing direction of the remote operator of the marine vessel are configured to determine the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized. In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel camera feed from the marine vessel comprises two or more prioritized feeds.
According to another aspect, there is provided a marine vessel control system of optimizing data transmissions from a marine vessel. The marine vessel control system comprises means for receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, means for determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing transmission of the marine vessel camera feed to the remote operator system.
In an embodiment, alternatively or in addition to the above described embodiments, the prioritized feed data has a higher resolution and/or a higher frame rate than non-prioritized feed data.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel control system further comprises means for causing transmission of at least one secondary marine vessel camera feed from the marine vessel to the remote operator system at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed. In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel control system further comprises means for combining multiple marine vessel camera feeds into a combined marine vessel camera feed, wherein the means for causing transmission are configured to cause transmission of the combined marine vessel camera feed to the remote operator system, where the part of the combined camera feed relating to the viewing direction has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
In an embodiment, alternatively or in addition to the above described embodiments, the marine vessel control system further comprises means for obtaining transmission capacity information, and means for determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
According to another aspect, there is provided a computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to determine a focusing point in displayed camera feed data provided from the marine vessel, cause transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel relating to the viewing direction, receive a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause display of the received prioritized feed data in the viewing direction.
In an embodiment, the computer program is embodied on a computer-readable medium.
According to another aspect, there is provided a computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to receive a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determine a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause transmission of the marine vessel camera feed to the remote operator system.
In an embodiment, the computer program is embodied on a computer-readable medium.
According to another aspect, there is provided a control system comprising at least one processing unit and at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control system to determine a focusing point in displayed camera feed data provided from the marine vessel, cause transmission of a request to the marine vessel, the request identifying the viewing direction relating to the focusing and requesting prioritized feed data from the marine vessel relating to the viewing direction, receive a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause display of the received prioritized feed data in the viewing direction.
According to another aspect, there is provided a control system comprising at least one processing unit and at least one memory, wherein the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control system to receive a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting prioritized feed data from the marine vessel relating to the viewing direction, determine a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and cause transmission of the marine vessel camera feed to the remote operator system. BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are included to provide a further understanding of the invention and constitute a part of this specification, illustrate embodiments of the invention and together with the description help to explain the principles of the invention. In the drawings:
FIG. 1 illustrates an operating environment of a remote operator of a marine vessel in accordance with an example embodiment .
FIG. 2 is an example block diagram of a remote operator system in accordance with an example embodiment .
FIG. 3 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment.
FIG. 4 is an example block diagram of a marine vessel control system in accordance with an example embodiment .
FIG. 5 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment FIG. 6 illustrates a marine vessel in accordance with an example embodiment .
FIG. 7 is an example block diagram of a remote operator system in accordance with an example embodiment .
FIG. 8 is an example block diagram of a marine vessel control system in accordance with an example embodiment .
Like reference numerals are used to designate like parts in the accompanying drawings.
DETAILED DESCRIPTION
Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings. The detailed description provided below in connection with the appended drawings is intended as a description of the present examples and is not intended to represent the only forms in which the present example may be constructed or utilized. The description sets forth the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be accomplished by different examples.
FIG. 1 illustrates an example operating environment 100 of a remote operator 102 of a marine vessel in accordance with an example embodiment. The remote operator 102 handles various functions relating to the marine vessel remotely. As an example, the remote operator 102 may have a complete steering control of the marine vessel. In order to be able steer the marine vessel, a plurality of display screens 104A-104H may be provided that show camera feeds from the marine vessel. In one embodiment, the display screens 104A-104H provide a full 360 degree view from the bridge of the marine vessel. In another embodiment, for example, only a 180 degree view from the bridge of the marine vessel is provided. Further, the display screens may be configured to turn together with the remote operator 102. In other words, when the remote operator 102 turns, the display screens may also turn and the view on the display screens may also change .
FIG. 2 is an example block diagram of a remote operator system 200 in accordance with an example embodiment. The remote operator system 200 comprises a control unit 202 configured to provide on overall control of a marine vessel for a remote operator. The control unit 202 may comprise at least one processing unit and at least one memory, where the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control unit 202 to enable remote operation of the marine vessel.
The remote operator system 200 may also comprise a tracking system 206 and a display system 204. The tracking system 206 may be configured to track the viewing direction of the remote operator 102. The display system 204 is configured to display a marine vessel camera feed or feeds received from the marine vessel. The display system 204 may be implemented using any appropriate technology. The marine vessel camera feed or feeds may comprise a normal camera feed, an infrared camera feed or a night vision camera feed or any combination of these.
In an embodiment, at least one of the control unit 202, the display system 204, the tracking system 206 and the data interface 206 constitute means for determining a focusing point in displayed camera feed data provided from the marine vessel, means for causing transmission of a request to the marine vessel, the request identifying a viewing direction relating to the focusing point and requesting a prioritized marine vessel camera feed data from the marine vessel relating to the viewing direction, means for receiving the a prioritized marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing display of the received prioritized marine vessel camera feed data in the viewing direction.
FIG. 3 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment. In the following, FIG. 2 and FIG. 3 will be discussed in more detail together.
At 300 a focusing point in displayed camera feed data provided from the marine vessel. The focusing point may be determined, for example, by an intelligent decision support system. In one embodiment, the focusing point may also relate to a point in the camera feed data based on which zoom in/out operation is to be performed. In another embodiment, the focusing point may be determined based on a primary viewing direction of a remote operator of the marine vessel. The control unit 202 may be configured to control the tracking system 206 to perform measurements based on which the primary viewing direction of the remote operator 102 can be determined. The term "primary viewing direction" refers generally to the direction to which the remote operator 102 focuses his/her gaze. The tracking system 206 may also comprise means for tracking the eyes or the head of the remote operator 102. The means for tracking are implemented, for example, using a camera system that detects the face of the remote operator 102 and is able to track the remote operator's 102 eye or head. In another embodiment, the tracking system 206 may comprise a head-mounted unit worn by the remote operator 102. Based on information from the tracking system 206, the control unit 202 is able determine the primary viewing direction. As a simple example, a specific viewing direction may be set as a zero direction. The zero direction may be linked to a marine vessel camera feed that shows camera feed directly from ahead from the marine vessel. In one embodiment, the marine vessel may provide to the remote operator system 200 multiple camera feeds together providing a 360 degree view from the bridge of the marine vessel.
At 302 a request is caused to be transmitted to the marine vessel control system 208 at the marine vessel. The control unit 202 may cause the transmission via a data interface 206 of the remote operator system 200. The request identifies the viewing direction relating to the focusing point and requests prioritized feed data from the marine vessel relating to the viewing direction. As a simple example, the viewing direction may be identified using a numeral value indicating the viewing direction of the remote operator 102 with respect to a default zero direction. However, it is evident that also other viewing direction identification solutions may be used .
At 304 the remote operator system 200 receives a marine vessel camera feed from the marine vessel. The marine vessel camera feed comprises the prioritized feed data relating to the viewing direction. In an embodiment, marine vessel camera feed comprises the prioritized feed data relating to the primary viewing direction of the remote operator. The term "prioritized feed data" refers to a camera feed or camera feeds or part of the camera feed that is somehow prioritized over other camera feeds relating to other viewing directions or the remaining part of the camera feed. In one embodiment, the prioritized feed data has a higher resolution and/or a higher frame rate than other camera feeds or parts of the camera feed. Alternatively or in addition, the marine vessel control system 208 may also first transmit the marine vessel camera feed relating to the primary viewing direction first and only after that the other camera feeds. In one embodiment, at least one secondary marine vessel camera feed is received at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
In an embodiment, the marine vessel camera feed received from the marine vessel control system 208 comprises a combined marine vessel camera feed comprising multiple marine vessel camera feeds. A part or a feed of the combined marine vessel camera feed relating to the primary viewing direction of the remote operator may have been prioritized. In another embodiment, the marine vessel camera feed may comprise a single feed comprising a 360 degree view. A part of the single feed relating to the primary viewing direction of the remote operator may have been prioritized. The prioritization may mean, for example, that the part or feed relating to the primary viewing direction of the remote operator it has a higher resolution and/or a higher frame rate than other feeds or parts. The other part of feeds may have been compressed to a lower resolution by the marine vessel control system 208 before transmission to the remote operator system 200.
At 306 the received prioritized feed data is caused to be displayed in the viewing direction by the display system 204. In an embodiment, at 306 the received prioritized feed data is caused to be displayed in the primary viewing direction of the remote operator by the display system 204. If the marine vessel transmits to the remote operator system 200 at least one secondary marine vessel camera feed, this feed or feeds may be caused to be displayed by the display system 204 in viewing directions other than the viewing direction.
When the remote operator 102 turns his/her head to a new viewing direction, the process of determining the primary viewing direction may be performed again as the new viewing direction may relate to a different marine vessel camera feed.
Further, in one embodiment, in response to the request sent at 302, more than one marine vessel camera feed may be received from the marine vessel control system as prioritized marine vessel camera feeds. This may happen, for example, when the focusing point or the primary viewing direction of the remote operator 102 is close to a border area of two or more marine vessel camera feeds. If only one camera feed is prioritized (e.g. by having a higher resolution and/or frame rate than non-prioritized feed data) , the remote operator 102 may perceive a notable quality difference between two adjacent displays of the display system 204.
When the marine vessel camera feed relating to the viewing direction or primary viewing direction is prioritized, communication capacity is used effectively as the communication capacity between the marine vessel and the remote operator system is not always sufficient to stream live camera feeds, for example, at high resolution and/or high frame rate for large angles (for example, for a complete 360 degree view) . The prioritization ensures that the primary field of view to which the remote operator 102 is able to focus is of higher quality. Other live camera feeds can be of lower quality as the remote operator 102 cannot adequately focus his/gaze to them while viewing towards the primary viewing direction.
FIG. 4 is an example block diagram of a marine vessel control system 208 in accordance with an example embodiment . The marine vessel control system 208 comprises a control unit 400 configured to control the marine vessel. The control unit 400 may comprise at least one processing unit and at least one memory, where the at least one memory stores program instructions that, when executed by the at least one processing unit, cause the control unit to control operations of the marine vessel.
The marine vessel control system 208 comprises also a camera system 402. The camera system 402 may be configured to provide multiple marine vessel camera feeds using multiple cameras. The cameras may be configured to record live outside views from the bridge of the marine vessel so that altogether a complete 360 degree live view is provided by the cameras. The camera system 402 may also comprises a single camera that is able to provide a 360 degree view. The camera system 402 may comprise a normal camera, an infrared camera and/or a night vision camera or any combination of these.
In one embodiment, the control unit 400 is configured to combine multiple camera feeds into a combined marine vessel camera feed. The control unit 400 may determine a part or a feed of the combined marine vessel camera feed that is prioritized over other parts of feeds of the combined marine vessel camera feed. In another embodiment, the marine vessel camera feed may comprise a single feed comprising a 360 degree view. A part of the single feed relating to a viewing direction relating to a focusing point in camera feed data provided from the marine vessel may have been prioritized. The prioritized part or feed has, for example, a higher resolution and/or higher frame rate than other parts of feeds of the marine vessel camera feed. The other part of feeds may be compressed to a lower resolution. In an embodiment, the marine vessel control system 208 may obtain transmission capacity information 408 relating to a communication system that is used to wirelessly transmit data from the marine vessel, for example, to the remote operator system 200. Based on the obtained information the marine vessel control system 208 determines transmission characteristics for the marine vessel camera feed. The transmission characteristics refer, for example, resolution and/or frame rate that can be used for the prioritized feed data and the remaining non-prioritized parts of the marine vessel camera feed .
In an embodiment, at least one of the control unit 400, the camera system 402 and the data interface 406 constitute means for receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requesting a prioritized marine vessel camera feed data from the marine vessel relating to the viewing direction, means for determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction, and means for causing transmission of the marine vessel camera feed relating to the viewing direction of the remote operator as a prioritized marine vessel camera feed to the remote operator system.
FIG. 5 is a flow diagram illustrating a method of optimizing data transmissions from a marine vessel in accordance with an example embodiment. In the following, FIG. 4 and FIG. 5 will be discussed in more detail together.
At 500 a request a request is received from the remote operator system 200 via a data interface 406. The request identifies a viewing direction relating to a focusing point in camera feed data provided from the marine vessel and requests prioritized feed data from the marine vessel relating to the viewing direction. In an embodiment, the request identifies a primary viewing direction of the remote operator 102 of the marine vessel and requests prioritized feed data from the marine vessel relating to the primary viewing direction of the remote operator 102. As described earlier in the description of FIG. 3, the viewing direction may be identified using a numeral value indicating the viewing direction with respect to a zero direction. However, it is evident that also other primary viewing direction identification solutions may be used.
At 502 a marine vessel camera feed comprising the prioritized camera feed relating to the viewing direction is determined. In an embodiment, at 502 a marine vessel camera feed comprising the prioritized camera feed relating to the primary viewing direction of the remote operator 102 is determined. The determination may be based on the identification of the primary viewing direction in the request. The marine vessel control system 208 may store correspondence data between viewing directions and marine vessel cameras. In other words, a specific viewing direction value may map to a specific camera. It is evident that this is only one possible embodiment for determining the marine vessel camera feed relating to the primary viewing direction of the remote operator 102.
At 504 transmission of the marine vessel camera feed is caused to the remote operator system 200 via the data interface 506.
The term "prioritized feed data" refers to a camera feed or part of the camera feed that is somehow prioritized over other camera feeds relating to other viewing directions or the remaining part of the camera feed. In one embodiment, the prioritized feed data has a higher resolution and/or a higher frame rate than other camera feeds or part of the camera feed. Alternatively or in addition, the marine vessel control system 208 may also transmit the marine vessel camera feed relating to the primary viewing direction first and only after that the other camera feeds. In one embodiment, the marine vessel control system 208 transmits to the remote operator system 200 at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the marine vessel camera feed.
FIG. 6 illustrates a marine vessel 600 in accordance with an example embodiment. Herein, a marine vessel refers to any craft designed for transportation on water. Examples of a marine vessel include but are not limited to a cargo ship and a ferry .
The marine vessel 600 comprises the marine vessel control system 208 configured to control the marine vessel 600 via remote operation instructions originating from the remote operator system 200. The marine vessel control system 208 comprises the camera system 402 that is able to provide a 360 degree view 602 from the bridge of the marine vessel 600.
FIG. 7 is a more detailed schematic block diagram of the remote operator system 200. It should also be appreciated that at least some of the components described below may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of FIG. 7. As such, among other examples, the remote operator system 200 could be any computer device, such as any suitable servers, workstations, personal computers, laptop computers or a system comprising several separate subsystems . The illustrated remote operator system 200 includes a controller or a processor 700 (i.e. a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions. An operating system 720 controls the allocation and usage of the components of the remote operator system 200 and supports for one or more application programs 722. The application programs 722 can include vessel control and operation related applications, or any other application.
The illustrated remote operator system 200 includes one or more memory components, for example, a non-removable memory 702 and/or removable memory 704. The non-removable memory 702 may include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies. The removable memory 704 may include flash memory (such as one or more removable flash drives) or smart cards. The one or more memory components may be used for storing data and/or code for running the operating system 720 and the applications 722. Example of data may include text, images, sound files, image data, video data, or other data sets to be sent to and/or received from one or more network servers, other devices or marine vessels via one or more wired or wireless networks.
The remote operator system 200 can support one or more input devices 708 and one or more output devices 716. Examples of the input devices 708 may include, but are not limited to, a touchscreen 710 (i.e., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone 712 (i.e., capable of capturing voice input), and a physical keyboard 714. Examples of the output devices 716 may include, but are not limited to a speaker 718 and a display system 204. The display system 204 is configured to display marine vessel camera feed or feeds received from a marine vessel. Other possible output devices (not shown) can include piezoelectric or other haptic output devices. Some devices can serve more than one input /output function.
The remote operator system 200 can further include one or more input/output interfaces 706. The illustrated components are not required or all- inclusive, as any of the components shown can be deleted and other components can be added.
The remote operator system 200 can further include or have a connection to a tracking system 206. The tracking system 206 is configured to track the viewing direction of the remote operator 102. The tracking system 206 may comprise means for tracking the eyes or head of the remote operator 102. The means for tracking are implemented, for example, using a camera system that detects the face of the remote operator 102 and is able to track the remote operator's 102 eye or head. In another embodiment, the tracking system 206 may comprise a head-mounted unit worn by the remote operator 102.
FIG. 8 is a more detailed schematic block diagram of the marine vessel control system 208. It should also be appreciated that at least some of the components described below may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of FIG. 8. As such, among other examples, the marine vessel control system 208 could be any computer device, such as any suitable servers, workstations, personal computers, laptop computers or a system comprising several separate subsystems . The illustrated marine vessel control system 208 includes a controller or a processor 800 (i.e. a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, input/output processing, power control, and/or other functions. An operating system 816 controls the allocation and usage of the components of the marine vessel control system 208 and supports for one or more application programs 818. The application programs 818 can include vessel control and operation related applications, or any other application.
The illustrated marine vessel control system 208 includes one or more memory components, for example, a non-removable memory 802 and/or removable memory 804. The non-removable memory 802 may include RAM, ROM, flash memory, a hard disk, or other well- known memory storage technologies. The removable memory 804 may include flash memory (such as one or more removable flash drives) or smart cards. The one or more memory components may be used for storing data and/or code for running the operating system 816 and the applications 818. Example of data may include text, images, sound files, image data, video data, or other data sets to be sent to and/or received from one or more network servers, other devices, marine vessels or the remote operator system 200 via one or more wired or wireless networks.
The marine vessel control system 208 can support one or more input devices 808 and one or more output devices 816. Examples of the input devices 808 may include, but are not limited to, a touchscreen 810 (i.e., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone 812 (i.e., capable of capturing voice input), and a physical keyboard 814. Examples of the output devices 816 may include, but are not limited to a speaker 818 and a display or displays 820.
The marine vessel control system 208 can further include one or more input/output interfaces 806. The illustrated components are not required or all-inclusive, as any of the components shown can be deleted and other components can be added.
The marine vessel control system 208 can further include or have a connection to a camera system 402. The camera system 402 is configured to provide multiple marine vessel camera feeds using multiple cameras or a single camera providing a 360 degree view. The cameras may be configured to record live outside views from the bridge of the marine vessel 600 so that altogether a complete 360 degree live view is provided by the cameras. Naturally, a camera of cameras may alternatively or additionally be arranged also in other locations, for example, the mast, the roof above the bridge, the bow, the stern etc .
Aspects of any of the embodiments described above may be combined with aspects of any of the other embodiments described to form further embodiments without losing the effect sought.
The term 'comprising' is used herein to mean including the method, blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.
The exemplary embodiments can include, for example, any suitable servers, workstations, personal computers, laptop computers, other devices, and the like, capable of performing the processes of the exemplary embodiments. The devices and subsystems of the exemplary embodiments can communicate with each other using any suitable protocol and can be implemented using one or more programmed computer systems or devices.
One or more interface mechanisms can be used with the exemplary embodiments, including, for example, Internet access, telecommunications in any suitable form (e.g., voice, modem, and the like), wireless communications media, and the like. For example, employed communications networks or links can include one or more satellite communications networks, wireless communications networks, cellular communications networks, 3G communications networks, 4G communications networks, Public Switched Telephone Network (PSTNs), Packet Data Networks (PDNs), the Internet, intranets, a combination thereof, and the like.
It is to be understood that the exemplary embodiments are for exemplary purposes, as many variations of the specific hardware used to implement the exemplary embodiments are possible, as will be appreciated by those skilled in the hardware and/or software art(s) . For example, the functionality of one or more of the components of the exemplary embodiments can be implemented via one or more hardware and/or software devices.
The exemplary embodiments can store information relating to various processes described herein. This information can be stored in one or more memories, such as a hard disk, optical disk, magneto- optical disk, RAM, and the like. One or more databases can store the information used to implement the exemplary embodiments of the present inventions. The databases can be organized using data structures (e.g., records, tables, arrays, fields, graphs, trees, lists, and the like) included in one or more memories or storage devices listed herein. The processes described with respect to the exemplary embodiments can include appropriate data structures for storing data collected and/or generated by the processes of the devices and subsystems of the exemplary embodiments in one or more databases.
All or a portion of the exemplary embodiments can be conveniently implemented using one or more general purpose processors, microprocessors, digital signal processors, micro-controllers, and the like, programmed according to the teachings of the exemplary embodiments of the present inventions, as will be appreciated by those skilled in the computer and/or software art(s) . Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the exemplary embodiments, as will be appreciated by those skilled in the software art. In addition, the exemplary embodiments can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be appreciated by those skilled in the electrical art(s) . Thus, the exemplary embodiments are not limited to any specific combination of hardware and/or software.
Stored on any one or on a combination of computer readable media, the exemplary embodiments of the present inventions can include software for controlling the components of the exemplary embodiments, for driving the components of the exemplary embodiments, for enabling the components of the exemplary embodiments to interact with a human user, and the like. Such software can include, but is not limited to, device drivers, firmware, operating systems, development tools, applications software, and the like. Such computer readable media further can include the computer program product of an embodiment of the present inventions for performing all or a portion (if processing is distributed) of the processing performed in implementing the inventions. Computer code devices of the exemplary embodiments of the present inventions can include any suitable interpretable or executable code mechanism.
As stated above, the components of the exemplary embodiments can include computer readable medium or memories for holding instructions programmed according to the teachings of the present inventions and for holding data structures, tables, records, and/or other data described herein. Computer readable medium can include any suitable medium that participates in providing instructions to a processor for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, and the like. Non-volatile media can include, for example, optical or magnetic disks, magneto-optical disks, and the like. Volatile media can include dynamic memories, and the like. Common forms of computer-readable media can include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CD±R, CD±RW, DVD, DVD-RAM, DVD±RW, DVD±R, HD DVD, HD DVD-R, HD DVD-RW, HD DVD-RAM, Blu-ray Disc, any other suitable optical medium, punch cards, paper tape, optical mark sheets, any other suitable physical medium with patterns of holes or other optically recognizable indicia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge, a carrier wave or any other suitable medium from which a computer can read.
While the present inventions have been de- scribed in connection with a number of exemplary embodiments, and implementations, the present inventions are not so limited, but rather cover various modifications, and equivalent arrangements, which fall within the purview of prospective claims.

Claims

1. A method of optimizing data stream transmissions from a marine vessel (600), characterized in that the method comprises :
determining (300) a focusing point in displayed camera feed data provided from the marine vessel ;
transmitting (302) a request to the marine vessel (600), the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel (600) relating to the viewing direction;
receiving (304) a marine vessel camera feed from the marine vessel, the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction and non-prioritized feed data, the prioritized feed data having a higher resolution and/or a higher frame rate than the non-prioritized feed data; and
causing (306) display of the received prioritized feed data in the viewing direction.
2. The method according to claim 1, wherein: determining a focusing point in camera feed data provided from the marine vessel (600) comprises determining a primary viewing direction of a remote operator of the marine vessel (600); and
causing display of the received prioritized feed data comprises causing display of the received prioritized feed data in the primary viewing direction of the remote operator.
3. The method according to any of claims 1 - 2, further comprising:
receiving at least one secondary marine vessel camera feed from the marine vessel (600); and causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
4. The method according to claim 3, wherein receiving at least one secondary marine vessel camera feed from the marine vessel (600) comprises receiving the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
5. The method according to any of claims 2 - 4, wherein determining the primary viewing direction of the remote operator of the marine vessel (600) comprises determining the primary viewing direction of the remote operator based on eye tracking or head tracking .
6. The method according to any of claims 2 - 4, wherein determining the primary viewing direction of the remote operator of the marine vessel comprises determining the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
7. The method according to any of claims 2 - 6, wherein the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized.
8. The method according to any of claims 2 - 6, wherein the marine vessel camera feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized .
9. The method according to any of claims 1 - 8, wherein the marine vessel camera feed from the marine vessel (600) comprises two or more prioritized feeds.
10. A method of optimizing data stream transmissions from a marine vessel (600), characterized in that the method comprises :
receiving (500) a request from a remote operator system (200), the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel (600) and requesting prioritized feed data from the marine vessel (600) relating to the viewing direction;
determining (502) a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction and non-prioritized feed data, the prioritized feed data having a higher resolution and/or a higher frame rate than the non-prioritized feed data; and
transmitting (504) the marine vessel camera feed to the remote operator system.
11. The method according to claim 10, further comprising :
transmitting at least one secondary marine vessel camera feed from the marine vessel (600) at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
12. The method according to any of claims 10 - 11, further comprising:
combining multiple marine vessel camera feeds into a combined marine vessel camera feed;
wherein transmitting the marine vessel camera feed to the remote operator system comprises transmitting the combined marine vessel camera feed, where the part of the combined camera feed relating to the viewing direction has been prioritized.
13. The method according to any of claims 10 - 12, wherein the marine vessel camera feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the viewing direction has been prioritized.
14. The method according to any of claims 10
- 12, further comprising:
obtaining transmission capacity information; and
determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
15. The method according to any of claims 10
- 12, wherein the marine vessel camera feed from the marine vessel (600) comprises two or more prioritized feeds .
16. A remote operator system (200) of optimizing data stream transmissions from a marine vessel (600), characterized in that the remote operator system comprises:
means for determining a focusing point in displayed camera feed data provided from the marine vessel (600);
means for transmitting a request to the marine vessel (600), the request identifying a viewing direction relating to the focusing point and requesting prioritized feed data from the marine vessel (600) relating to the viewing direction;
means for receiving a marine vessel camera feed from the marine vessel (600), the marine vessel camera feed comprising the prioritized feed data relating to the viewing direction and non-prioritized feed data, the prioritized feed data having a higher resolution and/or higher frame rate than the non- prioritized feed data; and
means for causing display of the received prioritized feed data in the viewing direction.
17. The remote operator system (200) according to claim 16, wherein:
the means for determining a focusing point in camera feed data provided from the marine vessel (600) are configured to determine a primary viewing direction of a remote operator of the marine vessel (600); and
the means for causing display of the received prioritized feed data are configured to cause display of the received prioritized feed data in the primary viewing direction of the remote operator.
18. The remote operator system (200) according to claim 16 - 17, further comprising:
means for receiving at least one secondary marine vessel camera feed from the marine vessel (600); and
means for causing display of the received at least one secondary marine vessel camera feed in viewing directions other than the viewing direction.
19. The remote operator system (200) according to claim 18, wherein the means for receiving at least one secondary marine vessel camera feed from the marine vessel (600) are configured to receive the at least one secondary marine vessel camera feed at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
20. The remote operator system (200) according to any of claims 16 - 19, wherein the means for determining the primary viewing direction of the remote operator of the marine vessel (600) are configured to determine the primary viewing direction of the remote operator based on eye tracking or head tracking .
21. The remote operator system (200) according to any of claims 16 - 19, wherein the means for determining the primary viewing direction of a remote operator of the marine vessel (600) are configured to determine the primary viewing direction of the remote operator based on information from a device attached to the head of the remote operator.
22. The remote operator system (200) according to any of claims 16 - 21, wherein the marine vessel camera feed comprises a combined camera feed comprising multiple marine vessel camera feeds, wherein the part of the combined camera feed relating to the primary viewing direction of the remote operator has been prioritized.
23. The remote operator system (200) according to any of claims 16 - 22, wherein the marine vessel camera feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
24. The remote operator system (200) according to any of claims 16 - 23, wherein the marine vessel camera feed from the marine vessel (600) comprises two or more prioritized feeds.
25. A marine vessel control system (208) of optimizing data stream transmissions from a marine vessel (600), characterized in that the marine vessel control system comprises:
means for receiving a request from a remote operator system, the request identifying a viewing direction relating to a focusing point in camera feed data provided from the marine vessel (600) and requesting prioritized feed data from the marine vessel relating to the viewing direction;
means for determining a marine vessel camera feed comprising the prioritized feed data relating to the viewing direction and non-prioritized feed data, the prioritized feed data having a higher resolution and/or higher frame rate than the non-prioritized feed data; and
means for transmitting the marine vessel camera feed to the remote operator system.
26. The marine vessel control system (208) according to claim 25, further comprising:
means for transmitting at least one secondary marine vessel camera feed from the marine vessel (600) to the remote operator system at a lower resolution and/or lower frame rate than that of the prioritized marine vessel camera feed.
27. The marine vessel control system (208) according to any of claims 25 - 26, further comprising :
means for combining multiple marine vessel camera feeds into a combined marine vessel camera feed;
wherein the means for transmitting are configured to transmit the combined marine vessel camera feed to the remote operator system, where the part of the combined camera feed relating to the viewing direction has been prioritized.
28. The marine vessel control system (208) according to any of claims 25 - 26, wherein the marine vessel camera feed comprises a single feed comprising a 360 degree view, wherein the part of the single feed relating to the primary viewing direction of the remote operator has been prioritized.
29. The marine vessel control system (208) according to any of claims 25 - 28, further comprising:
means for obtaining transmission capacity information; and
means for determining transmission characteristics for the marine vessel camera feed based on the transmission capacity information.
30. The marine vessel control system (208) according to any of claims 25 - 29, wherein the marine vessel camera feed from the marine vessel (600) comprises two or more prioritized feeds.
31. A computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to perform the method of any of claims 1 - 9.
32. A computer program comprising program code instructions, which when executed by at least one processor, cause the at least one processor to perform the method of any of claims 10 - 15.
PCT/FI2017/050862 2016-12-05 2017-12-04 Optimizing data stream transmissions from marine vessel WO2018104582A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FI20165925A FI20165925L (en) 2016-12-05 2016-12-05 Optimizing data stream transmissions from marine vessel
FI20165925 2016-12-05

Publications (1)

Publication Number Publication Date
WO2018104582A1 true WO2018104582A1 (en) 2018-06-14

Family

ID=60923516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/FI2017/050862 WO2018104582A1 (en) 2016-12-05 2017-12-04 Optimizing data stream transmissions from marine vessel

Country Status (2)

Country Link
FI (1) FI20165925L (en)
WO (1) WO2018104582A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3688536B1 (en) 2017-09-28 2022-04-27 A.P. Møller - Mærsk A/S A method and system for operating a ship
US11815886B2 (en) 2017-09-28 2023-11-14 A.P. Møller Mærsk A/S Method and system for operating a ship

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4405943A (en) * 1981-08-19 1983-09-20 Harris Corporation Low bandwidth closed loop imagery control and communication system for remotely piloted vehicle
JPH09266548A (en) * 1995-11-27 1997-10-07 Canon Inc System, device, and method for camera control, and memory
US20080036875A1 (en) * 2006-08-09 2008-02-14 Jones Peter W Methods of creating a virtual window
WO2015184416A1 (en) * 2014-05-29 2015-12-03 Nextvr Inc. Methods and apparatus for delivering content and/or playing back content

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4405943A (en) * 1981-08-19 1983-09-20 Harris Corporation Low bandwidth closed loop imagery control and communication system for remotely piloted vehicle
JPH09266548A (en) * 1995-11-27 1997-10-07 Canon Inc System, device, and method for camera control, and memory
US20080036875A1 (en) * 2006-08-09 2008-02-14 Jones Peter W Methods of creating a virtual window
WO2015184416A1 (en) * 2014-05-29 2015-12-03 Nextvr Inc. Methods and apparatus for delivering content and/or playing back content

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3688536B1 (en) 2017-09-28 2022-04-27 A.P. Møller - Mærsk A/S A method and system for operating a ship
US11789443B2 (en) 2017-09-28 2023-10-17 A.P. Møller Mærsk A/S Method and system for operating a ship
US11815886B2 (en) 2017-09-28 2023-11-14 A.P. Møller Mærsk A/S Method and system for operating a ship

Also Published As

Publication number Publication date
FI20165925L (en) 2018-06-06

Similar Documents

Publication Publication Date Title
US20210390765A1 (en) Output of virtual content
US7450840B2 (en) Image distribution apparatus, image receiving apparatus, management method for image distribution apparatus, and setting method for image receiving apparatus
US11900578B2 (en) Gaze direction-based adaptive pre-filtering of video data
US20190331914A1 (en) Experience Sharing with Region-Of-Interest Selection
US8019817B2 (en) Collaboration support system and control method thereof
US8817122B2 (en) Image pickup apparatus
US20200349355A1 (en) Method for determining representative image of video, and electronic apparatus for processing the method
CN113826059A (en) Techniques to set focus in a camera in a mixed reality environment with hand gesture interaction
WO2004032107A3 (en) Image display system
JP2016143919A (en) Image processing apparatus and control method therefor
CN105306859A (en) Information-processing device and information-processing method
US11463670B2 (en) Method and apparatus for buffer management in cloud based virtual reality services
WO2018104582A1 (en) Optimizing data stream transmissions from marine vessel
WO2018138411A1 (en) Modelling a vessel and its vicinity in real-time
JP2005348371A (en) Electronic album display system, electronic album display method, and electronic album display program
US8781302B2 (en) Moving image reproducing apparatus and control method therefor, and storage medium
CN101848352B (en) Image supplying device, image supplying system and image supplying method
US10560727B2 (en) Server structure for supporting multiple sessions of virtualization
KR101706215B1 (en) Image play and backup apparatus
JP2009296536A (en) Moving image relay distributing device, moving image relay distributing method and program
JP6781445B1 (en) Information processing method
US11134236B2 (en) Image processing device and system
JP2004032560A (en) Image processing apparatus, image processing method, and program
US8060702B2 (en) Information reproducing apparatus and information reproducing method
US10880534B2 (en) Electronic device and method for tiled video multi-channel playback

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17825272

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17825272

Country of ref document: EP

Kind code of ref document: A1