EP3905236A1 - Traitement vidéo - Google Patents

Traitement vidéo Download PDF

Info

Publication number
EP3905236A1
EP3905236A1 EP20275081.6A EP20275081A EP3905236A1 EP 3905236 A1 EP3905236 A1 EP 3905236A1 EP 20275081 A EP20275081 A EP 20275081A EP 3905236 A1 EP3905236 A1 EP 3905236A1
Authority
EP
European Patent Office
Prior art keywords
video
video data
safety
security
hierarchical display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP20275081.6A
Other languages
German (de)
English (en)
Inventor
designation of the inventor has not yet been filed The
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BAE Systems PLC
Original Assignee
BAE Systems PLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BAE Systems PLC filed Critical BAE Systems PLC
Priority to EP20275081.6A priority Critical patent/EP3905236A1/fr
Priority to PCT/GB2021/051032 priority patent/WO2021220002A1/fr
Priority to EP21723896.3A priority patent/EP4143818A1/fr
Priority to US17/919,143 priority patent/US20230158887A1/en
Publication of EP3905236A1 publication Critical patent/EP3905236A1/fr
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/395—Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen
    • G09G5/397—Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/08—Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/20—Details of the management of multiple sources of image data
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2380/00—Specific applications
    • G09G2380/12—Avionics applications

Definitions

  • the present invention relates to video processing.
  • a related issue is the drive to reduce the number of user displays to a minimum whilst increasing the size to allow for flexibility. This decrease in independent displays, however, can cause problems where regulations and standards require separation of differing display integrity levels.
  • Embodiments of the present invention are intended to address at least some of the above technical problems.
  • a video processing system comprising:
  • the at least one controller may be configured to generate the output video by merging the plurality of video data in in a top down manner such that the video data assigned a higher said hierarchical display layer is arranged on top of the video data assigned a lower said hierarchical display layer.
  • the at least one controller may be further configured to obtain video display control data comprising information describing how the plurality of hierarchical display layers are assigned dependent on the plurality of video data sources.
  • the video display control data may further comprise image manipulation instructions for each of the plurality of hierarchical display layers.
  • the image manipulation instructions may be selected from a set comprising: combine, merge, scale and/or translate instructions.
  • the image manipulation instructions may identify at least one said portion of the video data to be manipulated.
  • the system may comprise a first processing component comprising a first CPU and a first GPU, and a second processing component comprising a second CPU and a second GPU.
  • the first processing component may be configured to be connected to at least said one video data source and to process safety/security related video data.
  • the second processing component may be configured to process non-safety/security related video data.
  • the first processing component may further comprise a separate video switching, mixing and control component configured to process the plurality of video data.
  • the video switching, mixing and control component may comprise a Field Programmable Gate Array, FPGA, or an Application Specific Integrated Circuit, ASIC.
  • the CPU of the first processing component may be configured to provide graphics generation instructions to the GPU of the first processing component, wherein the GPU comprises one of the video data sources and uses the graphics generation instructions to generate one of the plurality of safety/security related video data.
  • the CPU of the second processing component may provide graphics generation instructions to the GPU of the second processing component, wherein the GPU comprises one of the video data sources and uses the graphics generation instructions to generate one of the plurality of non-safety/security related video data.
  • the CPU of the first processing component may provide the image manipulation instructions to the video switching, mixing and control component.
  • At least one of the plurality of video data may be received from an external source via a video input/interface.
  • a predetermined pixel value is set as a chroma-key colour value and pixels of the video data having the chroma-key colour value in a said assigned hierarchical display layer may be replaced by values of corresponding pixels in a lower hierarchical display layer.
  • interference from lower layer video data may prevented by using a non-chroma key colour in the video data to ensure a solid background is displayed for higher layers of the video.
  • the safety levels may comprise DAL or SILs.
  • the system may further comprise a display device or an interface for a display device.
  • a (computer-implemented) method of processing video comprising:
  • a controller configurable to transfer instructions to a video switching, mixing and control component substantially as described herein.
  • a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out methods substantially as described herein.
  • a computer device configured to execute a method substantially as described herein.
  • a vehicle such as an aircraft, comprising a video data processing system substantially as described herein.
  • Figure 1 is a diagram of an example video processing system 100.
  • the system is designed to take data from avionics equipment, such as radios, inertial navigation sensors, radio navigation equipment and height sensors, etc, and process it for display to a pilot so that they may control the appropriate equipment or use the displayed information to fly the aircraft.
  • avionics equipment such as radios, inertial navigation sensors, radio navigation equipment and height sensors, etc.
  • the example system 100 of Figure 1 comprises a controller/processor unit 102 configured as a general purpose computing and graphics processor that can perform processing, including switching/merging operations, on video data.
  • the processor unit can be associated at least one internal memory (e.g. Random Access Memory) and can also have one or more interfaces that allow communication with other devices via any suitable wired/wireless interface and/or communications network.
  • the internal memory can store data and instructions for processing by the processor(s).
  • the processor unit may also include/be associated with other conventional features, such as non-volatile storage device(s), a user interface, and so on, which need not be described herein in detail.
  • Video output generated by the processor unit 102 can be displayed on one or more display device 104.
  • the display device(s) can be based on any suitable technology, e.g. LED, OLED, LCD, etc.
  • the processor unit can receive a plurality of video data from a respective plurality of internal and/or external video data sources.
  • the external video data sources may be connected to the processor unit by any suitable wired or wireless interface(s), e.g. HDMI.
  • first 106A and second 106B external video data sources are connected to the processor unit via first 107A and second 107B respective interfaces. These sources may comprise external video feeds from equipment such as a video camera or a Forward Looking Infra-Red Camera. It will be understood that any number of video sources/interfaces can be used in embodiments.
  • the video sources/interfaces may be of the same or different types.
  • the processor unit 102 also processes video data that is generated using signals from one or more avionic equipment components 108A - 108D that are in communication with it by means of a data bus 110.
  • the processor unit may be configured to combine one or more video/graphics channels generated internally (by General Purpose Computing) and the external video inputs into one output video for display on the display device 104 in the manner described herein.
  • the video data may be received from the various sources substantially simultaneously.
  • the data received by the processor unit 102 may be processed based on recognized safety or security standards. For instance, data received from one or more of the data sources may be treated in a manner that corresponds to a particular level of an international standard that defines a plurality of hierarchical levels. Data from a particular source may be in accordance with one of those standard levels, which affects the manner in which it is processed/displayed. For instance, the received data can have differing safety integrity levels (SILs) as apportioned through a system safety analysis process following recognised standards, such as ARP-4754/ARP-4761 or DEF-STAN 00-055.
  • SILs safety integrity levels
  • standards DO-254 and EUROCAE ED-80 define five Design Assurance Levels, commonly referred to as DAL, that describe how critical components are for safe flight.
  • DAL Design Assurance Levels
  • the different DAL levels progressively describe components whose importance ranges from extremely important to trivial for safe flight.
  • DAL A the highest importance/integrity level, describes flight electronics hardware whose failure or malfunction could cause a catastrophic, hazardous, or severe condition that would result in the deaths of everyone aboard the aircraft.
  • the data may be processed based on data security standards that define hierarchical levels.
  • Examples of alternative safety/security standards that can be used in alternative embodiments include: DO-326A/ED-202A, NIST Special Publication 800-53 (Security), and IEC 61508, IEC 62279, IEC 61511, IEC 61513 (Safety).
  • FIG. 2 schematically illustrates the logical internals of the processor unit 102 configured to operate General Purpose Computing and Graphics Processing, where a Graphics Processing Unit (GPU) is used to perform computation in applications conventionally performed by a Central Processing Unit (CPU). GPU processing is very efficient, high performance and low latency and may avoid negatively affecting the performance of existing applications.
  • the processor unit can provide a switching/merging device/capability, including Video Switching, Mixing & Control (VSM&C).
  • VSM&C Video Switching, Mixing & Control
  • the processor unit 102 can contain computer processing and graphics generation components within two physically separate hardware components 202A, 202B.
  • Each component 202A, 202B can comprise a minimum of one circuit board (i.e. all on the same board), but may also be comprised of multiple circuit boards, e.g. CPU and GPU contained on separate circuit boards.
  • the first hardware component 202A can be intended to process safety/security-related data (e.g. data that should be processed in accordance with a recognised safety/security standard), whilst the second hardware component 202B can process non-safety/security data. This design can prevent the no-integrity, non-safety/security processing from interfering with the safety/security processing, which would invalidate safety requirements.
  • the safety/security-related processing component 202A comprises a CPU 206A and a GPU 208A, as well as a component 209 configured to provide a safety/security qualified video switching and merging capability.
  • the CPU may be run a certifiable operating system, such as Wind RiverTM VxWorks653 or Green HillsTM Integrity-178, for example.
  • the component 209 may comprise an FPGA or ASIC, or any other suitable controller or processor, configured to perform video processing operations on a plurality of received video data.
  • embodiments may use any suitable means, including circuits, processors, microprocessors, microcontrollers, ASICs, FPGAs, etc, for implementing system components.
  • the non-safety/security-related processing component 202B comprises its own CPU 206B and GPU 208B. Video data from the non-safety/security related GPU 208B and the external video source 106 associated with the safety/security-related processing component 202B can also be transferred to the component 209.
  • the component 209 contained within the safety/security-related processing hardware 202A can be configured to provide a video manipulation capability that is able to combine, merge, scale and translate different video inputs into a single output video. This capability can be controlled by safety/security-related software running on the CPU 206A.
  • the hardware within the safety/security-related component is normally developed to the highest integrity level as allocated from the system safety process, e.g. DO-254 DAL A, using known commercial video techniques and mechanisms.
  • Embodiments can advantageously provide the separation of concerns for the processing and graphics generation along with the use of the Safety/Security Qualified Video Switching and Merging Capability (via FPGA/ASIC or equivalent) controlled by the safety/security related processor to configure any required mixing scenario.
  • the CPU 206A of the safety/security processing component 202A can produce graphics generation commands that it transfers to the GPU 208A. These commands can control how the GPU generates output video, such as real-time navigational displays, that is based on data received from one or more of the avionics components 108.
  • the GPU 208A can generate video data that is transferred to the component 209 and is treated as a safety/security-related video data source.
  • the CPU 206B of the non-safety/security processing component 202B can control the GPU 208B to generate video data that is also transferred to the component 209, but is treated as a non-safety/security-related video data source.
  • the component 209 receives the plurality of video data received from the GPU 208A, the GPU 208B and any external video sources 106A, 106B and processes the data to generate output video that can be displayed on the display device 104.
  • FIG. 3 is a flowchart illustrating steps that can be performed by the video processing system 100.
  • the steps will typically be controlled by a processor in the processor unit 102, e.g. the CPU 206A of the safety/security processing component 202A, but it will be understood that one or more other processor(s) may perform at least one of the steps in alternative embodiments.
  • the processor unit 102 can obtain video display control data that can be used to control how output video data for display is generated.
  • the video display control data may be stored in an internal memory associated with the processor unit, or may be transferred from an external storage, device or communication link.
  • the video display control data may be generated by, for example, an administrator user of the video processing system 100 using an application or data editor. This can allow the administrator user to at least assign a display layer to the video data received from each of the various (internal/external) video data sources and, optionally, to specify additional video manipulation instructions that can be executed on the video data to generate the output video.
  • At least some of the hierarchical display layers can correspond to data that is treated according to a particular safety/security level defined by a safety/security standard.
  • a particular video data source is known to provide video data that is based on information treated in accordance with a particular level of the standard then the hierarchical display layer that has been designed to correspond to that standard level can be assigned to it.
  • a video data source that deals with the highest level of data according to the standard e.g. DAL A
  • the video data source that deals with the second highest level of data according to the standard e.g. DAL B
  • the second highest display layer e.g. DAL B
  • the video display control data will define at least a plurality of hierarchical display layers.
  • Layers are known in image/video editing and can separate different elements of a video/image. Layers can represent a part of an image/video, either as pixels or as modification instructions, and may be stacked/ordered to determine the appearance of the final image/video. Operations can be performed on layers, including stacking and merging. Layers can be partially obscured allowing portions of images within a layer to be hidden or shown in a translucent manner within another image. Layers can also be used to combine two or more images into a single image.
  • the video display control data can contain information that is used to assign one of the plurality of hierarchical display layers to video data that is received from different video data sources of the system.
  • Safety/security-related GPU 208A (graphics generation commands from CPU 206A based on signals from one or more of avionic equipment components 108A - 108D) 2 Safety/security-related external video source 106A 3 Non-safety/security-related GPU 208B (graphics generation commands from CPU 206B) 4 Non-safety/security-related external video source 106B
  • the video display control data may further comprise additional data that can be used to control how the plurality of video data will be incorporated into the output video data.
  • the additional data may indicate how portions of individual ones of the video data can be arranged, such as shown in the examples discussed below.
  • the processor unit 102 can receive the plurality of video data to be processed.
  • the video data may be received from various sources internal or external to the processor unit, e.g. as indicated in the example table above.
  • the processor unit 102 can assign one of the plurality of hierarchical display layers to each of the received plurality of video data using the video display control data, e.g. as indicated in the example table above.
  • the processor unit 102 can generate an output video for display.
  • the output video typically generated by the component 209, comprises at least a portion of each the plurality of video data ordered/arranged/layered according to the assigned hierarchical display layers.
  • the output video generation can also involve additional video processing based on the additional information included in the video display control data. For instance, operations such as merge or combine may be performed on at least a portion of at least one of the plurality of video data, such as shown in the examples discussed below.
  • Figure 4 illustrates a first example of how a plurality of video data received from both internal and external sources can be manipulated using operations, such as cropping, scaling, translation, chroma keying, etc, to generate an output video 400.
  • first video data 402 can comprise video images received from the GPU 208A and generated based on signals from one or more of the avionics components 108A - 108B.
  • the first video data is designated the highest/topmost layer 1 amongst the hierarchical display layers of the output video and corresponds to safety/security level DAL-B as defined by the relevant safety/security standard (DAL-B, rather than DAL-A, is set as the highest safety/security level expected to be processed by the system according to the video display control data of the example).
  • the video display control data can also contain information regarding further operations that can be performed on the received video data before/when it is added to the output video.
  • first 404A and second 404B portions of the first video data 402 should appear in layer 1 of the output video after chroma key and merge operations have been performed on them.
  • the video display control data can contain details for such manipulation operations, e.g. coordinates of the portion(s), parameters for the specific operations, and so on.
  • the CPU 206A can then use this information to provide instructions to the component 209 for it to generate the output video.
  • Second video data 406 can comprise video data from the first external video source 106A.
  • the second video data is designated as layer 2 amongst the hierarchical display layers of the output video and a portion 408 of the second video should be inserted in the output video.
  • the external video source may have an associated safety/security level, e.g. DAL, and therefore be safety related.
  • the display layer may not directly correspond to a particular safety/security level of the standard, but the administrator user uses knowledge of the system to create display control data where a particular video data source is the second most important source in terms of safety/security.
  • FIG. 4 also shows how portions of further received video data 413 - 418 (to be designated as layers 3 - 8 amongst the hierarchical display layers of the output video) can be manipulated and merged to generate the output video 400. If video data is not received from a particular source then that layer can be skipped/omitted from the output video.
  • Figure 5 illustrates another example of an output video 500 being generated.
  • Figures 4 and 5 do not show the same scenario as they combine different numbers of video data inputs/channels.
  • the merging of layers based on received input video data can be done in a top down priority manner such that the higher safety/security level information should be always arranged on top so that lower, or no integrity graphics, cannot overwrite or interfere with higher priority graphics/information.
  • layer 1 which corresponds to DAL A information
  • 500 overlays layer 2, 504 (which corresponds to DAL B information) at location 502 and this will be layered on top of layer 3, 508 (which corresponds to DAL E information).
  • Interference from background/lower layer video graphics can be prevented where appropriate by using a non-chroma key colour to ensure a solid background is displayed for the higher integrity/level portions of the displayed video.
  • the chroma key colour is applied using the generating application's graphics generations commands and the component 209 replaces that colour with the lower layer's colours on a pixel by pixel basis. This, in effect for each pixel, works from the lowest layer replacing each pixel as required up through to the top layer, if applicable. If no chroma-key is present then this replaces the pixel.
  • RGB colour of a pixel is replaced for a different value to provide a different background; for example, RGB (0, 255, 0) might be used as a chroma key and any pixel in the layers of the video having that value can be replaced by the corresponding pixel in the lower layer.
  • more than one video source may be based on information that is at the same standard level (e.g. multiple videos all based on DAL E level information as shown in Figure 5 ).
  • Embodiments can control how the video data from such multiple sources will be displayed in the output video. For instance, specific (e.g. non-overlapping) portions of the different videos may be combined to form a single layer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)
EP20275081.6A 2020-04-30 2020-04-30 Traitement vidéo Ceased EP3905236A1 (fr)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP20275081.6A EP3905236A1 (fr) 2020-04-30 2020-04-30 Traitement vidéo
PCT/GB2021/051032 WO2021220002A1 (fr) 2020-04-30 2021-04-29 Traitement vidéo
EP21723896.3A EP4143818A1 (fr) 2020-04-30 2021-04-29 Traitement vidéo
US17/919,143 US20230158887A1 (en) 2020-04-30 2021-04-29 Video processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP20275081.6A EP3905236A1 (fr) 2020-04-30 2020-04-30 Traitement vidéo

Publications (1)

Publication Number Publication Date
EP3905236A1 true EP3905236A1 (fr) 2021-11-03

Family

ID=70483068

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20275081.6A Ceased EP3905236A1 (fr) 2020-04-30 2020-04-30 Traitement vidéo

Country Status (1)

Country Link
EP (1) EP3905236A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1729256A1 (fr) * 2005-05-30 2006-12-06 Harman Becker Automotive Systems GmbH Processeur d'images
EP3082127A1 (fr) * 2015-04-17 2016-10-19 Freescale Semiconductor, Inc. Contrôleur d'affichage, système d'affichage d'image frontal et procédé associé
US20190286115A1 (en) * 2018-03-15 2019-09-19 Xinxin Wang Method and system for preventing and detecting hazardously misleading information on safety-critical display

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1729256A1 (fr) * 2005-05-30 2006-12-06 Harman Becker Automotive Systems GmbH Processeur d'images
EP3082127A1 (fr) * 2015-04-17 2016-10-19 Freescale Semiconductor, Inc. Contrôleur d'affichage, système d'affichage d'image frontal et procédé associé
US20190286115A1 (en) * 2018-03-15 2019-09-19 Xinxin Wang Method and system for preventing and detecting hazardously misleading information on safety-critical display

Similar Documents

Publication Publication Date Title
US9691359B2 (en) Vehicle display system with transparent display layer
US8462018B1 (en) Systems and method for controlling the simultaneous display of multi-level classified information on the same surface of an aircraft display unit
RU2702295C2 (ru) Способ общего представления информации, критичной для безопасности, и информации, не критичной для безопасности, и дисплейное устройство
US20140359564A1 (en) System for Scheduling Tasks to Control the Execution of Warning Procedures on an Aircraft
US20120050320A1 (en) Device and method for generating variable priority multiwindow images
Latham The dictionary of computer graphics and virtual reality
US6147688A (en) Method and apparatus for defining and selectively repeating unit image cells
DE102019102009A1 (de) Reduzierung des rauschens während des renderings durch parallele path-space-filterung unter verwendung von hashing
US9156543B2 (en) Control system for an aircraft
WO2012112090A1 (fr) Affichage de données de vol
CN105786426A (zh) 用于操控显示设备和显示系统的方法和设备
EP1556835A1 (fr) Procede pour enlever la couleur de fond dans une image composite de porter et duff
US9715356B2 (en) Method, apparatus and system for determining a merged intermediate representation of a page
Read Developing the next generation cockpit display system
EP3905236A1 (fr) Traitement vidéo
US20030214508A1 (en) System for alpha blending and method thereof
US20230158887A1 (en) Video processing
CN109213563B (zh) 支持多用户同步操作的座舱显示系统
CN1203406A (zh) 提供自动控制显示单元备份的飞行管理系统
US8520015B2 (en) Method for intermixing graphics applications using display intermix server for cockpit displays
GB2594493A (en) Video processing
Barladian et al. Visualization component for the aircraft real-time operating system JetOS
DE102021113178A1 (de) Techniken für zugriff auf komprimierte daten und ihre statusinformationen sowie nutzung der daten und statusinformationen
DE102022128966A1 (de) Parallelverarbeitung zur kombinatorischen optimierung
KR20150004479A (ko) 선박 모델링 시스템 그 모델링 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

B565 Issuance of search results under rule 164(2) epc

Effective date: 20200701

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20220222