WO2022228683A1 - Modular display apparatus and method for controlling thereof - Google Patents

Modular display apparatus and method for controlling thereof Download PDF

Info

Publication number
WO2022228683A1
WO2022228683A1 PCT/EP2021/061287 EP2021061287W WO2022228683A1 WO 2022228683 A1 WO2022228683 A1 WO 2022228683A1 EP 2021061287 W EP2021061287 W EP 2021061287W WO 2022228683 A1 WO2022228683 A1 WO 2022228683A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
display modules
image processor
raster image
address number
Prior art date
Application number
PCT/EP2021/061287
Other languages
French (fr)
Inventor
Lachezar Petkanchin
Original Assignee
Nrg Tech Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nrg Tech Ltd. filed Critical Nrg Tech Ltd.
Priority to PCT/EP2021/061287 priority Critical patent/WO2022228683A1/en
Publication of WO2022228683A1 publication Critical patent/WO2022228683A1/en

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/22Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources
    • G09G3/30Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels
    • G09G3/32Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1446Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2300/00Aspects of the constitution of display devices
    • G09G2300/02Composition of display devices
    • G09G2300/026Video wall, i.e. juxtaposition of a plurality of screens to create a display screen of bigger dimensions
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/06Adjustment of display parameters
    • G09G2320/0693Calibration of display systems
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/02Details of power systems and of start or stop of display operation
    • G09G2330/026Arrangements or methods related to booting a display
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00Aspects of display data processing
    • G09G2340/04Changes in size, position or resolution of an image
    • G09G2340/0492Change of orientation of the displayed image, e.g. upside-down, mirrored
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2356/00Detection of the display position w.r.t. other display screens
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00Aspects of the architecture of display systems
    • G09G2360/14Detecting light within display terminals, e.g. using a single or a plurality of photosensors
    • G09G2360/145Detecting light within display terminals, e.g. using a single or a plurality of photosensors the light originating from the display screen
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00Aspects of data communication
    • G09G2370/02Networking aspects
    • G09G2370/022Centralised management of display operation, e.g. in a server instead of locally
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00Aspects of data communication
    • G09G2370/02Networking aspects
    • G09G2370/025LAN communication management
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00Aspects of data communication
    • G09G2370/04Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00Aspects of data communication
    • G09G2370/04Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
    • G09G2370/042Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller for monitor identification
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00Aspects of data communication
    • G09G2370/10Use of a protocol of communication by packets in interfaces along the display data pipeline
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/006Electronic inspection or testing of displays and display drivers, e.g. of LED or LCD displays

Definitions

  • the invention relates to a modular display apparatus and a controlling method thereof. Particularly, the invention relates to a modular display apparatus with a screen that can be extended, and a controlling method thereof.
  • Such a screen can be used in big concert events and in smaller venues like conference rooms. Assembly and disassembly should be quick and with minimal possibility for errors on the part of the technical team in charge.
  • the screen could not be resized after being assembled once.
  • the screen needs a manual input from a skilled engineer after being resized.
  • More recent devices can automatically determine the position of every individual module of the screen by magnetic field sensors and electromagnets mounted on the sides of the display modules. By running a complicated algorithm to switch electromagnets on and off, and reading electromagnetic sensors, the exact topology of the entire screen can be determined. In most cases it is assumed that the large screen will have only a rectangular shape and all display modules need to be oriented vertically in strictly the same way. It is an object of the invention to overcome these shortcomings and provide a modular display apparatus with a screen that can be extended, and a controlling method thereof.
  • a modular display apparatus comprising a plurality of display modules, connected in arbitrary order, and a raster image processor connected to the to the plurality of display modules.
  • the raster image processor is configured to send its unique address number to all display modules, and each display module is configured to display both its unique address number and the address number of the raster image processor received.
  • a digital camera which is configured to photograph the display modules with their address numbers visualized and submit the obtained control image to the raster image processor.
  • the raster image processor is configured to analyze the control image by optical character recognition software and determine the orientation in space, the address number and the location of each display module relative to neighboring display modules, and record these data in a set-up table.
  • the raster image processor is further configured to extract a portion of an input image to be displayed and, based on the set-up table, send it to the specified display module with its unique address number and properly rotated so that the specified display module recognizes its address number and acquires the portion of the input image and displays it so that it appears on proper location with correct orientation.
  • the raster image processor is configured to repeat the process for all of the remaining display modules.
  • the display modules are assembled together regardless of any order or orientation space. The only condition is to properly align the sides of neighboring display modules. Every display module of the plurality of display modules has its own address number stored in the memory inside it, together with other data, specific for the particular display module. Address number and other specific data is programmed into every display module memory during manufacturing.
  • the plurality of display modules are connected to the raster image processor in series. Star or wireless connections are also possible embodiments. No consideration is taken of the order of connection of the plurality of display modules to the raster image processor. No manual input is fed to the raster image processor of the number of display modules connected or the order of connection. Then initial set-up procedure is initiated by the raster image processor.
  • the raster image processor sends over the serial connection line its address number and a universal command to all display modules of the plurality of display modules connected to visualize their set-up graphic.
  • the set-up graphic is individual for every display of the plurality of display modules and comprises the display’s address number and other individual data.
  • the set-up graphic may comprise symbols, figures and/or QR codes containing a unique address number for each display module, together with the address number of the raster image processor acquired over the serial connection and other data, related to the display module.
  • All display modules in the plurality of display modules recognize the universal command and visualize their set-up graphics.
  • the composite image of all displays of the plurality of display modules with all their set-up graphics displayed on them is then photographed by a digital camera and this control image is fed back to the raster image processor.
  • the control image contains the set-up graphics visualized by all displays of the plurality of display modules.
  • the control image of the plurality of display modules is analyzed by the raster image processor and every display module’s location and orientation in space is determined by analyzing the set-up graphics displayed.
  • the set-up graphic may contain a figure which is orientation sensitive, like a right-angle triangle.
  • the set-up graphics are present in the control image taken of the plurality of display modules.
  • a specialized algorithm running on the raster image processor identifies the set-up graphics in the control image, as well as their orientation in space, which indicates the orientation of every display module of the plurality of display modules.
  • Specialized software on the raster image processor scans the control image and locates all orientation sensitive figures present in it and records their relative position and orientation in a set-up table. The position of every module is recorded in the set-up table in terms of relative coordinates, wherein every step is the length or width of one display module. This information is indicative of the position and orientation of all display modules of the plurality of display modules.
  • the address numbers of the display modules are also visualized on the set-up graphic and therefore are also present in the control image of the plurality of display modules. This address number is displayed in a precise position relative to the orientation sensitive figure. Taking into account the orientation and position of all orientation sensitive figures, the set-up graphics of all address numbers and the address number of the raster image processor are cropped from the control image and fed to an optical character recognition (OCR) software which extracts exact address numbers of all display modules of the plurality of display modules, together with the address number the raster image processor.
  • OCR optical character recognition
  • a record is created in the set-up table, containing the address number, location coordinates, orientation in space as well as angle and other data of the display module.
  • the raster image processor runs a test procedure. Consecutively all display modules are addressed with their address numbers as from the filled set-up table followed by status request command. In the same sequence the display modules recognize their address numbers and reply with “ok” or an error message. If certain display module did not reply then the raster image processor issues an error message.
  • the raster image processor is further configured to crop an input image received from an external device, based on the identified locations of the display modules, and display each of the cropped image frames on each of the plurality of display modules based on the identified locations and the address number of the respective display modules. This enables to display a large image on a plurality of display modules which are located in arbitrary order and orientation. Every display module of the plurality of display modules is configured to receive packets of data from the raster image processor by recognizing its address number, which is attached to every packet of data. Every display module is further configured to visualize digital images received from the raster image processor in one or more data packets over the series connection.
  • a method for controlling a modular display apparatus comprising: identifying the number of display modules in the plurality of display modules and locating the position, orientation and unique address number of every display module by extracting information from a control image of the plurality of display modules.
  • the method may further comprise the step of cropping an input image, received from external apparatus, into as many image frames as there are display modules, rotating the image frame if necessary, and sending every cropped image frame to a particular display module, based on location, orientation and address number of the display modules, wherein the position of the display in the modular display apparatus is the same as the position of the cropped image in the input image.
  • Fig. 1 is a diagram illustrating a display apparatus according to an embodiment
  • Fig. 2 is a diagram illustrating a part of a display apparatus according to an embodiment
  • Fig. 3 is a diagram illustrating a large-size display apparatus according to an embodiment
  • Fig. 4 is an exemplary diagram of how display modules position in space is determined
  • Fig. 5 is a flow chart illustrating how the raster image processor crops portions of the input image and sends it to a display with particular address number, associated with its location and orientation in space;
  • Fig. 6 is an exemplary set-up table, containing description of all connected display modules with their address numbers, location as (X, Y) coordinates and their orientation as an angle;
  • Fig. 7 is a diagram illustrating an example of display modules visualizing an exemplary input image;
  • Fig. 8a - 8f is a set of diagrams illustrating the images the raster image processor sends to individual display modules, so that the exemplary image from Fig. 7 appears correctly on the plurality of display modules;
  • Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module in the middle;
  • Fig. 10 is the set-up table, describing the configuration of display modules with one missing module in the middle;
  • Figs. 1 - 2 are diagrams illustrating a single display module according to an embodiment.
  • a display module 1 may include a plurality of illuminated pixels 11 , may be implemented as an LED display module including light emitting diodes (LEDs).
  • the LED of an LED display module according to an embodiment may be implemented as a micro-LED, representing one pixel 11 on the display module 1.
  • an LED display module as described above is just an example, and a display module may also be implemented as an organic LED (OLED), an active-matrix OLED (AMOLED), a plasma display panel (PDP), and the like.
  • OLED organic LED
  • AMOLED active-matrix OLED
  • PDP plasma display panel
  • the number of pixels 11 horizontally equals the number of pixels 11 vertically and the front look of the display module 1 is square.
  • Each side of the display module 1 can be aligned and mechanically attached to any side of a similar neighboring display module without consideration of the orientation of any of the attached display modules 1.
  • Every pixel 11 is associated with portion of the video memory in the display module. By varying the content of the video memory, the pixels emit accordingly varying wave length light.
  • Fig. 3 illustrates an exemplary configuration of a modular display apparatus, comprising six display modules 1 , all connected by serial connection 8 to a raster image processor
  • the raster image processor 7 After the configuration is assembled and connected, the raster image processor 7 initiates the set-up procedure.
  • the raster Image Processor 7 transmits over the serial connection 8 its unique address number followed by universal commands to the plurality of display modules 1 to visualize their set-up graphic.
  • An exemplary set-up graphic comprises a right-angle triangle 2 and a nine-digit number 3.
  • the right-angle triangles 2 indicate the orientation in space of each display module of the plurality of six display modules connected in the example configuration.
  • the nine-digit numbers 3 comprises the address number of the raster image processor 7 received over the serial connection
  • the address number of the raster image processor 7 is 310.
  • the remaining six digits of the nine-digit numbers 3 contain the individual address numbers of all display modules connected.
  • the address numbers of the connected display modules are:
  • Fig. 4 visualizes an exemplary method to analyze the control image received from the camera 9. All pixels of the control image from the camera 9 are scanned horizontally from top to bottom. The line 6 indicates a horizontal line of pixels being scanned at a certain moment. Pixels of the control image from camera 9 are scanned in lines from left to right and top to bottom in two programming cycles - one for horizontal scan and one for vertical scan. The arrow 5 indicates the direction where the next horizontal lines of scanning will appear.
  • the right-angle triangles 2a, 2b and the nine-digit numbers 3 are displayed in dark color on the plurality of display modules 1. Therefore, on the control image from camera 9 the right-angle triangles 2a, 2b will appear as a set of triangle shaped dark pixels.
  • the triangle 2a will register sections of dark pixels 4a, increasing linearly to the left until the line scan reaches the right-angle tip. With every line of scan the dark pixels will start earlier and will end roughly at the same number of pixels scanned. After that the section of dark pixels 4a will decrease linearly from the left. If such rule applies, the related display module 1 is registered as +90° rotated. The tips of the triangle are stored in a memory.
  • the triangle 2b will register sections of dark pixels 4b, increasing linearly to the left and to the right until the line scan reaches the right-angle triangle hypotenuse. If such rule applies, the related display module 1 is registered as 0° rotated. Similar rules apply when display modules rotated at -90° and 180° are registered (not shown).
  • the nine-digit numbers 3 are visualized in exemplary set-up graphics centered below the hypotenuse of the triangles. After determining orientation of every display module 1 of the plurality of display modules, the raster image processor 7 locates the pixels where every nine-digit number 3 is located on the control image from the camera 9.
  • the images of the nine-digit numbers 3 are cropped from the control image of the camera 9 and fed to OCR (optical character recognition) software which recognizes every digit and all nine-digit numbers 3 as a sequence of decimal digits.
  • OCR optical character recognition
  • the data is recorded in a set-up table as described with respect to Fig. 6 with the corresponding orientation of every display module 1 , its nine-digit number and position on the control image received from the camera 9.
  • Fig. 5 is a flowchart of the algorithm to analyze the control image received from the camera 9.
  • the set-up table as described in Fig. 6, describing the configuration of the plurality of display modules is filled in.
  • the algorithm uses data extracted from the scan of the control image from the camera 9 and creates a simple (X, Y) coordinate system, where every display module 1 is assigned coordinates, such that the leftmost display modules have their X coordinate as 1 , and the lowermost display modules have their Y coordinate as 1.
  • the maximum value of X is determined as the number of steps from the leftmost display module to the rightmost display module.
  • the maximum value for Y is determined as the number of steps from the lowermost display module to the uppermost display module.
  • Fig. 6 is an exemplary set-up table, derived from the exemplary configuration of display modules 1 shown in Fig. 3 after processing the control image from the camera 9 with the algorithm shown in the flowchart of Fig. 5
  • Fig. 7 is an example of an input image of six letters to be displayed on the plurality of display modules 1.
  • the input image is received by the raster image processor 7 from an external source.
  • the raster image processor 7 After the input image is received by the raster image processor 7, it is scaled to fit into the total image area of the modular display apparatus and then divided according to the data in the set-up table, so that a separate portion of the image is prepared for every display module 1 in the modular display apparatus. This process involves properly rotating every portion of the image and breaking it down into bits of data to control every pixel on all display modules 1 in the modular display apparatus.
  • Figures 8a to 8f show how the desired input image from Fig. 7 is divided into 6 parts, one part for every display module 1 of the configuration as shown on Fig. 3. Each of those six parts is rotated according to the record in the set-up table from Fig. 6.
  • the display module has address 010331 and is rotated at 180° as recorded in the set-up table (Fig. 6). In other words, it is mounted upside down. Therefore, the image on Fig. 8a is rotated at 180°, to compensate for the way the display module had been rotated when mounted. Then it is sent to address 010331.
  • the image from Fig. 8b is rotated at 180° and sent to address 010483; the image from Fig. 8c is sent to address 003063 without rotation; the image from Fig. 8d is sent to address 002172 without rotation; the image from Fig. 8e is rotated at -90° and sent to address 024191 ; the image from Fig. 8f is rotated at +90° and sent to address 024191.
  • the desired image will appear correctly on the modular display apparatus, regardless of the fact that individual display modules 1 are mounted in arbitrary fashion.
  • Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module 10 in the middle. Such configuration may be used in combination with a real object to be placed right in the middle of the modular display apparatus. Advertising is an exemplary application.
  • Fig. 10 is the set-up table, generated after running the algorithm from Fig. 5 on the control image of the configuration on Fig. 9. The set-up table describes the configuration of display modules with one missing module in the middle, as shown on Fig. 9. The set - up table has a specific record, stating that the spot with coordinates (2,2) is empty.

Abstract

A modular display apparatus is provided. The modular display apparatus includes a plurality of display modules, a single connector on every display module for serial type connection to a single raster image processor configured to crop portions the image to be displayed into sub images, addressed to individual display modules. The exact position, orientation in space and unique address information of every individual display module is determined by digital optical recognition of a digital control image of the entire display module configuration after the plurality of display modules are mechanically assembled into a large screen and electrically connected.

Description

Modular display apparatus and method for controlling thereof
This application is filed in the name of NRG Tech Ltd., j.k. Bokar bl. 43, ap. 1 , 1404 SOFIA (BG), VAT-No. BG201220396.
The invention relates to a modular display apparatus and a controlling method thereof. Particularly, the invention relates to a modular display apparatus with a screen that can be extended, and a controlling method thereof.
Technological advance in recent years gave rise to variety of multimedia devices, big and small. In particular, extra-large display devices, consisting of many identical modules put together to appear as one large multimedia screen are being developed.
Very often there is a need to alter both the size and the shape of such a modular large screen. For example, such a screen can be used in big concert events and in smaller venues like conference rooms. Assembly and disassembly should be quick and with minimal possibility for errors on the part of the technical team in charge.
Flowever, there are problems with such modular large screen devices in the related art. In some cases, the screen could not be resized after being assembled once. In other examples the screen needs a manual input from a skilled engineer after being resized.
More recent devices can automatically determine the position of every individual module of the screen by magnetic field sensors and electromagnets mounted on the sides of the display modules. By running a complicated algorithm to switch electromagnets on and off, and reading electromagnetic sensors, the exact topology of the entire screen can be determined. In most cases it is assumed that the large screen will have only a rectangular shape and all display modules need to be oriented vertically in strictly the same way. It is an object of the invention to overcome these shortcomings and provide a modular display apparatus with a screen that can be extended, and a controlling method thereof.
According to the invention, there is provided a modular display apparatus comprising a plurality of display modules, connected in arbitrary order, and a raster image processor connected to the to the plurality of display modules. The raster image processor is configured to send its unique address number to all display modules, and each display module is configured to display both its unique address number and the address number of the raster image processor received. There is further provided a digital camera, which is configured to photograph the display modules with their address numbers visualized and submit the obtained control image to the raster image processor.
The raster image processor is configured to analyze the control image by optical character recognition software and determine the orientation in space, the address number and the location of each display module relative to neighboring display modules, and record these data in a set-up table.
The raster image processor is further configured to extract a portion of an input image to be displayed and, based on the set-up table, send it to the specified display module with its unique address number and properly rotated so that the specified display module recognizes its address number and acquires the portion of the input image and displays it so that it appears on proper location with correct orientation. The raster image processor is configured to repeat the process for all of the remaining display modules.
Initially, the display modules are assembled together regardless of any order or orientation space. The only condition is to properly align the sides of neighboring display modules. Every display module of the plurality of display modules has its own address number stored in the memory inside it, together with other data, specific for the particular display module. Address number and other specific data is programmed into every display module memory during manufacturing. In a preferred embodiment the plurality of display modules are connected to the raster image processor in series. Star or wireless connections are also possible embodiments. No consideration is taken of the order of connection of the plurality of display modules to the raster image processor. No manual input is fed to the raster image processor of the number of display modules connected or the order of connection. Then initial set-up procedure is initiated by the raster image processor.
During the initial set-up procedure, the raster image processor sends over the serial connection line its address number and a universal command to all display modules of the plurality of display modules connected to visualize their set-up graphic. The set-up graphic is individual for every display of the plurality of display modules and comprises the display’s address number and other individual data. The set-up graphic may comprise symbols, figures and/or QR codes containing a unique address number for each display module, together with the address number of the raster image processor acquired over the serial connection and other data, related to the display module.
All display modules in the plurality of display modules recognize the universal command and visualize their set-up graphics. The composite image of all displays of the plurality of display modules with all their set-up graphics displayed on them is then photographed by a digital camera and this control image is fed back to the raster image processor. The control image contains the set-up graphics visualized by all displays of the plurality of display modules. The control image of the plurality of display modules is analyzed by the raster image processor and every display module’s location and orientation in space is determined by analyzing the set-up graphics displayed. In particular the set-up graphic may contain a figure which is orientation sensitive, like a right-angle triangle.
The set-up graphics are present in the control image taken of the plurality of display modules. A specialized algorithm running on the raster image processor identifies the set-up graphics in the control image, as well as their orientation in space, which indicates the orientation of every display module of the plurality of display modules. Specialized software on the raster image processor scans the control image and locates all orientation sensitive figures present in it and records their relative position and orientation in a set-up table. The position of every module is recorded in the set-up table in terms of relative coordinates, wherein every step is the length or width of one display module. This information is indicative of the position and orientation of all display modules of the plurality of display modules. The address numbers of the display modules are also visualized on the set-up graphic and therefore are also present in the control image of the plurality of display modules. This address number is displayed in a precise position relative to the orientation sensitive figure. Taking into account the orientation and position of all orientation sensitive figures, the set-up graphics of all address numbers and the address number of the raster image processor are cropped from the control image and fed to an optical character recognition (OCR) software which extracts exact address numbers of all display modules of the plurality of display modules, together with the address number the raster image processor.
For every display module of the plurality of display modules, a record is created in the set-up table, containing the address number, location coordinates, orientation in space as well as angle and other data of the display module. After all records, one for every display module, are added to the set-up table, the raster image processor runs a test procedure. Consecutively all display modules are addressed with their address numbers as from the filled set-up table followed by status request command. In the same sequence the display modules recognize their address numbers and reply with “ok” or an error message. If certain display module did not reply then the raster image processor issues an error message.
The raster image processor is further configured to crop an input image received from an external device, based on the identified locations of the display modules, and display each of the cropped image frames on each of the plurality of display modules based on the identified locations and the address number of the respective display modules. This enables to display a large image on a plurality of display modules which are located in arbitrary order and orientation. Every display module of the plurality of display modules is configured to receive packets of data from the raster image processor by recognizing its address number, which is attached to every packet of data. Every display module is further configured to visualize digital images received from the raster image processor in one or more data packets over the series connection.
According to the invention, there is provided a method for controlling a modular display apparatus, the method comprising: identifying the number of display modules in the plurality of display modules and locating the position, orientation and unique address number of every display module by extracting information from a control image of the plurality of display modules. The method may further comprise the step of cropping an input image, received from external apparatus, into as many image frames as there are display modules, rotating the image frame if necessary, and sending every cropped image frame to a particular display module, based on location, orientation and address number of the display modules, wherein the position of the display in the modular display apparatus is the same as the position of the cropped image in the input image.
The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
Fig. 1 is a diagram illustrating a display apparatus according to an embodiment;
Fig. 2 is a diagram illustrating a part of a display apparatus according to an embodiment;
Fig. 3 is a diagram illustrating a large-size display apparatus according to an embodiment;
Fig. 4 is an exemplary diagram of how display modules position in space is determined; Fig. 5 is a flow chart illustrating how the raster image processor crops portions of the input image and sends it to a display with particular address number, associated with its location and orientation in space;
Fig. 6 is an exemplary set-up table, containing description of all connected display modules with their address numbers, location as (X, Y) coordinates and their orientation as an angle; Fig. 7 is a diagram illustrating an example of display modules visualizing an exemplary input image;
Fig. 8a - 8f is a set of diagrams illustrating the images the raster image processor sends to individual display modules, so that the exemplary image from Fig. 7 appears correctly on the plurality of display modules;
Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module in the middle;
Fig. 10 is the set-up table, describing the configuration of display modules with one missing module in the middle;
While the embodiments of the disclosure will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, it should be noted that the embodiments are not for restricting or limiting the scope of the disclosure to a specific embodiment. hereinafter, the disclosure will be described in detail with reference to the accompanying drawings.
Figs. 1 - 2 are diagrams illustrating a single display module according to an embodiment. Referring to Figures 1 and 2 a display module 1 according to an embodiment may include a plurality of illuminated pixels 11 , may be implemented as an LED display module including light emitting diodes (LEDs). The LED of an LED display module according to an embodiment may be implemented as a micro-LED, representing one pixel 11 on the display module 1. However, an LED display module as described above is just an example, and a display module may also be implemented as an organic LED (OLED), an active-matrix OLED (AMOLED), a plasma display panel (PDP), and the like. Hereinafter, for the convenience of explanation, a display module according to an embodiment will be explained based on the assumption that it is an LED display module. In a preferred embodiment the number of pixels 11 horizontally equals the number of pixels 11 vertically and the front look of the display module 1 is square. Each side of the display module 1 can be aligned and mechanically attached to any side of a similar neighboring display module without consideration of the orientation of any of the attached display modules 1. Every pixel 11 is associated with portion of the video memory in the display module. By varying the content of the video memory, the pixels emit accordingly varying wave length light.
Fig. 3 illustrates an exemplary configuration of a modular display apparatus, comprising six display modules 1 , all connected by serial connection 8 to a raster image processor
7.
After the configuration is assembled and connected, the raster image processor 7 initiates the set-up procedure. The raster Image Processor 7 transmits over the serial connection 8 its unique address number followed by universal commands to the plurality of display modules 1 to visualize their set-up graphic. An exemplary set-up graphic comprises a right-angle triangle 2 and a nine-digit number 3. The right-angle triangles 2 indicate the orientation in space of each display module of the plurality of six display modules connected in the example configuration. The nine-digit numbers 3 comprises the address number of the raster image processor 7 received over the serial connection
8, concatenated with the address number of the display module which displays the particular set-up graphic.
In the exemplary configuration depicted in Fig. 3, the address number of the raster image processor 7 is 310. The remaining six digits of the nine-digit numbers 3 contain the individual address numbers of all display modules connected. In the exemplary configuration of Fig. 3, the address numbers of the connected display modules are:
010331 , 010483, 003063, 002172, 024191 , and 024332. The set-up graphics of the plurality of display modules 1 are photographed by a camera 9 and the control image from the camera 9 is sent to the raster image processor 7 for analysis. Fig. 4 visualizes an exemplary method to analyze the control image received from the camera 9. All pixels of the control image from the camera 9 are scanned horizontally from top to bottom. The line 6 indicates a horizontal line of pixels being scanned at a certain moment. Pixels of the control image from camera 9 are scanned in lines from left to right and top to bottom in two programming cycles - one for horizontal scan and one for vertical scan. The arrow 5 indicates the direction where the next horizontal lines of scanning will appear. The right-angle triangles 2a, 2b and the nine-digit numbers 3 are displayed in dark color on the plurality of display modules 1. Therefore, on the control image from camera 9 the right-angle triangles 2a, 2b will appear as a set of triangle shaped dark pixels.
When the line of scanning 6 crosses a triangle 2a, 2b, a section of dark pixels 4a, 4b is registered. In case the figure is a triangle the section of dark pixels 4a, 4b will increase or decrease in number of dark pixels registered with every new line of scan, but this increase or decrease will be linear. The increase or decrease of number of dark pixels 4a, 4b in the scanned line will be roughly the same with every new line of scan 6. This rule will not apply to the dark pixels of the nine-digit numbers 3.
In the left side of Fig. 4, the triangle 2a will register sections of dark pixels 4a, increasing linearly to the left until the line scan reaches the right-angle tip. With every line of scan the dark pixels will start earlier and will end roughly at the same number of pixels scanned. After that the section of dark pixels 4a will decrease linearly from the left. If such rule applies, the related display module 1 is registered as +90° rotated. The tips of the triangle are stored in a memory.
Similarly, in the right side of Fig. 4, the triangle 2b will register sections of dark pixels 4b, increasing linearly to the left and to the right until the line scan reaches the right-angle triangle hypotenuse. If such rule applies, the related display module 1 is registered as 0° rotated. Similar rules apply when display modules rotated at -90° and 180° are registered (not shown). The nine-digit numbers 3 are visualized in exemplary set-up graphics centered below the hypotenuse of the triangles. After determining orientation of every display module 1 of the plurality of display modules, the raster image processor 7 locates the pixels where every nine-digit number 3 is located on the control image from the camera 9. The images of the nine-digit numbers 3 are cropped from the control image of the camera 9 and fed to OCR (optical character recognition) software which recognizes every digit and all nine-digit numbers 3 as a sequence of decimal digits. The data is recorded in a set-up table as described with respect to Fig. 6 with the corresponding orientation of every display module 1 , its nine-digit number and position on the control image received from the camera 9.
Fig. 5 is a flowchart of the algorithm to analyze the control image received from the camera 9. As result, the set-up table as described in Fig. 6, describing the configuration of the plurality of display modules is filled in. The algorithm uses data extracted from the scan of the control image from the camera 9 and creates a simple (X, Y) coordinate system, where every display module 1 is assigned coordinates, such that the leftmost display modules have their X coordinate as 1 , and the lowermost display modules have their Y coordinate as 1. The maximum value of X is determined as the number of steps from the leftmost display module to the rightmost display module. Similarly, the maximum value for Y is determined as the number of steps from the lowermost display module to the uppermost display module.
Fig. 6 is an exemplary set-up table, derived from the exemplary configuration of display modules 1 shown in Fig. 3 after processing the control image from the camera 9 with the algorithm shown in the flowchart of Fig. 5
Fig. 7 is an example of an input image of six letters to be displayed on the plurality of display modules 1. The input image is received by the raster image processor 7 from an external source. After the input image is received by the raster image processor 7, it is scaled to fit into the total image area of the modular display apparatus and then divided according to the data in the set-up table, so that a separate portion of the image is prepared for every display module 1 in the modular display apparatus. This process involves properly rotating every portion of the image and breaking it down into bits of data to control every pixel on all display modules 1 in the modular display apparatus.
Figures 8a to 8f show how the desired input image from Fig. 7 is divided into 6 parts, one part for every display module 1 of the configuration as shown on Fig. 3. Each of those six parts is rotated according to the record in the set-up table from Fig. 6. Fig. 8a is the image to be sent to the display module with the coordinates (X=1 , Y=1). The display module has address 010331 and is rotated at 180° as recorded in the set-up table (Fig. 6). In other words, it is mounted upside down. Therefore, the image on Fig. 8a is rotated at 180°, to compensate for the way the display module had been rotated when mounted. Then it is sent to address 010331. The display module with coordinates (X=1 , Y=1 ) has exactly address 010331 and it acquires the image from Fig. 8a and visualizes it. As the orientation of the display module is compensated beforehand by the raster image processor 7, the image is visualized correctly as the letter O’.
Similarly, the image from Fig. 8b is rotated at 180° and sent to address 010483; the image from Fig. 8c is sent to address 003063 without rotation; the image from Fig. 8d is sent to address 002172 without rotation; the image from Fig. 8e is rotated at -90° and sent to address 024191 ; the image from Fig. 8f is rotated at +90° and sent to address 024191. As a result, the desired image will appear correctly on the modular display apparatus, regardless of the fact that individual display modules 1 are mounted in arbitrary fashion.
Fig. 9 is a diagram illustrating an example of a configuration of display modules where there is a missing display module 10 in the middle. Such configuration may be used in combination with a real object to be placed right in the middle of the modular display apparatus. Advertising is an exemplary application. Fig. 10 is the set-up table, generated after running the algorithm from Fig. 5 on the control image of the configuration on Fig. 9. The set-up table describes the configuration of display modules with one missing module in the middle, as shown on Fig. 9. The set - up table has a specific record, stating that the spot with coordinates (2,2) is empty.
The disclosed embodiment is not limited to the aforementioned specific to the embodiment, and it is apparent that various modifications can be made by those having ordinary skill in the art to which the disclosure belongs, without departing from the spirit and gist of the disclosure as claimed by the appended claims. Also, it is intended that such modifications are not to be interpreted independently from the technical idea or prospect of the disclosure.

Claims

Claims
1. A modular display apparatus, comprising a. a plurality of display modules (1 ), each with an electronic memory wherein an individual address number (3) is stored, and b. a raster image processor (7), which is connected to all display modules
(1) and configured to send an instruction to all display modules (1) to display its unique address number, their address number (3) and an orientation-sensitive set-up graphic (2), characterized in that c. it comprises a digital camera (9), which is configured to photograph all display modules (1) with their address numbers (3) and set-up graphics
(2) visualized and submit the obtained control image to the raster image processor (7), and d. the raster image processor (7) is configured to analyze the control image and to determine the spatial location, address number, and spatial orientation of every display module (1).
2. A modular display apparatus according to claim 1 , characterized in that the raster image processor (7) is further configured to store information describing all display modules (1 ) of the plurality of display modules with their address number, location and orientation in a set-up table.
3. A modular display apparatus according to claim 1 or 2, characterized in that the raster image processor (7) is configured to receive an input image to be displayed from an external apparatus, crop portions of the input image, and send those cropped portions with proper orientation to a particular display module (1) by addressing it with its address number.
4. A modular display apparatus according to claim 3, characterized in that the plurality of display modules (1) are configured to recognize their address numbers and only then to acquire the following cropped portions of the input image to be visualized and to visualize it.
5. A computer-implemented method to control a modular display apparatus with a plurality of display modules (1), each with an electronic memory wherein an individual address number (3) is stored, by a raster image processor (7), which is connected to all display modules (1), characterized in that a. the raster image processor (7) sends instructions to all display modules (1) to display its unique address number, their address number (3) and an orientation-sensitive set-up graphic (2), b. a digital camera (9) photographs all display modules (1 ) with their address numbers (3) and set-up graphics (2) visualized and submits the obtained control image to the raster image processor (7), and c. the raster image processor (7) analyzes the control image and to determines the spatial location, address number, and spatial orientation of every display module (1 ).
6. A computer-implemented method according to claim 5, characterized in that the raster image processor (7) stores information describing all display modules (1) of the plurality of display modules with their address number, location and orientation in a set-up table.
7. A computer-implemented method according to claim 6, characterized in that the raster image processor (7) receives an input image to be displayed from an external apparatus, crops portions of the input image into as many frames as there are display modules (1), and submits those cropped portions with the proper orientation and location taken from the set-up table to a particular display module (1) by addressing it with its address number, wherein the position of the display modules (1) in the modular display apparatus is the same as the position of the cropped portion in the input image.
8. A computer-implemented method according to claim 7, characterized in that the display modules (1) only acquire and visualize the cropped portions of the input image from the raster image processor (7) when they recognize their respective address number.
9. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 5 to 8.
10. A raster image processor adapted to be connected to a plurality of display modules (1) and further adapted to perform the steps of the method of claims 5 to 8.
PCT/EP2021/061287 2021-04-29 2021-04-29 Modular display apparatus and method for controlling thereof WO2022228683A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2021/061287 WO2022228683A1 (en) 2021-04-29 2021-04-29 Modular display apparatus and method for controlling thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2021/061287 WO2022228683A1 (en) 2021-04-29 2021-04-29 Modular display apparatus and method for controlling thereof

Publications (1)

Publication Number Publication Date
WO2022228683A1 true WO2022228683A1 (en) 2022-11-03

Family

ID=75787075

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/061287 WO2022228683A1 (en) 2021-04-29 2021-04-29 Modular display apparatus and method for controlling thereof

Country Status (1)

Country Link
WO (1) WO2022228683A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140193037A1 (en) * 2013-01-08 2014-07-10 John Fleck Stitzinger Displaying an Image on Multiple Dynamically Located Displays
US20160086582A1 (en) * 2014-09-18 2016-03-24 Aten International Co., Ltd. Automatic installation method for video wall and related system
US20180113664A1 (en) * 2016-01-08 2018-04-26 Boe Technology Group Co., Ltd. Display device, method and device for adjusting information channels thereof
US20200133615A1 (en) * 2018-10-26 2020-04-30 Samsung Electronics Co., Ltd. Electronic device and control method thereof
US20200241828A1 (en) * 2017-07-26 2020-07-30 Barco N.V. Calibration method and system for tiled displays
US20200348898A1 (en) * 2017-11-09 2020-11-05 Samsung Electronics Co., Ltd. Electronic apparatus, display apparatus, and multivision setting method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140193037A1 (en) * 2013-01-08 2014-07-10 John Fleck Stitzinger Displaying an Image on Multiple Dynamically Located Displays
US20160086582A1 (en) * 2014-09-18 2016-03-24 Aten International Co., Ltd. Automatic installation method for video wall and related system
US20180113664A1 (en) * 2016-01-08 2018-04-26 Boe Technology Group Co., Ltd. Display device, method and device for adjusting information channels thereof
US20200241828A1 (en) * 2017-07-26 2020-07-30 Barco N.V. Calibration method and system for tiled displays
US20200348898A1 (en) * 2017-11-09 2020-11-05 Samsung Electronics Co., Ltd. Electronic apparatus, display apparatus, and multivision setting method
US20200133615A1 (en) * 2018-10-26 2020-04-30 Samsung Electronics Co., Ltd. Electronic device and control method thereof

Similar Documents

Publication Publication Date Title
JP6489262B2 (en) Guided image capture user interface
US9852356B2 (en) Image acquisition user interface for linear panoramic image stitching
WO2020211499A1 (en) Self-checkout method and device for commodities
EP4071704A1 (en) System and method for digital markups of custom products
CN103189827A (en) Object display device and object display method
JP2001325069A (en) Device and method for detecting position
JP2008503121A (en) Image sensors and display devices that function in various aspect ratios
WO2021147219A1 (en) Image-based text recognition method and apparatus, electronic device, and storage medium
CN111325104A (en) Text recognition method, device and storage medium
CN106446888A (en) Camera module multi-identifier identification method and camera module multi-identifier identification equipment
JP6635140B2 (en) Realogram to Planogram user interface
US11263785B2 (en) Object detection method, electronic apparatus and object detection system
US20190370988A1 (en) Document imaging using depth sensing camera
WO2022228683A1 (en) Modular display apparatus and method for controlling thereof
JP2017097622A (en) Information processing apparatus, information processing method, and program
JP5618719B2 (en) Image processing apparatus, method and program thereof
CN110524538A (en) Image processing apparatus, robot and robot system
US9158968B2 (en) Apparatus for extracting changed part of image, apparatus for displaying changed part of image, and computer readable medium
JP2005250628A (en) Pattern, device and method for camera calibration
TW202331655A (en) Label integrity self-adaptive detection method and system
CN110996026B (en) OSD display method, device, equipment and storage medium
KR101098641B1 (en) Sub-component based rendering of objects having spatial frequency dominance parallel to the striping direction of the display
CN113269045A (en) Chinese artistic word detection and recognition method under natural scene
CN112150559A (en) Calibration method of image acquisition device, computer equipment and storage medium
CN115843375A (en) Logo labeling method and device, logo detection model updating method and system and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21723189

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE