WO2018228353A1 - Procédé et appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et dispositif terminal - Google Patents

Procédé et appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et dispositif terminal Download PDF

Info

Publication number
WO2018228353A1
WO2018228353A1 PCT/CN2018/090744 CN2018090744W WO2018228353A1 WO 2018228353 A1 WO2018228353 A1 WO 2018228353A1 CN 2018090744 W CN2018090744 W CN 2018090744W WO 2018228353 A1 WO2018228353 A1 WO 2018228353A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
hardware time
time code
initial
camera system
Prior art date
Application number
PCT/CN2018/090744
Other languages
English (en)
Chinese (zh)
Inventor
崔永太
Original Assignee
深圳市瑞立视多媒体科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市瑞立视多媒体科技有限公司 filed Critical 深圳市瑞立视多媒体科技有限公司
Publication of WO2018228353A1 publication Critical patent/WO2018228353A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • H04N5/06—Generation of synchronising signals
    • H04N5/067—Arrangements or circuits at the transmitter end
    • H04N5/073—Arrangements or circuits at the transmitter end for mutually locking plural sources of synchronising signals, e.g. studios or relay stations
    • H04N5/0733—Arrangements or circuits at the transmitter end for mutually locking plural sources of synchronising signals, e.g. studios or relay stations for distributing synchronisation pulses to different TV cameras

Definitions

  • the invention belongs to the technical field of multi-camera systems, and in particular relates to a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system.
  • the multi-camera system is a system based on the principle of computer vision, which combines multiple cameras, light sources, storage devices, etc., and is commonly used in 3D reconstruction, motion capture, multi-view video, and the like.
  • optical motion capture is a technique for capturing motion by monitoring and tracking target feature points from different angles by multiple high-speed cameras based on the principle of computer vision. For any point in space, as long as it is seen by both cameras at the same time, the position of the point in space at this moment can be determined. When the camera continuously shoots at a high enough rate, the image sequence can be obtained. The trajectory of the point, if a plurality of points are marked on one object, and the object is photographed simultaneously by multiple cameras, the motion trajectory of the object can be obtained.
  • the existing method of synchronizing multiple cameras usually uses a synchronization signal to trigger multiple cameras, and transmits a synchronization trigger signal to each camera through a wired or wireless manner, but in reality, network transmission delay, network instability, External environmental factors such as temperature and humidity can cause multiple cameras to capture images without being completely synchronized.
  • the present invention provides a method, a device and a terminal device for controlling synchronous exposure of a multi-camera system, which can improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in a complex scene.
  • a first aspect of the present invention provides a method for controlling simultaneous exposure of a multi-camera system, the control method comprising:
  • a control device for synchronous exposure of a multi-camera system comprising:
  • An acquiring module configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes
  • a determining module configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera value;
  • a sending module configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system
  • the exposure time of all cameras is synchronized.
  • a terminal device includes a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor executing the computer
  • the steps of the method provided by the first aspect above are implemented at the time of the program.
  • a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method provided by the first aspect above.
  • the technical solution provided by the present invention after receiving an instruction of exposure synchronization, acquiring an initial hardware time code of each camera, and determining a synchronization reference value according to the plurality of initial hardware time codes, and according to the synchronization reference value and each An initial hardware time code of the camera, determining a frame compensation value corresponding to each camera in the multi-camera system, and transmitting the frame compensation value to each camera corresponding to the multi-camera system, thereby enabling each camera in the multi-camera system to
  • the determined frame compensation value corresponds to the frame length of the adjusted image, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image, which improves the accuracy of exposure synchronization of multiple cameras in the multi-camera system.
  • FIG. 1 is a schematic flow chart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention
  • FIG. 2 is a schematic flow chart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention
  • FIG. 3 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 4 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 5 is a schematic flow chart of an embodiment of step 202 in FIG. 2;
  • FIG. 6 is a schematic flow chart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • FIG. 7 is a schematic block diagram of a first embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention.
  • FIG. 8 is a schematic block diagram of a second embodiment of a control device for synchronous exposure of a multi-camera system provided by the present invention.
  • FIG. 9 is a schematic block diagram of an embodiment of a terminal device provided by the present invention.
  • Figure 10 is a diagram showing the relationship between the control device and the multi-camera system provided by the present invention.
  • the term “if” can be interpreted as “when” or “on” or “in response to determining” or “in response to detecting” depending on the context. .
  • the phrase “if determined” or “if detected [condition or event described]” may be interpreted in context to mean “once determined” or “in response to determining” or “once detected [condition or event described] ] or “in response to detecting [conditions or events described]”.
  • the hardware time code is the minimum timing unit of the camera sensor input clock (MCLK) crystal oscillation period, that is, every crystal clock cycle
  • MCLK camera sensor input clock
  • the hardware time codes passed by all the cameras to capture one frame of image are the same.
  • the image sensor models of all cameras are different (and other hardware and software configurations are the same)
  • the image resolution of the camera is different at this time, but the hardware time code of all the cameras capturing one frame of image is still the same.
  • the oscillation period of the actual crystal oscillator of the camera is constantly changing due to some factors such as temperature and humidity.
  • the crystal oscillation period of each camera is not necessarily equal, so the time elapsed when each camera captures one frame of images in a multi-camera system is not necessarily the same, which is the root cause of camera exposure unsynchronization in a multi-camera system.
  • FIG. 1 is a schematic flowchart of a first embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 1 may include the following steps:
  • Step S101 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Embodiments of the present invention are for controlling simultaneous exposure photography of multiple cameras in a multi-camera system.
  • the method can be applied to a control device for controlling the simultaneous exposure of each camera in a multi-camera system.
  • the instruction for exposure synchronization may be issued by a camera in a multi-camera system or may be issued by the control device itself.
  • the exposure synchronization command may be a timer inside the camera, and an exposure synchronization command is issued each time the timer count ends.
  • the clock phase will drift for one exposure period (1 ms) in about 30 seconds, so it is necessary to perform one exposure synchronization within 30 seconds, and then it can be set every 20 seconds. Issue an instruction to synchronize the exposure.
  • control device If the instruction for the exposure synchronization is issued by the control device itself, the control device also has a timer internally, which functions the same as the timer described above. As another embodiment, it is also possible to directly set an instruction that the control device itself issues an exposure synchronization every time a predetermined time elapses.
  • the exposure time of each camera in the multi-camera system is theoretically the same.
  • the oscillation period of the actual crystal oscillator is also changing, and is not completely constant, so that the problem of unsynchronized exposure occurs.
  • the hardware time code of each camera in the multi-camera system needs to be acquired, so that the exposure time of each camera of the multi-camera system can be adjusted according to the acquired hardware time code, so that The exposure time of all cameras is synchronized.
  • each camera has a time code register inside, which is used to read and write the hardware time code of the current camera.
  • the hardware timer module starts timing.
  • the exposure start time and exposure duration of each frame of the camera are timed by the hardware time code recorded by the hardware timer.
  • the process of the control device specifically acquiring the initial hardware time code of each camera in the multi-camera system may be: assuming that there are a total of N cameras in the multi-camera system, the control device respectively sends a hardware time code acquisition request to each of the multi-camera systems, Each camera receives the request, reads its own hardware time code through its own time code register and returns it to the control device.
  • the control device receives the hardware time code sent by each camera in the multi-camera system and counts as Initial hardware time code T 1 , T 2 , T 3 ... T N . At this time, initial hardware time codes T 1 , T 2 , T 3 ... T N corresponding to each camera in the multi-camera system are obtained.
  • Step S102 determining a synchronization reference value according to the plurality of initial hardware time codes, and determining a frame compensation value corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera.
  • the synchronization reference value may be determined according to the plurality of initial hardware time codes.
  • any one of the plurality of initial hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of initial hardware time codes may be selected as the synchronization reference value, of course,
  • An average of the plurality of initial hardware time codes is calculated and used as a synchronization reference value.
  • a frame compensation value for each camera is then determined based on the synchronization reference value and the plurality of initial hardware time codes obtained.
  • the frame extension register calculates a corresponding frame compensation value for each camera based on the deviation of the synchronization reference value from the camera's initial hardware time code.
  • Step S103 transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems
  • the camera's exposure time is synchronized.
  • the control device calculates the frame compensation value corresponding to each camera and transmits the frame compensation value to the corresponding camera in the multi-camera system. After each camera in the multi-camera system receives the corresponding frame compensation value, the frame length of the image stored in the frame extension register inside the camera is updated so that each camera in the multi-camera system can align the exposure time when acquiring the image. It should be noted that if the determined synchronization reference value does not deviate from the initial hardware time code of the camera, it means that the camera does not need to perform frame compensation during the current exposure synchronization.
  • the frame length of the image when the frame length of the image is adjusted according to the frame compensation value, the frame length of the current image frame can be adjusted, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is to say, The next frame image is the image to be synchronized.
  • the camera can also adjust the frame length of the next frame image according to the frame compensation value, so that the camera in the multi-camera system can realize the exposure synchronization when acquiring the next frame image, that is, the next frame image is to be synchronized. Image, and so on.
  • the method for synchronously exposing a multi-camera system acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and determines a synchronization reference value according to the plurality of initial hardware time codes, and according to each Deviation between the initial hardware time code of the camera and the synchronization reference value, determining a frame compensation value of the image, and transmitting the determined frame compensation value to a corresponding camera in the multi-camera system so that each camera in the multi-camera system can
  • the frame length of the image is adjusted correspondingly according to the received frame compensation value, so that each camera in the multi-camera system synchronizes the exposure time when acquiring the image.
  • the reason for causing the multiple camera exposure times in the multi-camera system to be out of sync is that there are other factors besides the crystal oscillation period is not equal.
  • the network delay when acquiring the initial time code of each camera in the multi-camera system the operation delay when acquiring the initial time code of each camera, and the hardware cumulative delay value of each camera itself in the multi-camera system. Since the obtained initial hardware time code includes the above delay values, in order to further improve the synchronization accuracy of the multi-camera system, after the initial hardware time code is acquired, before the frame compensation value is determined, the acquired initial hardware time code may also be acquired. Perform delay correction processing.
  • the delay correction processing may include at least one of a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing.
  • the corrected hardware time code is obtained when the synchronization reference value is obtained, and the corrected hardware time code is also used when determining the frame compensation value of each camera.
  • the delay correction processing including the network delay correction processing, the operation delay correction processing, and the hardware cumulative correction processing will be described in detail as an example.
  • FIG. 2 is a schematic flowchart of a second embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 2 may include the following steps:
  • Step S201 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Step S202 performing hardware accumulation correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes in sequence.
  • the hardware timer starts to count after the camera sensor starts to work. After a certain period of operation, the crystal oscillator generates a certain cumulative error. To eliminate this error, it can be used in each frame of the image.
  • the preset position (such as the position of the 100th pixel) is set to trigger the CPU hardware interrupt. When the interrupt is triggered, the hardware timer will automatically latch the time code of the current time. Each time an interrupt is triggered, a time code is latched.
  • step 202 when hardware initial correction processing is performed on a plurality of initial hardware time codes, the operation flow may be performed according to the flow shown in FIG.
  • step 202 when the flowchart of the hardware cumulative correction processing in step 202 is performed, the following steps may be included:
  • Step 301 Corresponding to acquiring an interrupt time code of the last interrupt trigger time of each camera.
  • Step 302 Perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtain a corrected hardware time code of each camera.
  • the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, by taking the difference between the obtained initial hardware time code and the interrupt time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
  • the hardware cumulative correction processing operation for each camera in the multi-camera system can be performed by the control device, or of course, by each camera in the multi-camera system.
  • each camera needs to send it after reading its own initial hardware time code and the interruption time code of its own last interruption time.
  • the control device compares the initial hardware time code of each camera in the multi-camera system with the interrupt time code of each camera, and obtains the corrected hardware time code of each camera after the hardware cumulative correction processing.
  • the hardware cumulative correction processing operation is performed.
  • the corrected hardware time code K i T i - T i0 of the following camera , where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the most recent interruption of camera i, i ranges from 1 to N.
  • network delay correction processing is performed on the initial hardware time code.
  • the camera in the multi-camera system When acquiring the initial hardware time code of the camera, it is necessary to send a request for acquiring a hardware time code to each camera, and the camera in the multi-camera system passes its own time after receiving the acquisition request of the hardware time code sent by the control device.
  • the code register reads its own hardware time code and sends it to the control device.
  • the time delay from the time the control device sends a request to the time the camera in the multi-camera system receives the request is the network delay in the embodiment of the present invention.
  • the operation may be specifically performed according to the flow shown in FIG. 4.
  • FIG. 4 it is a schematic flowchart of an embodiment of performing network delay correction processing on an initial hardware time code, and the specific method may include:
  • Step 401 Acquire a network delay value between each camera in the local and multi-camera system.
  • Step 402 Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera in the multi-camera system.
  • the specific operation method may be, for example:
  • the network delay between each camera in the local and multi-camera systems can be measured by the IEEE 1588 Precision Clock Synchronization Protocol and converted in units of the sensor clock crystal period. Network latency between each camera in the local and multi-camera systems.
  • the hardware time code after the hardware cumulative correction processing should be used in the difference calculation in this step. Where i ranges from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the network delay values between each of the acquired local and multi-camera systems are Y 1 , Y 2 , Y 3 ... Y N , respectively .
  • the operation delay correction processing is performed on the initial hardware time code.
  • FIG. 5 it is a schematic flowchart of an embodiment of performing an operation delay correction process on an initial hardware time code, and includes the following steps:
  • Step 501 When acquiring the initial hardware time code of each camera, reading the local instantaneous hardware time code.
  • Step 502 Perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when the initial hardware time code of the first camera is obtained, to obtain a local and each camera Operation delay value.
  • Step 503 Calculate a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained operation delay value between the local and each camera.
  • step 501 the hardware time code of the control device itself is read once before the initial hardware time code acquisition request is sent to each camera, and is recorded as the instantaneous hardware time code. This also requires that the control device itself also contains a time code register for reading and writing the hardware time code of the current control device. Then, an initial hardware time code acquisition request is sent to each camera in the multi-camera system, and the camera in the multi-camera system reads the hardware time code of the camera and returns the hardware time code of the camera to the control after receiving the request. Device.
  • the operation delay value C i T 1i - T 11 .
  • T 1i is the instantaneous hardware time code local to the control device read before the initial hardware time code of the i-th camera is acquired
  • T 11 is the instantaneous hardware local to the control device read when the initial hardware time code of the first camera is read.
  • the camera hardware time corrected code N i T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N.
  • the corrected hardware time code N i T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
  • the hardware cumulative correction processing and the network delay correction processing have been performed on the initial hardware delay code in step 202, the hardware cumulative correction processing and network extension should be used in the difference calculation in this step.
  • the hardware time code after processing is corrected, that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, when the difference calculation is performed in this step, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the operational delay values obtained between each of the cameras in the multi-camera system are C 1 , C 2 , C 3 ... C N , respectively .
  • the above embodiment adopts three delay correction processings of network delay correction processing, operation delay correction processing, and hardware cumulative correction processing, in practical applications, only one of them may be selected or Two delay correction processing. Which delay correction processing is selected in the process of correcting the hardware time code by performing the delay correction processing on the initial hardware time code, and correspondingly, which delay correction result obtained by the delay correction processing is calculated together with the initial hardware time code Obtaining the modified hardware time code, the embodiment of selecting only one of the delay correction processing in the above three delay correction processing and the embodiment for selecting any two of the delay correction processing are not illustrated herein, and can be referred to The calculation process of the above embodiment is obtained.
  • Step 203 Determine the synchronization reference value according to the plurality of modified hardware time codes, and determine frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and the corrected hardware time code of each camera. value.
  • the synchronization reference value when the synchronization reference value is determined, it is necessary to determine the synchronization reference value based on the corrected plurality of corrected time codes. Specifically, when determining the synchronization reference value, any one of the plurality of modified hardware time codes may be selected as the synchronization reference value, and the maximum value or the minimum value of the plurality of modified hardware time codes may be selected as the synchronization reference value. It is of course also possible to calculate an average of the plurality of modified hardware time codes and use the average as a synchronization reference value.
  • the synchronization reference value needs to be determined according to the hardware time code after the hardware cumulative correction processing, and so on.
  • the hardware cumulative correction processing, the network delay correction processing, and the operation delay correction processing are performed on the initial hardware time code, then the hardware cumulative correction processing, the network delay correction processing, and the operation delay should be performed according to the hardware.
  • the corrected hardware time code of each camera obtained after processing determines the synchronization reference value. After selecting the sync reference value, all cameras are aligned with the camera corresponding to the selected sync reference value.
  • Step S204 sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value so that all the multi-camera systems
  • the camera's exposure time is synchronized.
  • the corrected hardware time code of the camera is greater than the synchronization reference value, the hardware time code of the camera is relatively faster than the synchronization reference value, and the current camera needs to be The frame image or the next frame image is extended for a certain period of time, so that the time code of the camera corresponding to the synchronization reference value can be caught up, so that the exposure time is synchronized when the lower frame or the lower frame image is acquired; if the corrected hardware time code of the camera is smaller than the synchronization
  • the reference value indicates that the time code of the camera is slow, and it is necessary to catch up with the time code of the camera corresponding to the synchronization reference value.
  • the camera's time code can be caught up so that the exposure is synchronized when the next frame or the next frame is captured.
  • the method for synchronously exposing a multi-camera system acquires an initial hardware time code of each camera after receiving an instruction of exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes to obtain a plurality of Correcting a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining a frame compensation corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera And transmitting a frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value so that all of the multi-camera system
  • the camera's exposure time is synchronized.
  • Embodiments of the present invention may apply the method to a control device for controlling simultaneous synchronization of each camera in a multi-camera system, and the method may also be applied to one camera in a multi-camera system, that is, the control may be
  • the device is integrated into one of the cameras in the multi-camera system.
  • One camera in the multi-camera system can be selected as the main camera, the other is named as the slave camera, the control device is integrated into the main camera, or the method is directly applied to the main camera.
  • the acquired control device and the delay value of each camera are replaced by the delay values of the main camera and each camera.
  • the delay value of the main camera and itself can be replaced by the value 0.
  • the main camera is used as a control device to realize synchronous exposure of each camera in the multi-camera system, which can be obtained by the above content, which is not exemplified herein.
  • FIG. 6 is a schematic flowchart of a third embodiment of a method for controlling synchronous exposure of a multi-camera system according to the present invention.
  • the control method for synchronous exposure of the multi-camera system shown in FIG. 6 may include the following steps:
  • Step S601 after receiving the instruction of the exposure synchronization, acquiring an initial hardware time code of each camera in the multi-camera system, to obtain a plurality of initial hardware time codes.
  • Step S602 sequentially performing hardware cumulative correction processing, network delay processing, and operation delay processing on the obtained plurality of initial hardware time codes to obtain a plurality of modified hardware time codes.
  • Step S603 determining a synchronization reference value according to the plurality of modified hardware time codes.
  • Step S605 sending the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system.
  • step S604 When determining the frame compensation value, the formula in step S604 is specifically employed.
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents one of each camera in the multi-camera system.
  • the size of the frame image in units of oscillation periods, and % indicates the modulo operation.
  • the specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code.
  • the modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
  • the determined synchronization reference value may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value.
  • the minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
  • the camera in the multi-camera system can adjust the length of the current frame or the next frame image according to the frame compensation value, so that the camera can realize when acquiring the lower frame or the lower frame image. Exposure sync.
  • the size of the serial number of each step does not mean the order of execution sequence, and the order of execution of each step should be determined by its function and internal logic, and should not constitute any implementation process of the embodiment of the present invention. limited.
  • FIG. 1 to FIG. 6 describe in detail the control method of the synchronous exposure of the multi-camera system.
  • the control decoration of the multi-camera system synchronous exposure, the terminal device and the computer-readable storage medium will be described in detail below with reference to the accompanying drawings. To avoid redundancy, the terms already described above may not be repeatedly described below.
  • FIG. 7 is a structural block diagram of a control apparatus 700 provided by the present invention.
  • the control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant.
  • the synchronous exposure apparatus 700 includes:
  • the obtaining module 701 is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes.
  • a determining module 702 configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine a frame corresponding to each camera in the multi-camera system according to the synchronization reference value and an initial hardware time code of each camera Compensation value
  • the sending module 703 is configured to send the frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value, so that the multi-camera system The exposure time of all cameras is synchronized.
  • FIG. 8 it is a structural block diagram of a control apparatus 700 for simultaneous exposure of a multi-camera system provided by the present invention, and for convenience of explanation, only parts related to the embodiment of the present invention are shown.
  • the control device 700 may be a software unit, a hardware unit or a combination of a soft and a hard unit built in a terminal device (mobile phone, computer, computer), or may be integrated into the terminal device as a separate pendant; or may be built in A software unit, a hardware unit, or a combination of soft and hard units within the main camera may also be integrated into the main camera as a separate pendant.
  • the control device 700 includes an acquisition module 701, a determination module 702, and a transmission module 703.
  • the difference between the embodiment of the present invention and the embodiment shown in FIG. 7 is that the correction module 704 is further included.
  • the correction module 704 is configured to sequentially perform hardware cumulative correction processing, network delay processing, and operation delay processing on the plurality of initial hardware time codes obtained by the obtaining module 701.
  • the correction module 704 may specifically include: an obtaining unit, and a correcting unit. Specifically, when the correction module 704 performs hardware cumulative correction processing on a plurality of initial hardware time codes, the specific operation method may be:
  • the acquiring unit is configured to obtain an interrupt time code of the last interrupt trigger time of each camera, and the correcting unit performs a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and obtains a corrected hardware time of each camera. code.
  • the hardware timer can automatically latch a time code when an interrupt is generated. Therefore, after reading each of its own initial hardware time code, each camera also reads the hardware time code of the most recent interrupt trigger time, which is recorded as the interrupt time code. Then, the read initial hardware time code and the interrupt time code are sent to the acquiring unit, and the acquiring unit transmits the initial hardware time code and the interrupt time code acquired from the camera to the correcting unit process. By correcting the initial hardware time code of the correction unit and the interruption time code, the error generated by the crystal oscillator can be eliminated, and the corrected hardware time code after the hardware cumulative delay processing is obtained.
  • the correction module 704 corrects the initial hardware time code of the camera
  • the initial hardware time code of the camera is T i
  • the acquired interrupt time code of the latest interrupt of the camera is T i0
  • the hardware is accumulated.
  • Correct the corrected hardware time code K i T i - T i0 of the camera after the processing operation, where T i is the initial hardware time code of camera i, and T i0 is the interrupt time code of the last interrupt of camera i, and the value range of i is 1 to N.
  • the specific operation method may be:
  • the obtaining unit acquires a network delay value between each camera in the local and multi-camera system, and the correcting unit associates the plurality of initial hardware time codes with a network delay value between the local and each camera acquired by the acquiring unit The difference operation corresponds to the corrected hardware time code of each camera.
  • the operation method may be: measuring the network delay between each camera and the camera through the IEEE 1588 precision clock synchronization protocol, and performing conversion in units of the sensor clock crystal period, thereby obtaining At this point, the network delay between the control device itself and each camera.
  • the correction unit performs the difference correction after the network delay correction processing is performed on the network delay value between the plurality of initial hardware time codes and the acquired control device and each camera.
  • the hardware time code M i T i - Y i , where T i is the initial hardware time code of camera i, Y i is the network delay value of camera i, and i ranges from 1 to N.
  • the hardware time code after the hardware cumulative correction processing should be used when performing the difference calculation. Values range from 1 to N. If the hardware cumulative correction processing is not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the specific operation method may be:
  • the obtaining unit When acquiring the initial hardware time code of each camera, the obtaining unit reads the instantaneous hardware time code local to the control device, and the correction unit will respectively read the instantaneous hardware time code local to the control device and acquire the first camera.
  • the instantaneous hardware time code local to the control device read at the initial hardware time code is subjected to a difference operation to obtain an operation delay value between the control device and each camera, and according to the plurality of initial hardware time codes and acquired locations.
  • the operation delay value is calculated to obtain the corrected hardware time code of each camera.
  • the acquisition unit Before the initial hardware time code acquisition request is sent to each camera, the acquisition unit first reads the hardware time code local to the control device, which is counted as the instantaneous hardware time code. An acquisition request for the initial hardware time code is then sent to each camera, and each camera receives the request and reads the hardware time code of each camera itself and sends it to the control device.
  • T 1i is the instantaneous hardware time code of the control device itself read before acquiring the initial hardware time code of the i-th camera
  • T 11 is the instantaneous hardware time of the control device itself read when reading the hardware time code of the first camera code.
  • i is in the range of 1 to N, and when i is equal to 1, it represents the operational delay value between the control device and the first camera.
  • the camera hardware time corrected code N i T i - C i , where T i is the initial time code camera hardware i, the camera operation latency value C i, i, i is taken Values range from 1 to N.
  • the corrected hardware time code N i T i - C i of the camera after the operation delay correction processing, wherein i ranges from 1 to N.
  • the hardware cumulative correction processing and the network delay correction processing should be used when performing the difference calculation.
  • the hardware time code that is, M i , where i ranges from 1 to N. If the hardware cumulative correction processing and the network delay correction processing are not performed in the foregoing, then when performing the difference calculation, the initial hardware time code T i should be used, where i ranges from 1 to N.
  • the control device for synchronous exposure of the multi-camera system of the embodiment of the present invention obtains the initial hardware time code of each camera after receiving the instruction of the exposure synchronization, and performs delay correction processing on the plurality of initial hardware time codes. Determining a hardware time code, and determining a synchronization reference value according to the plurality of modified hardware time codes, and determining, corresponding to each camera in the multi-camera system, according to the synchronization reference value and an initial hardware time code of each camera Frame compensation value; finally transmitting the frame compensation value corresponding to each camera to each camera corresponding to the multi-camera system, so that each camera can adjust the length of the image according to the corresponding frame compensation value such that the multi-camera system
  • the exposure time of all cameras is synchronized.
  • the determining module 702 of the embodiment of the present invention specifically includes: a synchronization reference value unit and a frame compensation value determining unit.
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents a frame image of each camera in the multi-camera system.
  • the size in units of oscillation periods, % represents the modulo operation.
  • the specific frame compensation value is the size of the camera one frame image in units of the oscillation period of the crystal oscillator, that is, the size converted into the time code.
  • the modulo operation is performed so that the frame compensation value of the camera does not exceed the frame length of one frame of image (the length of time of one frame of image converted into time code), as long as the phase of the clock is aligned.
  • the synchronization reference value determined by the synchronization reference value determining unit may be a minimum value among the plurality of modified hardware time codes, that is, a minimum value is selected from the plurality of modified hardware time codes as the synchronization reference value.
  • the minimum worth of choosing to correct the hardware timecode is to simplify the calculations and also to get better results. At this time, all cameras only need to lengthen the length of the current frame or the next frame image.
  • each functional module in the embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit or module, and the integrated unit or module may adopt hardware.
  • the form is implemented in the form of a software functional unit.
  • the specific names of the respective functional modules are only for the purpose of distinguishing from each other, and are not intended to limit the scope of protection of the present application.
  • FIG. 9 is a schematic block diagram of a terminal device according to an embodiment of the present invention.
  • the terminal device can also be a camera in a multi-camera system, such as a main camera.
  • the terminal device 9 of this embodiment includes: one or more processors 90, a memory 91, and a computer program 92 stored in the memory 91 and operable on the processor 90.
  • the processor 90 implements the functions of the modules/units in the terminal device embodiment when the computer program 92 is executed, such as the functions of the modules 701 to 703 shown in FIG.
  • the computer program 92 can be partitioned into one or more modules/units that are stored in the memory 91 and executed by the processor 90 to complete this invention.
  • the one or more modules/units may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program 92 in the terminal device 9.
  • the computer program 92 can be partitioned into an acquisition module, a determination module, and a transmission module.
  • the acquiring module is configured to acquire an initial hardware time code of each camera in the multi-camera system after receiving an instruction of exposure synchronization, to obtain a plurality of initial hardware time codes;
  • the determining module is configured to determine a synchronization reference value according to the plurality of initial hardware time codes, and determine, according to the synchronization reference value and an initial hardware time code of each camera, each camera in the multi-camera system Frame compensation value;
  • the sending module is configured to send a frame compensation value corresponding to each camera to each camera in the multi-camera system, so that each camera adjusts the length of the image according to the corresponding frame compensation value to make the multi-camera
  • the exposure time of all cameras in the system is synchronized.
  • it may also include:
  • a correction module configured to perform delay correction processing on the plurality of initial hardware time codes to obtain a plurality of modified hardware time codes
  • the delay correction processing includes at least one of the following: a network delay correction processing, an operation delay correction processing, and a hardware cumulative correction processing;
  • the determining module is specifically configured to:
  • the modification module includes:
  • An obtaining unit configured to acquire a network delay value between the local and each camera in the multi-camera system
  • a correction unit configured to obtain a corrected hardware time code of each camera according to the plurality of initial hardware time codes and the obtained network delay value between the local and each camera.
  • the acquiring unit is further configured to: when acquiring an initial hardware time code of each camera, read a local instantaneous hardware time code;
  • the correcting unit is further configured to perform a difference operation between the local instantaneous hardware time code read each time and the local instantaneous hardware time code read when acquiring the initial hardware time code of the first camera, to obtain local and per Operation delay value between cameras;
  • a corrected hardware time code for each camera is obtained based on the plurality of initial hardware time codes and the resulting operational delay value between the local and each camera.
  • the acquiring unit is further configured to: correspondingly acquire an interrupt time code of a last interrupt trigger time of each camera;
  • the modifying unit is further configured to perform a difference operation between the plurality of initial hardware time codes and the acquired interrupt time code, and correspondingly obtain a corrected hardware time code of each camera.
  • the determining module is specifically configured to:
  • B i represents the frame compensation value of the camera i
  • N i represents the corrected hardware time code of the camera i after the delay correction processing
  • S represents the synchronization reference value
  • FrameLength represents a frame image of each camera in the multi-camera system.
  • the size in units of oscillation periods, % represents the modulo operation.
  • the terminal device includes, but is not limited to, a processor 90 and a memory 91. It will be understood by those skilled in the art that FIG. 9 is only an example of the terminal device 9, and does not constitute a limitation on the terminal device 9, which may include more or less components than those illustrated, or combine some components, or Different components, such as the terminal device, may also include input devices, output devices, network access devices, buses, and the like.
  • the memory 91 is configured to store software programs, modules, units, and data information required in the terminal device, and the processor 90 executes various functional applications by running software programs, modules, and units stored in the memory 91. And data processing to improve the accuracy of simultaneous exposure of multiple cameras in a multi-camera system in complex scenes.
  • the memory 91 can include read only memory and random access memory and provides instructions and data to the processor 90.
  • a portion of the memory 91 may also include a non-volatile random access memory.
  • the memory 91 can also store information of the device type.
  • the processor 90 can be a central processing unit (Central) Processing Unit (CPU), which can also be other general-purpose processors, digital signal processors (DSPs), and application specific integrated circuits (Application). Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the processor is a control center of the terminal device, and connects various parts of the entire terminal device by using various interfaces and lines.
  • the processor 90 and the memory 91 described in the embodiments of the present invention may implement the implementation manner described in the embodiment of the method for controlling the synchronous exposure of the multi-camera system provided by the embodiment of the present invention, and may also execute the multi-camera system.
  • the implementation described in the embodiment of the synchronous exposure control device will not be described herein.
  • control device 7 is used to control a multi-camera system, and the multi-camera system includes a plurality of cameras 101.
  • the control device 7 can represent the control device described in any of the above control devices, and can also represent any of the above embodiments.
  • an embodiment of the present invention further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the application provided by the embodiment of the present invention is implemented.
  • the steps of the control method for simultaneous exposure of a multi-camera system are not limited to:
  • control method control device, and terminal device may be implemented in other manners.
  • the embodiments described above are merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the medium includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), and a random access memory (RAM, Random Access).
  • ROM read-only memory
  • RAM random access memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un procédé et un appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et un dispositif terminal associé, le procédé de commande comprenant : après réception d'une instruction d'exposition synchrone, acquisition de codes de temps matériels initiaux des appareils de prise de vues dans le système à appareils de prise de vues multiples de façon à obtenir une pluralité de codes de temps matériels initiaux ; détermination d'une valeur de référence synchrone selon la pluralité de codes de temps matériels initiaux ; détermination d'une valeur de compensation de trame correspondant à chaque appareil de prise de vues dans le système à appareils de prise de vues multiples en fonction de la valeur de référence synchrone et du code de temps matériel initial de chaque appareil de prise de vues ; et envoi de la valeur de compensation de trame correspondant à chaque appareil de prise de vues à chaque appareil de prise de vues correspondant dans le système à appareils de prise de vues multiples, de telle sorte que chaque appareil de prise de vues règle la longueur d'une image en fonction de la valeur de compensation de trame correspondante de celle-ci, de sorte que le temps d'exposition de tous les appareils de prise de vues dans le système à appareils de prise de vues multiples soit synchrone. L'utilisation de la solution décrite peut améliorer la précision d'exposition synchrone d'une pluralité d'appareils de prise de vues dans le système à appareils de prise de vues multiples dans une scène complexe.
PCT/CN2018/090744 2017-06-12 2018-06-12 Procédé et appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et dispositif terminal WO2018228353A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710439586.5 2017-06-12
CN201710439586.5A CN107277385B (zh) 2017-06-12 2017-06-12 一种多相机系统同步曝光的控制方法、装置及终端设备

Publications (1)

Publication Number Publication Date
WO2018228353A1 true WO2018228353A1 (fr) 2018-12-20

Family

ID=60066084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/090744 WO2018228353A1 (fr) 2017-06-12 2018-06-12 Procédé et appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et dispositif terminal

Country Status (2)

Country Link
CN (1) CN107277385B (fr)
WO (1) WO2018228353A1 (fr)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107277385B (zh) * 2017-06-12 2020-04-17 深圳市瑞立视多媒体科技有限公司 一种多相机系统同步曝光的控制方法、装置及终端设备
CN107911575B (zh) * 2017-11-02 2018-10-26 中国矿业大学 一种高精度低功耗多相机同步触发系统
CN110740251B (zh) * 2018-07-20 2021-09-24 杭州海康机器人技术有限公司 一种多相机同步取流方法、装置及系统
CN111343401B (zh) * 2018-12-18 2021-06-01 华为技术有限公司 帧同步方法及其装置
CN109743501A (zh) * 2019-01-15 2019-05-10 上海曼恒数字技术股份有限公司 一种多相机同步触发方法、装置、设备及存储介质
CN109842737B (zh) * 2019-02-01 2021-04-09 初速度（苏州）科技有限公司 一种图像曝光方法及装置、车载终端
WO2020181494A1 (fr) * 2019-03-12 2020-09-17 深圳市大疆创新科技有限公司 Procédé de synchronisation de paramètres, appareil de capture d'images et plateforme mobile
CN110186957B (zh) * 2019-05-30 2021-08-31 哈尔滨工业大学 一种同步信息采集处理系统及方法
CN110290287B (zh) * 2019-06-27 2022-04-12 上海玄彩美科网络科技有限公司 多摄像头帧同步方法
CN113096128B (zh) * 2021-06-08 2021-09-14 上海启迪睿视智能科技有限公司 一种基于软件的确保相机图片对齐的方法
CN114371722B (zh) * 2021-12-03 2024-05-10 深圳供电局有限公司 数据采集方法、装置、无人机和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103108125A (zh) * 2013-01-07 2013-05-15 华中科技大学 一种多相机系统的取像同步控制设备及其方法
JP2013098877A (ja) * 2011-11-02 2013-05-20 Sharp Corp ステレオ撮像装置、同期撮像方法および電子情報機器
CN103792841A (zh) * 2014-01-23 2014-05-14 中国科学院长春光学精密机械与物理研究所 一种空间相机图像对时信息生成系统
CN107277385A (zh) * 2017-06-12 2017-10-20 深圳市瑞立视多媒体科技有限公司 一种多相机系统同步曝光的控制方法、装置及终端设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101959022B (zh) * 2009-07-17 2012-11-28 深圳泰山在线科技有限公司 一种同步电路和摄像装置及其同步曝光控制方法
JP6241467B2 (ja) * 2015-09-11 2017-12-06 カシオ計算機株式会社 撮像装置、撮像制御装置、撮像方法、撮像制御方法及びプログラム
CN105680969A (zh) * 2015-12-31 2016-06-15 浙江中控技术股份有限公司 一种时钟同步方法及装置
CN105681632B (zh) * 2015-12-31 2019-06-28 深圳市华途数字技术有限公司 多目摄像机及其帧同步的方法
CN105680975B (zh) * 2016-03-07 2018-06-15 浙江大学 一种主从结构多节点网络的时间同步方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013098877A (ja) * 2011-11-02 2013-05-20 Sharp Corp ステレオ撮像装置、同期撮像方法および電子情報機器
CN103108125A (zh) * 2013-01-07 2013-05-15 华中科技大学 一种多相机系统的取像同步控制设备及其方法
CN103792841A (zh) * 2014-01-23 2014-05-14 中国科学院长春光学精密机械与物理研究所 一种空间相机图像对时信息生成系统
CN107277385A (zh) * 2017-06-12 2017-10-20 深圳市瑞立视多媒体科技有限公司 一种多相机系统同步曝光的控制方法、装置及终端设备

Also Published As

Publication number Publication date
CN107277385A (zh) 2017-10-20
CN107277385B (zh) 2020-04-17

Similar Documents

Publication Publication Date Title
WO2018228353A1 (fr) Procédé et appareil de commande pour l'exposition synchrone d'un système à appareils de prise de vues multiples, et dispositif terminal
WO2018228352A1 (fr) Procédé et appareil d'exposition synchrone, et dispositif terminal
CN109104259B (zh) 一种多传感器对时同步系统和方法
CN107439000B (zh) 一种同步曝光的方法、装置及终端设备
KR101389789B1 (ko) 촬상 장치, 촬상 시스템, 촬상 방법, 및 컴퓨터 판독 가능한 기록 매체
JP5817664B2 (ja) 撮像装置、撮像方法及びプログラム
US20160088209A1 (en) Synchronous photographing system that controls synchronous photographing by pluraltiy of image capture apparatus
JP6075357B2 (ja) 撮像装置、撮像制御方法及びプログラム
Ansari et al. Wireless software synchronization of multiple distributed cameras
JP2019087791A (ja) 情報処理装置、情報処理方法、及び、プログラム
CN109391799B (zh) 一种监控视频同步方法、装置及视频采集设备
US20160088210A1 (en) Photographing control apparatus that controls synchronous photographing by plurality of image capture apparatus
WO2020125643A1 (fr) Procédé et appareil de synchronisation de trames
WO2024060763A1 (fr) Dispositif portable intelligent sans fil et procédé d'acquisition d'image associé
US20230412918A1 (en) System for wireless synchronized capturing, and smart device
CN107455006B (zh) 一种同步曝光的方法、装置及终端设备
JP5802727B2 (ja) 同期式カメラ
WO2019183759A1 (fr) Procédé de fusion d'image, appareil de capture d'image et système de plateforme mobile
JP2022096504A (ja) 通信装置、通信装置の制御方法およびプログラム
JP2005253056A (ja) 撮像装置
CN117336419A (zh) 拍摄方法、装置、电子设备及可读存储介质
TWM634644U (zh) 影格對齊系統
WO2019010626A1 (fr) Procédé de capture et terminal
JP2023078537A (ja) 通信装置
KR20210117635A (ko) 카메라 동기화 장치 및 그 동기화 처리 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18818182

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18818182

Country of ref document: EP

Kind code of ref document: A1