WO2004098204A1 - Procede et appareil de photographie stereoscopique - Google Patents

Procede et appareil de photographie stereoscopique Download PDF

Info

Publication number
WO2004098204A1
WO2004098204A1 PCT/JP2003/005617 JP0305617W WO2004098204A1 WO 2004098204 A1 WO2004098204 A1 WO 2004098204A1 JP 0305617 W JP0305617 W JP 0305617W WO 2004098204 A1 WO2004098204 A1 WO 2004098204A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
scanning line
imaging
stereoscopic
scanning
Prior art date
Application number
PCT/JP2003/005617
Other languages
English (en)
Japanese (ja)
Inventor
Seijiro Tomita
Original Assignee
Seijiro Tomita
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Seijiro Tomita filed Critical Seijiro Tomita
Priority to AU2003231376A priority Critical patent/AU2003231376A1/en
Priority to JP2004571328A priority patent/JPWO2004098204A1/ja
Priority to PCT/JP2003/005617 priority patent/WO2004098204A1/fr
Publication of WO2004098204A1 publication Critical patent/WO2004098204A1/fr

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals

Definitions

  • the present invention relates to a three-dimensional image capturing method and apparatus, and particularly to a three-dimensional image capturing method that can easily output two images that can be correctly recognized in three dimensions from an image capturing device when capturing a three-dimensional image with two image capturing devices. And equipment.
  • the two cameras are fixed to the base as a single unit, so they are fixed to the horizontal base as a single unit.However, if the base for fixing the two cameras is tilted, people can view this image. It has a great sense of discomfort when viewed with both eyes.
  • Fig. 8 let us consider a case where images are taken by two cameras 1L and 1R.
  • a mountain B is at a long distance (for example, several km away).
  • both cameras 1L and 1R are slightly inclined, as shown in Fig.
  • the displacement of person A on the images 41L and 41R can be reduced to several tens of lines.
  • the image of the distant mountain B is enlarged and the difference ⁇ becomes several hundred lines.
  • the viewer can recognize the images AL and AR of the person as the same thing in their heads and see them stereoscopically, but see two images BL and BR of the same image for mountain B. It cannot be recognized as an image and will be perceived as a double image.
  • Fig. 7 when an image of only the near view without a distant view is taken, a difference of several scanning lines occurs in the image of the person of interest, AL and AR, as shown in Fig. 5 in the image.
  • the observer who is paying attention to this person A is greatly conscious and becomes an obstacle to comfortable stereoscopic vision.
  • an object of the present invention is to provide a stereoscopic image photographing method and apparatus capable of easily measuring the inclination of a camera in order to align the horizontal base axes of two cameras at a scanning line level.
  • the present invention described in claim 1 includes a three-dimensional image capturing apparatus that includes at least two image capturing apparatuses that perform main scanning in the horizontal direction and sub-scans in the vertical direction, and that captures the same object with these image capturing apparatuses.
  • a stereoscopic video photographing method comprising comparing video information of scanning lines defined by video captured by two imaging devices, and detecting the positions of the two imaging devices based on the comparison result.
  • You. ADVANTAGE OF THE INVENTION According to this invention, the inclination of two imaging devices can be measured accurately and quickly by electronically processing the data of the scanning line obtained from an imaging device, without using a mechanical device such as a level.
  • a scanning line to be compared is selected from a central portion of the image.
  • ADVANTAGE OF THE INVENTION According to this invention, the inclination of an imaging device can be measured accurately based on the scanning line of the location with the largest amount of information.
  • an absolute value of a difference between scanning line information is calculated, and an inclination of the two cameras is measured. Things. According to the present invention, a simple operation And an accurate inclination can be obtained.
  • the stereoscopic image capturing method according to the first aspect, wherein an added value of a plurality of scanning lines is used as video information of a scanning line of each imaging unit. is there.
  • accurate measurement can be performed by canceling noise generated in an image.
  • a target portion to be compared is extracted by performing image processing on scanning line information of each imaging unit. Things.
  • the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.
  • the stereoscopic video imaging method according to any one of the first to fifth aspects, wherein the imaging is performed using an imaging device that captures an area larger than a display range, and the image to be output The tilt is corrected by outputting the range of the image as an image captured in the specific range of the image sensor.
  • the inclination can be corrected electronically one operation line at a time precisely and quickly.
  • the present invention described in claim 7 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and a stereoscopic image obtained by photographing the same object with these imaging devices.
  • the tilt angle measuring device in the stereoscopic video photographing device that measures the inclination of the plurality of image capturing devices extracts a predetermined scanning line from the images photographed by the two photographing means.
  • the data of the scanning line obtained from a camera can be processed electronically, without using a mechanical device, such as a level, and can measure the inclination angle of an imaging device accurately and quickly.
  • the scanning line extracting means selects a scanning line from a central portion of the image. It is characterized by the following. According to the present invention, it is possible to accurately measure the inclination of two cameras based on the scanning line at the location having the largest amount of information.
  • the present invention described in claim 9 is the stereoscopic image photographing device according to claim 7, wherein the scanning line comparing means calculates an absolute value of the difference between the scanning line information, and the inclination angle calculating means calculates the difference of the difference.
  • the feature is to measure the tilt angle of two cameras on the basis of. According to the present invention, an accurate inclination value can be obtained by a simple calculation process.
  • the scanning line extracting unit extracts a plurality of scanning lines
  • the scanning line comparing unit extracts the sum value. It is characterized in that it is compared as image information of scanning lines of each photographing means. According to the present invention, accurate measurement can be performed by canceling out noise generated in an image.
  • the scanning line extracting means performs image processing on the scanning line information of each imaging means. It is characterized by selecting a target location to be compared by using. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.
  • the stereoscopic image photographing device according to any one of the seventh to eleventh aspects, wherein each imaging unit captures an area larger than a display range.
  • an output image correction means for correcting an image taken in a specific range of the image sensor to be an output image.
  • amendment of inclination can be performed electronically precisely and promptly one operation line at a time.
  • FIG. 1 is a block diagram showing a stereoscopic video photographing apparatus according to the present invention.
  • FIG. 2 is a flowchart showing the stereoscopic image capturing method shown in FIG. is there.
  • FIG. 3 is a diagram illustrating an example of a rough used for determining a horizontal direction in the stereoscopic video imaging method illustrated in FIG.
  • FIG. 4 is a diagram illustrating an image captured by the stereoscopic video capturing device.
  • FIG. 5 is a diagram showing a correction state of a captured image of the stereoscopic video imaging device shown in FIG.
  • FIG. 6 is a schematic diagram showing an image when two cameras are kept horizontal.
  • FIG. 7 is a schematic diagram showing an image when two cameras are tilted from horizontal by an angle.
  • FIG. 8 is a diagram showing an arrangement relationship of a stereoscopic image photographing device to which the present invention is applied.
  • FIG. 1 is a block diagram showing a stereoscopic video photographing apparatus according to the present invention
  • FIG. 2 is a flowchart showing a stereoscopic video photographing method shown in FIG. 1
  • FIG. 3 is a horizontal direction of the stereoscopic video photographing apparatus shown in FIG.
  • Fig. 4 is a diagram showing an example of a graph used to determine the image
  • Fig. 4 is a diagram showing an image including a distant view and a near view taken by the stereoscopic image photographing device
  • Fig. 5 is a photographing including a distant view and a near view of the stereoscopic image photographing device shown in Fig. Diagram showing the image correction state
  • Fig. 1 is a block diagram showing a stereoscopic video photographing apparatus according to the present invention
  • FIG. 2 is a flowchart showing a stereoscopic video photographing method shown in FIG. 1
  • FIG. 3 is a horizontal direction of the stereoscopic video photographing apparatus shown in FIG.
  • FIG. 6 is a schematic diagram showing the image when the two cameras are kept horizontal, and Fig. 7 shows the image when the two cameras are tilted from horizontal by an angle.
  • FIG. 8 is a schematic diagram showing a positional relationship of a stereoscopic image photographing apparatus to which the present invention is applied.
  • the tilt angle measuring device in the stereoscopic image photographing device is composed of scanning line extracting means 2 connected to left and right cameras 1 L and 1 R as photographing means, and scanning line comparing means 3. And inclination angle calculation means 4 and output image correction means.
  • the cameras 1L and 1R are fixed to the base, shoot the subject, output video with parallax, and output video data for displaying stereoscopic video (Fig. 4, Fig. 2, S1).
  • the cameras 1L and 1R are horizontally on the same horizontal base axis, correct left and right images without vertical displacement are obtained, and the subject A and the mountain B are also at the same horizontal position.
  • the positions of the subjects A and B are shifted vertically in the left and right images.
  • the shift amount ⁇ between the left and right images AL and AR of the person A in the foreground is small (for example, several tens of lines), and the shift amount BL and BR for the left and right images of the distant mountain ⁇ is large (for example, several hundred lines). .
  • the scanning line extracting means 2 extracts the image extraction ranges 41 L and 41 R (for example, scanning line numbers 1 to N) (S 2). This is, for example, about 1/3 of the vertical dimension at the top of the image. If there is parallax between the images of both cameras 1L and 1R and there is a shift in the left and right direction, determine the extraction range by detecting the subject.
  • the scanning line comparison means 3 sequentially calculates the absolute value of the difference between the luminance data (
  • a n -Bm I (m l to N) (S 3).
  • the absolute value of the difference between the luminance signals is represented by a curve, and the minimum value is i An — B pi It can be seen that it can be obtained. It can be seen that the n-th scanning line of the left image 41 L coincides with the p-th scanning line of the right image 41 R. Note that this minimum value is not always 0 due to the effects of noise and the like.
  • the tilt angle calculating means 4 can calculate the angle 0 from the distance between the cameras 1L and 1R and the dimensions of the scanning line (S4). Then, the output image correcting means 5 corrects the output image (S5). In this correction, if a correction corresponding to is performed, a correction for this is performed (S4).
  • This correction is larger than the actual output image 52 L, 52 R (for example, 640 horizontal pixels X 480 vertical pixels) as the image pickup devices 51 L and 51 R of the photographing device ( For example, by using 100 pixels horizontally and 600 pixels vertically, for example, the right output image 52R is moved or rotated up, down, left, and right.
  • the right output image May be moved.
  • the camera tilt can be measured accurately and quickly.
  • the horizontal position of the camera can be adjusted based on this value.
  • the scanning line extracting means extracts a plurality of scanning lines, for example, three lines, and the scanning line comparing means can use the added value as video information of the scanning lines of each camera. In this way, accurate measurements can be made by canceling out noise in the video.
  • the brightness is extracted using the same number of scanning lines, the camera brightness is changed while comparing the two brightness data, and at the point where the brightness difference becomes minimum, the force camera matches the horizontal reference. It can be.
  • the cameras 1L and 1R are fixed to the base, photograph the subject A as a subject, output an image with a parallax amount, and output image data for displaying a stereoscopic image (FIG. 2). , S 1).
  • the scanning line extracting means 2 extracts the image extraction ranges 11 L and 11 R (for example, scanning line numbers 1 to N) (S 2). This is, for example, about one-third the size of the top, bottom, left and right in the center of the video. If there is a parallax between the images of both cameras 1L and 1R and there is a deviation in the left and right direction, determine the extraction range by detecting the subject.
  • the image extraction ranges 11 L and 11 R for example, scanning line numbers 1 to N
  • the scanning line comparison unit 3 focuses on the scanning line 12 L (the n-th scanning line from the top) in the scanning range 11 L, and calculates the luminance data of the scanning line. Are sequentially acquired in the horizontal direction and stored (these luminance data are referred to as An).
  • the scanning line comparison means 3 sequentially calculates the absolute value of the difference between the luminance data (
  • the absolute value of the difference It is shown by the line, and it can be seen that the minimum value is obtained by IA n-B p I. Also, it can be seen that the n-th scanning line of the left image 11L and the p-th scanning line of the right image 11R coincide. Note that this minimum value is not always 0 due to the influence of noise and the like.
  • the tilt angle calculating means 4 calculates the angle 0 from the distance between the cameras 1 and 1R and the size of the scanning line. If the correction for the parallax has been performed, the correction for this is performed (S4).
  • the camera tilt can be measured accurately and quickly.
  • the horizontal position of the camera can be adjusted based on this value.
  • the scanning line extracting means extracts a plurality of, for example, three rows of scanning lines, and the scanning line comparing means 3 can use the added value as image information of the scanning line of each force lens. In this way, accurate measurements can be made by canceling the noise generated in the video.
  • the luminance is extracted using the same number of scanning lines, and the two luminance data are compared, and the inclination of the camera is changed. At the point where the luminance difference is minimized, it is determined that the camera matches the horizontal reference. can do.
  • a range in which no tilt occurs can be output by using a tilt driving device for the camera or using an image sensor larger than the actual image capturing range as in the above-described example.
  • the present invention described in claim 1 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and captures the same object using these imaging devices.
  • a three-dimensional video imaging method comprising: comparing video information of a predetermined scanning line of video captured by two imaging devices; and detecting a position of the two imaging devices based on the comparison result. is there.
  • scanning line data obtained from an imaging device can be processed electronically accurately and quickly without using a mechanical device such as a level. The inclination of two imaging devices can be measured.
  • a scanning line to be compared is selected from a central portion of the image.
  • ADVANTAGE OF THE INVENTION According to this invention, the inclination of an imaging device can be measured accurately based on the trajectory of the location with the largest amount of information.
  • an absolute value of a difference between scanning line information is calculated, and a tilt of two cameras is measured. It is. According to the present invention, an accurate inclination can be obtained by a simple calculation.
  • the stereoscopic image photographing method according to the first aspect, wherein an added value of the plurality of scanning lines is used as image information of a scanning line of each imaging unit. It is. According to the present invention, accurate measurement can be performed by canceling noise generated in an image.
  • a target portion to be compared is extracted by performing image processing on scanning line information of each imaging unit. Things. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.
  • the imaging is performed using an imaging element that captures a range larger than a display range.
  • the tilt is corrected by outputting a range of an image to be output as an image captured in a specific range of the image sensor.
  • amendment of inclination can be performed electronically one operation line precisely and promptly.
  • the present invention described in claim 7 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and a stereoscopic image obtained by photographing the same object with these imaging devices.
  • a predetermined scanning line is extracted from the video captured by the two capturing units.
  • Stereoscopic image photographing comprising: scanning line extracting means; comparing means for comparing image information of the extracted scanning lines; and inclination angle calculating means for measuring the inclination of the photographing device based on the comparison result.
  • the inclination angle of an imaging device can be measured accurately and quickly by electronically processing the data of the scanning line obtained from a camera, without using a mechanical device, such as a level.
  • the present invention described in claim 8 is characterized in that, in the stereoscopic image photographing device according to claim 7, the scanning line extracting means selects a scanning line from a central portion of the image. According to the present invention, it is possible to accurately measure the inclination of two cameras based on the scanning line at the location having the largest amount of information.
  • the present invention described in claim 9 is the stereoscopic image photographing device according to claim 7, wherein the scanning line comparing means calculates an absolute value of the difference between the scanning line information, and the inclination angle calculating means calculates the difference of the difference.
  • the feature is to measure the tilt angle of two cameras on the basis of. According to the present invention, an accurate inclination value can be obtained by a simple calculation process.
  • the scanning line extracting unit extracts a plurality of scanning lines
  • the scanning line comparing unit calculates the added value.
  • the comparison is made as image information of the scanning lines of the photographing means. According to the present invention, accurate measurement can be performed by canceling noise generated in an image.
  • the scanning line extracting means compares the scanning line information of each imaging means by performing image processing. It is characterized by selecting the target place to be. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.
  • each imaging unit captures an area larger than a display range.
  • the range of the output image is An output image correcting means for correcting an image captured in a specific range of the image sensor to be an output image is provided. According to the present invention, inclination correction can be performed electronically precisely and quickly for each operation line.

Abstract

L'invention concerne un procédé et un appareil de photographie stéréoscopique comprenant au moins deux unités d'imagerie qui effectuent un balayage principal dans la direction horizontale ainsi qu'un sous-balayage en direction verticale et qui photographient un objet identique, une image étant générée lorsque les axes horizontaux de ces deux unités d'imagerie sont alignés au niveau de la ligne de balayage. Des informations vidéo relatives à une ligne de balayage prédéterminée sont comparées sur deux vidéos prises par les unités d'imagerie, et l'image produite par ces unités d'imagerie est corrigée, en fonction des résultats de la comparaison, de façon à être adaptée à une visualisation stéréoscopique.
PCT/JP2003/005617 2003-05-02 2003-05-02 Procede et appareil de photographie stereoscopique WO2004098204A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU2003231376A AU2003231376A1 (en) 2003-05-02 2003-05-02 Stereoscopic photographing method and apparatus
JP2004571328A JPWO2004098204A1 (ja) 2003-05-02 2003-05-02 立体映像撮影方法および装置
PCT/JP2003/005617 WO2004098204A1 (fr) 2003-05-02 2003-05-02 Procede et appareil de photographie stereoscopique

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2003/005617 WO2004098204A1 (fr) 2003-05-02 2003-05-02 Procede et appareil de photographie stereoscopique

Publications (1)

Publication Number Publication Date
WO2004098204A1 true WO2004098204A1 (fr) 2004-11-11

Family

ID=33398160

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2003/005617 WO2004098204A1 (fr) 2003-05-02 2003-05-02 Procede et appareil de photographie stereoscopique

Country Status (3)

Country Link
JP (1) JPWO2004098204A1 (fr)
AU (1) AU2003231376A1 (fr)
WO (1) WO2004098204A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009109090A1 (fr) * 2008-03-03 2009-09-11 华为技术有限公司 Procédé et système de correction d'image
JP2010128575A (ja) * 2008-11-25 2010-06-10 Nec System Technologies Ltd ステレオ画像処理装置、ステレオ画像処理方法、および、プログラム
JP2011095431A (ja) * 2009-10-28 2011-05-12 Victor Co Of Japan Ltd 立体映像撮像装置および立体映像撮像方法
JP2012014019A (ja) * 2010-07-01 2012-01-19 Canon Inc 撮像装置及びその制御方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1127703A (ja) * 1997-06-30 1999-01-29 Canon Inc 表示装置及びその制御方法
JP2001128195A (ja) * 1999-10-29 2001-05-11 Atr Ningen Joho Tsushin Kenkyusho:Kk 立体画像補正装置、立体画像表示装置および立体画像補正処理プログラムを記録した記録媒体
JP2002077947A (ja) * 2000-09-05 2002-03-15 Sanyo Electric Co Ltd 立体画像補正方法及びこれを用いた立体画像装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1127703A (ja) * 1997-06-30 1999-01-29 Canon Inc 表示装置及びその制御方法
JP2001128195A (ja) * 1999-10-29 2001-05-11 Atr Ningen Joho Tsushin Kenkyusho:Kk 立体画像補正装置、立体画像表示装置および立体画像補正処理プログラムを記録した記録媒体
JP2002077947A (ja) * 2000-09-05 2002-03-15 Sanyo Electric Co Ltd 立体画像補正方法及びこれを用いた立体画像装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009109090A1 (fr) * 2008-03-03 2009-09-11 华为技术有限公司 Procédé et système de correction d'image
US8849010B2 (en) 2008-03-03 2014-09-30 Huawei Technologies Co., Ltd. Method and system for rectifying images
JP2010128575A (ja) * 2008-11-25 2010-06-10 Nec System Technologies Ltd ステレオ画像処理装置、ステレオ画像処理方法、および、プログラム
JP2011095431A (ja) * 2009-10-28 2011-05-12 Victor Co Of Japan Ltd 立体映像撮像装置および立体映像撮像方法
JP2012014019A (ja) * 2010-07-01 2012-01-19 Canon Inc 撮像装置及びその制御方法

Also Published As

Publication number Publication date
AU2003231376A1 (en) 2004-11-23
JPWO2004098204A1 (ja) 2006-07-13

Similar Documents

Publication Publication Date Title
US11223820B2 (en) Augmented reality displays with active alignment and corresponding methods
US10869024B2 (en) Augmented reality displays with active alignment and corresponding methods
JP5715735B2 (ja) 3次元測定方法、装置、及びシステム、並びに画像処理装置
JP5858433B2 (ja) 注視点検出方法及び注視点検出装置
US20120275667A1 (en) Calibration for stereoscopic capture system
US20130188834A1 (en) Gaze point detection method and gaze point detection device
US20160295194A1 (en) Stereoscopic vision system generatng stereoscopic images with a monoscopic endoscope and an external adapter lens and method using the same to generate stereoscopic images
US20010043739A1 (en) Image forming method and apparatus
US20140293007A1 (en) Method and image acquisition system for rendering stereoscopic images from monoscopic images
WO2011125937A1 (fr) Dispositif de sélection de données d'étalonnage, procédé de sélection, programme de sélection et dispositif de mesure de position tridimensionnelle
JP2011160421A (ja) 立体画像の作成方法、作成装置及びプログラム
JP2002232913A (ja) 複眼カメラ及び立体視画像観察システム
JP4193342B2 (ja) 3次元データ生成装置
JP2011141381A (ja) 立体画像表示装置及び立体画像表示方法
US10142544B1 (en) Real time wide angle video camera system with distortion correction
US10140687B1 (en) Real time wide angle video camera system with distortion correction
CN111292380B (zh) 图像处理方法及装置
WO2004098204A1 (fr) Procede et appareil de photographie stereoscopique
JP2012227653A (ja) 撮像装置及び撮像方法
JP2013074473A (ja) パノラマ撮像装置
KR20110025083A (ko) 입체 영상 시스템에서 입체 영상 디스플레이 장치 및 방법
WO2012014695A1 (fr) Dispositif d'imagerie en trois dimensions et procédé d'imagerie correspondant
CN114390267A (zh) 立体图像数据合成方法、装置、电子设备及存储介质
JP6045280B2 (ja) 撮像装置
US9076215B2 (en) Arithmetic processing device

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2004571328

Country of ref document: JP

AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ OM PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase