WO2014017719A1 - Procédé et système de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle et support d'enregistrement lisible par ordinateur - Google Patents

Procédé et système de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle et support d'enregistrement lisible par ordinateur Download PDF

Info

Publication number
WO2014017719A1
WO2014017719A1 PCT/KR2013/001040 KR2013001040W WO2014017719A1 WO 2014017719 A1 WO2014017719 A1 WO 2014017719A1 KR 2013001040 W KR2013001040 W KR 2013001040W WO 2014017719 A1 WO2014017719 A1 WO 2014017719A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature point
camera
image
observer
image filtering
Prior art date
Application number
PCT/KR2013/001040
Other languages
English (en)
Korean (ko)
Inventor
박영성
한동숭
Original Assignee
전주대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 전주대학교 산학협력단 filed Critical 전주대학교 산학협력단
Publication of WO2014017719A1 publication Critical patent/WO2014017719A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S11/00—Systems for determining distance or velocity not using reflection or reradiation
    • G01S11/12—Systems for determining distance or velocity not using reflection or reradiation using electromagnetic waves other than radio waves

Definitions

  • the present invention relates to a visually impaired guide method, a system and a computer-readable recording medium using a multi-view camera having a parallel image filtering module, and more particularly, to a distance from an obstacle located in front of a plurality of cameras and
  • the present invention relates to a visually impaired guide method, a system and a computer-readable recording medium using a multi-view camera equipped with a parallel image filtering module that detects a direction and notifies the observer, so that even a blind person can walk independently.
  • the present invention aims to solve all the above-mentioned problems.
  • the present invention obtains a multi-view image using a plurality of cameras, by using the same to obtain the position and direction of the obstacle located in front of the observer to provide to the observer so that the observer can walk independently when the visually impaired It is an object of the present invention to provide a visually impaired guide method, a system and a computer-readable recording medium.
  • Another object of the present invention is to provide an image processing algorithm capable of processing a large amount of image data more quickly and accurately by connecting a plurality of image filtering modules in parallel.
  • a method for guiding a visually impaired person using a multi-view camera having a parallel image filtering module comprising: (a) extracting a common feature point from each input image from a plurality of cameras; Calculating a relative distance between the feature point and the observer, and (c) generating an alarm having a stimulus corresponding to the relative distance between the feature point and the observer.
  • a visually impaired guide system using a multi-view camera having a parallel image filtering module wherein a plurality of cameras and a common feature point are extracted from each input image from the camera
  • a system includes a central processing unit for calculating a relative distance of and an alarm device for generating an alarm having a stimulus corresponding to the relative distance between the feature point and the observer.
  • the present invention by using a plurality of cameras to detect the distance and the direction of the obstacle located in front and to notify the observer can be able to walk independently even in the case of the visually impaired.
  • a plurality of image filtering modules may be connected in parallel to process a large amount of image data more quickly and accurately.
  • FIG. 1 is a block diagram illustrating a visually impaired guide system using a multi-view camera having a parallel image filtering module according to an embodiment of the present invention.
  • FIG. 2 is a flowchart illustrating an object searching method using a multiview camera having a parallel image filtering module according to an embodiment of the present invention.
  • FIG. 3 is a diagram for describing a method of calculating a relative distance between a feature point and an observer common in an object searching system using a multiview camera having a parallel image filtering module according to an embodiment of the present invention.
  • FIG. 1 is a block diagram illustrating a visually impaired guide system using a multi-view camera having a parallel image filtering module according to an embodiment of the present invention.
  • the visually impaired guide system includes a plurality of cameras 100, a gap adjusting device 200 for adjusting a gap between the cameras 100, and a plurality of cameras connected in parallel with each other.
  • the image filtering module 300 may include a central processing unit 400 interoperating with the image filtering module 300.
  • the camera 100 may include an image memory 110 for temporarily storing an image captured therein, and the image filtering module 300 may include at least one graphic memory 310 and a graphic. And a central processing unit (400), wherein the central processing unit (400) includes a central processing unit (410), a central processing unit (420), a persistent storage unit (430), and a network communication unit (440). ), And an alarm device 450.
  • the central processing unit (400) includes a central processing unit (410), a central processing unit (420), a persistent storage unit (430), and a network communication unit (440).
  • an alarm device 450 includes a central processing unit (410), a central processing unit (420), a persistent storage unit (430), and a network communication unit (440).
  • the visually impaired guide system is a digital device, and includes a microprocessor and a memory means such as a personal computer (for example, a tablet computer, a laptop computer, etc.), a PDA, a web pad, a mobile phone, and the like.
  • a personal computer for example, a tablet computer, a laptop computer, etc.
  • PDA personal digital assistant
  • a web pad a mobile phone
  • Any portable digital device equipped with a computing power may be adopted as a visually impaired guide system according to the present invention.
  • the portable digital device may be manufactured in the form of accessories such as glasses or necklaces so that the visually impaired person can easily equip and move.
  • the camera 100 may be a digital camera 100 configured as a conventional general imaging device, for example, using a CCD and generating a digital image signal.
  • Image information input through the camera 100 may be temporarily stored in the image memory 110 of the camera 100, and may be stored in the center through a plurality of image filtering modules 300 connected in parallel with the camera 100. May be sent to the processing device 400.
  • the interval adjusting device 200 adjusts the interval between the cameras 100 to obtain a common feature point from each image input from the camera 100. Performs a function that enables extraction. That is, in the multi-view camera system, if the distance between the cameras 100 is sufficiently small, the distance between the respective cameras 100 is increased by using a possibility that a common feature point is extracted from an image input from each camera 100. By adjusting the common feature point can be extracted.
  • the feature point may be a portion of an obstacle located in front of the observer.
  • the image filtering module 300 may include at least one graphic memory 310 and a graphic computing device 320 therein, and the image filtering module 300 may include It is connected in parallel with the camera 100 and performs a function of receiving and filtering image information photographed using the camera 100 from the image memory 110.
  • the graphic memory 310 may receive and load a filtering program to be used for image filtering from the CPU 400, and the graphic computing device 320 may use the filtering program to load the camera 100. Performs a filtering operation on the image information input from the.
  • the computing device 320 may transmit and receive filtered image information by communicating with the central processing unit 400 in a bidirectional manner.
  • the image filtering module 300 is connected to the camera 100 in parallel and performs a function of processing two or more image information in real time.
  • the filtering operation through the image filtering module 300 requires various algorithms or various threshold processing according to the information to be extracted from the input image, and the time required for individual image processing is large so that two or more pieces of information can be simultaneously extracted. There is a limit.
  • the plurality of image filtering modules 300 may be connected to the camera 100 in parallel to satisfy a speed required for real-time image processing.
  • the graphic memory 310 and the graphic computing device 320 may be program modules in which at least some of them are in communication with the camera 100 or the central processing unit 400.
  • Such program modules may be included in the image filtering module 300 in the form of an operating system, an application module, and other program modules, and may be physically stored on various known storage devices.
  • program modules may be stored in a remote storage device that can communicate with the image filtering module 300.
  • program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.
  • the central processing unit 400 includes a central processing unit 410, a central processing unit 420, a persistent storage unit 430, a network communication unit 440, and an alarm unit. 450, and performs a function of controlling the flow of data between the camera 100 and the image filtering module 300. That is, the central processing unit 400 controls the flow of data to and from the outside or between the components of the visually impaired guide system, so that the camera 100 and the image filtering module 300 each have unique functions. Control to perform.
  • the central processing memory 410 is connected in parallel with the image filtering module 300 to store image information transmitted from the plurality of image filtering modules 300, and each processed and stored image. Loads a program that can aggregate information and extract other information.
  • the central processing unit 420 aggregates a plurality of image information input through a plurality of cameras 100 and filtered through the image filtering module 300, and is located in front of an observer. Calculates the position and direction of the camera, and configures the visually impaired guide system 100, the image filtering module 300, the central processing unit 410, the persistent storage device 430, the network in the central processing unit 400 It performs a function of controlling the communication device 440, the alarm device 450 and the like.
  • the persistent storage device 430 stores the data processed by the central processing unit 420.
  • the network communication device 440 performs a function of receiving data from the outside or transmitting data to the outside through a communication network.
  • the network communication device 440 may receive an externally created program through the communication network.
  • the communication network may be configured as an aspect of wireless communication and may be configured with various communication networks such as a mobile communication network and a satellite communication network. More specifically, the communication network according to the present invention includes a concept including all known wireless LAN (WLAN), code division multiple access (CDMA), wideband code division multiple access (WCDMA), global system for mobile communications (GSM), and the like. It should be understood that
  • the alarm device 450 functions to generate an alarm having a stimulus corresponding to the relative distance according to the relative distance between the obstacle and the observer calculated by the central processing unit 420. In addition, the alarm device 450 performs a function of notifying the observer of the direction in which the obstacle is located.
  • the central operation memory 410, the central processing unit 420, the persistent storage device 430, the network communication device 440, and the alarm device 450 is at least a portion thereof
  • the image filtering module 300 may be program modules communicating with an external system.
  • Such program modules may be included in the central processing unit 400 in the form of operating systems, application modules, and other program modules, and may be physically stored on a variety of known storage devices.
  • Such program modules may also be stored in a remote storage device that can communicate with the central processing unit 400.
  • program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.
  • FIG. 2 is a flowchart illustrating an object searching method using a multiview camera having a parallel image filtering module according to an embodiment of the present invention.
  • the guide method of the visually impaired person of the present invention photographs an image using a plurality of cameras 100 (S100), and extracts feature points from the photographed images (S110).
  • the feature point refers to a predetermined portion of a subject photographed using the camera 100 and may be set to, for example, a portion protruding most toward the observer of the subject or a point closest to the observer.
  • the process returns to step S100 to again use the plurality of cameras 100.
  • the video will be taken.
  • a SIFT or SURF algorithm may be used.
  • the Scale Invariant Feature Transform (SIFT) algorithm was proposed by David G. Lowe, which extracts feature points generated from edges or vertices of objects as vectors, and provides excellent matching for deformations due to image size changes, rotations, and light changes. Has performance.
  • the SIFT Descriptor maintains constant feature matching even with image transformation, scale, rotation, and noise effects.
  • the feature extraction process using SIFT can be divided into two stages: finding a feature point and generating a descriptor of the selected feature point. First, in the step of extracting the feature points, the candidate feature points in the scale space are extracted and the stability of the candidate feature points is examined to correct the positions of the stable feature points to the detailed positions. In the generating of the descriptor, the direction component is obtained through the gradient of the peripheral area around the selected points, and the descriptor is generated by resetting the region of interest based on the obtained direction component.
  • the Speeded Up Robust Features (SURF) algorithm is an algorithm for extracting feature points that are robust to changes in size and rotation, like the SIFT algorithm, and is used for object recognition, panorama image generation, and 3D image reconstruction.
  • the SURF algorithm generates image pyramids and extracts feature points from the generated image pyramids to represent a scale space for extracting feature points robust to a change in size.
  • the relative distance between the feature point and the observer is calculated using the common feature point (S140).
  • a method of calculating the relative distance between the feature point and the observer will be described in more detail with reference to FIG. 3.
  • the alarm device 450 of the central processing unit 400 generates an alarm having a stimulus corresponding to the relative distance between the feature point and the observer based on the calculated distance (S150).
  • the alarm may include a loudness or an intensity of vibration, and the closer the feature point is to the observer, the louder the sound or the stronger the vibration so as to inform the observer of the distance to the obstacle. More preferably, the observer may be notified of the direction of the feature point using the x and y coordinate values of the feature point obtained as described with reference to FIG. 3 below (S160).
  • FIG. 3 is a diagram for describing a method of calculating a relative distance between a feature point and an observer common in an object searching system using a multiview camera having a parallel image filtering module according to an embodiment of the present invention.
  • the coordinate F of the first camera is (u, v, w) and the coordinate F of the second camera. If '(u', v ', w') and the coordinate of the feature point P is (x, y, z), the feature point P is located within the shooting radius of the fan shape of the first and second cameras. May be a common feature point in images captured by the first and second cameras.
  • the feature point P forms an image at (px, py) spaced apart by a focal length f of the first camera, and similarly the feature point P in the image photographed using the second camera.
  • the image is formed at (p'x, p'y) separated by the focal length f 'of the second camera.
  • the coordinates (x, y, z) of the feature point P can be obtained.
  • the relative coordinates of the feature point and the minimum distance of the origin can be calculated to calculate the relative position between the observer and the obstacle.
  • Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium.
  • the computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.
  • Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts.
  • Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.
  • Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.
  • the hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention concerne, dans un aspect, un procédé de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle, qui comprend les étapes de : (a) extraction d'un point caractéristique commun à partir d'images, entrées respectivement depuis une pluralité de caméras ;(b) calcul d'une distance relative entre le point caractéristique et un observateur ; et (c) génération d'une alarme ayant une stimulation correspondant à la distance relative entre le point caractéristique et l'observateur.
PCT/KR2013/001040 2012-07-25 2013-02-08 Procédé et système de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle et support d'enregistrement lisible par ordinateur WO2014017719A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020120081301A KR101353860B1 (ko) 2012-07-25 2012-07-25 병렬 영상 필터링 모듈을 구비한 다시점 카메라를 이용한 시각 장애인 가이드 방법, 시스템 및 컴퓨터 판독 가능한 기록 매체
KR10-2012-0081301 2012-07-25

Publications (1)

Publication Number Publication Date
WO2014017719A1 true WO2014017719A1 (fr) 2014-01-30

Family

ID=49997505

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2013/001040 WO2014017719A1 (fr) 2012-07-25 2013-02-08 Procédé et système de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle et support d'enregistrement lisible par ordinateur

Country Status (2)

Country Link
KR (1) KR101353860B1 (fr)
WO (1) WO2014017719A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104021549A (zh) * 2014-05-19 2014-09-03 清华大学深圳研究生院 全仿射不变surf特征点检测方法及装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101578523B1 (ko) * 2014-03-20 2015-12-17 전주대학교 산학협력단 병렬 영상 필터링부가 구비된 다시점 카메라와 선형 광원을 이용하는 방법, 시스템 및 컴퓨터 판독 가능한 기록 매체

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06205291A (ja) * 1992-12-28 1994-07-22 Canon Inc 画像処理装置
JPH0843081A (ja) * 1994-07-29 1996-02-16 Sony Corp 距離演算装置
JP2001318594A (ja) * 2000-05-11 2001-11-16 Kumamoto Technopolis Foundation 視覚障害者用歩行支援システムおよび情報記録媒体
JP2003079685A (ja) * 2001-09-17 2003-03-18 Seiko Epson Corp 視覚障害者歩行用補助具
JP2010181919A (ja) * 2009-02-03 2010-08-19 Toyohashi Univ Of Technology 三次元形状特定装置、三次元形状特定方法、三次元形状特定プログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011250928A (ja) 2010-06-01 2011-12-15 Chuo Univ 視覚障害者用空間認識装置、方法およびプログラム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06205291A (ja) * 1992-12-28 1994-07-22 Canon Inc 画像処理装置
JPH0843081A (ja) * 1994-07-29 1996-02-16 Sony Corp 距離演算装置
JP2001318594A (ja) * 2000-05-11 2001-11-16 Kumamoto Technopolis Foundation 視覚障害者用歩行支援システムおよび情報記録媒体
JP2003079685A (ja) * 2001-09-17 2003-03-18 Seiko Epson Corp 視覚障害者歩行用補助具
JP2010181919A (ja) * 2009-02-03 2010-08-19 Toyohashi Univ Of Technology 三次元形状特定装置、三次元形状特定方法、三次元形状特定プログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104021549A (zh) * 2014-05-19 2014-09-03 清华大学深圳研究生院 全仿射不变surf特征点检测方法及装置

Also Published As

Publication number Publication date
KR101353860B1 (ko) 2014-01-23

Similar Documents

Publication Publication Date Title
CN110866480B (zh) 对象的跟踪方法及装置、存储介质、电子装置
US10659768B2 (en) System and method for virtually-augmented visual simultaneous localization and mapping
US8928736B2 (en) Three-dimensional modeling apparatus, three-dimensional modeling method and computer-readable recording medium storing three-dimensional modeling program
US10116867B2 (en) Method and apparatus for displaying a light field based image on a user's device, and corresponding computer program product
US10445924B2 (en) Method and device for processing DVS events
JP6406345B2 (ja) 画像処理装置、画像処理方法、および、画像処理プログラム
CN111145192B (zh) 图像处理方法及电子设备
CN112633306B (zh) 对抗图像的生成方法及装置
CN110544272A (zh) 脸部跟踪方法、装置、计算机设备及存储介质
WO2015037957A1 (fr) Appareil et procédé de stabilisation d'image au moyen d'une correction de flou d'image
KR102297217B1 (ko) 영상들 간에 객체와 객체 위치의 동일성을 식별하기 위한 방법 및 장치
WO2011078596A2 (fr) Procédé, système et support d'enregistrement lisible par ordinateur pour réalisation adaptative d'une adaptation d'image selon certaines conditions
EP3782359A1 (fr) Procédé de combinaison de contenu provenant de multiples trames et dispositif électronique associé
CN112270709B (zh) 地图构建方法及装置、计算机可读存储介质和电子设备
KR102528453B1 (ko) 영상 내의 특징 추출 방법, 이를 이용하는 영상 매칭 방법 및 영상 처리 방법
WO2019221340A1 (fr) Procédé et système de calcul de coordonnées spatiales d'une région d'intérêt et support d'enregistrement non transitoire lisible par ordinateur
CN105637852A (zh) 一种图像处理方法、装置及电子设备
CN111652933B (zh) 基于单目相机的重定位方法、装置、存储介质与电子设备
CN112802033A (zh) 图像处理方法及装置、计算机可读存储介质和电子设备
CN116468917A (zh) 图像处理方法、电子设备及存储介质
WO2014017719A1 (fr) Procédé et système de guidage d'une personne malvoyante à l'aide d'une caméra multivision ayant un module de filtrage d'image parallèle et support d'enregistrement lisible par ordinateur
CN111385481A (zh) 图像处理方法及装置、电子设备及存储介质
CN113936042B (zh) 一种目标跟踪方法、装置和计算机可读存储介质
CN113936064B (zh) 定位方法和装置
CN112950641B (zh) 图像处理方法及装置、计算机可读存储介质和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13822622

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13822622

Country of ref document: EP

Kind code of ref document: A1