WO2013133463A1 - Dispositif d'affichage d'image et procédé correspondant - Google Patents

Dispositif d'affichage d'image et procédé correspondant Download PDF

Info

Publication number
WO2013133463A1
WO2013133463A1 PCT/KR2012/001734 KR2012001734W WO2013133463A1 WO 2013133463 A1 WO2013133463 A1 WO 2013133463A1 KR 2012001734 W KR2012001734 W KR 2012001734W WO 2013133463 A1 WO2013133463 A1 WO 2013133463A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
pedestrian
image
body part
vehicle
Prior art date
Application number
PCT/KR2012/001734
Other languages
English (en)
Inventor
Jeongeun SHIN
Junbum PARK
Hansung Lee
Seungman KIM
Sungjea KO
Hyokak KIM
Bosang Kim
Keunyung BYUN
Original Assignee
Lg Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lg Electronics Inc. filed Critical Lg Electronics Inc.
Priority to PCT/KR2012/001734 priority Critical patent/WO2013133463A1/fr
Publication of WO2013133463A1 publication Critical patent/WO2013133463A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • B60R21/0134—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over responsive to imminent contact with an obstacle, e.g. using radar systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Definitions

  • the present invention relates to an image display device and a method thereof.
  • an image display device captures an image of a subject according to a user request and displays the captured image of the subject on a display unit.
  • a black box recording a driving state or a surrounding situation by using a camera mounted in a vehicle or a parking assistance system capturing an image of the rear side by using a camera installed at the rear side of a vehicle when the vehicle is backed up and outputting the captured image to a display device within the vehicle, and the like, are used, and this trend is increasing.
  • a system of capturing certain images by using wide angle cameras installed at front, rear, and left, and right sides of a vehicle, reconfiguring images captured by the cameras into images in such a form that the images are looked over immediately from an upper side of the vehicle, namely, in a downward direction, and outputting the reconfigured images to a display device of the vehicle has been proposed to enhance driver convenience.
  • This system is called a bird-eye view system in the aspect that it provides an image as if bird eyes view from the sky, an around view monitoring (AVM) system, or the like.
  • an aspect of the present invention provides an image display device and a method thereof capable of detecting a distance between a pedestrian included in a distorted image captured by a wide angle camera and a vehicle to thus prevent an occurrence of an accident in which the vehicle collides with the pedestrian in advance.
  • an image display device including: a storage unit configured to store previously captured images of body parts of a pedestrian; an image capture unit configured to capture images of a surrounding area of a vehicle; a controller configured to detect a pedestrian from the captured images on the basis of the images captured by the image capture unit and the images of body parts, and generate warning information when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and a display unit configured to display the generated warning information.
  • the image capture unit may include a plurality of wide angle cameras, and the previously captured images of body parts may be multi-view body part images previously captured through the plurality of wide angle cameras.
  • the body part images may be any one of images of an upper body of the pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian.
  • the controller may display a body part image which is the nearest (or the most adjacent) to the vehicle among the plurality of body part images, as the warning information.
  • the controller may further display a position of the most adjacent body part image and a distance between the vehicle and the most adjacent body part image on the display unit.
  • the controller in indicating (or displaying) a virtual parking line on the basis of a rotation angle of the wheel of the vehicle and a traveling direction on the display unit, the controller may display a virtual parking line avoiding the pedestrian on the display unit.
  • the controller may determine the detected body part images as pedestrians.
  • the controller may calculate distances between the vehicle and the determined respective pedestrians, select the smallest distance value from among the calculated distance values, and generate the warning information when the smallest distance value is equal to or smaller than the pre-set distance value.
  • the controller may divide the images captured by the image capture unit into pre-set equal parts, and detect pedestrians from the respective divided images on the basis of the respective divided images and the body part images corresponding to the respective images.
  • the controller may divide each of the captured images into a left image (or a left part) in which a subject is warped to the right side, a right image (or a right part) in which a subject is warped to the left side, and a central image (or a central part) in which a subject is less warped than those of the left and right images or is not warped, compare the right image with body part images of a pedestrian in which the pedestrian is warped to the left side, among the stored body part images, to detect a first pedestrian from the right image, compare the left image with body part images of a pedestrian in which the pedestrian is warped to the right side, among the stored body part images, to detect a second pedestrian from the left image, and compare the central image with body part images of a pedestrian in which the pedestrian is less warped than those of the left and right images or is not warped, among the stored body part images, to detect a third pedestrian from the central image.
  • an image display method including: storing previously captured body part images of a pedestrian; capturing images of a surrounding area through an image capture unit; detecting a pedestrian from the captured images on the basis of image captured by the image capture unit and the body part images, and generating warning information when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and displaying the generated warning information on a display unit.
  • FIG. 1 is a schematic block diagram showing the configuration of a mobile communication terminal employing an image display device according to embodiments of the present invention.
  • FIG. 2 is a view showing a vehicle navigation system employing an image display device according to embodiments of the present invention.
  • FIG. 3 is a schematic block diagram showing the configuration of a telematics terminal employing an image display device according to embodiments of the present invention.
  • FIG. 4 is a schematic block diagram showing the configuration of a navigation device (or a vehicle navigation device) employing an image display device according to embodiments of the present invention.
  • FIG. 5 is schematic block diagram of an image display device according to embodiments of the present invention.
  • FIG. 6 is a view showing wide angle cameras installed on a vehicle according to a first embodiment of the present invention.
  • FIG. 7 is a flow chart illustrating a process of an image display method according to the first embodiment of the present invention.
  • FIG. 8 is a view showing an image captured by an image capture unit according to the first embodiment of the present invention.
  • FIG. 9 is a view showing warning information displayed on a display unit according to the first embodiment of the present invention.
  • FIG. 10 is a flow chart illustrating a process of an image display method according to a second embodiment of the present invention.
  • FIG. 11 is a flow chart illustrating a process of an image display method according to a third embodiment of the present invention.
  • FIG. 12 is a view showing a receiver operating characteristic (ROC) curve according to the third embodiment of the present invention and an ROC curve according to the related art.
  • ROC receiver operating characteristic
  • an image display device and method capable of detecting an obstacle from distorted images (e.g., barrel images) captured by wide angle cameras which are able to secure a wide viewing angle without correcting the distorted images, and detecting a distance between a vehicle and the obstacle, to thus prevent an occurrence of an accident in which the vehicle collides with the obstacle and thus is damaged, in advance, according to embodiments of the present invention will be described in detail with reference to FIGS. 1 to 12.
  • distorted images e.g., barrel images
  • FIG. 1 is a view showing the configuration of a mobile communication terminal employing an image display device according to embodiments of the present invention.
  • a mobile communication terminal (or a mobile phone) 100 may be implemented in various forms such as mobile phones, smart phones, notebook computers, digital broadcast terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Player), etc.
  • the mobile communication terminal 100 includes a wireless communication unit 110, an A/V (Audio/Video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, an interface unit 170, a controller 180, and a power supply unit 190, etc.
  • FIG. 1 shows the mobile communication terminal 100 having various components, but it is understood that implementing all of the illustrated components is not a requirement.
  • the mobile communication terminal 100 may be implemented by greater or fewer components.
  • the wireless communication unit 110 typically includes one or more components allowing radio communication between the mobile communication terminal 100 and a wireless communication system or a network in which the mobile communication terminal is located.
  • the wireless communication unit may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115.
  • the broadcast receiving module 111 receives broadcast signals and/or broadcast associated information from an external broadcast management server (or other network entity) via a broadcast channel.
  • the broadcast channel may include a satellite channel and/or a terrestrial channel.
  • the broadcast management server may be a server that generates and transmits a broadcast signal and/or broadcast associated information or a server that receives a previously generated broadcast signal and/or broadcast associated information and transmits the same to a terminal.
  • the broadcast associated information may refer to information associated with a broadcast channel, a broadcast program or a broadcast service provider.
  • the broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and the like. Also, the broadcast signal may further include a broadcast signal combined with a TV or radio broadcast signal.
  • the broadcast associated information may also be provided via a mobile communication network and, in this case, the broadcast associated information may be received by the mobile communication module 112.
  • the broadcast signal may exist in various forms. For example, it may exist in the form of an electronic program guide (EPG) of digital multimedia broadcasting (DMB), electronic service guide (ESG) of digital video broadcast-handheld (DVB-H), and the like.
  • EPG electronic program guide
  • DMB digital multimedia broadcasting
  • ESG electronic service guide
  • DVD-H digital video broadcast-handheld
  • the broadcast receiving module 111 may be configured to receive signals broadcast by using various types of broadcast systems.
  • the broadcast receiving module 111 may receive a digital broadcast by using a digital broadcast system such as multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), digital video broadcast-handheld (DVB-H), the data broadcasting system known as media forward link only (MediaFLO ), integrated services digital broadcast-terrestrial (ISDB-T), etc.
  • the broadcast receiving module 111 may be configured to be suitable for every broadcast system that provides a broadcast signal as well as the above-mentioned digital broadcast systems. Broadcast signals and/or broadcast-associated information received via the broadcast receiving module 111 may be stored in the memory 160 (or anther type of storage medium).
  • the mobile communication module 112 transmits and/or receives radio signals to and/or from at least one of a base station (e.g., access point, Node B, etc.), an external terminal (e.g., other user devices) and a server (or other network entities).
  • a base station e.g., access point, Node B, etc.
  • an external terminal e.g., other user devices
  • a server or other network entities.
  • radio signals may include a voice call signal, a video call signal or various types of data according to text and/or multimedia message transmission and/or reception.
  • the wireless Internet module 113 supports wireless Internet access for the mobile communication terminal. This module may be internally or externally coupled to the terminal.
  • a wireless local area network (WLAN), Wi-Fi, wireless broadband (WiBro), world interoperability for microwave access (WiMAX), high speed downlink packet access (HSDPA), and the like may be used.
  • the short-range communication module 114 is a module for supporting short range communications.
  • Some examples of short-range communication technology include Bluetooth TM , Radio Frequency IDentification (RFID), Infrared Data Association (IrDA), Ultra-WideBand (UWB), ZigBee TM , and the like.
  • the location information module 115 is a module for checking or acquiring a location (or position) of the mobile communication terminal (when the mobile communication terminal is located in a vehicle, the location of the vehicle can be checked).
  • the location information module 115 may be embodied by using a GPS (Global Positioning System) module that receives location information from a plurality of satellites.
  • the location information may include coordinate information represented by latitude and longitude values.
  • the GPS module may measure an accurate time and distance from three or more satellites, and accurately calculate a current location of the mobile communication terminal according to trigonometry on the basis of the measured time and distances. A method of acquiring distance and time information from three satellites and performing error correction with a single satellite may be used.
  • the GPS module may acquire an accurate time together with three-dimensional speed information as well as the location of the latitude, longitude and altitude values from the location information received from the satellites.
  • a Wi-Fi position system and/or a hybrid positioning system may be used as the location information module 115.
  • the A/V input unit 120 is configured to receive an audio or video signal.
  • the A/V input unit 120 may include a camera 121 (or other image capture device) and a microphone 122 (or other sound pick-up device).
  • the camera 121 processes image data of still pictures or video obtained by an image capturing device in a video capturing mode or an image capturing mode.
  • the processed image frames may be displayed on a display unit 151 (or other visual output device).
  • the image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration of the mobile communication terminal.
  • the microphone 122 may receive sounds (audible data) via a microphone (or the like) in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sounds into audio data.
  • the processed audio (voice) data may be converted for output into a format transmittable to a mobile communication base station (or other network entity) via the mobile communication module 112 in case of the phone call mode.
  • the microphone 122 may implement various types of noise canceling (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.
  • the user input unit 130 may generate key input data from commands entered by a user to control various operations of the mobile communication terminal.
  • the user input unit 130 allows the user to enter various types of information, and may include a keypad, a dome switch, a touch pad (e.g., a touch sensitive member that detects changes in resistance, pressure, capacitance, etc. due to being contacted) a jog wheel, a jog switch, and the like.
  • a touch pad e.g., a touch sensitive member that detects changes in resistance, pressure, capacitance, etc. due to being contacted
  • a jog wheel e.g., a jog wheel
  • a jog switch e.g., a jog wheel
  • the sensing unit 140 detects a current status (or state) of the mobile communication terminal 100 such as an opened or closed state of the mobile communication terminal 100, a location of the mobile communication terminal 100, the presence or absence of user contact with the mobile communication terminal 100 (i.e., touch inputs), the orientation of the mobile communication terminal 100, an acceleration or deceleration movement and direction of the mobile communication terminal 100, etc., and generates commands or signals for controlling the operation of the mobile communication terminal 100.
  • the sensing unit 140 may sense whether the slide phone is opened or closed.
  • the sensing unit 140 can detect whether or not the power supply unit 190 supplies power or whether or not the interface unit 170 is coupled with an external device.
  • the interface unit 170 serves as an interface by which at least one external device may be connected with the mobile communication terminal 100.
  • the external devices may include wired or wireless headset ports, an external power supply (or battery charger) ports, wired or wireless data ports, memory card ports, ports for connecting a device having an identification module, audio input/output (I/O) ports, video I/O ports, earphone ports, or the like.
  • the identification module may be a memory chip (or other element with memory or storage capabilities) that stores various information for authenticating user's authority for using the mobile communication terminal 100 and may include a user identity module (UIM), a subscriber identity module (SIM) a universal subscriber identity module (USIM), and the like.
  • the device having the identification module may take the form of a smart card. Accordingly, the identifying device may be connected with the terminal 100 via a port or other connection means.
  • the interface unit 170 may be used to receive inputs (e.g., data, information, power, etc.) from an external device and transfer the received inputs to one or more elements within the mobile communication terminal 100 or may be used to transfer data within the mobile communication terminal to an external device.
  • the output unit 150 is configured to provide outputs in a visual, audible, and/or tactile manner (e.g., audio signal, video signal, alarm signal, vibration signal, etc.).
  • the output unit 150 may include the display unit 151, an audio output module 152, an alarm unit 153, and the like.
  • the display unit 151 may display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display unit 151 may display a User Interface (UI) or a Graphic User Interface (GUI) associated with a call or other communication (such as text messaging, multimedia file downloading, etc.). When the mobile terminal 100 is in a video call mode or image capturing mode, the display unit 151 may display a captured image and/or received image, a UI or GUI that shows videos or images and functions related thereto, and the like.
  • UI User Interface
  • GUI Graphic User Interface
  • the display unit 151 may include at least one of a Liquid Crystal Display (LCD), a Thin Film Transistor-LCD (TFT-LCD), an Organic Light Emitting Diode (OLED) display, a flexible display, a three-dimensional (3D) display, or the like.
  • the mobile terminal 100 may include two or more display units (or other display means) according to its particular desired embodiment.
  • the mobile terminal may include both an external display unit (not shown) and an internal display unit (not shown).
  • the display unit 151 may function as both an input device and an output device.
  • the touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, and the like.
  • the touch sensor may be configured to convert the pressure applied to a particular portion of the display unit 151 or a change in capacitance generated at a particular portion of the display unit 151 into an electrical input signal.
  • the touch sensor may be configured to detect a touch input pressure as well as a touch input position and a touch input area. When there is a touch input with respect to the touch sensor, the corresponding signal(s) are sent to a touch controller (not shown).
  • the touch controller processes the signal(s) and transmits corresponding data to the controller 180. Accordingly, the controller 180 can recognize a touched region of the display unit 151.
  • a proximity sensor 141 may be may be disposed within the mobile terminal covered by the touch screen or near the touch screen.
  • the proximity sensor 141 refers to a sensor for detecting the presence or absence of an object that accesses a certain detect surface or an object that exists nearby by using the force of electromagnetism or infrared rays without a mechanical contact.
  • the proximity sensor 141 has a longer life span compared with a contact type sensor, and it can be utilized for various purposes.
  • the example of the proximity sensor 141 may be a transmission type photo sensor, a direct reflection type photo sensor, a mirror-reflection type photo sensor, an RF oscillation type proximity sensor, a capacitance type proximity sensor, a magnetic proximity sensor, an infrared proximity sensor.
  • the touch screen is an electrostatic type touch screen, an approach of the pointer is detected on the basis of a change in an electric field according to the approach of the pointer.
  • the touch screen may be classified as a proximity sensor.
  • recognition of the pointer positioned to be close to the touch screen without being contacted will be called a 'proximity touch'
  • recognition of actual contacting of the pointer on the touch screen will be called a 'contact touch'.
  • the pointer when the pointer is in the state of the proximity touch, it means that the pointer is positioned to correspond vertically to the touch screen.
  • the proximity sensor 141 detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, or the like), and information corresponding to the detected proximity touch operation and the proximity touch pattern can be outputted to the touch screen.
  • a proximity touch and a proximity touch pattern e.g., a proximity touch distance, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, or the like
  • the audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode, a record mode, a voice recognition mode, a broadcast reception mode, and the like. Also, the audio output module 152 may provide audible outputs related to a particular function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100.
  • the audio output module 152 may include a receiver, a speaker, a buzzer, etc.
  • the alarm unit 153 outputs a signal for informing about an occurrence of an event of the mobile terminal 100.
  • Events generated in the mobile terminal may include call signal reception, message reception, key signal inputs, and the like.
  • the alarm unit 153 may output signals in a different manner, for example, to inform about an occurrence of an event.
  • the alarm unit 153 may output a signal in the form of vibration.
  • the alarm unit 153 may vibrate the mobile terminal through a vibration means.
  • the alarm unit 153 may vibrate the mobile terminal 100 through a vibration means as a feedback with respect to the key signal input. Through the vibration, the user may recognize the occurrence of an event.
  • a signal for notifying about the occurrence of an event may be output to the display unit 151 or to the voice output module 152.
  • a haptic module 154 generates various tactile effects the user may feel.
  • a typical example of the tactile effects generated by the haptic module 154 is vibration.
  • the strength and pattern of the haptic module 154 can be controlled. For example, different vibrations may be combined to be outputted or sequentially outputted.
  • the haptic module 154 may generate various other tactile effects such as an effect by stimulation such as a pin arrangement vertically moving with respect to a contact skin, a spray force or suction force of air through a jet orifice or a suction opening, a contact on the skin, a contact of an electrode, electrostatic force, etc., an effect by reproducing the sense of cold and warmth using an element that can absorb or generate heat.
  • an effect by stimulation such as a pin arrangement vertically moving with respect to a contact skin, a spray force or suction force of air through a jet orifice or a suction opening, a contact on the skin, a contact of an electrode, electrostatic force, etc.
  • the haptic module 154 may be implemented to allow the user to feel a tactile effect through a muscle sensation such as fingers or arm of the user, as well as transferring the tactile effect through a direct contact. Two or more haptic modules 154 may be provided according to the configuration of the mobile terminal 100. The haptic module 154 may be provided to a place which is frequently in contact with the user. For example, the haptic module 154 may be provided to a steering wheel, a gearshift, a lever, a seat, and the like.
  • the memory 160 may store software programs used for the processing and controlling operations performed by the controller 180, or may temporarily store data (e.g., a map data, phonebook, messages, still images, video, etc.) that are inputted or outputted.
  • data e.g., a map data, phonebook, messages, still images, video, etc.
  • the memory 160 may include at least one type of storage medium including a Flash memory, a hard disk, a multimedia card micro type, a card-type memory (e.g., SD or DX memory, etc), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.
  • the mobile terminal 100 may be operated in relation to a web storage device that performs the storage function of the memory 160 over the Internet.
  • the interface unit 170 serves as an interface with every external device connected with the mobile terminal 100.
  • the external devices may transmit data to an external device, receives and transmits power to each element of the mobile terminal 100, or transmits internal data of the mobile terminal 100 to an external device.
  • the interface unit 170 may include wired or wireless headset ports, external power supply ports, wired or wireless data ports, memory card ports, ports for connecting a device having an identification module, audio input/output (I/O) ports, video I/O ports, earphone ports, or the like.
  • the identification module may be a chip that stores various types of information for authenticating the authority of using the mobile terminal 100 and may include a user identity module (UIM), a subscriber identity module (SIM) a universal subscriber identity module (USIM), and the like.
  • the device having the identification module (referred to as 'identifying device', hereinafter) may take the form of a smart card. Accordingly, the identifying device may be connected with the terminal 100 via a port.
  • the interface unit 170 may be used to receive inputs (e.g., data, information, power, etc.) from an external device and transfer the received inputs to one or more elements within the mobile terminal 100 or may be used to transfer data between the mobile terminal and an external device.
  • the interface unit 170 may serve as a passage to allow power from the cradle to be supplied therethrough to the mobile terminal 100 or may serve as a passage to allow various command signals inputted by the user from the cradle to be transferred to the mobile terminal therethrough.
  • Various command signals or power inputted from the cradle may operate as signals for recognizing that the mobile terminal is properly mounted on the cradle.
  • the controller 180 typically controls the general operations of the mobile terminal. For example, the controller 180 performs controlling and processing associated with voice calls, data communications, video calls, and the like.
  • the controller 180 may include a multimedia module 181 for reproducing multimedia data.
  • the multimedia module 181 may be configured within the controller 180 or may be configured to be separated from the controller 180.
  • the controller 180 may perform a pattern recognition processing to recognize a handwriting input or a picture drawing input performed on the touch screen as characters or images, respectively.
  • the power supply unit 190 receives external power or internal power and supplies appropriate power required for operating respective elements and components under the control of the controller 180.
  • Various embodiments described herein may be implemented in a computer-readable or its similar medium using, for example, software, hardware, or any combination thereof.
  • the embodiments described herein may be implemented by using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, electronic units designed to perform the functions described herein.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • processors controllers, micro-controllers, microprocessors, electronic units designed to perform the functions described herein.
  • controller 180 itself.
  • the voice recognition module 182 recognizes a voice pronounced by the user and performs a corresponding function according to the recognized voice signal.
  • a navigation session 300 applied to the mobile terminal 100 displays a travel route on map data.
  • the image display device employed in the mobile communication terminal 100 may include: a memory 160 configured to store previously captured body part images (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of a pedestrian (a human body); a camera 121 configured to capture images of a surrounding area of a vehicle and outputting the captured images corresponding to the surrounding area; a controller 180 configured to detect a pedestrian from the captured images on the basis of the images captured by the camera 121 and the stored body part images, and generate warning information (e.g., an alarm message, an alarm voice, an alarm image, or the like) when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and a display unit 151 configured to display the generated warning information.
  • a memory 160 configured to store previously captured body part images (e.g., images of an upper body of
  • FIG. 2 is a view showing a vehicle navigation system employing an image display device according to embodiments of the present invention.
  • a vehicle navigation system includes an information providing center 30 providing traffic information and various types of data (e.g. a program, an execution file, and the like); and a telematics terminal 200 mounted within a vehicle, receiving traffic information through a wide area wireless communication network 20 and/or a local area wireless communication network, and providing a road guidance service on the basis of an artificial satellite 10.
  • the communication network may further include a wired/wireless communication network such as a local area network (LAN) or a wide area network (WAN).
  • LAN local area network
  • WAN wide area network
  • various types of traffic information e.g., road traffic information, information regarding a point of interest (POI)
  • POI point of interest
  • an information providing center 30 e.g., a server
  • TPEG Transmission Protocol Expert Group
  • the broadcast station inserts the traffic information including the information regarding a traffic light into a broadcast signal and broadcasts it to the vehicle 200.
  • the information regarding a traffic light may be received from a server (not shown) installed in the traffic light when the vehicle approaches the traffic light.
  • the server reconfigures various types of traffic information, which are collected through various paths connected to the communication network, for example, according to an operator input, through the wired/wireless Internet, from digital broadcast services such as a TDC (Transparent Data Channel) or a MOC (Multimedia Object Transport), from a different server, or from a probe car, into a traffic information format such as a format in conformity with a TPEG (Transport Protocol Expert Group) standard, namely, for example, a standard for a traffic information service, and transmits the same to the broadcast station.
  • digital broadcast services such as a TDC (Transparent Data Channel) or a MOC (Multimedia Object Transport)
  • TDC Transmission Data Channel
  • MOC Multimedia Object Transport
  • TPEG Transmission Protocol Expert Group
  • the server may generate a traffic information format on the basis of the TPEG standard including traffic light information, and transmit the same to the broadcast station.
  • the broadcast station includes the traffic information including traffic light information, which has been received from the server, in a broadcast signal and wirelessly transmits the same, so that a traffic information reception terminal, e.g., a navigation device, mounted in the vehicle 200, or the like, can receive the same.
  • the traffic information includes the traffic light information, and may include information regarding various traffic conditions required for vehicle driving in roads, sea traffic, airline flights, such as accidents, a road situation, traffic congestion, road construction, road blockage (or road closure), a public transportation network delay, air transportation holdup, and the like.
  • the broadcast station receives the processed traffic information including traffic light information from the server and transmits it through digital signals in conformity with various digital broadcast standards to the vehicle 200.
  • the broadcast standards include a European digital audio broadcasting (DAB) standard on the basis of Eureca-147 [ETSI EN 300 401), a terrestrial or satellite digital multimedia broadcasting (DMB) standard, a terrestrial digital video broadcasting (DVB-T) standard, a mobile digital video broadcasting-handheld (DVB-H) standard, media forward link only (MFLO) standard, and the like.
  • DMB European digital audio broadcasting
  • DMB terrestrial or satellite digital multimedia broadcasting
  • DVD-T terrestrial digital video broadcasting
  • DVD-H mobile digital video broadcasting-handheld
  • MFLO media forward link only
  • the broadcast station may transmit the traffic information including the traffic light information through a wired/wireless network such as the wired/wireless Internet.
  • the vehicle 200 refers to every carrier, such as general automobiles, buses, trains, ships, airplanes, and the like, implemented by using a mechanic, electronic device for the purpose of transporting people or objects.
  • the vehicle 200 includes a traffic information reception terminal mounted therein, receives traffic light information from the broadcast station by using the mounted traffic information reception terminal, processes the traffic light information, and transfers the processed traffic light information to a corresponding user through graphics, text, and/or audio.
  • FIG. 3 is a schematic block diagram showing the configuration of a telematics terminal employing an image display device according to embodiments of the present invention.
  • the telematics terminal 200 includes a main board 210 including a controller (e.g., a central processing unit (CPU)) 212 for controlling the telematics terminal 200 on the whole, a memory 213 for storing various types of information, a key controller 211 for controlling various key signals, and a liquid crystal display (LCD) controller 214 for controlling an LCD.
  • a controller e.g., a central processing unit (CPU)
  • CPU central processing unit
  • memory 213 for storing various types of information
  • a key controller 211 for controlling various key signals
  • LCD liquid crystal display
  • the memory 213 stores map information (map data) for displaying road guidance information on a digital map. Also, the memory 213 stores a traffic information collecting control algorithm for inputting traffic information according to the situation of a road in which the vehicle currently travels (runs), and information for controlling the algorithm.
  • the main board 210 includes a code division multiple access (CDMA) module 206, i.e., a mobile communication terminal having a unique device number as assigned and installed in the vehicle, a global position system (GPS) module 207 for guiding a location of the vehicle, receiving a GPS signal for tracking a travel route from a start point to a destination, or transmitting traffic information collected by the user, as a GPS signal, a CD deck 208 for reproducing a signal recorded in a CD (Compact Disk), a gyro sensor 209, and the like.
  • CDMA code division multiple access
  • GPS global position system
  • the CDMA module 206 and the GPS module 207 transmit or receive signals via antennas 204 and 205.
  • a broadcast receiving module 222 is connected with the main board 210 and receives a broadcast signal via an antenna 223.
  • a display unit (i.e., an LCD) 201 under the control of the LCD controller 214, a front board 202 under the control of the key controller 211, and a camera 227 for capturing the interior and/or the exterior of a vehicle are connected to the main board 210 via an interface board 203.
  • the display unit 201 displays various video signals and character signals
  • the front board 202 includes buttons for various key signal inputs and provides a key signal corresponding to a button selected by the user to the main board 210.
  • the display unit 201 includes a proximity sensor and a touch sensor (touch screen) of FIG. 2.
  • the front board 202 includes a menu key for directly inputting traffic information.
  • the menu key may be configured to be controlled by the key controller 211.
  • An audio board 217 is connected with the main board 210 and processes various audio signals.
  • the audio board 217 includes a microcomputer 219 for controlling the audio board 217, a tuner 218 for receiving a radio signal, a power source unit 216 for supplying power to the microcomputer 219, and a signal processing unit 215 for processing various voice signals.
  • the audio board 217 also includes a radio antenna 220 for receiving a radio signal and a tape deck 221 for reproducing an audio tape.
  • the audio board 217 may further include a voice output unit (e.g., an amplifier) 226 for outputting a voice signal processed by the audio board 217.
  • a voice output unit e.g., an amplifier
  • the voice output unit (amplifier) 226 is connected to a vehicle interface 224. Namely, the audio board 217 and the main board 210 are connected to the vehicle interface 224.
  • a handsfree 225a for inputting a voice signal, an airbag 225b configured for the security of a passenger, a speed sensor 225c for detecting a speed of the vehicle, and the like, may be connected to the vehicle interface 224.
  • the speed sensor 225c calculates a vehicle speed and provides the calculated vehicle speed information to the CPU 212.
  • the navigation session 300 applied to the telematics terminal 200 generates road guidance information on the basis of the map data and current location information of the vehicle and provides the generated road guidance information to a user.
  • the display unit 201 detects a proximity touch within a display window via a proximity sensor. For example, when a pointer (e.g., user's finger or a stylus) is proximity-touched, the display unit 201 detects the position of the proximity touch and outputs position information corresponding to the detected position to the controller 212.
  • a proximity sensor e.g., user's finger or a stylus
  • a voice recognition device (or a voice recognition module) 301 recognizes a voice pronounced by the user and performs a corresponding function according to the recognized voice signal.
  • the navigation session 300 applied to the telematics terminal 200 displays a travel route on map data, and when the location of the mobile terminal 100 is within a pre-set distance from a blind spot included in the travel route, the navigation session 300 automatically forms a wireless network with a terminal mounted in an adjacent vehicle (e.g., a vehicle navigation device) and/or a mobile communication terminal carried by a nearby pedestrian through wireless communication (e.g., a short-range wireless communication network), to receive location information of the adjacent vehicle from the terminal mounted in the adjacent vehicle and receive location information of the nearby pedestrian from the mobile communication terminal carried by the nearby pedestrian.
  • an adjacent vehicle e.g., a vehicle navigation device
  • wireless communication e.g., a short-range wireless communication network
  • the image display device employed in the telematics terminal 200 may include: a memory 213 configured to store previously captured body part images (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of a pedestrian (a human body); a camera 227 configured to capture images of a surrounding area of a vehicle and outputting the captured images corresponding to the surrounding area; a controller 210 configured to detect a pedestrian from the captured images on the basis of the images captured by the camera 227 and the stored body part images, and generate warning information (e.g., an alarm message, an alarm voice, an alarm image, or the like) when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and a display unit 201 configured to display the generated warning information.
  • a memory 213 configured to store previously captured body part images (e.g., images of
  • FIG. 4 is a schematic block diagram showing the configuration of a navigation device (or a vehicle navigation device) 400 employing the image display device according to embodiments of the present invention.
  • the navigation device 400 is classified into an in-dash type navigation device and an on-dash type navigation device according to an installation form in the vehicle 200.
  • the in-dash type navigation device (or an in-dash type vehicle navigation device) is inserted in a certain space allocated within a dash board of the vehicle 200 and fixedly mounted.
  • the on-dash type navigation device (or an on-dash type vehicle navigation device) is mounted on a dash board of the vehicle 200 or installed in the vicinity of the dash board by using a support.
  • the on-dash type navigation device (or the on-dash type vehicle navigation device) is detachably mounted, so it can be separated from the vehicle 200 and carried around.
  • the navigation device 400 includes the in-dash type navigation device and the on-dash type navigation device.
  • the navigation device 400 according to the present embodiment may include any information processing device that can receive and/or process traffic information such as various portable terminals performing a navigation function by interworking with a GPS receiver that receives a navigation message transmitted from a GPS satellite within the vehicle.
  • the navigation device 400 includes a GPS module 401 for receiving a GPS signal from a satellite and generating first vehicle location data of the navigation device (which is regarded to be located at the same location as that of the telematics terminal 200 or the mobile communication terminal 100) on the basis of the received GPS signal; a DR (Dead-Reckoning) sensor 402 for generating second vehicle location data on the basis of a travel direction and the speed of a vehicle; a storage unit (or a memory) 404 for storing map data and various types of information; a map matching unit 403 for generating an estimated vehicle location on the basis of the first vehicle location data and the second vehicle location data, matching the generated estimated vehicle location and a link (map matching link or a map matching road) in the map data stored in the storage unit 404, and outputting the matched map information (map matching results); a communication unit 408 for receiving real time traffic information from an information providing center and/or an adjacent vehicle via a wireless communication network 500, receiving traffic light information, and performing call communication
  • the communication unit 408 receives a call signal and/or a text message received through a base station, and may include a handsfree having a Bluetooth module and may receive a broadcast signal including traffic information in a TPEG format via an antenna from a broadcast station.
  • the broadcast signal may include traffic information according to a traffic information (TPEG) service or a binary format for scene (BIFS) data service and supplementary information such as various supplementary data, as well as video and audio data in conformity with various standards such as a terrestrial or satellite digital multimedia broadcasting (DMB), digital audio broadcasting (DAB), digital video broadcasting (DVB-T, DVB-H), and the like.
  • the communication unit 408 tunes a signal band at which traffic information is provided, demodulates the tuned signal, and outputs the demodulated signal to a TPEG decoder (included in the controller 407).
  • the TPEG decoder decodes the traffic information in the TPEG format and provides various types of information such as the traffic light information included in the traffic information to the controller 407.
  • the road guidance information may include various types of information related to traveling such as lane information, travel (running) limit speed information, turn-by-turn information, traffic safety information, traffic guidance information, vehicle information, road search information, as well as the map data.
  • the signal (i.e., the location information of the terminal) received via the GPS module 401 may be provided to the navigation device 400 by using a wireless communication scheme such as 802.11, a standard of a wireless network LAN including infrared communication (or infrared data association (IrDA)), and the like, 802.15, a standard for a wireless personal area network (PAN) including Bluetooth TM , UWB, ZigBee, and the like, 802.16, a standard for a wireless metropolitan area network (MAN) broadband wireless access (BWA) including a fixed wireless access (FWA), and the like, 802.20, a standard for the mobile Internet with respect to a mobile broadband wireless access (MBWA) including WiBro, WiMAX, and the like, proposed by IEEE (Institute of Electrical and Electronics Engineers).
  • a wireless communication scheme such as 802.11, a standard of a wireless network LAN including infrared communication (or infrared data association (IrDA)), and the like
  • 802.15 a standard for
  • the navigation device 400 may further include an input unit.
  • the input unit may select a user-desired function or receive information, and various devices such as a keypad, a touch screen, a jog shuttle, a microphone, and the like, may be used as the input unit.
  • the map matching unit 403 generates a vehicle estimated location on the basis of the first location data and the second location data, and reads map data corresponding to a travel route from the storage unit 404.
  • the map matching unit 403 matches the vehicle estimated location and a link (road) included in the map data, and outputs the matched map information (map matching results) to the controller 407.
  • the map matching unit 403 generates the vehicle estimated location on the basis of the first location data and the second location data, matches the generated vehicle estimated location and links in the map data stored in the storage unit 404 according to a link sequence, and outputs the matched map information (map matching results) to the controller 407.
  • the map matching unit 403 may output information regarding road attribute information such as one-storied road, duplex-storied road, and the like, included in the matched map information (map matching results).
  • the function of the map matching unit 403 may also be implemented in the controller 407.
  • the storage unit 404 stores the map data.
  • the stored map data includes geographic coordinates (or longitude/latitude coordinates) representing the latitude and longitude by DMS (Degree/Minute/Second) unit.
  • DMS Degree/Minute/Second
  • UDM universal transverse mercator
  • UPS universal polar system
  • TM transverse mercator
  • the storage unit 404 stores various types of information such as various menu screen images, a point of interest (POI), function characteristic information according to a particular position of map data, and the like.
  • POI point of interest
  • the storage unit 404 stores various user interfaces (UIs) and/or graphic UIs (GUIs).
  • UIs user interfaces
  • GUIs graphic UIs
  • the storage unit 404 stores data and programs required for the navigation device 400 to operate.
  • the storage unit 404 stores destination information input from the user via the input unit.
  • the destination information may be a destination or one of a destination and a start point.
  • the display unit 405 displays image information (or the road guidance map) included in the road guidance information generated by the controller 407.
  • the display unit 405 includes a touch sensor (touch screen) and/or a proximity sensor.
  • the road guidance information may include various types of information in relation to traveling (running or driving) such as lane information, running limit speed information, turn-by-turn information, traffic safety information, traffic guidance information, vehicle information, road search information, and the like, as well as the map data.
  • the display unit 405 may display various contents such as various menu screen images, road guidance information, and the like, by using the user interface and/or the graphic user interface included in the storage unit 404.
  • the contents displayed on the display unit 405 may include a menu screen image including various text or image data (including map data or various types of information data), and data such as icons, list menus, combo boxes, and the like.
  • the voice output unit 406 outputs voice information included in road guidance information (or a voice message with respect to the road guidance information) generated by the controller 407.
  • the voice output unit 406 may be an amplifier or a speaker.
  • the controller 407 generates the road guidance information on the basis of the matched map information and outputs the generated road guidance information to the display unit 405 and the voice output unit 406. Then, the display unit 405 displays the road guidance information.
  • the controller 407 receives real time traffic information from the information providing center and/or the terminal (or the vehicle navigation device) mounted in an adjacent vehicle) and generates road guidance information.
  • the controller 407 may be connected to a call center via the communication unit 408 to perform call communication, or transmit or receive information between the navigation device 400 and the call center.
  • the communication unit 408 may include a handsfree module having a Bluetooth TM function using a short-range radio communication scheme.
  • the image display device employed in the navigation device 400 includes; a memory 404 configured to store previously captured body part images (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of a pedestrian (a human body); a camera (not shown) configured to capture images of a surrounding area of a vehicle and outputting the captured images corresponding to the surrounding area; a controller 407 configured to detect a pedestrian from the captured images on the basis of the images captured by the camera and the stored body part images, and generate warning information (e.g., an alarm message, an alarm voice, an alarm image, or the like) when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and a display unit 405 configured to display the generated warning information.
  • a memory 404 configured to store previously captured body part images (e.g., images of an upper body of
  • the image display device and method thereof according to embodiments of the present invention can be variably applicable to smart phones, desktop computers, notebook computers, digital broadcast terminals, televisions, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Player), or the like, as well as to the mobile communication terminal (or a mobile phone) 100, the telematics terminal 200, and the navigation device 400.
  • PDAs Personal Digital Assistants
  • PMPs Portable Multimedia Player
  • FIG. 5 is schematic block diagram of the image display device according to embodiments of the present invention.
  • the image display device 600 includes: a storage unit 602 configured to store previously captured body part images (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of a pedestrian (a human body); an image capture unit 604 configured to capture images of a surrounding area of a vehicle and outputting the captured images corresponding to the surrounding area; a controller 601 configured to detect a pedestrian from the captured images on the basis of the images captured by the image capture unit 604 and the stored body part images, and generate warning information (e.g., an alarm message, an alarm voice, an alarm image, or the like) when a distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value; and an output unit (e.g., a speaker and/or a display unit) 603 configured to output the generated warning information
  • a storage unit 602 configured to store previously captured
  • the image capture unit 604 may be a wide angle camera capable of securing a wide viewing angle.
  • the body part images may be images captured by wide angle cameras capable of securing a wide viewing angle, and the images captured by the wide angle cameras may include distorted images (e.g., barrel images).
  • the stored body part images may be images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian previously captured by the wide angle cameras.
  • the stored body part images may be body part images previously captured by a plurality of wide angle cameras from multiple viewpoints.
  • the controller 601 detects a pedestrian from the images captured by the image capture unit 604 by comparing the images (e.g., barrel images) captured by the image capture unit 604 and the body part images (e.g., barrel images) stored in the storage unit 602.
  • the controller 601 may compare the distorted images (e.g., barrel images) captured by the wide angle cameras and the body part images (e.g., barrel images) stored in the storage unit 602 to detect a subject which is similar to the stored body part images (i.e., a subject having high similarity) (e.g., a subject having similarity of 85% to 90% or higher over the stored body part images) from the images captured by the image capture unit 604 and determine (or detect) the detected subject as a pedestrian (or an obstacle).
  • a subject which is similar to the stored body part images i.e., a subject having high similarity
  • a subject having similarity of 85% to 90% or higher over the stored body part images e.g., a subject having similarity of 85% to 90% or higher over the stored body part images
  • the controller 601 When a distance between the vehicle and the pedestrian is equal to or smaller than the pre-set distance value (e.g., 30 centimeters to 1 meter), the controller 601 generates warning information (e.g., a warning message, a warning voice, a warning image, or the like) and outputs the generated warning information to the output unit (e.g., the speaker and/or the display unit) 603.
  • the pre-set distance value may be changed by the user or a designer.
  • the image capture unit 604 may include first to fourth wide angle cameras.
  • the body part images may be body part images previously captured through four wide angle cameras mounted in the vehicle.
  • FIG. 6 is a view showing wide angle cameras installed on a vehicle according to a first embodiment of the present invention.
  • a first wide angle camera 6-1 is fixedly installed at the center of a front bumper of the vehicle to capture a front image of the vehicle.
  • a second wide angle camera 6-2 is fixedly installed at the center of a rear bumper of the vehicle to capture a rear image of the vehicle.
  • a third wide angle camera 6-3 is fixedly installed at the center of a left side mirror of the vehicle to capture a left image of the vehicle.
  • a fourth wide angle camera 6-4 is fixedly installed at the center of a right side mirror of the vehicle to capture a right image of the vehicle.
  • the first to fourth cameras may be installed at various positions by a designer.
  • the image capture unit 604 outputs images corresponding to four directions of the vehicle (e.g., a front image, a rear image, a leftward directional image, a rightward directional image of the vehicle) to the controller 601 in real time.
  • images corresponding to four directions of the vehicle e.g., a front image, a rear image, a leftward directional image, a rightward directional image of the vehicle.
  • FIG. 7 is a flow chart illustrating a process of an image display method according to the first embodiment of the present invention.
  • the image capture unit 604 captures an image of a surrounding area of a vehicle and outputs the captured image of the surrounding area to the controller 601 (step S11).
  • the controller 601 may operate the image capture unit 604.
  • the controller 601 receives vehicle gear shifting information through a vehicle interface, determine whether or not a reverse gear or a forward gear of the vehicle has been selected on the basis of the vehicle gear shifting information, and operate the rear camera 6-2 of the vehicle to capture a rear image when the reverse gear has been selected or automatically operate the front camera 6-1 to capture a front image when the forward gear of the vehicle has been selected.
  • FIG. 8 is a view showing an image captured by an image capture unit according to the first embodiment of the present invention.
  • the image of the surrounding area captured by the image capture unit 604 may include body part images 8-1 (e.g., an image of the upper body of a pedestrian, an image of the lower body of the pedestrian, an image of the left side of the body of the pedestrian, and the like) of the pedestrian.
  • body part images 8-1 e.g., an image of the upper body of a pedestrian, an image of the lower body of the pedestrian, an image of the left side of the body of the pedestrian, and the like
  • the controller 601 receives the images (e.g., the images in four directions) output from the image capture unit 604, and detects the pedestrian from the received images on the basis of the received images and the body part images of the pedestrian (step S12). For example, the controller 601 compares the images (e.g., the images in the four directions) output from the image capture unit 604 with the body part images of the pedestrian and determines the body part images (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of the pedestrian detected from the received images, as a pedestrian.
  • the body part images e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian
  • the controller 601 may detect a whole body (or an entire body) from the image through a related art object recognition program, and determine the detected whole body as a pedestrian. Also, when the whole body is not detected from the images through the related art object recognition program, the controller 601 may perform step S12. For example, when a user (or a driver) attempt to park a vehicle in a parking lot, if a portion of a particular body part is blocked by the pillar of a building, or the like, the pedestrian cannot be recognized through the related art object (a vehicle, a human body, or the like) recognition program. In this case, step S12 may be performed.
  • the controller 601 calculates a distance between the vehicle (the position of the camera mounted on the vehicle may be regarded to be identical to the position of the vehicle) and the pedestrian (step S13).
  • the method of detecting the distance between the camera and the subject (e.g., the pedestrian) whose image has been captured by the camera is a known art, so a detailed description thereof will be omitted.
  • the controller 601 determines whether or not the distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value (step S14).
  • the controller 601 When the distance value between the vehicle and the pedestrian is equal to or smaller than the pre-set distance value, the controller 601 generates warning information (e.g., a warning message and/or a warning sound), a warning image, or the like) and outputs the generated warning information to the output unit 603 (step S15).
  • the controller 601 may display a warning message and/or a warning image of Risk of collision on the display unit 603, and output the same through a warning voice speaker (not shown).
  • the output unit 603 may include a speaker and/or a display unit.
  • FIG. 9 is a view showing warning information displayed on a display unit according to the first embodiment of the present invention.
  • the controller 601 when the distance value between the vehicle and the pedestrian (e.g., an image of a particular body part of the pedestrian) is equal to or smaller than the pre-set distance value, the controller 601 generates a warning message 9-1, and displays the generated warning message 9-1 on the display unit 603.
  • the controller 601 when the distance value between the vehicle and the pedestrian (e.g., an image of a particular body part of the pedestrian) is equal to or smaller than the pre-set distance value, the controller 601 generates a warning message 9-1, and displays the generated warning message 9-1 on the display unit 603.
  • the controller 601 displays a first image 9-2 on the display unit 603.
  • the controller 601 may display a distance value (a collision anticipation distance, CAD) 9-3 between the vehicle and the pedestrian on the display unit 603.
  • the controller 601 may display a collision anticipation position(CAP) (the rear side) 9-4 on the display unit 603.
  • the controller 601 may display a collision anticipation position (the front side) on the display unit 603.
  • the controller 601 may display a collision anticipation position (the left side) on the display unit 603.
  • the controller 601 may display a collision anticipation position (the right side) on the display unit 603.
  • the controller 601 may display a body part image closest to the vehicle, among the plurality of body part images, as the warning information on the display unit 603.
  • the controller 601 may display the position of the closest body part image (e.g., any one of the front side, rear side, the left side, and the right side of the vehicle) and the distance between the vehicle and the closest body part image, as well as the closest body part image, on the display unit 603.
  • the controller 601 may display a virtual parking line avoiding the pedestrian included in the distorted images captured by the wide angle cameras on the display unit 603.
  • the image display device and method thereof by detecting the distance between the pedestrian included in the distorted images captured by the wide angle cameras and the vehicle, an occurrence of an accident in which the vehicle collides with the pedestrian can be prevented.
  • a stable, reliable parking system can also be provided to the user.
  • FIGS. 5 to 10 An image display device and method thereof according to a second embodiment of the present invention will be described with reference to FIGS. 5 to 10.
  • FIG. 10 is a flow chart illustrating a process of an image display method according to a second embodiment of the present invention.
  • the image capture unit 604 captures an image of a surrounding area of a vehicle, and outputs the captured image of the surrounding area to the controller 601 (step S21).
  • the controller 601 may operate the image capture unit 604.
  • the controller 601 receives vehicle gear shifting information through a vehicle interface, determine whether or not a reverse gear or a forward gear of the vehicle has been selected on the basis of the vehicle gear shifting information, and operate the rear camera 6-2 of the vehicle to capture a rear image when the reverse gear has been selected or automatically operate the front camera 6-1 to capture a front image when the forward gear of the vehicle has been selected.
  • the image of the surrounding area captured by the image capture unit 604 may include the body part images 8-1 (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of the pedestrian.
  • the body part images 8-1 e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian
  • the controller 601 receives the images (e.g., the images in four directions) output from the image capture unit 604, detects body part image(s) from the received images, and determines whether or not a plurality of the detected body part images are included in the received images (step S22). For example, the controller 601 compares the images output from the image capture unit 604 with the stored body part images and detects body part image(s) (e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian) of the pedestrian included in the received images, as a pedestrian.
  • body part image(s) e.g., images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian
  • the images output from the image capture unit 604 may include any one or more of the images of an upper body of a pedestrian, images of a lower body of the pedestrian, images of the left side of the body of the pedestrian, images of the face of the pedestrian, images of legs of the pedestrian, and images of the trunk of the pedestrian.
  • the controller 601 may detect the whole body from the images through a related art object (an obstacle such as a vehicle, a human body, and the like) recognition program, and determine the detected whole body as a pedestrian (an obstacle). Also, when the whole body is not detected from the images through the related art object recognition program, the controller 601 may perform step S22.
  • a related art object an obstacle such as a vehicle, a human body, and the like
  • the controller 601 determines whether or not a position relation between the detected body part images is identical to a pre-set position relation (step S23). For example, when the previously captured images of the face and legs of the pedestrian are detected from the images output from the image capture unit 604, the controller 601 determines whether or not a position relation between the face and the legs is identical to the pre-set position relation. Namely, the controller 601 may determine whether or not the face is located at a position higher than that of the legs, determine whether or not the face is spaced apart from the legs by a pre-set distance, or determine whether or not the position relation between the face and the legs is identical to the pre-set position relation through combination thereof.
  • the controller 601 determines the detected body part images as pedestrians (or obstacles), respectively, and calculates distances between the vehicle (the positions of the cameras mounted on the vehicle may be considered to be identical to the location of the vehicle) and the respective pedestrians (step S24).
  • the controller 601 determines whether or not calculated distance values are equal to or smaller than the pre-set distance values (step S25). For example, the controller 601 selects the smallest distance value from among the calculated distance values, and determine whether or not the smallest distance value is equal to or smaller than the pre-set distance value.
  • the controller 601 When the distance values between the vehicle and the pedestrians are equal to or smaller than the pre-set distance value, the controller 601 generates warning information (e.g., a warning message and/or a warning sound, a warning image, or the like) and outputs the generated warning information to the output unit 603 (step S26).
  • the controller 601 may display a warning message and/or a warning image of Risk of collision on the display unit 603, and output the same through a warning voice speaker (not shown).
  • the output unit 603 may include a speaker and/or a display unit.
  • the controller 601 may detect a position relation between pedestrians (obstacles) included in the distorted images captured by the wide angle cameras and display a virtual parking line avoiding the pedestrians (obstacles) on the display unit 603.
  • the image display device and method thereof by detecting the position relation between the body part images included in the distorted images captured by the wide angle cameras to accurately detect pedestrians, the distances between the pedestrians and the vehicle can be accurately detected.
  • FIG. 11 is a flow chart illustrating a process of an image display method according to a third embodiment of the present invention.
  • the image capture unit 604 captures an image of a surrounding area of a vehicle and outputs the captured image of the surrounding area to the controller 601 (step S31).
  • the controller 601 may operate the image capture unit 604.
  • the controller 601 receives vehicle gear shifting information through a vehicle interface, determine whether or not a reverse gear or a forward gear of the vehicle has been selected on the basis of the vehicle gear shifting information, and operate the rear camera 6-2 of the vehicle to capture a rear image when the reverse gear has been selected or automatically operate the front camera 6-1 to capture a front image when the forward gear of the vehicle has been selected.
  • the image capture unit 604 may be a wide angle camera securing a wide viewing angle.
  • An image captured by the wide angle camera includes a distorted image (e.g., a barrel image).
  • a left image of a distorted image (e.g., a barrel image) captured by the wide angle camera includes a subject warped to the right side
  • a right image of the distorted image includes a subject warped to the left side
  • a central image of the distorted image includes a subject which is less warped than those of the left image and the right image or a subject which is not warped.
  • the warped degree of the subject may vary according to types of wide angle cameras.
  • the controller 601 divides the image captured by the image capture unit 604 into pre-set equal parts (step S32). For example, the controller 601 divides the image captured by the image capture unit 604 into equal parts (e.g., three equal parts) in a vertical direction.
  • the pre-set equal parts may be two equal parts or three equal parts or greater.
  • the distorted image e.g., the barrel image
  • the wide angle camera is divided into three equal parts (a left image, a central image, and a right image)
  • the left image has a subject warped to the right side
  • the right image has a subject warped to the left side
  • the central image has a subject which is less warped than those of the left image and the right image or is not warped.
  • the controller 601 detects a pedestrian from the divided images on the basis of the divided respective images (e.g., the left image, the right image, and the central image) and the body part images of the pedestrian stored in the storage unit 602 (step S33). For example, the controller 601 compares the divided left image with the body part images (e.g., the first distorted body part images in which the pedestrian is warped to the right side) corresponding to the left image, among the body part images of the pedestrian stored in the storage unit 602, to detect the body part image(s) (e.g., an image of the upper body warped to the right side, an image of the lower body warped to the right side, an image of the left side of the body warped to the right side, an image of the face warped to the right side, an image of legs warped to the right side, an image of the trunk warped to the right side, and the like) from the left images, and determines the detected body part images as obstacles or pedestrians.
  • the body part images e.g., the first
  • the controller 601 compares the divided right image with the body part images (e.g., second distorted body part images in which the pedestrian is warped to the left side) corresponding to the right image, among the body part images of the pedestrian stored in the storage unit 602, to detect the body part image(s) (e.g., an image of the upper body warped to the left side, an image of the lower body warped to the left side, an image of the left side of the body warped to the left side, an image of the face warped to the left side, an image of legs warped to the left side, an image of the trunk warped to the left side, and the like) from the right image, and determines the detected body part images as obstacles or pedestrians.
  • the body part images e.g., second distorted body part images in which the pedestrian is warped to the left side
  • the body part image(s) e.g., an image of the upper body warped to the left side, an image of the lower body warped to the left side, an image of the left side of the
  • the controller 601 compares the divided central image with the body part images (e.g., the body part images in which the pedestrian is less warped than the subjects of the left and right images or is not warped) corresponding to the central image, among the body part images of the pedestrian stored in the storage unit 602, to detect the body part image(s) from the central image, and determines the detected body part images as obstacles or pedestrians.
  • the body part images e.g., the body part images in which the pedestrian is less warped than the subjects of the left and right images or is not warped
  • the controller 601 may detect the whole body (or the entire body) from the divided images through the related art object recognition program and determine the detected whole body as a pedestrian. Also, when the whole body is not detected from the images through the related art object recognition program, the controller 601 may perform steps S32 and S33. For example, when the user (or the driver) attempt to park a vehicle in a parking lot, if a portion of a pedestrian is blocked by the pillar of a building, or the like, the pedestrian cannot be recognized through the related art object (a vehicle, a human body, or the like) recognition program. In this case, the controller may perform steps S32 and S33.
  • the controller 601 calculates a distance between the vehicle (the position of the camera mounted on the vehicle may be regarded to be identical to the position of the vehicle) and the pedestrian (step S34).
  • the method of detecting the distance between the camera and the subject (e.g., the pedestrian) whose image has been captured by the camera is a known art, so a detailed description thereof will be omitted.
  • the controller 601 determines whether or not the distance value between the vehicle and the pedestrian is equal to or smaller than a pre-set distance value (step S35).
  • the controller 601 When the distance value between the vehicle and the pedestrian is equal to or smaller than the pre-set distance value, the controller 601 generates warning information (e.g., a warning message and/or a warning sound, a warning image, or the like) and outputs the generated warning information to the output unit 603 (step S36).
  • the controller 601 may display a warning message and/or a warning image of Risk of collision on the display unit 603, and output the same through a warning voice speaker (not shown).
  • the output unit 603 may include a speaker and/or a display unit.
  • FIG. 12 is a view showing a receiver operating characteristic (ROC) curve according to the third embodiment of the present invention and an ROC curve according to the related art.
  • ROC receiver operating characteristic
  • obstacle detection (recognition) performance when distorted images captured by wide viewing angles are corrected and an obstacle (e.g., a pedestrian) is detected from the corrected images
  • obstacle detection (recognition) performance e.g., the third embodiment
  • obstacle detection (recognition) performance when the distorted images are compared with the previously stored body part images, without correcting the distorted images captured by the wide angle cameras, to detect an obstacle (e.g., a pedestrian) are further improved.
  • the controller 601 when the distance value between the vehicle and the pedestrian (e.g., an image of a particular body part of the pedestrian) is equal to or smaller than the pre-set distance value, the controller 601 generates a warning message 9-1, and displays the generated warning message 9-1 on the display unit 603.
  • the controller 601 when the distance value between the vehicle and the pedestrian (e.g., an image of a particular body part of the pedestrian) is equal to or smaller than the pre-set distance value, the controller 601 generates a warning message 9-1, and displays the generated warning message 9-1 on the display unit 603.
  • the controller 601 displays a first image 9-2 on the display unit 603.
  • the controller 601 may display a distance value (a collision anticipation distance) 9-3 between the vehicle and the pedestrian on the display unit 603.
  • the controller 601 may display a collision anticipation position (the rear side) 9-4 on the display unit 603.
  • the controller 601 may display a collision anticipation position (the front side) on the display unit 603.
  • the controller 601 may display a collision anticipation position (the left side) on the display unit 603.
  • the controller 601 may display a collision anticipation position (the right side) on the display unit 603.
  • the controller 601 may display a body part image closest to the vehicle, among the plurality of body part images, as the warning information on the display unit 603.
  • the controller 601 may display the position of the closest body part image (e.g., any one of the front side, rear side, the left side, and the right side of the vehicle) and the distance between the vehicle and the closest body part image, as well as the closest body part image, on the display unit 603.
  • the controller 601 may display a virtual parking line avoiding the pedestrian included in the distorted images captured by the wide angle cameras on the display unit 603.
  • the distance between the pedestrian and the vehicle can be accurately detected.
  • the image display device and method thereof by dividing the distorted images captured by the wide angle cameras and comparing the respective divided images with the body part images to detect a pedestrian from the respective divided images, an occurrence of an accident in which the vehicle collides with the pedestrian can be prevented.
  • a stable, reliable parking system can also be provided to the user.
  • a distance between a pedestrian included in distorted images captured by wide angle cameras and a vehicle is detected, whereby an occurrence of an accident in which the pedestrian collides with the obstacle can be prevented.
  • a distance between a pedestrian included in distorted images captured by wide angle cameras and a vehicle is detected, whereby a stable, reliable parking system can be provided to a user.
  • a position relation between body part images included in distorted images captured by wide angle cameras is detected, whereby a pedestrian can be precisely detected from the distorted images.
  • a position relation between body part images included in distorted images captured by wide angle cameras is detected to precisely detect a pedestrian from the distorted images, whereby a distance between the pedestrian and the vehicle can be precisely detected.
  • each of distorted images captured by wide angle cameras is divided and the respective divided images are compared with body part images to detect a pedestrian from each of the divided images, whereby a distance between the pedestrian and the vehicle can be accurately detected.
  • each of distorted images captured by wide angle cameras is divided and the respective divided images with body part images are compared to detect a pedestrian from each of the divided images, whereby an occurrence of an accident in which a vehicle and a pedestrian collide can be prevented.
  • each of distorted images captured by wide angle cameras is divided and the respective divided images with body part images are compared to detect a pedestrian from each of the divided images and accurately detect a distance between the pedestrian and the vehicle, whereby a stable, reliable parking system can be provided to a user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Traffic Control Systems (AREA)

Abstract

L'invention concerne un dispositif d'affichage d'image et un procédé correspondant aptes à détecter une distance entre un piéton inclus dans une image déformée capturée par une caméra à grand angle et un véhicule pour ainsi empêcher à l'avance la survenue d'un accident dans lequel le véhicule entre en collision avec le piéton. Le dispositif d'affichage d'image comprend : une unité de stockage configurée pour stocker des images capturées précédemment de parties corporelles d'un piéton ; une unité de capture d'image configurée pour capturer des images d'une zone environnante d'un véhicule ; un dispositif de commande configuré pour détecter un piéton à partir des images capturées sur la base des images capturées par l'unité de capture d'image et des images de parties corporelles, et générer des informations d'avertissement lorsqu'une valeur de distance entre le véhicule et le piéton est égale ou plus petite qu'une valeur de distance préréglée ; et une unité d'affichage configurée pour afficher les informations d'avertissement générées.
PCT/KR2012/001734 2012-03-09 2012-03-09 Dispositif d'affichage d'image et procédé correspondant WO2013133463A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/001734 WO2013133463A1 (fr) 2012-03-09 2012-03-09 Dispositif d'affichage d'image et procédé correspondant

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/001734 WO2013133463A1 (fr) 2012-03-09 2012-03-09 Dispositif d'affichage d'image et procédé correspondant

Publications (1)

Publication Number Publication Date
WO2013133463A1 true WO2013133463A1 (fr) 2013-09-12

Family

ID=49116926

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/001734 WO2013133463A1 (fr) 2012-03-09 2012-03-09 Dispositif d'affichage d'image et procédé correspondant

Country Status (1)

Country Link
WO (1) WO2013133463A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3109098A1 (fr) * 2015-06-15 2016-12-28 Lg Electronics Inc. Appareil d'assistance au conducteur de véhicule
EP3093820A4 (fr) * 2014-01-09 2017-08-16 Clarion Co., Ltd. Dispositif de reconnaissance des alentours d'un véhicule
CN108583571A (zh) * 2018-04-28 2018-09-28 深圳市商汤科技有限公司 碰撞控制方法及装置、电子设备和存储介质
EP3706038A1 (fr) * 2019-03-07 2020-09-09 MAN Truck & Bus SE Véhicule automobile doté d'une pluralité de dispositifs capteurs destinés à la détection d'environnement

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030010891A (ko) * 2001-07-27 2003-02-06 주식회사 현대오토넷 자동차 후방 감시장치
US20070222565A1 (en) * 2006-03-27 2007-09-27 Mazda Motor Corporation Pedestrian detecting device for vehicle
JP2007251257A (ja) * 2006-03-13 2007-09-27 Fujitsu Ten Ltd 画像認識装置、車両制御装置、画像認識方法および車両制御方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20030010891A (ko) * 2001-07-27 2003-02-06 주식회사 현대오토넷 자동차 후방 감시장치
JP2007251257A (ja) * 2006-03-13 2007-09-27 Fujitsu Ten Ltd 画像認識装置、車両制御装置、画像認識方法および車両制御方法
US20070222565A1 (en) * 2006-03-27 2007-09-27 Mazda Motor Corporation Pedestrian detecting device for vehicle

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3093820A4 (fr) * 2014-01-09 2017-08-16 Clarion Co., Ltd. Dispositif de reconnaissance des alentours d'un véhicule
EP3109098A1 (fr) * 2015-06-15 2016-12-28 Lg Electronics Inc. Appareil d'assistance au conducteur de véhicule
US10029639B2 (en) 2015-06-15 2018-07-24 Lg Electronics Inc. Driver assistance apparatus for vehicle and vehicle
CN108583571A (zh) * 2018-04-28 2018-09-28 深圳市商汤科技有限公司 碰撞控制方法及装置、电子设备和存储介质
EP3706038A1 (fr) * 2019-03-07 2020-09-09 MAN Truck & Bus SE Véhicule automobile doté d'une pluralité de dispositifs capteurs destinés à la détection d'environnement

Similar Documents

Publication Publication Date Title
WO2013133464A1 (fr) Dispositif d'affichage d'image et procédé correspondant
WO2016076587A1 (fr) Appareil de fourniture d'informations et procédé associé
WO2011136456A1 (fr) Procédé et appareil d'affichage vidéo
WO2011093560A1 (fr) Appareil d'affichage d'informations et son procede
WO2014010879A1 (fr) Appareil et procédé de reconnaissance vocale
WO2012133983A1 (fr) Traitement d'image dans un dispositif d'affichage d'image monté sur véhicule
WO2012036323A1 (fr) Terminal de communications et son procédé de commande
WO2013054957A1 (fr) Appareil de contrôle d'interface d'entrée et procédé associé
WO2015178574A1 (fr) Système de fourniture d'informations et procédé associé
WO2014189200A1 (fr) Appareil d'affichage d'image et procédé de fonctionnement d'appareil d'affichage d'image
WO2016028007A1 (fr) Dispositif portable et son procédé de commande
WO2017039047A1 (fr) Véhicule et procédé de commande associé
WO2017217578A1 (fr) Dispositif de commande de véhicule et procédé de commande associé
WO2017034287A1 (fr) Système de prévention des collisions avec des piétons, et procédé de fonctionnement dudit système
WO2012133982A1 (fr) Dispositif de traitement d'image et procédé de commande du dispositif de traitement d'image
WO2017061653A1 (fr) Procédé pour empêcher la conduite en état d'ivresse et dispositif auxiliaire de véhicule pour le fournir
WO2017022881A1 (fr) Véhicule et procédé de commande associé
WO2016117790A1 (fr) Appareil de commutation de mode de conduite de véhicule et procédé associé
WO2014038809A1 (fr) Dispositif électronique et son procédé de commande
EP3012588A1 (fr) Terminal mobile et procédé pour sortir des notifications en fonction des priorités
WO2017022879A1 (fr) Assistance de conduite de véhicule et véhicule la comprenant
WO2021040060A1 (fr) Dispositif électronique embarqué et son procédé de fonctionnement
WO2014112659A1 (fr) Dispositif électronique et procédé de commande dudit dispositif électronique
WO2016186355A1 (fr) Dispositif d'affichage et procédé de fonctionnement associé
WO2019139310A1 (fr) Appareil de conduite autonome et procédé de conduite autonome d'un véhicule

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12870461

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12870461

Country of ref document: EP

Kind code of ref document: A1