JP5646918B2 - Portable electronic device and method of using portable electronic device - Google Patents

Description

  The present invention relates to a portable electronic device having a function of displaying an image and a method of using the portable electronic device.

  Conventionally, as a device for projecting an image on a wall surface or a screen, there is a so-called projector. This projector is mainly a so-called stationary apparatus that is supplied with power from a commercial power source and is used in a fixed state. This stationary projector projects an image on a fixed wall or screen in a fixed state.

  In contrast, in recent years, portable projectors that are small and easy to carry have been proposed. For example, Patent Document 1 includes a projector-equipped mobile phone that includes an upper cabinet, a lower cabinet, and a hinge portion that rotatably connects the upper cabinet and the lower cabinet to each other, and is equipped with a projector having a lens and a light source. The terminal is listed. Moreover, as a portable electronic device, there is a portable electronic device that displays an image on a display unit such as a display.

JP 2007-96542 A

  Here, in a portable electronic device, a map may be displayed. However, the map cannot be fully utilized simply by displaying the map on the display unit of the portable electronic device.

  An object of this invention is to provide the portable electronic device which can display a map image with high effectiveness simply more appropriately, and the usage method of a portable electronic device.

  In order to solve the above-described problems and achieve the object, a portable electronic device according to the present invention includes a housing, a photographing unit that obtains a map image by photographing, and information included in the map image obtained by the photographing unit. And a processing unit that acquires information of a specific image corresponding to the map image, and an image projection unit that is provided in the housing and projects the specific image in a projection region. .

  Here, it is preferable that the processing unit obtains an image that can be added to the map image as the specific image, and causes the image projection unit to project the specific image.

  In addition, the housing further includes a display unit that displays an image, and the processing unit acquires an image that can be added to the map image as the specific image, and the specific image, It is preferable that the map image captured by the imaging unit is added and displayed on the display unit.

  The specific image associates the position information of the map image and the position information of the reference map data based on the map image and the reference map data, and It is preferable to include an image created by associating information included in the data with the coordinates of the map image.

  Moreover, it is preferable that the specific image includes an image obtained by converting the coordinates on the reference map data in association with the coordinates of the map image.

  Moreover, it has a position information acquisition part which acquires own position information, and the said process part compares with the said map image based on the said map information and the own position information acquired by the said position information acquisition part. It is preferable to determine a range of map data serving as the reference.

  Moreover, it is preferable that the said process part analyzes the said map image, specifies an absolute position from each element contained in a map image, and acquires the specific image corresponding from the specified absolute position.

  The communication unit further includes a communication unit that communicates with another communication device, and the processing unit outputs the map image from the communication unit to another communication device, and acquires a specific image from the other communication device. Is preferred.

  The communication unit further includes a communication unit that communicates with another communication device, the processing unit analyzes the map image, specifies an absolute position of each element included in the map image, and stores information on the specified absolute position. It is preferable that the communication unit outputs the image to another communication device and acquires a specific image from the other communication device.

  In order to solve the above-described problems and achieve the object, the present invention is a method of using the portable electronic device according to any one of the above, and captures a map arranged in a capturing area of the capturing unit. Thus, a map image is obtained, information on a specific image corresponding to the map image is acquired based on the map image, and the acquired specific image is projected by an image projection unit.

  Here, when the map is arranged in the shooting area and the specific image is projected by the image projection unit, the position of the map is confirmed by the shooting unit, and the map and the specific image are It is preferable to project the specific image while correcting the projection position so as to be added.

  In order to solve the above-described problems and achieve the object, a portable electronic device according to the present invention includes a housing, a photographing unit that obtains a map image by photographing, and information included in the map image obtained by the photographing unit. And a display unit that displays information on the specific image corresponding to the map image and a display unit that displays the map image and the specific image.

  Here, it is preferable that the display unit displays the map image and the specific image together.

  INDUSTRIAL APPLICABILITY The portable electronic device and the method of using the portable electronic device according to the present invention have an effect that a map image with higher effectiveness can be appropriately displayed.

FIG. 1 is a schematic diagram illustrating a schematic configuration of a communication system including the portable electronic device according to the embodiment. FIG. 2 is a front view illustrating a schematic configuration of one embodiment of the portable electronic device according to the embodiment. FIG. 3 is a side view of the portable electronic device shown in FIG. FIG. 4 is a block diagram illustrating a schematic configuration of functions of the portable electronic device illustrated in FIGS. 2 and 3. FIG. 5 is an explanatory diagram showing a state in which an image is displayed by the projector of the portable electronic device shown in FIG. FIG. 6 is a schematic diagram illustrating an example of a map. FIG. 7 is a schematic diagram illustrating an example of an image projected on a map. FIG. 8 is an explanatory diagram illustrating an example of a state in which an image is displayed by the projector of the portable electronic device. FIG. 9 is a flowchart for explaining an example of the operation of the mobile electronic device. FIG. 10 is an explanatory diagram for explaining the operation of the portable electronic device. FIG. 11 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 12 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 13 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 14 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 15 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 16 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 17 is an explanatory diagram for explaining the operation of the mobile electronic device. FIG. 18 is an explanatory diagram for explaining another example of the operation of the mobile electronic device. FIG. 19 is an explanatory diagram for explaining another example of the operation of the mobile electronic device. FIG. 20 is an explanatory diagram for explaining another example of the operation of the mobile electronic device. FIG. 21 is an explanatory diagram for explaining another example of the operation of the mobile electronic device. FIG. 22 is an explanatory diagram for explaining an example of a screen displayed on the display. FIG. 23 is an explanatory diagram for explaining another example of a screen displayed on the display.

  Hereinafter, the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following description. In addition, constituent elements in the following description include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those in a so-called equivalent range. In the following, a cellular phone is taken as an example of a portable electronic device, but the application target of the present invention is not limited to a cellular phone. For example, a PHS (Personal Handy-phone System), a PDA, a portable navigation device, a notebook The present invention can also be applied to personal computers, game machines, and the like.

  FIG. 1 is a schematic diagram illustrating a schematic configuration of a communication system including the portable electronic device according to the embodiment. A communication system 1 shown in FIG. 1 includes a server 2, an Internet communication network 3, a database 4, a server 5, a wireless communication device 6, a GPS satellite 7, and a portable electronic device 10. In addition, the number of each part which comprises the communication system 1 is not specifically limited, Each part may be provided with the some apparatus. For example, a plurality of portable electronic devices 10 may be provided for one server 2.

  The server 2 has various data such as information (telephone number and address) specifying each communication device, and communicates with the portable electronic device 10 via the communication network to supply various information. The server 2 and the portable electronic device 10 are formed by a base station and a relay station, and communicate via a communication network that transmits and receives communication waves wirelessly. In addition, the communication network can use various communication methods as long as communication can be performed between the server 2 and the portable electronic device 10. For example, communication may be performed using a satellite line.

  The server 2 also receives information from the portable electronic device 10 and relays information based on the information to communicate information to other communication devices via the Internet communication network 3. That is, the server 2 collects and stores information sent from each communication device, and supplies (provides) the collected information to the communication device as necessary. The server 2 can also process (process) the collected information and supply it to the communication device.

  The Internet communication network (hereinafter also referred to as “Internet”) 3 includes an exchange and a wired / wireless communication line. The Internet communication network 3 performs information communication between a communication device and another communication device using a wired or wireless communication line. Specifically, the Internet 3 performs information communication between the server 2 and the server 5 and information communication between the server 2 and the database 4.

  The database 4 is a storage device and stores various data necessary for processing in the portable electronic device 10 such as map information and image processing information. The database 4 supplies various stored information to the server 2 or the server 5 via the Internet 3.

  The server 5 communicates with the portable electronic device 10 via the wireless communication device 6 to receive and supply various information. Note that the server 5 differs only in the communication method with the portable electronic device 10, and other configurations are basically the same as those of the server 2.

  The wireless communication device 6 is a communication device that performs direct wireless communication with the portable electronic device 10. As the wireless communication device 6, for example, a communication unit that communicates with another communication device using a short-range wireless communication technology can be used. As short-range wireless communication technologies, infrared communication (IrDA (registered trademark), IrMC (registered trademark), IrSimple (registered trademark)), visible light communication, Bluetooth (registered trademark), magnetic field communication (RFID, Radio Frequency Identification) technology can be used.

  The GPS satellite 7 is a satellite that transmits (provides) position information of the communication device to a communication device (communication terminal, communication device) having a GPS (Global Positioning System) communication function. Although only one GPS satellite 7 is shown in FIG. 1, the GPS satellites 7 are arranged in a number (mainly three or four) necessary for specifying the position of the communication device. Each of the plurality of GPS satellites 7 transmits position information to the mobile electronic device 10.

  The portable electronic device 10 is a portable communication device having a function of capturing an image and a function of displaying an image. The portable electronic device 10 acquires an image of the map 9 that is a photographing target, performs processing based on the acquired image of the map 9, and displays the generated image. As a display method, there are a method of displaying on a display unit and a method of projecting an image. Hereinafter, the portable electronic device 10 will be described in detail.

  FIG. 2 is a front view showing a schematic configuration of one embodiment of the portable electronic device according to the embodiment, and FIG. 3 is a side view of the portable electronic device shown in FIG. A portable electronic device 10 shown in FIGS. 2 and 3 is a mobile phone having a wireless communication function, an output unit, a voice acquisition unit, and an imaging unit. The portable electronic device 10 includes a housing 10C that includes a plurality of housings. Specifically, the housing 10C is configured to be openable and closable by a first housing 10CA and a second housing 10CB. That is, the portable electronic device 10 has a foldable housing. In addition, the housing | casing of the portable electronic device 10 is not limited to such a structure. For example, the housing of the mobile electronic device 10 may be a sliding housing that allows one housing and the other housing to slide from each other in a state in which both housings are overlapped, It may be a rotary type in which one casing is rotated around an axis along the overlapping direction, or both casings are connected via a biaxial hinge.

  1st housing 10CA and 2nd housing 10CB are connected by the hinge mechanism 18 which is a connection part. By connecting the first housing 10CA and the second housing 10CB with the hinge mechanism 18, the first housing 10CA and the second housing 10CB rotate together around the hinge mechanism 18 and move away from each other and approach each other. It is comprised so that it can rotate to a direction (direction shown by arrow R of FIG. 3). When the first housing 10CA and the second housing 10CB rotate in a direction away from each other, the portable electronic device 10 opens, and when the first housing 10CA and the second housing 10CB rotate in a direction approaching each other, the portable electronic device 10 closes. Thus, it is in a folded state (a state indicated by a dotted line in FIG. 3).

  The first casing 10CA is provided with a display 12 shown in FIG. 2 as a display unit. The display 12 displays a standby image when the portable electronic device 10 is waiting for reception, or displays a menu image used to assist the operation of the portable electronic device 10. In addition, the first housing 10CA is provided with a receiver 16 that is an output unit that outputs sound during a call of the mobile electronic device 10.

  The second chassis 10CB is provided with a plurality of operation keys 13A for inputting characters at the time of creating a mail, for example, the telephone number of the other party, and selecting and determining menus displayed on the display 12 and scrolling the screen. For example, a direction key and a determination key (direction and determination key) 13B are provided for easily executing the above processing. The operation key 13 </ b> A, the direction key, and the determination key 13 </ b> B constitute the operation unit 13 of the mobile electronic device 10. In addition, the second casing 10CB is provided with a microphone 15 that is a voice acquisition unit that receives voice during a call of the portable electronic device 10. The operation unit 13 is provided on the operation surface 10PC of the second casing 10CB shown in FIG. The surface opposite to the operation surface 10PC is the back surface 10PB of the portable electronic device 10.

  An antenna is provided inside the second housing 10CB. The antenna is a transmission / reception antenna used for wireless communication, and is used for transmission / reception of radio waves (electromagnetic waves) related to calls, electronic mails, and the like between the portable electronic device 10 and a base station. The second casing 10CB is provided with a microphone 15. The microphone 15 is disposed on the operation surface 10PC side of the portable electronic device 10 shown in FIG.

  On the side opposite to the hinge mechanism 18 of the first casing 10CA, a projector 34 that is an image projection unit and a camera 40 that captures (acquires) an image of a surface on which the image is projected are provided. Note that the light emitting portion of the projector 34 and the photographing window of the camera 40 are exposed to the outside of the first housing 10CA. With such a configuration, the projector 34 can project an image onto the projection target. Further, by acquiring the distance from the image of the area where the image is projected by the camera 40, the image projected by the projector 34 can be automatically focused. Note that the size and focal length of the image to be projected can be adjusted by acquiring an image of the area where the image is projected by the camera 40.

  FIG. 4 is a block diagram illustrating a schematic configuration of functions of the portable electronic device illustrated in FIGS. 2 and 3. As shown in FIG. 4, the mobile electronic device 10 includes a processing unit 22, a storage unit 24, an external storage unit 25, a transmission / reception unit 26, an operation unit 13, an audio processing unit 30, a display unit 32, A projector 34, a short-range communication unit 36, and a camera 40 are included. The processing unit 22 has a function of comprehensively controlling the overall operation of the mobile electronic device 10. In other words, the processing unit 22 transmits and receives various processes of the mobile electronic device 10 so as to be executed in an appropriate procedure according to the operation of the operation unit 13 and the software stored in the storage unit 24 of the mobile electronic device 10. The operations of the unit 26, the voice processing unit 30, the display unit 32, the short-range communication unit 36, the position information acquisition unit 42, and the like are controlled.

  Various processes of the portable electronic device 10 include, for example, a voice call performed via a circuit switching network, creation and transmission / reception of an electronic mail, browsing of an Internet Web (World Wide Web) site, and the like. Examples of operations of the transmission / reception unit 26, the audio processing unit 30, and the display unit 32 include transmission / reception of signals by the transmission / reception unit 26, input / output of audio by the audio processing unit 30, display of images by the display unit 32, and the like. .

  The processing unit 22 executes processing based on a program (for example, an operating system program, application program, etc.) stored in the storage unit 24. The processing unit 22 is configured by, for example, a microprocessor unit (MPU: Micro Processing Unit), and executes various processes of the above-described portable electronic device 10 according to procedures instructed by software. That is, the processing unit 22 sequentially reads instruction codes from an operating system program, application program, or the like stored in the storage unit 24 and executes processing.

  The processing unit 22 has a function of executing a plurality of application programs. Examples of application programs executed by the processing unit 22 include an application program that controls driving of the projector 34 and the camera 40, an application program that reads and decodes various image files (image information) from the storage unit 24, and decoding. There are a plurality of application programs such as an application program for displaying the image obtained in this way on the display unit 32 or projecting it on the projector 34.

  In the present embodiment, the processing unit 22 is based on a projector control unit 22 a that controls the operation of the projector 34, a camera control unit 22 b that controls the operation of the camera 40, an image acquired by the camera 40, and an input from the operation unit 13. A condition determination unit 22c that determines the conditions of the projection region, an attitude / position calculation unit 22d that calculates and calculates the posture and position of the housing 10C with respect to the projection plane, an image displayed on the display unit 32, and an image displayed on the projector 34 An image processing unit 22e for processing and generating the image. The projector control unit 22a, the camera control unit 22b, the condition determination unit 22c, the posture / position calculation unit 22d, and the image processing unit 22e each have a hardware resource constituted by the processing unit 22 and the storage unit 24. This is realized by executing a task assigned by 22 control units. Here, a task is a processing unit that cannot be executed at the same time among the entire processing performed by the application software or the processing performed by the same application software.

  The storage unit 24 stores software and data used for processing in the processing unit 22, and includes a task for operating the application program for controlling the driving of the projector 34 and the camera 40, and an image processing program. The task for operating is stored. In addition to these tasks, for example, communication, downloaded voice data, software used by the processing unit 22 to control the storage unit 24, the telephone number or mail address of the communication partner, etc. are described in the storage unit 24. The address book to be managed, voice files such as dial tone and ring tone, temporary data used in the process of software, and the like are stored.

  Note that computer programs and temporary data used in the software processing process are temporarily stored in the work area assigned to the storage unit 24 by the processing unit 22. The storage unit 24 includes, for example, a nonvolatile storage device (nonvolatile semiconductor memory such as ROM: Read Only Memory, a hard disk device, etc.), and a readable / writable storage device (for example, SRAM: Static Random Access Memory, DRAM: Dynamic Random Access Memory).

  The external storage unit 25 is a storage device that includes an external memory and connection terminals, and the external memory is detachable from the housing 10C. Various software and data are stored in the external memory in the same manner as the storage unit 24. The external storage unit 25 connects the external memory and the processing unit 22 via a connection terminal, and writes and reads information to and from the external memory, similarly to the storage unit 24. In addition, since the external memory is detachable from the external storage unit 25, the external memory connected to the processing unit 22 can be exchanged. As the external memory, various storage media such as an SD card (registered trademark), a memory stick (registered trademark), smart media, and a USB memory can be used.

  The transmission / reception unit 26 has an antenna 26a, establishes a radio signal line by a CDMA (Code Division Multiple Access) method or the like with the base station via a channel assigned by the base station, and makes a telephone call with the base station. Perform communication and information communication. The operation unit 13 includes, for example, an operation key 13A to which various functions such as a power key, a call key, a numeric key, a character key, a direction key, a determination key, and a call key are assigned, and a direction key and a determination key 13B. Is done. When these keys are input by a user operation, a signal corresponding to the operation content is generated. The generated signal is input to the processing unit 22 as a user instruction.

  The audio processing unit 30 executes processing of an audio signal input to the microphone 15 and an audio signal output from the receiver 16 and the speaker 17. That is, the sound processing unit 30 amplifies the sound input from the microphone 15 and executes AD conversion (Analog Digital conversion), and then performs signal processing such as encoding to convert the sound into digital sound data. Output to the processing unit 22. Further, the audio data sent from the processing unit 22 is subjected to processing such as decoding, DA conversion (Digital Analog conversion), and amplification to convert it into an analog audio signal, which is then output to the receiver 16 and the speaker 17. Here, the speaker 17 is disposed in the housing 10C of the portable electronic device 10, and outputs a ringtone, a mail transmission sound, and the like.

  The display unit 32 includes the display 12 described above, and displays a video corresponding to the video data supplied from the processing unit 22 and an image corresponding to the image data on the display panel. The display 12 is configured by a display panel configured by, for example, a liquid crystal display (LCD) or an organic EL (Organic Electro-Luminescence) panel. The display unit 32 may include a sub display in addition to the display 12.

  The projector 34 includes a light source and an optical system that switches whether to project the light emitted from the light source based on image data. In the present embodiment, the projector 34 includes a visible light irradiation device (visible light irradiation means) 31 that is a light source, a drawing device 35 that is an optical system, and a focus adjustment device 39. The visible light irradiation device 31 emits visible laser light. The light in the visible light region is light of 360 nm to 400 nm on the short wavelength side and light of 760 nm to 830 nm on the long wavelength side. In this embodiment, the visible light irradiation device 31 includes R (Red), G (Green), B Irradiate light of three colors (Blue).

  The drawing device 35 combines the three colors of light emitted from the visible light irradiation device 31 and irradiates the image projection target. The drawing device 35 includes a switching element that switches whether to transmit light emitted from the light source, and a mirror that performs raster scanning of the light that has passed through the switching element. The drawing device 35 changes the angle of the laser light emitted from the visible light irradiation device 31 by the mirror and causes the image projection target to scan the laser light, thereby projecting the image onto the image projection target.

  As the mirror, for example, a MEMS (Micro Electro Mechanical System) mirror is used. A MEMS mirror drives a mirror using a piezoelectric element, scans the visible light irradiated from the visible light irradiation apparatus 31, and produces | generates a visible image and an invisible image. In this case, by changing the angle of the light emitted from the light source by the mirror and scanning the light emitted from the light source over the entire surface of the image projection target, a visible image or an invisible image can be projected onto the image projection target. it can. Thus, the projector 34 is a scan-type projector. Note that the configuration of the projector 34 is not limited to that using the laser described above as a light source. For example, the projector 34 may be a projector having a configuration in which a halogen light, an LED light source, or an LD light source is used as a light source, and an LCD (Liquid Crystal Display) or DMD (Digital Micro-mirror Device) is provided in the optical system.

  The focus adjustment device 39 has a function (focus adjustment function) for forming a visible image projected from the drawing device 35 on an image projection target in accordance with a command from the projector control unit 22a. The focus adjustment device 39 includes a focus adjustment mechanism including, for example, a movable lens, and realizes the focus adjustment function by moving the lens. Further, the focus adjustment device 39 may realize the focus adjustment function by performing predetermined image processing on the image data projected by the projector 34 by the image processing unit 22e. Further, the focus adjustment device 39 may realize the focus adjustment function by a focus adjustment function and image processing.

  As described above, the projector 34 of the portable electronic device 10 is an image projection unit that projects an image, and the image projection surface is exposed to the outside of the housing 10 </ b> C of the portable electronic device 10. The portable electronic device 10 projects an image from the projector 34, and as shown in FIG. 5, a predetermined number of image projection targets (for example, a wall surface or a screen) at a position facing the image projection surface of the projector 34 are predetermined. An image can be projected onto the area (projection area) PA. The operation of the projector 34 is controlled by the processing unit 22, and various images sent from the processing unit 22, such as still images and moving images, are projected and displayed on the projection area PA. Further, the example shown in FIG. 5 is a case where, for example, the area where the map 100 such as the map guidance display is displayed is the projection area PA.

  Next, the short-range communication unit 36 is a communication unit that communicates with other communication devices using a short-range wireless communication technology. As short-range wireless communication technologies, infrared communication (IrDA (registered trademark), IrMC (registered trademark), IrSimple (registered trademark)), visible light communication, Bluetooth (registered trademark), magnetic field communication (RFID, Radio Frequency Identification) technology can be used. For example, when using the magnetic field communication, the short-range communication unit 36 includes an IC tag that identifies itself and a reading unit that reads an IC tag included in another communication device.

  The position information acquisition unit 42 is a communication unit that receives a GPS signal transmitted from the GPS satellite 7. Further, the position information acquisition unit 42 calculates the latitude and longitude of the mobile electronic device 10 from the received GPS signal, and sends the calculated latitude and longitude information to the processing unit 22.

  As described above, the camera 40 is an imaging mechanism that is provided near the light emitting unit of the projector 34 and acquires an image of an area including a projection area. That is, the camera 40 acquires an image in the direction in which the projector 34 emits light. The camera 40 is a shooting mechanism that takes an image with a wider angle of view than the projection angle of the image projected by the projector 34, and takes an image of an area wider than the projection area onto which the image is projected by the projector 34. can do. The portable electronic device 10 is basically configured as described above.

  Next, the operation of the portable electronic device will be described with reference to FIGS. FIG. 6 is a schematic diagram illustrating an example of a map. FIG. 7 is a schematic diagram illustrating an example of an image projected on a map. FIG. 8 is an explanatory diagram showing an example of a state in which an image is displayed by the projector of the portable electronic device shown in FIG.

  First, the mobile electronic device 10 captures and acquires the map 100 shown in FIG. When the mobile electronic device 10 acquires the image of the map 100, the mobile electronic device 10 analyzes the image of the map 100 and identifies which area the captured map 100 represents. Then, if the portable electronic device 10 specifies the area | region which the map 100 shows, the specific image corresponding to the map 100 will be acquired. For example, the specific image 102 shown in FIG. 7 is an image on which an index 104 indicating that a predetermined position on the map 100 is a destination is displayed.

  As illustrated in FIG. 8, when the portable electronic device 10 acquires the specific image 102, the acquired specific image 102 is projected onto the map 100 by the projector 34. Thereby, the destination index 104 can be displayed on the map 100.

  Hereinafter, the operation of the portable electronic device will be described with reference to FIGS. Here, FIG. 9 is a flowchart showing an example of the operation of the portable electronic device. FIGS. 10 to 17 are explanatory diagrams for explaining the operation of the portable electronic device. The operation illustrated in FIG. 9 may be performed whenever the projector 34 or the camera 40 is activated, or may be performed when an execution instruction for a mode for performing the operation illustrated in FIG. 9 is input. 9 may be processed by executing an application stored in the storage unit 24, or may be processed by executing an application stored in the external storage unit 25. That is, the program for executing the operation shown in FIG. 9 may be stored in any area. The program can be acquired by downloading from the outside, and can also be acquired by reading from the external storage unit 25. Further, the mobile electronic device 10 may send part of the processing information to the server 2 or the server 5 so that the server 2 or the server 5 performs the processing.

  First, the process part 22 of the portable electronic device 10 image | photographs a map image as step S12. That is, the processing unit 22 captures an image including a map such as a guidance display board with the camera 40. Note that the adjustment of the shooting area may be performed by the user or automatically. When the processing unit 22 captures the map image in step S12, in step S14, the processing unit 22 performs image analysis processing, that is, analyzes the map image acquired in step S12. Specifically, the shape of the road and the shape of the section on the map are extracted from the captured map image. After analyzing the image in step S14, the processing unit 22 performs a modeling process on the analyzed map image in step S16.

  Specifically, when the processing unit 22 captures and acquires an image of the map 100 as shown in FIG. 6 in step S12, the processing unit 22 analyzes the image of the map 100 in step S14, for example, performs binarization processing. A road 111 can be extracted from the image of the map 100 as shown in an image 110 of FIG. At this time, the processing unit 22 binarizes the color histogram shown in FIG. 11 with the color of the color range 114 corresponding to the color of the road 111 being black and the other colors being white. Further, by extracting an area corresponding to the road from the binarized image data, the road 111 can be extracted like the image 110. Thereafter, as shown in an image 112 in FIG. 10, the processing unit 22 thins the road 111 of the image 110 into a road 113. Next, the processing unit 22 extracts feature points from the thinned road 113 as shown in the image 112. Specifically, as shown in an image 120 in FIG. 12, a connection point between the road 122 and the road 122 and an end point of the road 122 are extracted as feature points 124.

  When the processing unit 22 analyzes the image and extracts feature points as shown in the image 120, the processing unit 22 performs modeling as step S16. Specifically, the processing unit 22 models the feature points 124 extracted from the image 120 as shown in an image 126. That is, the number of branches is calculated for each feature point 128. The numbers shown in the circles of the feature points 128 are the number of branch points. Thereafter, the processing unit 22 assigns an index (identification number) 132 to each feature point 128 as shown in the image 130. Thereafter, the processing unit 22 lists each feature point as shown in FIG. The list items include an index (Index), the number of branches, an index connected to the index, and index coordinates (relative coordinates in the image). Note that the processing unit 22 performs the processing shown by the images in FIGS. 10 and 11 by calculation processing on the image data.

  Next, the process part 22 extracts area data as step S18. That is, data of an area (region) corresponding to the map indicated by the map image is extracted. Note that the processing unit 22 of the present embodiment acquires the position information of the mobile electronic device 10 from the GPS satellite 7 by the position information acquisition unit 42, and narrows down the area corresponding to the map based on the acquired position information. That is, an area centered on the acquired position information is extracted. Further, the processing unit 22 extracts map data (area data) corresponding to the extracted area. In addition, even if the area data is acquired from the database 4 by communicating with the servers 2 and 5 through the transmission / reception unit 26 or the short-range communication unit 36, the storage unit 24 and the external storage itself are provided. You may extract from the map data memorize | stored in the part 25. FIG.

  After extracting the area data in step S18, the processing unit 22 performs a modeling process on the area data in step S20. That is, the processing unit 22 models the map data corresponding to the extracted area. Specifically, the processing unit 22 thins the path of the map data. The map data may be data thinned at the time of acquisition. Thereafter, as illustrated in an image 140 in FIG. 14, the processing unit 22 extracts feature points 144 from the road 142 and calculates the number of branches for each feature point 144. The numbers shown in the circles of the feature points 144 are also the number of branch points. Thereafter, the processing unit 22 assigns an index (identification number) 146 to each feature point 144 as shown in the image 145. Thereafter, the processing unit 22 lists each feature point as shown in FIG. The list items include an index, the number of branches, an index connected to the index, and index coordinates. Note that latitude and longitude are associated with the feature points extracted from the area data as coordinates.

  After performing the modeling process in step S20, the processing unit 22 performs a data matching process as step S22. That is, the processing unit 22 collates the map image data modeled in step S16 with the area data modeled in step S20, and determines which position in the area data the map image represents. Identify. Specifically, the processing unit 22 compares the list of feature points of the map image shown in FIG. 13 with the list of feature points of the area data shown in FIG. The feature point to be detected is detected. That is, an area where the modeled road shape matches between the map image and the area data is detected. Here, in the present embodiment, the feature points having the indexes of 1 to 9 correspond to each other.

  After performing the data matching process in step S22, the processing unit 22 creates a coordinate conversion table as step S24. Specifically, the processing unit 22 associates the coordinates of the map image with the coordinates of the area data, and calculates a conversion rule for converting the coordinates of the area data into the coordinates of the map image. Specifically, as shown in FIG. 16, the processing unit 22 associates the map image index with the area data index, and the coordinates of the map image of the index indicating the same feature point and the coordinates of the area data Create a list that associates. The list shown in FIG. 16 is data in which the coordinates of the map image and the coordinates of the area data are associated with the data surrounded by the frame 150. The processing unit 22 creates a coordinate conversion table based on the data surrounded by the frame 150.

  After creating the coordinate conversion table in step S24, the processing unit 22 performs processing as step S26. That is, the processing unit 22 performs processing for processing the area data based on the created conversion table. Specifically, as shown in the list of FIG. 17, the processing unit 22 converts the coordinates of a predetermined target stored in the area data into the coordinates of the map image. Here, in the list shown in FIG. 17, the coordinates of the display position when the destination A, the landmark A, and the landmark B included in the area data are displayed on the map image are calculated. That is, as indicated by a frame 152, the coordinates of the area data are converted into the coordinates of the map image.

  After performing the processing in step S26, the processing unit 22 determines in step S28 whether to project the image on the map, that is, whether to project an image on the map to be imaged by the projector 34. If the processing unit 22 determines in step S28 that the image is not projected on the map (No in step S28), the processing unit 22 proceeds to step S34. If it is determined in step S28 that the image is projected on the map (Yes in step S28), the processing unit 22 creates a projection image in step S30. In other words, the processing unit 22 performs the processing process in step S26 as a projection image, and creates an image to be displayed on the map (that is, an image that can be added to the map) based on the generated image. After creating the projection image in step S30, the processing unit 22 projects the image created by the projector 34 in step S32. The processing unit 22 captures an image projection area with the camera 40 and projects the image so that the projection image overlaps a predetermined position on the map.

  If it is determined No in step S28 or if the process of step S32 is performed, the processing unit 22 determines whether to display an image, that is, to display the image on the display unit 32, as step S34. If it is determined in step S34 that the image is not displayed (No in step S34), the processing unit 22 ends the process. If it is determined that the image is to be displayed in step S34 (Yes in step S34), the processing unit 22 creates a composite image in step S36. That is, the processing unit 22 performs a processing process in step S26 as a composite image, and creates an image in which a predetermined processed image is superimposed on the map image captured in step S12 based on the generated image. After creating the composite image in step S36, the processing unit 22 projects the created composite image on the display unit 32 in step S38. When the processing unit 22 displays an image in step S38, the processing ends.

  The communication system 1 and the mobile electronic device 10 analyze the map image acquired by the camera 40 as described above, and the analyzed map image and a prestored reference (for example, latitude and longitude are associated). Compare the map data. Further, the communication system 1 and the portable electronic device 10 specify the position of the map image acquired by the camera 40, and the correspondence between each part of the map image acquired by the camera 40 and each part of the reference map data. calculate. Thereby, the processed image with respect to the image of the map acquired with the camera 40 can be produced easily. In addition, since the map data serving as the reference can be processed in correspondence with the map image acquired by the camera 40, it corresponds to the display rules (scale, degree of abstraction) of the map image acquired by the camera 40. Processed images can be created. Thus, when the processed image is superimposed on the map image acquired by the camera 40, the image can be easily viewed and understood.

  For example, the communication system 1 and the mobile electronic device 10 can display an index indicating the destination at an appropriate position even when the destination is to be displayed on the map image acquired by the camera 40 as described above. . In addition, the communication system 1 and the portable electronic device 10 project an image on a map acquired by a projector, thereby displaying a large display map or a map actually confirmed visually. Destination information can be displayed. Furthermore, the communication system 1 and the mobile electronic device 10 display the image obtained by superimposing the processed image on the acquired map on the display unit 32, thereby confirming the map and the destination even at a position away from the place where the map is taken. be able to. Also in this case, the destination and the current position can be confirmed with an image using a map that has been confirmed once (image of the photographed map). For this reason, the user can easily understand the map.

  Here, in the said embodiment, although the image process and the calculation process were performed with the portable electronic device 10, this invention is not limited to this. For example, some image processing and calculation processing may be performed by the server 2 or the server 5. In this case, the mobile electronic device 10 may send image data or data necessary for arithmetic processing to the server 2 or the server 5 and receive a result of the processing from the server 2 or the server 5. That is, the mobile electronic device 10 may perform part or all of the processing from step S14 to step S26, step S30, and step S36 in the server 2 or the server 5 in the processing of FIG. 9 described above. For example, the mobile electronic device 10 may send a captured image to the server 2 or the server 5, receive image data from the server 2 or the server 5, and display or project the received image. In addition, the portable electronic device 10 may perform modeling processing of a captured image, and then send the modeled image data to the server 2 or the server 5 and receive the processed image data. Further, the mobile electronic device 10 may send the position information to the server 2 or 5 and receive the modeled area data from the server 2 or 5.

  Moreover, in the said embodiment, although positional information was acquired based on the information acquired by communication with the GPS satellite 7, this invention is not limited to this. For example, position information may be acquired based on information acquired through communication with a base station. The target range can also be narrowed down to a certain range by acquiring position information through communication with the base station. Further, by using communication with the base station in this way, a map can be specified without using a GPS satellite, and the location of the destination can be accurately grasped. The communication system 1 preferably acquires position information and limits the target range of the map. However, the present invention is not limited to this, and without extracting area data, the reference map data and The area indicated by the map image may be specified by comparing with the map image. Further, the communication system 1 may extract the area data based on information instructed by the user by character input or the like.

  Further, the method for comparing the map image acquired by the camera 40 and the area data, and the method for specifying the map image acquired by the camera 40 are not limited to the above embodiment. Hereinafter, another example of a method for comparing with area data and a method for specifying a map image acquired by the camera 40 will be described with reference to FIGS. Here, FIGS. 18 to 21 are explanatory diagrams for explaining another example of the operation of the portable electronic device. Note that the examples shown in FIGS. 18 to 21 are also examples when the map 100 shown in FIG. 6 is taken as the map image.

  First, the mobile electronic device 10 analyzes a photographed map image and extracts a road 204 as shown in an image 200 in FIG. Thereafter, the mobile electronic device 10 extracts a point that becomes the starting point 202 from the image 200, and extracts vector information of the road 204 with reference to the starting point 202. As vector information, coordinates, angles, lengths, widths, and information of other connected vectors as starting points of the vectors are extracted. The portable electronic device 10 further associates an index 206 with the extracted vector information of the road 204. The portable electronic device 10 organizes the extracted vector information for each index 206 and forms a list as shown in FIG.

  Further, the mobile electronic device 10 extracts area data obtained by extracting a certain area from the reference map data. Thereafter, the portable electronic device 10 also extracts the extracted area data using the road 210 as a vector as shown in FIG. Thereafter, the mobile electronic device 10 compares the vector information extracted from the map image shown in FIG. 18 with the vector information of the area data shown in FIG. Thereby, the portable electronic device 10 can extract and specify the area 212 included in the map image from the area data path 210 as shown in FIG.

  In addition, as a comparison method with area data and a method for specifying a map image acquired by the camera 40, the shape of a block divided by a road is extracted, the shape of the block is compared, and the position of the map image Can also be specified. Further, characters (building name, station name, place name, road name) included in the map image may be extracted, and the position of the map image may be specified based on the name. Moreover, you may combine the various methods mentioned above.

  Further, the communication system 1 and the portable electronic device 10 can perform various processes as the processing performed after the map image is specified. That is, various images can be created as images to be displayed superimposed on the map image. For example, various images such as an image displaying a destination and an image showing a route can be created.

  When the destination is displayed, the destination may be extracted based on a character string included in the mail, such as text 230 shown in FIG. 22 or text 232 shown in FIG. Since the area data can include address data in addition to the coordinates, the location of the destination on the map image can be specified and displayed by using the address.

  In addition, as a map to be photographed, a map for guidance display set on a station or the like (a map displayed on a signboard) or a map printed on a book such as a guidebook can be used. In addition, when using the map printed on the book, the photographed image is analyzed, the telephone number or address information described on the page on which the map is displayed is acquired, the area data is specified, A destination to be displayed as a specific image may be specified.

  As described above, the portable electronic device and the method of using the portable electronic device according to the present invention include a device that displays an image such as a display, or a device that can project an image such as a projector, and takes an image. It is useful for those equipped with a device to perform.

DESCRIPTION OF SYMBOLS 10 Portable electronic device 10C Housing | casing 12 Display 13 Operation part 13A Operation key 13B Direction key and determination key 22 Processing part 22a Projector control part 22b Camera control part 22c Condition determination part 22d Posture / position calculation part 22e Image processing part 24 Storage part 25 External storage unit 30 Audio processing unit 31 Visible light irradiation device 32 Display unit 34 Projector 36 Short-range communication unit 40 Camera

Claims (11)

A housing,
A shooting unit for acquiring a map image by shooting;
A processing unit that acquires information of a specific image corresponding to the map image based on information included in the map image acquired by the photographing unit;
An image projection unit provided in the housing for projecting the specific image in a projection region;
A portable electronic device comprising:   The portable electronic device according to claim 1, wherein the processing unit acquires an image that can be added to the map image as the specific image, and causes the specific image to be projected from the image projection unit. A display unit provided on the housing for displaying an image;
The processing unit obtains an image that can be added to the map image as the specific image, and adds the specific image and the map image captured by the imaging unit to display on the display unit. The portable electronic device according to claim 1 or 2, characterized in that   The specific image associates the position information of the map image with the position information of the reference map data based on the map image and the reference map data, and the reference map data The portable electronic device according to any one of claims 1 to 3, further comprising an image created by associating the contained information with the coordinates of the map image.   5. The portable electronic device according to claim 4, wherein the specific image includes an image obtained by converting the coordinates on the reference map data in association with the coordinates of the map image. . It further has a position information acquisition unit that acquires its own position information,
The processing unit determines a range of reference map data to be compared with the map image based on the map information and the position information acquired by the position information acquisition unit. Item 6. The portable electronic device according to Item 4 or 5.   7. The processing unit according to claim 1, wherein the processing unit analyzes the map image, specifies an absolute position from each element included in the map image, and acquires a corresponding specific image from the specified absolute position. The portable electronic device according to any one of the above. A communication unit that communicates with other communication devices;
The said processing part outputs the said map image from the said communication part to another communication apparatus, and acquires a specific image from the said other communication apparatus, The any one of Claim 1 to 6 characterized by the above-mentioned. Portable electronic devices. A communication unit that communicates with other communication devices;
The processing unit analyzes the map image, specifies an absolute position of each element included in the map image, outputs information on the specified absolute position from the communication unit to another communication device, and outputs the other communication The mobile electronic device according to claim 1, wherein a specific image is acquired from the device. A method of using the portable electronic device according to any one of claims 1 to 9,
Obtain a map image by shooting a map placed in the shooting area of the shooting unit,
Based on the map image, obtain information of a specific image corresponding to the map image,
A method of using a portable electronic device, wherein the acquired specific image is projected by an image projection unit. Place the map in the shooting area,
When projecting the specific image by the image projection unit, the position of the map is confirmed by the photographing unit, and the specific image is corrected while correcting the projection position so that the map and the specific image are added together. The method of using a portable electronic device according to claim 10, wherein:

Images