JP2013258751A - Communication terminal and information transmission source identifying program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication terminal capable of easily identifying a visible light communication source.SOLUTION: A portable terminal 10 includes a camera control circuit 36 and an image sensor 38 and photographs a through image and a through still image including a visible light communication source. The through still image including the visible light communication source is displayed by an LCD monitor 26. A CPU 20 raster-scans the through still image to identify the position of the visible light communication source included in the through still image and displays an icon (IC) corresponding to the visible light communication source on the LCD monitor 26 on the basis of the identified position.

Description

  The present invention relates to a communication terminal, and more particularly to a communication terminal that performs data communication, for example.

  2. Description of the Related Art Conventionally, communication terminals that perform data communication, for example, are known, and an example of this type of device is disclosed in Patent Document 1. This background information communication system includes a PC server, a control unit, an information unit, and a portable terminal, and the PC server issues a command to the control unit to output data to the information unit. A plurality of optical beacons are arranged in this information unit, and each optical beacon transmits ID data including predetermined data to a portable terminal by emitting ID data for identifying itself instead of a blinking pattern. To do. And a portable terminal can receive several ID data by image | photographing several ID data changed into the blink pattern of each optical beacon with an ID recognition camera.

JP2004-349766

  However, in the information communication system in Patent Document 1, when the mobile terminal captures a plurality of optical beacons, it is not possible to narrow down (select) one optical beacon that receives ID data. Also, there is a method of determining an optical beacon that receives ID data by displaying a list of optical beacons on an image and displaying a GUI capable of selecting an arbitrary optical beacon on a portable terminal as in a conventional wireless LAN. However, the following new problems occur.

  First, in an interface in a mobile terminal, since the GUI is often operated using a plurality of operation keys, the operation of the GUI is complicated. Furthermore, since the displayed list and the optical beacon are associated with each other, information included in the list and displayed information increase, and it becomes difficult to understand the operation of the GUI.

  Therefore, a main object of the present invention is to provide a novel communication terminal and an information source specifying program applied to a processor of such a communication terminal.

  Another object of the present invention is to provide a communication terminal capable of easily specifying a visible light communication source and an information transmission source specifying program applied to a processor of such a communication terminal.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate the corresponding relationship with the embodiments described in order to help understanding of the present invention, and do not limit the present invention.

  According to a first aspect of the present invention, there is provided a photographing means for photographing an image including a light source for information transmission, a display device for displaying an image photographed by the photographing means, a specifying means for specifying a position of a light source included in the image, and a specifying means The communication terminal includes icon display means for displaying an icon at the position of the light source.

  In the first invention, the imaging means (36, 38) of the communication terminal (10) captures an image (through image, through still image) including an information transmission light source (visible light communication source) existing in the object scene. The photographed image is displayed on a display device (26) such as an LCD monitor. The specifying means (20, S11) specifies the position (coordinates) of the light source in the image displayed on the display device, and the icon display means (20, S17) displays an icon based on the specified position of the light source. indicate. As a result, an icon corresponding to the light source is displayed on the display device.

  According to the first aspect, by displaying the icon corresponding to the light source for performing visible light communication, the user can easily identify the visible light communication source. Note that the icons of the present invention include graphics, words, symbols, numbers, or any combination thereof. In addition to an icon linked to the program, a simple display not linked to the program may be used.

  The second invention is dependent on the first invention, and the related information display for displaying the related information based on the optical signal transmitted from the light source set by the receiving means and the setting means for receiving the optical signal from the light source. Means.

  In the second invention, the receiving means including the image sensor, the camera control circuit, and the CPU receives the optical signal including the index information from the light source by photographing the light source. The related information display means (20, S15, S17) displays related information indicating the type of received information (data) in the type display area (60) in the icon based on the index information included in the optical signal. . The information types include text data, music data, image data, and the like.

  According to the second aspect, the user can easily grasp the type of information transmitted from the visible light communication source.

  A third invention is dependent on the first invention or the second invention, and is provided in the display device, the touch operation detecting means for detecting the touch operation in the touch reaction area, and the touch detected by the touch operation detecting means. When the operation is an operation of selecting an icon, setting means is further provided for setting so as to acquire an optical signal from a light source corresponding to the icon.

  In the third invention, the touch operation detecting means (34) provided in the display device detects touch operations such as touch and slide in the touch reaction area. And a setting means (20, S47) will set so that the optical signal from the light source corresponding to the selected icon may be acquired, if the touch operation which selects the displayed icon is performed.

  According to the third invention, it is possible to arbitrarily select a light source for acquiring information by visible light communication.

  A fourth invention is dependent on the third invention, and further comprises data display means for displaying data corresponding to the optical signal transmitted from the light source set by the setting means.

  In the fourth invention, the data display means (20, S49) displays the contents of the text data by the text viewer function or the like when the set visible light communication source is transmitting the text data.

  According to the fourth aspect, the user can check the content of information (data) transmitted from the visible light communication source.

  A fifth invention is dependent on any one of the first to fourth inventions, and the light source emits visible light.

  In the fifth invention, visible light (54a, 54b, 54c) is radiated from a light source such as a fluorescent lamp and LED illumination.

  According to the fifth aspect, the user can recognize the visible light existing in the object scene and photograph the information transmission light source.

  According to a sixth aspect of the present invention, there is provided a communication terminal processor including an imaging unit (36, 38) for imaging an image including an information transmission light source and a display device (26) for displaying an image captured by the imaging unit. An information transmission source specifying program that functions as specifying means (S11) for specifying the position of the included light source and icon display means (S17) for displaying an icon at the position of the light source specified by the specifying means.

  In the sixth invention, similarly to the first invention, by displaying an icon corresponding to the light source for performing visible light communication, the user can easily identify the position of the visible light communication source. become.

  According to the present invention, the user can easily identify the visible light communication source by the icon displayed on the display device.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is a block diagram showing a communication terminal of the present invention. FIG. 2 is an illustrative view showing an appearance of the communication terminal shown in FIG. FIG. 3 is an illustrative view showing one example of raster scan processing by the CPU shown in FIG. FIG. 4 is an illustrative view showing a position display example of a through image displayed on the LCD monitor shown in FIG. FIG. 5 is an illustrative view showing one example of data displayed on the LCD monitor shown in FIG. FIG. 6 is an illustrative view showing types of icons displayed on the LCD monitor shown in FIG. FIG. 7 is an illustrative view showing one example of a configuration of visible light communication source data stored in the RAM shown in FIG. FIG. 8 is an illustrative view showing one example of a memory map of the RAM shown in FIG. FIG. 9 is a flowchart showing the visible light communication source search process of the CPU shown in FIG. FIG. 10 is a flowchart showing the light source selection processing of the CPU shown in FIG.

Referring to FIG. 1, communication terminal 10 includes a CPU (also referred to as a processor or a computer) 20, a key input device 22, and a touch panel (touch operation detecting means) 34 controlled by touch panel control circuit 32. The CPU 20 controls the radio communication circuit 14 corresponding to the CDMA system and outputs a call signal. The output call signal is transmitted from the antenna 12 and transmitted to the mobile communication network including the base station. When the other party performs a response operation, a call ready state is established.

  When a call end operation is performed by the key input device 22 or the touch panel 34 after the transition to the call enabled state, the CPU 20 controls the wireless communication circuit 14 to transmit a call end signal to the other party. After transmitting the call end signal, the CPU 20 ends the call process. Also when the call end signal is received from the other party first, the CPU 20 ends the call process. Also, the CPU 20 ends the call process when a call end signal is received from the mobile communication network regardless of the call partner.

  When the call signal from the other party is captured by the antenna 12 while the communication terminal 10 is activated, the wireless communication circuit 14 notifies the CPU 20 of the incoming call. The CPU 20 controls the LCD monitor 26 which is a display device by the LCD driver 24 and causes the LCD monitor 26 to display the transmission source information described in the incoming call notification. Further, the CPU 20 outputs a ring tone from an incoming call notification speaker (not shown).

  In the call ready state, the following processing is executed. The modulated audio signal (high frequency signal) sent from the other party is received by the antenna 12. The received modulated audio signal is subjected to demodulation processing and decoding processing by the wireless communication circuit 14. The received voice signal thus obtained is output from the speaker 18. The transmitted voice signal captured by the microphone 16 is subjected to encoding processing and modulation processing by the wireless communication circuit 14. Thus, the generated modulated audio signal is transmitted to the other party using the antenna 12 as described above.

  The touch panel 34 is a pointing device for the user to indicate an arbitrary position within the screen of the LCD monitor 26. When the touch panel 34 is operated by pressing, sliding (striking), or touching the upper surface with a finger, the touch panel 34 detects the operation. When the touch panel 34 detects a touch, the touch panel control circuit 32 specifies the position of the operation and outputs coordinate data of the operated position to the CPU 20. That is, the user can input an operation direction, a figure, or the like to the communication terminal 10 by pressing, sliding (rubbing), or touching the upper surface of the touch panel 34 with a finger.

  The touch panel 34 is a method called a capacitance method that detects a change in capacitance between electrodes generated when a finger approaches the surface of the touch panel 34, and one or more fingers touch the touch panel 34. Detect that. Specifically, the touch panel 34 has a projection-type capacitance method that detects a change in capacitance between electrodes caused by the approach of a finger by forming an electrode pattern on a transparent film or the like. It has been adopted. As the detection method, a surface type capacitance method may be employed, or a resistance film method, an ultrasonic method, an infrared method, an electromagnetic induction method, or the like may be used.

Here, an operation in which the user touches the upper surface of the touch panel 34 with a finger is referred to as “touch”. On the other hand, the operation of releasing the finger from the touch panel 34 is referred to as “release”. The operation of rubbing the surface of the touch panel 34 is referred to as “slide”. The coordinates indicated by the touch are referred to as “touch points”, and the coordinates of the operation end position indicated by the release are referred to as “release points”. Further, an operation in which the user touches the upper surface of the touch panel 34 and subsequently releases it is referred to as “touch and release”. An operation performed on the touch panel 34 such as touch, release, slide, and touch-and-release is generally referred to as “touch operation”. The operation on the touch panel 34 is not limited to a finger, and may be performed with a stick having a thin tip such as a pen. In addition, a dedicated touch pen or the like may be provided to perform the operation. When the touch point is touched with a finger, the center of gravity in the area of the finger touching the touch panel 34 becomes the touch point.

  The communication terminal 10 can execute a camera application. The camera application stores photographic image data focused on the subject by autofocus processing. Specifically, when an operation for executing a camera application is performed by the key input device 22 or the touch panel 34, the CPU 20 gives a command necessary for executing the camera application to the camera control circuit 36. The camera control circuit 36 controls the image sensor 38 and the focus lens 40 to convert the optical image of the show field acquired by the image sensor 38 into photographic image data. Then, the CPU 20 converts the photographic image data obtained from the camera control circuit 36 and focused on the subject into compressed image data and stores it in the flash memory 28. In this camera application, the size of a photograph image to be taken can be selected from WQVGA (240 × 400), VGA (640 × 480), and UXGA (1600 × 1200). Of course, the size of the photographic image taken by the camera application of the present invention is not limited to these, and may be other sizes as appropriate.

  Further, in a state where the camera application is executed, a process of displaying a real-time moving image of the object scene (hereinafter referred to as a through image) on the LCD monitor 26 is performed. Specifically, the CPU 20 activates an image sensor driver built in the camera control circuit 36 and instructs the image sensor driver to perform an exposure operation and a charge readout operation corresponding to a designated readout region.

  The image sensor driver performs exposure of the imaging surface of the image sensor 38 and reading of electric charges generated by the exposure. As a result, a raw image signal is output from the image sensor 38. The raw image signal output from the image sensor 38 is input to the camera control circuit 36. The camera control circuit 36 performs processing such as color separation, white balance adjustment, and YUV conversion on the input raw image signal to generate image data in YUV format. The CPU 20 receives YUV format image data. In YUV format image data, Y means luminance, U means a color difference obtained by subtracting luminance from blue, and V means a color difference obtained by subtracting luminance from red. That is, the YUV format image data is composed of luminance signal (Y) data, blue color difference signal (U) data, and red color difference signal (V) data.

  The YUV format image data is temporarily stored in the RAM 30 by the CPU 20. When the YUV format image data is stored in the RAM 30, the stored YUV format image data is provided from the RAM 30 to the LCD driver 24 via the CPU 20. At the same time, the CPU 20 issues a thinning-out read command corresponding to the camera setting display position stored in the RAM 30 to the LCD driver 24. Then, the LCD driver 24 outputs YUV format image data to the LCD monitor 26 in accordance with the thinning readout command issued from the CPU 20. Therefore, a low-resolution through image representing the object scene is reproduced on the LCD monitor 26.

  Note that the through image update cycle is 1000 fps, and the image sensor 38 is a CMOS image sensor that can output a raw image signal at high speed.

2A and 2B are illustrative views showing the appearance of the communication terminal 10. 2A is a front external view of the communication terminal 10, and FIG. 2B is a back external view of the communication terminal 10. FIG. Referring to FIG. 2A, communication terminal 10 has a case C formed in a plate shape. A microphone 16 and a speaker 18 (not shown in FIG. 2A) are built in the case C. The opening OP2 leading to the built-in microphone 16 is provided on one main surface in the length direction of the case C, and the opening OP1 leading to the built-in speaker 18 is provided on the other main surface in the length direction of the case C. That is, the user listens to the sound output from the speaker 18 through the opening OP1, and inputs the sound to the microphone 16 through the opening OP2.

  The key input device 22 includes three types of keys, a call key 22a, a menu key 22b, and an end call key 22c. Each key is provided on the main surface of the case C. The LCD monitor 26 is attached so that the monitor screen is exposed on the main surface of the case C. Further, a touch panel 34 is provided on the upper surface of the LCD monitor 26.

  The user performs a response operation by operating the call key 22a, and performs a call end operation by operating the call end key 22c. Further, the user operates the menu key 22b to display the menu screen on the LCD monitor 26. Then, the power on / off operation of the communication terminal 10 can be performed by long-pressing the end call key 22c.

  Referring to FIG. 2B, the image sensor 38 and the focus lens 40 (not shown in FIG. 2B) are built in the case C, and the openings leading to the built-in image sensor 38 and the focus lens 40 are provided. OP3 is provided on the other side of the other side in the length direction of the case C. That is, the user can confirm the through image including the subject on the LCD monitor 26 by directing the opening OP3 provided on the back side of the LCD monitor 26 toward the subject. The opening OP3 may be covered with a transparent plastic cover or the like.

  Here, in this embodiment, visible light communication is performed by photographing a visible light communication source, which is an information transmission light source, by a camera control circuit 36 and an image sensor 38 which are photographing means. Note that the camera control circuit 36 and the image sensor 38 controlled by the CPU 20 function as receiving means in visible light communication.

  Specifically, it is determined whether or not the light source that emits (emits) light with a luminance equal to or higher than a predetermined value included in the through image is a visible light communication source. Displayed on the monitor 26. And the visible light communication source which receives information (data) can be determined by selecting the displayed icon by touch operations, such as touch and release.

  First, a procedure for detecting a visible light communication source included in a through image will be described. Here, in order to detect a visible light communication source in the through image, a raster scan is performed from the upper left end, and visible light having a luminance of a predetermined value or more is detected. In addition, the through image to be raster scanned is not a moving image but an arbitrarily selected still image. Hereinafter, each still image constituting the through image is referred to as a through still image.

  FIG. 3A is an illustrative view showing a display example of the LCD monitor 26 that displays a through image taken by the camera application. Referring to FIG. 3A, LCD monitor 26 includes a status display area 50 and a function display area 52. In the status display area 50, the radio wave reception status by the antenna 12, the remaining battery capacity of the rechargeable battery, the current date and time, and the like are displayed. In the function display area 52, a through image captured by the camera application is displayed.

The through image illustrated in FIG. 3A includes a plurality of light sources, and each of the plurality of light sources emits visible light including an optical signal. Hereinafter, visible light emitted from each of the plurality of light sources is referred to as visible light 54a, visible light 54b, and visible light 54c. In addition, FIG.
Visible light 54a and visible light 54b shown in FIG. However, the luminance of the visible light 54c is lower than the luminance of the visible light 54a and the visible light 54b. In addition, although a some light source is a fluorescent lamp, if it is illumination which can be utilized for visible light communications, such as LED illumination and organic EL illumination, another lighting fixture may be sufficient.

  In FIG. 3A, visible light 54a, visible light 54b, and visible light 54c are shown as circles for simplicity. In addition, the state display area 50, the function display area 52, and the visible light 54a-54c shown in FIG. 3A are the same in other drawings, and thus detailed description is omitted in the other drawings for the sake of simplicity. To do.

  Then, a raster scan is performed on the through still image, and a light source that emits light with a luminance equal to or higher than a predetermined value is searched. Further, when a light source that emits light with a luminance of a predetermined position or higher is found by raster scanning, it is determined by concentrated sampling whether the light source is a visible light communication source. In the raster scan, the visible light having the luminance equal to or higher than the predetermined value is searched by determining whether the luminance is higher than the predetermined value based on the luminance signal data for each pixel. For example, if the luminance of the visible light 54a is equal to or higher than a predetermined position, when searching up to the upper part S of the visible light 54a (see FIG. 3B), concentrated sampling is performed within a certain range from the upper part S of the visible light 54a. Is called.

  FIG. 3B is an illustrative view showing a state in which concentrated sampling is performed in region A in FIG. Referring to FIG. 3B, a rectangular concentrated sampling area CSA is set such that the upper part S indicating the current search position by the raster scan is the midpoint of a certain side. That is, the concentrated sampling area CSA is set with the upper part S as a reference. Then, the through image is updated only in the concentrated sampling area CSA, and it is determined whether or not the visible light 54a is blinking at a cycle necessary for visible light communication.

  Note that the updated through image is not displayed on the LCD monitor 26 only in the concentrated sampling area CSA. Further, although the shape of the concentrated sampling area CSA is a quadrangle, it may be another figure. Furthermore, in the visible light communication, communication is possible if the blinking of visible light can be identified, and therefore it is not necessary that all visible light is included in the concentrated sampling area CSA.

  And if it blinks with the period required for visible light communication, while acquiring the index information of the transmitted information (data), the center of gravity of the area where the luminance above the predetermined position is detected in the concentrated sampling area CSA Is calculated. This is because an icon is displayed based on the calculated coordinates of the center of gravity.

  For example, in FIG. 3B, if the visible light 54a blinks at a period necessary for visible light communication, the center of the visible light 54a is calculated as the center of gravity G and stored in the RAM 30 together with the acquired index information. . In the concentrated sampling area CSA, when the concentrated sampling is completed, an icon ICa (see FIG. 4) is displayed based on the calculated center of gravity G and the acquired index information.

  As described above, the user can recognize the visible light existing in the object scene and photograph the light source. And the user can recognize the visible light communication source for performing visible light communication in the through image including a plurality of light sources.

  When the icon ICa is drawn, the raster scan is restarted from the upper part S. At this time, the area coordinates of the concentrated sampling area CSA are temporarily stored in the RAM 30 so that the concentrated sampling area CSA is not raster scanned. When the raster scan is resumed, the latest through still image is used. In other words, any through still image selected to start the raster scan first is updated to the latest through still image.

  FIG. 4 shows a through image displayed on the LCD monitor 26 displayed after the raster scan is completed. Referring to FIG. 4, an icon ICa is displayed overlaid on visible light 54a, and an icon ICb is displayed overlaid on visible light 54b. That is, on the LCD monitor 26, the light source that emits the visible light 54a and the icon ICa are displayed in association with each other, and the light source that emits the visible light 54b and the icon ICb are displayed in association with each other. When there is no need to distinguish between the icon ICa and the icon ICb, they are referred to as icon ICs.

  Further, since the visible light 54c does not have a luminance higher than a predetermined value, the icon IC is not displayed in an overlapping manner. However, when the light source that emits the visible light 54c is a visible light communication source, when the user approaches the light source that emits the visible light 54c and the luminance of the visible light 54c included in the through image is greater than or equal to a predetermined value. The icon IC is displayed in an overlapping manner.

  When the raster scan is completed, the raster scan is performed again on the arbitrarily selected through still image. This is because the scene where the user shoots constantly changes, so the position for displaying the light source included in the through image also changes. That is, the display position of the icon IC to be displayed is also changed in accordance with the change in the position where the light source included in the through image is displayed.

  Here, when a touch-and-release touch operation is performed on the icon ICa shown in FIG. 4, visible light communication is performed with a visible light communication source that emits visible light 54a corresponding to the icon ICa, and the acquired information (data ) To switch the display of the LCD monitor 26.

  FIG. 5 is an illustrative view showing an embodiment in which data acquired by visible light communication is displayed on the LCD monitor 26. Referring to FIG. 5, when the text data of weather forecast is transmitted from a visible light communication source that emits visible light 54a, when the icon ICa is touched and released, the function display area 52 displays the weather forecast. The text (character string) is displayed. Specifically, intensive sampling is performed based on the coordinates of the center of gravity G used to display the icon ICa. When text data transmitted from a visible light communication source that emits visible light 54a is received, the text data is temporarily stored in the RAM 30, and the temporarily stored text data is displayed on the LCD monitor 26 by the text viewer function of the communication terminal 10. . Thus, the user can arbitrarily select a plurality of icon ICs. Then, the user can check the contents of information (data) transmitted from the visible light communication source.

  Note that when the LCD monitor 26 is operated to end the text viewer function shown in FIG. 5, the display returns to the through image display shown in FIG.

  In this embodiment, the type (related information) of the displayed icon IC changes according to data received by visible light communication. Hereinafter, the types of icon ICs will be described with reference to FIGS.

FIGS. 6A to 6C are illustrative views showing an example of image data indicating the type of icon IC. Referring to FIG. 6A, alphabet “T” is displayed in type display area 60a in icon IC. This is “T” which is an acronym of text data, and the icon IC shown in FIG. 6A indicates to the user that the type of data obtained by visible light communication is text data. . Subsequently, referring to FIG. 6B, a triangle is drawn in the type display area 60b of the icon IC. This means a triangle drawn on a playback key such as a music player. That is, the icon IC shown in FIG. 6B indicates to the user that the type of data obtained by visible light communication is music data. Next, FIG.
Referring to (C), a symbol is drawn in the type display area 60c in the icon IC. That is, the icon IC shown in FIG. 6C indicates to the user that the type of data obtained by visible light communication is image data. When there is no need to distinguish each of the type display areas 60a to 60c, the type display areas 60a to 60c will be referred to as a type display area 60.

  For example, referring to FIG. 4, alphabet “T” is displayed in type display area 60 included in icon ICa and icon ICb. This indicates that each visible light communication source that emits visible light 54a and visible light 54b corresponding to each of icon ICa and icon ICb is transmitting text data. Thus, the user can easily grasp the type of information (data) transmitted from the visible light communication source by confirming the icon IC.

  Note that the icon IC is not limited to only three types, and may include four or more types of icon images. Moreover, the character string, figure, and symbol displayed (drawn) in the type display area 60 shown in FIGS. 6A to 6C may be other character strings, graphics, and symbols. If the type of data obtained by visible light communication cannot be specified, an icon IC that does not include the type display area 60 is displayed.

  Here, the type of icon IC to be displayed is determined based on visible light communication source data stored in the RAM 30. FIG. 7 is an illustrative view showing a data structure of visible light communication source data. Referring to FIG. 7, the visible light communication source data includes a “NO” column, a “coordinate” column, and an “index information” column. The “NO” column stores a numeric string indicating the order of discovery found by the search, and “1” indicates that the visible light communication source is first discovered. The column of “coordinates” indicates the position of the visible light communication source at the display coordinates in the LCD monitor 26. That is, the calculated coordinates of the center of gravity are stored in the column of “coordinates”. The origin of the display coordinates of the LCD monitor 26 and the touch position coordinates of the touch panel 34 is the upper left corner. That is, the abscissa increases as it proceeds from the upper left corner to the upper right corner, and the ordinate increases as it proceeds from the upper left corner to the lower left corner.

  The “index information” column further includes an “extension” column and the like. In the “extension” column, an extension indicating the type of data transmitted from the visible light communication source is stored. For example, “txt” which is an extension indicating text data, “wav” which is an extension indicating music data, and the like are stored in the “extension” column. Although not shown, “jpg”, which is an extension indicating image data, is also stored in the “extension” column.

  For example, when the visible light 54a shown in FIG. 4 corresponds to “1” in the “NO” column, the coordinates of (X, Y) are the coordinates of the center of gravity G (see FIG. 3B), It can be seen that the index information transmitted from the visible light communication source that emits the visible light 54a includes the extension data “txt”. The type of icon IC to be displayed is determined based on the “extension” column. That is, as shown in FIG. 4, the icon IC shown in FIG. 6A is displayed based on the coordinates (X, Y) of the center of gravity G. The index information includes information transmitted from the visible light communication source. The file name of (data), metadata, etc. may be included. Further, the extension indicating music data is not limited to “wav” but may include “mp3” and “mid”. Furthermore, the extension indicating image data may include “gif” and “bmp”.

Here, a plurality of layers constituting the display of the LCD monitor 26 will be described. Specifically, two layers (the lowermost layer and the uppermost layer) are provided in an overlapping manner, and in the virtual space, the uppermost layer is provided on the starting point side (user), and the lowermost layer is arranged in a direction away from the starting point. . In this embodiment, a through image is displayed on the bottom layer, and an icon IC is displayed on the top layer. As a result, as shown in FIG. 4, the icon ICa can be easily drawn so as to overlap the visible light 54a. Note that the number of layers constituting the display of the LCD monitor 26 may be three or more.

  FIG. 8 is an illustrative view showing a memory map of the RAM 30. Referring to FIG. 8, a memory map 300 of RAM 30 includes a program storage area 302 and a data storage area 304. A part of the program and data is read from the flash memory 28 all at once or partially and sequentially as necessary, stored in the RAM 30, and then processed by the CPU 20 or the like.

  The program storage area 302 stores a program for operating the communication terminal 10. A program for operating the communication terminal 10 includes a visible light communication source search program 310, a light source selection program 312 and the like.

  The visible light communication source search program 310 is a program for searching for a visible light communication source by performing a raster scan on a through still image. The light source selection program 312 is a program for acquiring information (data) from a visible light communication source corresponding to the icon IC when the displayed icon IC is selected.

  Although illustration is omitted, programs for operating the communication terminal 10 include a camera application program, a text viewer function program, a music player function program, an image viewer function program, and the like.

  In the data storage area 304, a touch buffer 320, a through image buffer 322, a raster scan buffer 324, a concentrated sampling buffer 326, and a visible light communication buffer 328 are provided. The data storage area 304 stores touch coordinate map data 330, icon data 332, and visible light communication source data 334, and a touch flag 336 is provided.

  The touch buffer 320 is a buffer for temporarily storing an input result such as a touch detected by the touch panel 34, and temporarily stores, for example, coordinate data of a touch point and a release point. The through image buffer 322 is a buffer in which YUV format image data read from the camera control circuit 36 is temporarily stored (stored once). The raster scan buffer 324 is a buffer for temporarily storing the through still image and coordinates indicating the current search position when performing a raster scan on the through still image.

  The concentrated sampling buffer 326 is a buffer for temporarily storing coordinates serving as a reference for setting the concentrated sampling area CSA and coordinates (area coordinates) indicating the area of the concentrated sampling area CSA. The visible light communication buffer 328 is a buffer for temporarily storing information (data) acquired by visible light communication.

  The touch coordinate map data 330 is data for associating coordinates such as a touch point on the touch panel 34 specified by the touch panel control circuit 32 with a display position of the LCD monitor 26. That is, the CPU 20 can associate the result of the touch operation performed on the touch panel 34 with the display on the LCD monitor 26 based on the touch coordinate map data 330. Icon data 332 is image data of the icons shown in FIGS.

The visible light communication source data 334 is the visible light communication source data shown in FIG. 7, and is composed of coordinates and index information. The touch flag 336 is a flag for determining whether or not the touch panel 34 is touched (touched). For example, the touch flag 336 is composed of a 1-bit register. When the touch flag 336 is established (turned on), the data value “1” is set in the register, and when the touch flag 336 is not established (turned off), the data value “0” is set in the register.

  Although illustration is omitted, the data storage area 304 stores image files and the like, and is provided with other counters and flags necessary for the operation of the communication terminal 10. Each flag is set to “0” in the initial state.

  The CPU 20 executes a plurality of tasks including a visible light source search process shown in FIG. 9 and a light source selection process shown in FIG. 10 in parallel under the control of a real-time OS such as Symbian or Linux (registered trademark).

  For example, when the user performs an operation of acquiring a through image, the through image is acquired in step S1. That is, the YUV format image data output from the camera control circuit 36 is temporarily stored in the through image buffer 322. Subsequently, in step S3, the through still image is raster scanned. That is, a through still image arbitrarily selected from the through images temporarily stored in the through image buffer 322 is temporarily stored in the raster scan buffer 324, and raster scanning is started on the through still image.

  Subsequently, in step S5, it is determined whether or not luminance of a predetermined value or more is detected. That is, it is determined by raster scanning whether or not the brightness of each pixel of the through still image is equal to or higher than a predetermined value. If “NO” in the step S5, that is, if a luminance of a predetermined value or more cannot be detected, the process proceeds to a step S21. On the other hand, if “YES” in the step S5, that is, if a luminance of a predetermined value or more is detected, concentrated sampling is performed in a step S7. That is, as shown in FIGS. 3A and 3B, a concentrated sampling area CSA is set based on the coordinates of a pixel in which a luminance of a predetermined value or more is detected, and the through image is updated in the concentrated sampling area CSA. To do.

  Subsequently, in step S9, it is determined whether the source is a visible light communication source. That is, it is determined whether or not the light source included in the set concentrated sampling area CSA is blinking at a frequency for performing visible light communication. If “NO” in the step S9, that is, if it is not a visible light communication source, the process proceeds to a step S21. On the other hand, if “YES” in the step S9, that is, if the visible light communication is performed, the position of the visible light communication source is calculated in a step S11. For example, as shown in FIG. 3B, the coordinates of the center of gravity G are calculated in the concentrated sampling area CSA as the position of the visible light communication source. In step S13, it is determined whether there is a visible light communication source nearby. That is, it is determined whether or not the coordinates of the center of gravity calculated in step S11 are included in the distance within the threshold in each coordinate stored in the column of “coordinates” in the visible light communication source data 344.

  Specifically, the distance between the coordinates of the center of gravity calculated in step S11 and each coordinate stored in the column of “coordinates” in the visible light communication source data 344 is calculated by the three-square theorem. And it is judged whether each of the calculated distance is below a threshold value. In addition, CPU20 which performs the process of step S11 functions as a specific means.

If “NO” in the step S13, that is, if there is no visible light communication source nearby, the index information is acquired in a step S15. That is, the index information transmitted from the visible light communication source is received, and the coordinates of the center of gravity calculated in step S11 and the received index information are stored as visible light communication source data 344. Subsequently, in step S17, an icon is displayed based on the index information. That is, in the “extension” column in the visible light communication source data 334, the extension stored last is read, and the type of icon is determined based on the read extension.

  For example, referring to FIG. 7, if the column in which “1” is stored in the “NO” column is the last stored row, the last stored extension is “txt”, and the CPU 20 The type of icon IC is determined to be a text icon IC shown in FIG. Furthermore, since the coordinates (X, Y) of the center of gravity G are stored in the column of “coordinates” corresponding to “txt”, the icon ICa is displayed on the LCD monitor 26 as shown in FIG. .

  The CPU 20 that executes the process of step S17 functions as an icon display unit, and the CPU 20 that executes the processes of steps S15 and S17 functions as a related information display unit.

  If “YES” in the step S13, that is, if there is a visible light communication source nearby, the coordinates of the nearby visible light communication source are updated in a step S19, and the process proceeds to a step S17. That is, it is determined that the nearby visible light communication source is a visible light communication source detected by the previous raster scan. Therefore, in step S19, the index information is not acquired again, and the coordinates of the visible light communication source are updated. For example, if the newly calculated coordinates are (X ′, Y ′) and the distance from the coordinates (X, Y) shown in FIG. 7 is within a threshold, the coordinates of (X, Y) are (X ′, Y ′). , Y ′).

  In step S21, it is determined whether or not the raster scan is completed. That is, it is determined whether or not the coordinates of the pixel for determining whether or not the luminance is equal to or higher than a predetermined value are the coordinates of the lower right corner of the display coordinates of the LCD monitor 26. If “NO” in the step S21, that is, if the raster scan is not ended, the coordinates indicating the current search position by the raster scan are read from the raster scan buffer 324, and the process returns to the step S3. In step S3, the raster scan is restarted from the read coordinates. On the other hand, if “YES” in the step S21, that is, if the raster scan is ended, the process returns to the step S1.

  FIG. 10 is a flowchart showing a light source selection process that is processed in parallel with the visible light communication source search process described above. CPU20 judges whether it was touched by step S41. That is, it is determined whether or not the touch flag 336 is turned on. If “NO” in the step S41, that is, if the touch flag 336 is turned off, the process of the step S41 is repeatedly executed. On the other hand, if “YES” in the step S41, that is, if the touch flag 336 is turned on, the touch point is stored in a step S43. That is, the touch point detected when the touch flag 336 is turned on is temporarily stored in the touch buffer 320.

  Subsequently, in step S45, it is determined whether or not the icon IC has been operated. That is, it is determined whether or not the touch point is included in the display coordinates of the icon IC. If “NO” in the step S45, that is, if the icon IC is not operated, the process proceeds to a step S51. On the other hand, if “YES” in the step S45, that is, if the icon IC is operated, data is acquired from the light source corresponding to the icon IC selected in the step S47. For example, when the icon ICa shown in FIG. 4 is operated, a concentrated sampling area CSA is newly set, and information transmitted from a visible light communication source corresponding to the icon ICa is received. Then, the acquired data is temporarily stored in the visible light communication buffer 328. Note that the display area of the icon IC may be a concentrated sampling area CSA. The CPU 20 that executes the process of step S47 functions as a setting unit.

Subsequently, in step S49, processing according to the acquired data is performed, and the light source selection processing is terminated. That is, the data temporarily stored in the visible light communication buffer 328 is read and displayed on the LCD monitor 26. For example, if the acquired (received) data is text data, the contents of the acquired text data are displayed on the LCD monitor 26 by the text viewer function. If the acquired data is music data, the music data is played back by the music player function. Further, if the acquired data is image data, the image data is displayed on the LCD monitor 26 by the image viewer function. In addition, CPU20 which performs the process of step S49 functions as a data display means.

  If “NO” is determined in the step S45, a process corresponding to the touch is executed in a step S51. For example, the through image is enlarged or reduced with the coordinate indicated by the touch point as the center.

  In this embodiment, the icon IC is displayed every time a visible light communication source is found. However, the icon IC may be displayed collectively after the raster scan is completed. Specifically, step S17 in the visible light communication source search process shown in FIG. 9 may be executed after “YES” is determined in step S21. Thus, after the raster scan is completed, each icon IC is displayed based on the index information.

  As can be seen from the above description, the mobile terminal 10 includes the camera control circuit 36 and the image sensor 38 and captures a through image and a through still image including a visible light communication source. Further, the through still image including the visible light communication source is displayed on the LCD monitor 26. The CPU 20 performs a raster scan on the through still image, specifies the position of the visible light communication source included in the through still image, and sets the icon IC corresponding to the visible light communication source on the LCD monitor 26 based on the specified position. To display.

  As a result, the user can recognize the icon IC corresponding to the visible light communication source for performing visible light communication, and can easily identify the visible light communication source.

  In this embodiment, the through image update cycle is set to 1000 fps, but may be set to 60 fps in order to reduce the processing load on the CPU 20, and the update cycle may be set to 1000 fps only when concentrated sampling is performed.

  Further, by providing the image sensor 38 capable of photographing infrared light, visible light communication may be performed using a light source that emits infrared light. Further, the image sensor 38 may be a CCD image sensor instead of a CMOS image sensor.

  Further, header information including extension information may be received instead of the index information. Further, if the index information including the extension information is not transmitted from the visible light communication source, the type of the icon IC may be determined from the file name including the extension.

  Further, the communication method of the communication terminal 10 is not limited to the CDMA method, but may be a W-CDMA method, a TDMA method, a PHS method, a GSM (registered trademark) method, or the like. Not only the communication terminal 10 but also a portable information terminal such as a PDA (Personal Digital Assistant) provided with a camera application may be used. Furthermore, visible light communication may be performed by attaching a WEB camera, a mobile camera, or the like to the communication terminal 10 that does not include a camera application and installing camera application software.

10 Communication terminal 20 CPU
22 Key input device 26 LCD monitor 30 RAM
34 Touch Panel 36 Camera Control Circuit 38 Image Sensor

Claims (6)

Photographing means,
A display device for displaying an image photographed by the photographing means;
Specifying means for specifying a position of an information transmission light source that emits light with a luminance of a predetermined level or more included in the image; and an icon display means for displaying an icon at the position of the information transmission light source specified by the specification means, Communication terminal.   2. The communication terminal according to claim 1, wherein the display unit displays an information transmission light source that emits light with a luminance less than a predetermined level in distinction from an information transmission light source that emits light with a luminance of the predetermined level or higher.   The communication terminal according to claim 1, wherein the display unit displays the icon at the position of the information transmission light source in the image so as to overlap the information transmission light source. Receiving means for receiving an optical signal from the information transmission light source, and related information display means for displaying related information based on the optical signal transmitted from the information transmission light source specified by the specifying means; The communication terminal according to any one of claims 1 to 3. Touch operation detecting means provided in the display device for detecting a touch operation in a touch reaction area; and when the touch operation detected by the touch operation detecting means is an operation for selecting the icon, the icon corresponds to the icon. The communication terminal according to any one of claims 1 to 4, further comprising a setting unit configured to acquire an optical signal from the information transmission light source.   6. The communication terminal according to claim 5, further comprising data display means for displaying data corresponding to an optical signal transmitted from the information transmission light source set by the setting means.

Images