JP2006332774A - Terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a terminal such as a mobile terminal, the built-in camera of which can effectively be utilized. <P>SOLUTION: The terminal with the built-in camera 30 includes: a generating circuit means as the camera 30 for carrying out photographing a near infrared ray region in addition to a visual region; a detection means for detecting a difference between a component of a particular color and a near infrared ray component in the visual region of a particular photographing region photographed by the camera 30; a conversion means for converting the difference detected by the detection means by a predetermined arithmetic operation; and a display means 23 for displaying a value converted by the conversion means or indication corresponding to the value. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a terminal device that is configured to be portable, such as a mobile phone terminal, for example. In particular, an imaging unit that captures a still image or a moving image, and a display unit that displays an image captured by the imaging unit. The present invention relates to a built-in terminal device.

  2. Description of the Related Art In recent years, various types of mobile phone terminals in which a photographing unit such as a CCD imager is incorporated and a still image or a moving image is photographed by the photographing unit have been widely used. The captured still image or moving image can be displayed on a display panel built in the terminal, and the image can be transmitted to another terminal or the like by using a wireless telephone function provided in the terminal.

  Other than mobile phone terminals, for example, a small information processing terminal such as a PDA in which photographing means is incorporated so that an image can be captured has been put into practical use.

Patent Document 1 discloses an example in which a photographing unit is built in a mobile phone terminal.
JP 2004-80442 A

  By the way, the photographing means built in the mobile phone terminal is required to be manufactured at a relatively low cost, and functions and performance tend to be inferior compared to a single camera device (so-called electronic still camera, video camera). There were problems that could not be said to be sufficient in function and performance. Portable terminal devices such as mobile phone terminals are often carried by users at all times, and if the shooting function becomes multi-functional, the shooting means can be used effectively, but in the past it has been fully used I couldn't say it was.

  The present invention has been made in view of such a point, and an object of the present invention is to enable effective use of photographing means incorporated in a terminal such as a mobile phone terminal.

  The present invention provides a terminal device incorporating a photographing means as a photographing means for photographing a near-infrared region in addition to a visible light region, and specifying a specific photographing region photographed by the photographing means within the visible light region. Detecting means for detecting the difference between the color component and the near-infrared component, a converting means for converting the difference value detected by the detecting means by a predetermined calculation set in advance, and a value converted by the converting means Or display means for performing display corresponding to the value.

  According to the present invention, a difference between a specific color component and a near-infrared component is detected using a signal photographed using a photographing means built in a terminal device such as a mobile phone terminal, and The measurement value corresponding to the freshness can be obtained and the measurement value can be displayed. If the terminal device of the present invention is possessed, the freshness of the plant can be easily determined anywhere.

  In this case, the display means displays the image of the visible light region photographed by the photographing means, and measures the freshness at which position by displaying a mark corresponding to the measured point in the display image. It is possible to determine directly from the display of the captured image, and the convenience during measurement is improved.

  Further, the specific color component and the near infrared component in the visible light region include an output imaged in a state where the optical filter provided in the imaging unit is disposed, and an output imaged in a state where the optical filter is not disposed. Therefore, each component can be easily obtained by switching the optical filter.

  The imaging unit includes a first pixel that detects a signal of a specific color in the visible light region and a second pixel that detects a signal in a band in the near infrared region, and the first and second pixels Is detected by the detection means, so that two signal components can be obtained and measured by one photographing.

  Further, when the first pixel and the second pixel are provided, the first pixel is a pixel selected from a plurality of pixel groups arranged in a matrix for image capturing, and the second pixel The pixel is a pixel arranged at a position different from the pixel group arranged in a matrix, so that the pixel arranged in a photographing means such as a general CCD imager can be used as the first pixel. Measurement can be performed only by preparing a dedicated second pixel.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
In this example, the present invention is applied to a mobile phone terminal device incorporating a camera as photographing means.

  FIG. 1 is a diagram showing a configuration example of the mobile phone terminal device of this example. First, the configuration of the reception system of the mobile phone terminal device 10 of this example will be described. An antenna 11 for wireless communication is connected to the reception processing unit 12, and the reception processing unit 12 receives a radio signal of a predetermined frequency. The received signal is sent to the demodulation processing unit 13, the signal modulated for transmission is demodulated, and the data obtained by the demodulation is supplied to the data processing unit 14. The data processing unit 14 extracts each data included in the received data. For example, the audio data is supplied to the digital / analog converter 15 to be an analog audio signal, and the analog audio signal is supplied to the speaker 16 for output. Data other than audio data, such as received control data and mail data, is supplied to the control unit 21 or the memory 22.

  Next, the configuration of the transmission system will be described. The audio signal output from the microphone 17 is converted into digital audio data by the analog / digital conversion unit 18, and the converted audio data is supplied to the data processing unit 14 for transmission. A trusted packet structure. The transmission packet obtained by the data processing unit 14 is supplied to the modulation processing unit 19 to be modulated for transmission, and the modulated signal is supplied to the transmission processing unit 20 to obtain a predetermined transmission frequency. Send it. When transmitting data other than voice data such as mail data and control data, the data processing unit 14 includes these data in the transmission packet.

  The control unit 21 included in the mobile phone terminal device 10 controls various processes in the terminal device 10 based on a program stored in the memory 22. Shooting with a camera block 30 to be described later and freshness measurement processing using an image shot with the camera block 30 are also executed under the control of the control unit 21. The control unit 21 also controls display on the display unit 23 included in the terminal device 10. In addition, operation data of the operation unit 24 including operation keys and the like is supplied to the control unit 21 and a control process based on a user operation is executed.

  The mobile phone terminal device 10 of this example includes a camera block 30 as a photographing unit. The camera block 30 includes a lens 31 (FIG. 2) that captures image light, and the image light imaged through the lens 31 is converted into an electrical signal (imaging signal) by a CCD imager 32. The imaging signal obtained by the CCD imager 32 is read by the imaging signal reading unit 33 and supplied to the imaging signal processing unit 34. In the imaging signal processing unit 34, the imaging signal read from the CCD imager 32 is converted into an image signal of a predetermined format, and the converted image signal is output from the output unit 35 and supplied to the memory 22 (FIG. 1). Let me remember. The image signal stored in the memory 22 can be displayed on the display unit 23 and can be transmitted to an arbitrary party using a wireless telephone function based on a user operation. The image that is captured and stored in the memory 22 or displayed on the display unit 23 may be either a still image or a moving image. As the CCD imager 32 of the present example, an imager having a detection sensitivity at a certain level not only for the visible light region but also for the signal component in the near infrared region is used. The CCD imager having such a configuration is a general imager built in a mobile phone terminal device.

  FIG. 3 is a diagram showing an example of the imaging pixel configuration of the CCD imager 32 of this example. In the CCD imager 32 of this example, pixels formed of light receiving elements are arranged on an image plane in a predetermined number of rows and columns. As shown in FIG. 3, as each pixel, a red pixel 32R, a green pixel 32G, and a blue pixel 32B for capturing a red component are arranged in order. It is as composition to obtain. In this example, only the pixel at a specific position such as the central portion in which the pixels are arranged in this way is used as a near-infrared pixel 32IR that receives and detects a near-infrared signal in the infrared region rather than the red component. is there. However, a signal picked up by the red pixel 32R also includes a near-infrared signal component. The color component imaged by each of the pixels 32R, 32G, 32B, and 32IR is determined by the characteristics of the optical filter disposed on the light receiving element that constitutes each pixel.

  The near-infrared signal that is captured and output by the near-infrared pixel 32IR is not used in normal mode shooting, but is used in a freshness measurement mode described later. Further, at the time of shooting in the normal mode, at least one of the red pixel 32R, the green pixel 32G, and the blue pixel 32B is lost at the pixel position where the near infrared pixel 32IR is disposed. For the imaging signal, it is necessary to perform an interpolation process using signal components of adjacent pixels of the same color. For this interpolation processing, for example, a technique that is normally performed in the interpolation processing of a defective pixel signal can be employed.

  In the example of FIG. 3, only one near-infrared pixel 32IR is arranged, but a plurality of pixels may be used as the near-infrared pixel 32IR.

  The imaging signal obtained by the camera block 30 configured as described above is read by the imaging signal reading unit 33 shown in FIG. In this case, the signal read from the near-infrared pixel 32IR is processed in a subsequent circuit as an imaging signal (detection signal) of a different system from the signals read from other pixels, and used for freshness detection processing described later.

  FIG. 4 is a diagram showing a configuration example of the outer shape of the mobile phone terminal device of this example. 4A is a diagram of the mobile phone terminal device 10 viewed from the front side, and FIG. 4B is a diagram of the mobile phone terminal device 10 viewed from the back side. As shown in FIG. 4A, a lens 31 provided in the camera block 30 is arranged on the surface of the mobile phone terminal device 10, and a light 36 is arranged in the vicinity of the lens 31. The light 36 is used when, for example, a light emitting diode is used as a light source and the illuminance is insufficient when photographing with the camera block 30. For example, the light 36 emits light under the control of the control unit 21. In this example, the light source used as the light 36 is a light source that emits white light. The white light includes a near infrared component (a near infrared component detected by the near infrared pixel 32IR of the imager 32 and a near infrared component). The same component) is included. The light 36 may always emit light under the control of the control unit 21 or the like, for example, when shooting in a freshness measurement mode described later.

  As shown in FIG. 4B, a display panel constituting the display unit 23, operation keys constituting the operation unit 24, a speaker 16, a microphone 17 and the like are arranged on the back surface of the mobile phone terminal device 10. As for the operation keys, some keys 24 a and 24 b are also arranged on the side surface of the terminal 10. Furthermore, an antenna 11 is arranged at the upper end of the terminal 10. As the keys 24a and 24b arranged on the side surfaces, here, the key 24a is used as a shutter button for photographing a camera, and the key 24b is used as a button for measuring freshness. The arrangement and function assignment of these keys 24a and 24b are an example. For example, the keys arranged on the back side of the terminal device 10 may be used as keys for photographing or freshness measurement.

  As a display on the display unit 23, a general display as a mobile phone, a phone number display at the time of outgoing or incoming calls, a battery remaining amount display, a reception level display, an e-mail display, a screen accessing a predetermined site Display is performed. When the camera block 30 performs shooting, the captured image is displayed on the display unit 23. Even at the time of freshness measurement, the entire image captured by the camera block 30 is displayed on the display unit 23, and the measurement values of the measurement points are displayed on the screen. Details of the display example will be described later.

  Next, processing for detecting the freshness of a subject from a signal photographed by the camera block 30 built in the mobile phone terminal device of this example will be described. The freshness measurement process of this example is executed by a process within the control unit 21. FIG. 5 is a functional block diagram illustrating a processing configuration related to freshness measurement in the control unit 21. The freshness measurement process itself using the image pickup signal is performed by executing a freshness measurement process program in the control unit 21. First, the process will be described as a functional block, and is stored in the memory 22 as shown in FIG. The red component detection unit 21a detects the level of the red component at the measurement point and the near infrared component detection unit 21b detects the level of the near infrared component at the measurement point. Specifically, the near-infrared component detection unit 21b detects the level of the signal output from the near-infrared pixel 32IR (FIG. 3), and the red component detection unit 21a detects the red pixel adjacent to the near-infrared pixel 32IR. The level of the signal output by 32R is detected.

  And the data of the level which each detection part 21a, 21b detected is supplied to the difference detection part 21c, and the difference detection process by the preset arithmetic expression is performed. An arithmetic expression for performing the difference detection process is an arithmetic expression for obtaining the freshness of a plant called an NDVI value from a ratio of a red component and a near infrared component. An arithmetic expression for calculating the NDVI value will be described later. The detected difference data is supplied to the measurement value calculation unit 21d to calculate a measurement value for display. The calculated measurement value is supplied to the display data generation unit 21e, and display data for displaying the measurement value and the like on the display unit 23 is created.

  FIG. 6 is a flowchart showing the freshness measurement process in the control unit 21. First, a shooting (measurement) mode in the camera block 30 is determined (step S11), and when the shooting mode is a normal mode, the process proceeds to another process (not shown) for normal shooting. If the shooting mode is the freshness measurement mode, the red component of the measurement point is extracted (step S12), the near infrared component of the measurement point is extracted (step S13), and the extracted signal components are respectively extracted. Then, the NDVI (Normalized Difference Vegetation Index) value is calculated (step S14). When the NDVI value is calculated, a process for updating the measured value to be displayed on the display unit 23 to the calculated value is performed (step S15). When the measurement is continuously performed, the process returns to step S12 to return to the NDVI value. The measurement and display processes are repeated.

  FIG. 7 is a diagram showing an example of the relationship between the freshness of a plant and each color component to be photographed. The horizontal axis in FIG. 7 indicates the wavelength (nm) of light, and the vertical axis indicates the level as a ratio (the maximum level is 100%). The characteristics a, b, and c show the level distribution detected when the leaves of the plant are irradiated with light. The characteristic a is a characteristic of a leaf having high freshness (energetic), and the characteristic b is slightly This is a characteristic of a leaf having a low freshness (no energy), and a characteristic c is a characteristic of a leaf having a worse freshness.

  In the visible light region, the characteristics a and b are high in the green component and small in the red component, and have almost the same change characteristics. However, in the near-infrared region of approximately 700 nm or more, the level of the characteristic “a” of the energetic leaf is extremely high, whereas the level of the characteristic “b” of the leaf whose freshness has deteriorated to some extent is high. . In the case of the leaf characteristic c having a very low freshness, the green level has dropped, and it can be seen that the freshness is poor from the standpoint of appearance.

In the case of this example, the difference between the characteristic a and the characteristic b can be understood from the detection of the difference between the red component and the near-infrared component, so that the freshness can be measured. Focusing on the relationship shown in FIG. 7, the NDVI value represents the freshness of the leaves as a numerical value. The NDVI value is calculated by the following formula.
NDVI value = (near infrared component value−red component) / (near infrared component value + red component)

Here, for example, in the case of the characteristic shown in FIG. 7, as the NDVI value in the case of the characteristic a having a high freshness,
NDVI value = (100-10) / (100 + 10) = 90/110 = 0.82
It becomes. In addition, as the NDVI value in the case of the characteristic b whose freshness has fallen even though it looks the same,
NDVI value = (50-10) / (50 + 10) = 40/60 = 0.67
It becomes. Furthermore, as the NDVI value in the case of the characteristic c in which the freshness is lowered,
NDVI value = (30-28) / (30 + 28) = 40/60 = 0.03
It becomes. Therefore, by detecting the NDVI value, the higher the freshness of the plant, the higher the numerical value, and it becomes possible to determine whether the freshness is high even when there is no apparent change.

  The plant exhibiting such change characteristics is particularly effective in the case of plants having green leaves, and in the case of vegetables, for example, is particularly effective for so-called leafy vegetables such as spinach. In the case of a plant such as a tree planted in an alley or the like, by measuring (photographing) the leaf, the freshness (the vigor of the leaf) can be determined, and the state of the tree can be determined.

  FIG. 8 shows a display processing example of the NDVI value measured in this way. In this example, the image taken by the camera block 30 is displayed on the panel of the display unit 23, and the freshness measurement point in the image is indicated by a cross mark or the like (the center of the cross is measured). Point) On the upper side of the display panel, the NDVI value is displayed as a numerical value. In this example, the red component value and the near-infrared component value are also displayed numerically. Note that the maximum value of the NDVI value after the current measurement is started may be displayed simultaneously. Further, the display as a cellular phone terminal such as the remaining battery level and the reception level is also performed at the same time.

  In the example of FIG. 8, the calculated numerical value is directly displayed as the NDVI value. However, a value converted into a value that can be easily understood by the user may be displayed. For example, it may be converted into a numerical value of 10 steps and the numerical value of the 10 steps may be displayed. Further, a mark corresponding to the calculated numerical value may be displayed. For example, a mark corresponding to the measurement value may be displayed by preparing a plurality of marks such as three stages of a state where the freshness is very good, a state where the freshness is normal, and a state where the freshness is bad.

  The NDVI value measured in this way is stored in the memory 22 together with the photographed image (still image or moving image) obtained at the time of measurement, and is displayed later or attached to an e-mail. You may make it transmit to arbitrary partners.

  As described above, according to the mobile phone terminal device 10 of the present example, the freshness of the plant can be measured by photographing using the built-in camera block, and the measured freshness can be directly confirmed by the display on the display unit 23. Very convenient. For example, whether the freshness is good or bad by photographing the vegetables displayed in the store or the vegetables that have been purchased and stored with the mobile phone terminal device 10 of this example, and measuring and displaying the freshness. It can be used as a judgment material for selecting a product and a judgment material for determining whether or not stored vegetables can be used. If you take a picture of a tree planted in an alley, you can determine whether the tree is fine or weak.

  Note that the imager included in the camera block built in the mobile phone terminal device has a sensitivity to detect near-infrared components at a certain level from the beginning, and its detection characteristics are worse than those for a camera dedicated to infrared photography. As shown in the example, it has a sufficient characteristic to be used for detecting the NDVI value, and it is relatively easy to prepare the pixel for detecting the near-infrared component shown in FIG.

  In the embodiment described so far, the dedicated imager having the structure shown in FIG. 3 is used. However, a configuration in which similar detection processing can be performed with other configurations may be used. For example, as shown in FIG. 9, as a mobile phone terminal device 10 ′, an auxiliary lens 41 is arranged beside the lens 31, and a near-infrared signal component is photographed with light input through the auxiliary lens 41 ( Detection) and obtaining the NDVI value from the difference from the red component imaged with the light input through the lens 31.

  FIG. 10 is a diagram showing a configuration example of the camera block in the case of the configuration shown in FIG. The image light of the subject input through the lens 31 is picked up by the CCD imager 32 and read out by the image pickup signal reading unit 33, and the read image pickup signal is processed by the image pickup signal processing unit 34. The light input through the auxiliary lens 41 is imaged by the infrared CCD imager 42 and read by the imaging signal reading unit 43, and the read imaging signal is supplied to the imaging signal processing unit 34. . The infrared CCD imager 42 only needs to have a measurement point pixel for measuring freshness, and does not need all pixels for one screen. Further, an optical filter for extracting a near-infrared signal component may be disposed between the lens 41 and the infrared CCD imager 42.

  The imaging signal processing unit 34 processes the signals read from the respective imagers 32 and 42 and outputs them from the output unit 35, and supplies them to the control unit 21 or the memory 22 to perform NDVI value calculation processing and display processing. Do. The NDVI value calculation process and display process may be similar to the processes described in the above-described embodiment, and the other parts of the mobile phone terminal device 10 ′ may be the mobile phone described in the above-described embodiment. The configuration is the same as that of the terminal device 10.

  With the configuration shown in FIGS. 9 and 10, the red component and the near-infrared component can be picked up and detected by different imagers, and the imager for detecting the red component can be an ordinary imager for visible light imaging. As an imager for imaging (detecting) near-infrared components, it is only necessary to provide a pixel (light receiving element) that can detect only the measurement point, and it can be an imager with a relatively simple configuration, and has two imagers. With this configuration, it is possible to easily detect the NDVI value.

  As another example, as shown in FIG. 11, the camera block may be configured such that a filter for extracting a near-infrared component is selectively arranged so that only one imager is required. That is, as shown in FIG. 11, as the mobile phone terminal device 10 ″, the lens 31 is disposed, and the optical filter 51 that allows only the near-infrared signal component to pass between the lens 31 and the CCD imager 32 is selectively used. A configuration that can be arranged.

  12 is a diagram showing the configuration of the camera block of the mobile phone terminal 10 ″ shown in FIG. 11. As shown in FIG. 12, an optical filter 51 is selected between the lens 31 and the CCD imager 32. The optical filter 51 is configured such that the optical filter 51 can be selected from a state in which it is disposed in the optical path and a state in which it is not disposed by driving by the filter driving unit 52. The driving by the filter driving unit 52 is, for example, the control unit 21. In the freshness measurement mode, photographing with the optical filter 51 disposed and photographing with the optical filter 51 not disposed are continuously performed.

  A signal imaged by the CCD imager 32 is read by the imaging signal reading unit 33, supplied to the imaging signal processing unit 34, output by the output unit 35, and supplied to the control unit 21 or the memory 22. In the freshness measurement mode, the control unit 21 uses a near-infrared signal component obtained by photographing with the optical filter 51 disposed and a red component in a signal photographed without the optical filter 51 being disposed, The NDVI value is obtained.

  With this configuration, it is possible to measure the freshness of a plant with a configuration using only one imager as a configuration for selectively arranging filters. In general, as a CCD imager provided in a camera block built in the mobile phone terminal device 10, a CCD imager having sensitivity up to near-infrared components is used, and a sufficient NDVI value can be obtained by such filter switching. As the optical filter 51, for example, a filter that extracts only the near-infrared component and a filter that extracts only the visible light component may be prepared, and the two filters may be switched. By doing so, a clear image taken with an existing CCD imager can be acquired, and it is revived as an imager that can understand the freshness of a plant.

  In the configuration of each embodiment described so far, a CCD imager is used as the imager provided in the camera block. However, it is needless to say that an imager having another configuration such as a MOS type imager may be used. It is.

  In each of the examples described so far, the camera function built in the mobile phone terminal device is used. However, the camera function and the photographed image are used in other small portable terminal devices of this type. The present invention is also applicable when a display function for displaying is incorporated.

It is the block diagram which showed the example of a structure of the terminal device by one embodiment of this invention. It is the block diagram which showed the structural example of the camera block by one embodiment of this invention. It is explanatory drawing which showed the structural example of the imager by one embodiment of this invention. It is the perspective view which showed the example of the shape of the terminal by one embodiment of this invention. It is the block diagram which showed the functional block of the freshness determination process by one embodiment of this invention. It is the flowchart which showed the example of the freshness determination process by one embodiment of this invention. It is a characteristic view showing an example of characteristics according to an embodiment of the present invention. FIG. 6 is a display example according to an embodiment of the present invention. FIG. It is the perspective view which showed the terminal structural example by other embodiment of this invention. It is a block diagram which shows the example of the camera block by the example of FIG. It is the perspective view which showed the terminal structural example by further another embodiment of this invention. It is a block diagram which shows the example of the camera block by the example of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10, 10 ', 10 "... Cell-phone terminal device, 11 ... Antenna, 12 ... Reception processing part, 13 ... Demodulation processing part, 14 ... Data processing part, 15 ... Digital / analog conversion part, 16 ... Speaker, 17 ... Microphone 18 ... Analog / digital conversion unit, 19 ... modulation processing unit, 20 ... transmission processing unit, 21 ... control unit, 22 ... memory, 23 ... display unit, 24 ... operation unit, 30 ... camera block, 31 ... lens, 32 DESCRIPTION OF SYMBOLS ... CCD imager, 33 ... Imaging signal reading part, 34 ... Imaging signal processing part, 35 ... Output part, 36 ... Light, 41 ... Auxiliary lens, 42 ... Infrared CCD imager, 43 ... Imaging signal reading part, 51 ... Optical filter 52: Filter drive unit

Claims (6)

In a terminal device incorporating a photographing means,
As the photographing means, means for photographing the near infrared region in addition to the visible light region,
Detecting means for detecting a difference between a component of a specific color and a near-infrared component in a visible light region of a specific imaging region captured by the imaging unit;
Conversion means for converting the value of the difference detected by the detection means by a predetermined calculation set in advance;
A terminal device comprising: a value converted by the conversion means or a display means for performing display corresponding to the value. The terminal device according to claim 1,
The terminal device characterized in that the conversion by the conversion means is conversion for obtaining a value corresponding to freshness when the subject imaged by the imaging means is a plant. The terminal device according to claim 1,
The display device displays an image of a visible light region photographed by the photographing device, and displays a mark indicating the specific photographing region superimposed on the display image. The terminal device according to claim 1,
The specific color component and near-infrared component in the visible light region include an output imaged in a state where an optical filter provided in the imaging unit is arranged, and an output imaged in a state where no optical filter is arranged. The terminal device characterized by detecting from. The terminal device according to claim 1,
The photographing means includes a first pixel that detects a signal of a specific color in the visible light region, and a second pixel that detects a signal in a band in the near infrared region,
A terminal device, wherein the detection means detects the outputs of the first and second pixels. The terminal device according to claim 5, wherein
The first pixel is a pixel selected from a plurality of pixel groups arranged in a matrix for imaging.
The terminal device, wherein the second pixel is a pixel arranged at a position different from the pixel group arranged in the matrix.