JP2013026920A - Portable terminal device, program, and illumination generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide: a portable terminal device that allows for generation of a user original illumination with user desired colors; a program; and an illumination generation method.SOLUTION: A portable telephone 1 includes: a display 21 to display an image on a display surface 21b; an illumination section 40 to display illumination according to an illumination pattern composed of a combination of a predetermined lighting pattern and predetermined colors; and a CPU 100. The CPU 100 sets an extraction area of color information with respect to a taken image 202 displayed on the display surface 21b, and extracts the color information contained in the extraction area. The CPU 100 also generates a display pattern on the basis of the extracted color information.

Description

  The present invention relates to a portable terminal device such as a cellular phone and a PDA (Personal Digital Assistant), and a program and illumination generation method suitable for the portable terminal device.

  2. Description of the Related Art Conventionally, a portable terminal device that blinks a light source such as an LED (Light Emitting Diode) in response to an incoming mail or voice is known. In this configuration, the illumination corresponding to the content of the incoming call is displayed by blinking the light source. By visually recognizing the illumination, the user can know various kinds of information such as an incoming mail or a voice incoming call or information on the transmission source.

  The following Patent Document 1 discloses a portable terminal device capable of selecting an illumination pattern.

JP 2008-035386 A

  However, according to the configuration of Patent Document 1, the user can select an illumination pattern, but cannot uniquely set a color for illumination display. If the illumination can be displayed in the color desired by the user, the identification and aesthetics of the illumination are improved.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a portable terminal device, a program, and an illumination generation method capable of generating a user-specific illumination with a color desired by the user.

  A portable terminal device according to a first aspect of the present invention includes a display unit that displays an image on a display surface, and an illumination unit that displays an illumination according to an illumination pattern composed of a combination of a predetermined lighting pattern and a predetermined color, A setting unit that sets an extraction region of color information for the image displayed on the display surface, an extraction unit that extracts color information included in the extraction region, and the color information extracted by the extraction unit. A generation unit configured to generate the illumination pattern based on the illumination pattern.

  In the portable terminal device according to the present invention, the illumination unit may display the illumination based on the generated illumination pattern in response to the generation of the illumination pattern by the generation unit.

  The mobile terminal device according to the present invention may further include a camera unit and a display control unit that controls the display unit. In this case, the display control unit performs control to display an image photographed by the camera unit on the display unit. Further, the setting unit sets the color information extraction region for the image taken by the camera unit.

  In the mobile terminal device according to the present invention, the setting unit may set a plurality of extraction regions for the image displayed on the display surface. In this case, the extraction unit extracts the color information from each extraction region, and the generation unit generates the illumination pattern based on the plurality of color information extracted by the extraction unit.

  Moreover, the portable terminal device which concerns on this invention WHEREIN: The said setting part can set the number of the said extraction area | regions in the said image according to the said lighting pattern.

  The second aspect of the present invention relates to a program. A program according to this aspect includes a display unit that displays an image on a display surface, and an illumination unit that displays an illumination according to an illumination pattern including a combination of a predetermined lighting pattern and a predetermined color. The computer has a function of setting an extraction area of color information for the image displayed on the display surface, a function of extracting color information included in the extraction area, and the color information extracted from the extraction area. And the function of generating the illumination pattern.

  A second aspect of the present invention is a portable terminal comprising: a display unit that displays an image on a display surface; and an illumination unit that displays an illumination according to an illumination pattern that is a combination of a predetermined lighting pattern and a predetermined color The present invention relates to an illumination generation method for an apparatus. The illumination generation method according to this aspect includes a step of extracting color information included in a predetermined area of the image displayed on the display surface, and a step of generating the illumination pattern based on the extracted color information. ,including.

  ADVANTAGE OF THE INVENTION According to this invention, the portable terminal device which can produce | generate a user's original illumination with the color which the user wanted, the program, and the illumination production | generation method can be provided.

  The effects and significance of the present invention will become more apparent from the following description of embodiments. However, the following embodiment is merely an example when the present invention is implemented, and the present invention is not limited to what is described in the following embodiment.

It is a figure which shows the external appearance structure of the mobile telephone based on Embodiment. It is a front view of the 1st cabinet of the mobile phone based on an embodiment. It is a block diagram which shows the whole structure of the mobile telephone which concerns on embodiment. It is a figure explaining the display pattern of the illumination which concerns on embodiment. It is a figure explaining the display pattern of the illumination which concerns on embodiment. It is a flowchart which shows the process sequence of the control for producing | generating the display pattern which concerns on embodiment. It is a figure for demonstrating the display content and illumination of a screen when operation for preparation of the display pattern which concerns on embodiment is made. It is a figure for demonstrating the display content and illumination of a screen when operation for preparation of the display pattern which concerns on embodiment is made. It is a figure for demonstrating the correspondence of the extraction area | region which concerns on the example 1 of a change, and a light source, and the display content and illumination of a screen when operation for creation of a display pattern is made. 12 is a flowchart illustrating a control processing procedure for generating a display pattern according to Modification Example 2. It is a figure for demonstrating the correspondence of the extraction area | region which concerns on the example 2 of a change, and the area | region in an illumination part. It is a figure for demonstrating the display content and illumination of a screen when operation for preparation of the display pattern which concerns on the example 2 of a change is made. It is a figure for demonstrating the display content and illumination of a screen when operation for creation of the display pattern which concerns on the other example of a change is made.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1A shows an external configuration of the mobile phone 1 in which the first cabinet 10 and the second cabinet 20 are arranged and opened. FIG. 1B shows an external configuration of the cellular phone 1 in a state where the first cabinet 10 and the second cabinet 20 are stacked and closed. FIG. 1C shows a rear view of the second cabinet 20.

  The mobile phone 1 has a first cabinet 10 and a second cabinet 20. The second cabinet 20 is rotatably connected to the first cabinet 10 by a hinge part 30.

  A key operation unit 11 and a microphone 12 are provided on the front surface of the first cabinet 10.

  On the front surface of the second cabinet 20, a liquid crystal panel 21a constituting the display 21 and a speaker 22 are arranged. A display surface 21b is provided in front of the liquid crystal panel 21a, and the display surface 21b appears outside.

  A panel backlight 21c constituting the display 21 is disposed behind the liquid crystal panel 21a. The panel backlight 21c includes an LED serving as a light source, and supplies light to the liquid crystal panel 21a.

  A camera module 23 is built in the upper side of the second cabinet 20. The camera module 23 includes a lens 23 a that captures an image of light for photographing a subject, and the lens 23 a desires the outside from a lens hole provided on the back surface of the second cabinet 20.

  The camera module 23 includes an image sensor such as a CCD or CMOS sensor that captures an image of light incident from the lens 23a. The camera module 23 digitizes the imaging signal output from the imaging device, performs various corrections such as gamma correction on the digital signal, and outputs the digital signal to a video encoder (FIG. 3) described later.

  The hinge part 30 includes a pair of rotating shafts 31 and a pair of bearing parts 32. The rotating shaft 31 extends from the connection side end of the second cabinet 20 to the left and right. The bearing portion 32 is formed at the connection side end portion of the first cabinet 10 and receives the rotation shaft 31.

  The key operation unit 11 includes a plurality of keys such as a numeric keypad, direction keys (up key, down key, right key, left key), an enter key, a call key, a call end key, and a key for starting a predetermined application. Is done.

  FIG. 2 is a front view of the first cabinet 10. The front surface of the first cabinet 10 mainly includes the above-described key operation unit 11 and the other plate unit 12. An illumination unit 40 is disposed behind the plate unit 12. The illumination unit 40 includes a plurality (14) of light sources 40 a to 40 n, and these light sources 40 a to 40 n are distributed on the back surface of the plate unit 12.

  The illumination unit 40 displays the illumination according to a predetermined illumination display pattern (described later). Thereby, the user is visually notified of incoming mail, incoming call, alarm, and the like.

  Each of the light sources 40a to 40n has a configuration in which LEDs that emit three primary colors are combined, and emits light having various brightnesses and various colors by combining the light of the three colors. When the light sources 40a to 40b emit light, the illumination unit 40 displays illumination according to a predetermined display pattern.

  In the present embodiment, the light sources 40a to 40n are configured to display a smaller number of colors (for example, 512 colors) than the number of colors that can be displayed on the display 21. Further, the light source may be composed of other light emitting members such as an organic EL as long as it can emit a plurality of colors.

  In FIG. 2, the positions where the light sources 40 a to 40 n are arranged are indicated by circles for convenience. However, since they are arranged behind the plate portion 12 as described above, the light sources 40 a to 40 n are externally provided. Cannot be seen. When the light sources 40a to 40n emit light, the emitted light passes through the plate portion 12 and is visible from the outside.

  As shown in FIG. 2, the four light sources 40a to 40d are arranged on the left side of the key operation unit 11 in order from the top in the vertical direction (Y-axis direction) at substantially equal intervals (see the vertical auxiliary line G1). . The numeric keys of the key operation unit 11 are arranged in three vertical columns, and the three light sources 40e to 40g are located approximately between the four light sources 40a to 40d between the first and second numeric keys from the left. They are arranged vertically at the height position in order from the top (see vertical auxiliary line G2). Between the second row and third row numeric keys from the left, three light sources 40h to 40j are arranged vertically in order from the top at the same height as the three light sources 40e to 40g. (See vertical auxiliary line G3). The four light sources 40k to 40n are arranged on the right side of the key operation unit 11 at the same height as the above four light sources 40a to 40d in a vertically aligned manner at substantially equal intervals from the top (vertical auxiliary). (See line G4).

  As a result, among the light sources 40a to 40n, the light sources 40a and 40k are arranged at the highest position (see the horizontal auxiliary line H1). Similarly, in order from the top, the light sources 40e and 40h are arranged at the second highest position (see the horizontal auxiliary line H2), and the light sources 40b and 40l are arranged at the third highest position (see the horizontal auxiliary line H3). The light sources 40f and 40i are arranged at the fourth highest position (see the horizontal auxiliary line H4), the light sources 40c and 40m are arranged at the fifth highest position (see the horizontal auxiliary line H5), and the light sources 40g and 40j are the sixth. (See the horizontal auxiliary line H6), and the light sources 40d and 40n are arranged at the seventh highest position (see the horizontal auxiliary line H7).

  FIG. 3 is a block diagram showing the overall configuration of the mobile phone 1.

  In addition to the components described above, the mobile phone 1 includes a CPU 100, a memory 101, a communication module 102, a key input circuit 103, an audio encoder 104, a backlight drive circuit 105, a video decoder 106, an audio decoder 107, a light source control circuit 108, A video encoder 109 is provided.

The communication module 102 converts a signal for a call from the CPU 100, data related to execution of an application, and the like into a radio signal, and transmits the signal to a base station via an antenna (not shown) built in the first cabinet 10. . The communication module 102 converts a radio signal received from the base station via the antenna into a signal for a call, data related to execution of an application, and the like, and outputs the signal to the CPU 100. Thereby, a signal for a call, data related to execution of an application, and the like are transmitted and received from a communication target device via the base station.

  The key input circuit 103 outputs an input signal corresponding to each key to the CPU 100 when each key of the key operation unit 11 is operated.

  The audio encoder 104 inputs an analog audio signal generated based on the audio collected by the microphone 12. The audio encoder 104 converts the input analog audio signal into a digital audio signal, encodes the digital audio signal, and outputs it to the CPU 100.

  The backlight drive circuit 105 supplies a voltage signal to the panel backlight 21c in accordance with a control signal from the CPU 100. The panel backlight 21 c is turned on or off by a voltage signal from the backlight drive circuit 105.

  The video decoder 106 converts the video signal from the CPU 100 into an analog or digital video signal that can be displayed on the liquid crystal panel 21a, and outputs it to the liquid crystal panel 21a. The liquid crystal panel 21a displays an image corresponding to the video signal on the display surface 21b.

  The audio decoder 107 decodes the audio signal from the CPU 100 and the sound signals of various notification sounds such as ringtones and alarm sounds, and further converts them into analog audio signals and sound signals for output to the speaker 22. The speaker 22 reproduces sound, notification sound, and the like based on the sound signal and sound signal from the sound decoder 107.

  The light source control circuit 108 controls the illumination unit 40 by supplying power to the illumination unit 40 so that each of the light sources 40a to 40n emits light of a predetermined color and brightness based on the light source control signal from the CPU 100. . The light source control signal includes RGB signals for individually causing each light source 40a to 40n to emit light in each color, and the illumination 40 unit supplies current so that the LEDs of each light source 40a to 40n emit light in a color corresponding to the RGB signal. To do.

  The video encoder 109 encodes the signal from the camera module 23 and outputs it to the CPU 100.

  The memory 101 is a storage unit including a ROM and a RAM. The memory 101 stores a control program for giving a control function to the CPU 100, various applications, an illumination display pattern, and the like. Here, the display pattern is a pattern that determines the transition of the light emission state including the color and brightness of illumination.

  The “display pattern” in the present embodiment corresponds to the “illumination pattern” recited in the claims.

  The CPU 100 generates an image of the photographed subject (hereinafter referred to as “captured image”) based on the function being executed, and displays the image on the display surface 21b. Here, the “photographed image” is an image generated by photographing a subject by the camera module 23 or another photographing device. Note that the captured image is not limited to a still image, and may be, for example, one frame image of a moving image. The captured image data may be acquired in any manner such as reading from the memory 101 or acquiring via the network.

  The CPU 100 outputs a light source control signal to the light source control circuit 108 so that each of the light sources 40a to 40n of the illumination unit 40 emits light according to the display pattern.

  FIG. 4A to FIG. 5B are diagrams illustrating illumination display patterns. These illuminations are displayed in a predetermined color or colors.

  FIG. 4A is an example of illumination (pattern 1) in which four lights flow back and forth up and down along the vertical auxiliary lines G1 to G4.

  Here, the light flowing in a certain direction means that the light emission transitions from a predetermined light source to another light source so that the light moves in that direction by changing the light emission state of the illumination.

4A to 5B, the lines drawn radially around the light sources 40a to 40n indicate the brightness of light emitted from the light sources 40a to 40n (the longer the line, the brighter the light). Schematic representation. The arrows schematically represent the direction of light movement. The same applies to the following figures. In the illumination of the pattern 1 illustrated in FIG. 4A, two lights flow up and down in the same tone along the vertical auxiliary lines G1 and G4 (see the upward arrow), and The two lights flow up and down in the same tone along the vertical auxiliary lines G2 and G3 so as to move in the opposite direction to the light flowing along the vertical auxiliary lines G1 and G4 (see the downward arrows).

  FIG. 4B is an example of illumination (pattern 2) in which four lights flow so as to circulate around a predetermined light source of the illumination unit 40 in a clockwise direction (see the four arrows) as indicated by the arrows in the figure. The four lights circulate while maintaining a synchronized state with the same period.

  FIG. 5A shows an example of illumination (pattern 3) in which six lights flow so as to reciprocate in one cycle as a whole along the six arrows pointing to the lower left as shown in the figure. In the illumination of pattern 3, the light sources arranged along the direction substantially orthogonal to the arrow emit light with substantially the same brightness at each moment. For example, the light sources 40c and 40g emit light with substantially the same brightness. The light sources 40b, 40f, 40j, and 40n emit light with substantially the same brightness, and the light sources 40a, 40e, 40i, and 40m emit light with substantially the same brightness.

  FIG. 5B is an example of illumination (pattern 4) in which four lights reciprocate along four arched arrows (see the arrows in the figure). In the illumination of the pattern 4, approximately along the first and third arrows from the top, the two lights flow so as to reciprocate in the left and right directions in the same tone, and move in the opposite directions. The remaining two lights flow so as to reciprocate in the left and right directions with the same tone, substantially along the second and fourth arrows from the top.

  FIG. 6 is a flowchart showing control processing according to the present embodiment. The process of FIG. 6 is a process for generating a user-specific display pattern using colors included in a captured image and displaying an illumination according to the generated display pattern.

  FIGS. 7A to 7B are diagrams showing a display example of a screen for selecting a lighting pattern used for generating a new display pattern and illumination according to the pattern.

  Here, the lighting pattern is a light emission brightness transition pattern when displaying illumination. An illumination display pattern is generated by combining a predetermined color with the lighting pattern. The CPU 100 can display the illumination on the illumination unit 40 in accordance with the generated display pattern, and can store the generated display pattern in the memory 101.

  A lighting pattern is a pattern for each light source 40a-40n to change a brightness according to time. For example, the brightness (for example, luminance or brightness) is represented by a numerical value 0 (not lit) to 1 (maximum brightness). When the color is represented by three numerical values of RGB, a display pattern is generated by multiplying the three numerical values of RGB by a numerical value representing the brightness of the lighting pattern. Such display pattern generation processing is executed by the CPU 100.

  In the process of FIG. 6, the four display patterns illustrated in FIGS. 4A to 5C are used as lighting patterns. That is, in this case, the CPU 100 generates a new display pattern by combining the predetermined colors in place of the colors originally specified in these display patterns.

  When the user performs a predetermined operation for starting display pattern generation, the display pattern generation process of FIG. 6 is started.

  In step S101 of FIG. 6, the CPU 100 displays a list screen 201 displaying the lighting pattern on the display surface 21b as shown in FIG. 7A, and accepts an operation for selecting the lighting pattern.

  The list screen 201 posts a plurality of lighting patterns by displaying the characters “Pattern 1”, “Pattern 2”,. The patterns 1 to 4 correspond to the four lighting patterns shown in FIGS. 4 (a) to 5 (b). In FIG. 7A, “Pattern 1” is currently focused (see the black circle and the underline of the character “Pattern 1”).

  The CPU 100 displays white light illumination according to the currently focused lighting pattern (pattern 1). Further, when the up key or down key is pressed, the CPU 100 switches the currently focused pattern in accordance with the pressed key. For example, when the down key is pressed, as shown in FIG. 7B, “Pattern 2” is focused, and illumination of white light is displayed according to Pattern 2 (FIG. 4B).

  When the user presses the enter key in the state shown in FIG. 7B, the lighting pattern corresponding to the pattern 2 that is currently focused is selected, and the CPU 100 uses this lighting pattern as the illumination display pattern. Set to be used to generate

  Thereafter, the CPU 100 drives the camera module 23 (S102), and displays a captured image on the display surface 21b based on a signal from the video encoder 109 (S103). Further, the CPU 100 sets a color (color information) extraction region for the captured image displayed on the display surface 21b and notifies the user of the set extraction region. The image is displayed on the captured image (S104). Here, the color extraction region is set to be circular and arranged in the center of the display surface 21b.

  Thereafter, the camera module 23 captures a subject and continues to output a signal based on the imaging signal to the video encoder 109 until the operation is later stopped (S116 described later).

  FIG. 7C, FIG. 8A, and FIG. 8B are diagrams showing display examples of captured images and illuminations displayed on the display surface 21b.

In FIG. 7C, a captured image 202 is displayed on the display surface 21b. Captured image 20
2 includes images of a plurality of colored pencils 203 to 207 that are subjects. In addition, a circular frame image 208 representing the extraction area 301 is displayed in the center of the display surface 21 b so as to be superimposed on the captured image 202.

  An image 209 of characters “Prev” and “Next” is displayed below the display surface 21b. The process for changing the lighting pattern relating to the character image 209 will be described in detail later with reference to FIGS.

  The colors of the color pencils 203 to 207 are blue, green, pink, light blue, and orange, respectively, from the top, and the side peripheral portions of the color pencils 203 to 207 are painted in the corresponding colors. The frame image 208 is displayed on the side periphery of the pink colored pencil 205.

  In FIG. 7C, the name of the color added to the right side of the numbers 203 to 207 indicating the color pencil is the color of the color pencil. The color name added to the right side of the number 208 indicating the frame image is the color of the pixel located at the center of the extraction area 301 (the center of the frame image 208). These color names are appended for reference. The same applies to the following drawings.

  Returning to FIG. 6, the CPU 100 extracts a color from an image of a portion on the extraction area 301 in the photographed image 202, and sets the extracted color as a color combined with the lighting pattern to generate a display pattern (S105). . Specifically, the RGB value of the color of the pixel located at the center of the extraction area 301 is extracted as color information included in the extraction area. For example, in the case of FIG. 7C, the pink color, more precisely, the RGB value corresponding to the pink color is extracted and set to the color combined with the lighting pattern corresponding to the pattern 2 (FIG. 4B).

  Further, the CPU 100 generates an illumination display pattern by combining the selected lighting pattern (pattern 2) and the extracted color (pink) (S106), and starts displaying the illumination according to the display pattern. (S107).

  As described above, the number of colors that the light sources 40a to 40n can display is smaller than the number of colors that the display 21 can display. Therefore, in the present embodiment, the fact that the extracted colors are combined with the lighting pattern is actually a color close to the extracted color (the set color) among the colors that can be displayed by the light sources 40a to 40n. The light is combined with the lighting pattern.

  Thereafter, the illumination unit 40 continues to display the illumination according to the generated display pattern based on the control by the CPU 100 until it is stopped later (S119 described later).

  As a result, as shown in FIG. 7C, the illumination that emits pink light is displayed in the illumination unit 40 of the first cabinet 10 according to the lighting pattern corresponding to the pattern 2.

  Thereafter, when any key of the key operation unit 11 is not pressed (S108: NO, S109: NO), the processes of steps S110 to S113 are repeatedly executed.

  The process of step S110 is the same as the process of step S103. That is, in step S <b> 111, the CPU 100 displays the frame image 208 so as to overlap the captured image 202.

In step S112, the CPU 100 extracts a color from the extraction area 301 in the same manner as in step S105, and updates and sets the extracted color to a color combined with the lighting pattern to generate a display pattern.

  In step S113, the CPU 100 generates a display pattern as in the process of step S106.

  That is, as a result of the processes in steps S110 to S113 being repeatedly executed, for example, as shown in FIG. 8A, the content of the photographed image 202 changes, and the side periphery of the light blue pencil 206 moves onto the extraction region 301. The light blue is updated and set as a color combined with the lighting pattern to generate the display pattern (S112). Thereby, a display pattern in which the light blue and the lighting pattern (pattern 2) are newly combined is generated (S113). As a result, the color of the illumination light changes from pink to light blue.

  In the state of FIG. 8A, for example, when the right key is pressed (S108: YES), the CPU 100 updates the selected lighting pattern (pattern 2) to the next lighting pattern (pattern 3) (S114). Further, a display pattern is generated in the same manner as in the process of step S106 (S115). That is, the lighting pattern used to generate the illumination display pattern is updated from pattern 2 to pattern 3 (FIG. 5A). As a result, the illumination lighting pattern is changed from pattern 2 to pattern 3. In this way, as shown in FIG. 8B, the illumination is displayed on the illumination unit 40 in accordance with the changed display pattern.

  When the left key is pressed (S108: YES), the CPU 100 updates the lighting pattern (pattern 2) to the previous lighting pattern (pattern 1) (S114). In this case, a light blue is combined with the lighting pattern of pattern 1 to generate a display pattern (S115), and illumination is displayed on the illumination unit 40 according to the generated display pattern.

  When the enter key is pressed in the state of FIG. 8B (S109: YES), the CPU 100 stops the operation of the camera module 23 (S116). Then, the CPU 100 stores the generated display pattern in the memory 101 (S117), and further displays a notification screen 210 for notifying that the display pattern has been stored in the memory 101 as shown in FIG. Is displayed on the display surface 21b (S118).

  In the case of FIG. 8 (c), the display pattern of the illumination that emits light blue light that has been set according to the lighting pattern (FIG. 5 (a)) corresponding to the pattern 3 that was selected when the enter key was pressed. Displayed on the memory 101.

  In step S118, CPU 100 cancels display of notification screen 210 after a predetermined time (for example, a time corresponding to the number of cycles of the display pattern) has elapsed since notification screen 210 was displayed. That is, the illumination according to the stored display pattern is displayed for the predetermined time. Thereafter, the CPU 100 stops displaying the illumination whose display has been started in step S107 (S119), and ends the process of FIG.

  In this way, the generated and stored display patterns correspond to predetermined events such as when a mail is received, when a voice is received, or when an alarm is activated based on a predetermined setting. When this event occurs later, the illumination according to the stored display pattern is displayed, and the user is notified of the event occurrence.

As described above, according to the configuration of the present embodiment, the color (color information) is extracted from the portion on the extraction region 301 arranged in the center of the display surface 21b in the captured image 202 displayed on the display surface 21b. The extracted colors are combined with the lighting pattern selected by the user to generate a display pattern. Therefore, the user can easily and intuitively specify the illumination color.

  The illumination display pattern generated in this way uses the color extracted from the image of the photographed subject, and is created by the conventional method of selectively creating a display pattern from an existing display pattern. Is difficult. That is, the user can easily create a user-specific illumination.

  Further, while the captured image 202 is displayed on the display surface 21b, a display pattern is generated as needed, and the illumination according to the generated display pattern is displayed on the illumination unit 40. Therefore, the user can directly confirm what kind of illumination the generated display pattern represents.

  An image captured by the camera module 23 is displayed as needed on the display surface 21b as a captured image 202. Based on the captured image 202, a display pattern is generated and illumination of the display pattern generated as needed is displayed. . Therefore, the user can easily generate an illumination display pattern that emits light of the color of the subject simply by photographing the desired subject.

  Furthermore, in the present embodiment, when a process for selecting a color from the captured image 202 is executed (see FIG. 7C), when a predetermined operation (pressing the right key or the left key) is performed, A configuration is adopted in which the lighting pattern is switched from the currently selected lighting pattern to another lighting pattern. The user can easily change the lighting pattern once selected, which is convenient. Since the illumination according to the display pattern generated based on the new lighting pattern is displayed according to the predetermined operation, the user can compare a plurality of lighting patterns while visually checking the displayed illumination.

<Modification 1>
In the above embodiment, a display pattern that emits light of one color is generated. However, in the first modification, a display pattern that emits light of a plurality of colors is generated.

  FIG. 9A is a diagram for explaining a correspondence relationship between the plurality of extraction regions 302a to 302n and the light sources 40a to 40n according to this modification. FIG. 9B is a diagram for explaining a display pattern to be generated and illumination according to the display pattern according to this modification.

  The control process according to this modification is the same as the process shown in the flowchart of FIG. However, as described below, in this modification, colors are extracted from the same number of extraction regions 302a to 302n as the light sources 40a to 40n, and these colors are associated with the light sources 40a to 40n, thereby turning on the light. The color combined with the pattern is set.

  With reference to Fig.9 (a), extraction area 301a-301n is set so that it may be in the substantially same arrangement | positioning state on the display surface 21b as the light sources 40a-40n. Each extraction region 301a to 301n corresponds to the light source 40a to 40n at the corresponding position, respectively. For example, the extraction area 301a corresponds to the light source 40a, the extraction area 301b corresponds to the light source 40b, and the extraction area 301c corresponds to the light source 40c.

With reference to FIGS. 9A and 9B, the CPU 100 sets the extraction areas 301a to 301n as described above, and further captures the frame images 211a to 211n surrounding the extraction areas 301a to 301n. (S104). Then, the CPU 100 extracts a color from each of the extraction areas 301a to 301n, and sets the extracted plurality of colors to colors combined with the lighting pattern (S105). Further, the CPU 100 generates a display pattern by combining the selected lighting pattern and the set plurality of colors (S106). Thereby, the display pattern which light-emits with the color extracted from each extraction area | region 301a-301n to which each light source 40a-40n respond | corresponds is produced | generated.

  For example, as illustrated in FIG. 9B, when the frame image 211a to 211n surrounding the photographed image 202 and the extraction regions 301a to 301n is displayed on the display surface 21b, the CPU 100 performs color from each of the extraction regions 301a to 301n. To extract. In the example of FIG. 9B, a plurality of colors are extracted as indicated on the right side of the frame image numbers 211a to 211n. Note that gray, which is the background color, is extracted from the extraction region 302a, and light brown, which is the color of the wooden shaft surrounding the core of the colored pencil, is extracted from the extraction regions 302j and 302m.

  As a result, the illumination according to the generated display pattern (S106, S113, S115) is displayed on the illumination unit 40. At this time, each of the light sources 40a to 40n emits light of a color extracted from the corresponding extraction region 302a to 302n. In the example of FIG. 9B, the light sources 40a to 40n emit light in gray, blue, green, pink, blue, pink, pink, green, pink, light brown, green, light blue, light brown, and orange, respectively.

  In steps S112 and S113, the CPU 100 extracts a plurality of colors from the extraction regions 301a to 301n and updates and sets the extracted plurality of colors as colors combined with the lighting pattern, as in step S106 described above. (S112). Further, the CPU 100 generates a display pattern by combining the selected lighting pattern and the updated plurality of colors (S113).

  As described above, according to the configuration of this modified example, it is possible to easily create an illumination display pattern that emits a plurality of colors using the captured image 202.

  Furthermore, the colors extracted from the extraction regions arranged in substantially the same arrangement state as the light sources 40a to 40n are designated for the light sources 40a to 40n, respectively. For this reason, the illumination pattern and color according to the display pattern are similar to the pattern and color of the captured image 202. Therefore, an illumination imitating a rough pattern or color of a subject is generated only by the user shooting a desired object as a subject.

<Modification 2>
In the modification example 2, as in the modification example 1, a display pattern that emits light of a plurality of colors is generated.

  FIG. 10 is a flowchart for explaining a control process according to this modification. In the flowchart of FIG. 10, the process of step S104 is replaced with the process of step S121 with respect to the flowchart of FIG. 6 in the first modification, and the process of step S122 is inserted between steps S114 and S115.

  Referring to the flowchart of FIG. 10, for example, when receiving an input for selecting the lighting pattern of pattern 2 in FIG. An extraction area is set, and a frame image representing the extraction area in this state is displayed.

FIGS. 11A to 11D are diagrams for explaining an extraction area of an arrangement state corresponding to a lighting pattern and a correspondence relationship between a plurality of colors extracted from the extraction area and light flowing according to the lighting pattern. is there.

  Referring to FIG. 11A, on the display surface 21b, four extraction regions 303a to 303d are provided in a predetermined arrangement state, that is, two rows and two columns, and the illumination unit 40 lights up the pattern 1 Illumination according to the pattern is shown. As described with reference to FIG. 4A, in the illumination of the pattern 1, light flows in the regions 401a to 401d surrounded by the one-dot chain line in FIG. 11A. The color of the light flowing in the areas 401a to 401d is set to the color extracted in the extraction areas 303a to 303d, respectively.

  Referring to FIG. 11B, extraction regions 303 a to 303 d are provided in the same arrangement state as described above, and illumination according to the lighting pattern of pattern 2 is shown in the illumination unit 40. As described with reference to FIG. 4B, in the illumination of the pattern 2, light flows in the regions 401a to 401d surrounded by the one-dot chain line in FIG. 11B. The color of the light flowing in the areas 402a to 402d is set to the color extracted in the extraction areas 303a to 303d, respectively.

  In the illumination of the pattern 2, the two lights flowing in the areas 401a and 401b pass through the light sources 40e and 40h. The colors of light flowing in the areas 401a and 401b are configured so that the colors designated for the areas 401a and 401b are appropriately combined, and the combined color light is emitted from the light sources 40e and 40h.

  Referring to FIG. 11C, six extraction areas 304a to 304f are provided in a predetermined arrangement state, that is, in 2 rows and 3 columns on the display surface 21b. Illumination according to the lighting pattern is shown. As described with reference to FIG. 4C, in the illumination of the pattern 3, light flows in the regions 403a to 403f surrounded by the one-dot chain line in FIG. 11C. The color of the light flowing in the areas 403a to 403f is set to the color extracted in the extraction areas 304a to 304f, respectively.

  Referring to FIG. 11D, extraction regions 303a to 303d are provided in the same arrangement state as in FIG. 11A, and illumination according to the lighting pattern of pattern 4 is shown in the illumination unit 40. ing. As described with reference to FIG. 4D, in the illumination of the pattern 4, light flows in the regions 404a to 404d surrounded by the one-dot chain line in FIG. 11D. The color of the light flowing in the areas 404a to 404d is set to the color extracted in the extraction areas 303a to 303d, respectively.

  FIGS. 12A to 12C are diagrams for explaining the generated display pattern and the illumination according to the display pattern according to this modification. With reference to the flowchart of FIG. 10 and FIGS. 12A to 12C, processing according to this modification will be described below.

  When the input for selecting the lighting pattern of pattern 2 is received as described above, the extraction regions 303a to 303d are set according to the lighting pattern based on the processing of step S121, and the captured image 202 and the display surface 21b are displayed. Frame images 212a to 212d corresponding to the lighting pattern of pattern 2 are displayed. Further, the CPU 100 generates a display pattern so that light of four colors extracted from the extraction regions 303a to 303d surrounded by the frame images 212a to 212d flows in the regions 402a to 402d, respectively (S105). To S106), as shown in FIG. 12A, the illumination according to this display pattern is displayed.

  In the example of FIG. 12A, illumination is displayed on the illumination unit 40 so that blue, light brown, green, and light blue light flow in the regions 402a to 402d, respectively.

  Here, for example, as shown in FIG. 12A, when the photographed image 202 is changed by bringing the mobile phone 1 close to the subject, color illumination based on the changed photographed image 202 is displayed. That is, in the example of FIG. 12B, the illumination is displayed on the illumination unit 40 so that green, pink, green, and light blue light flow in the regions 402a to 402d, respectively.

  In the state of FIG. 12B, for example, when the right key is pressed (S108: YES), the CPU 100 executes the process of step S122 following the process of step S114.

  In step S122, the CPU 100 updates and sets the extraction area according to the lighting pattern updated in the process of step S114. At this time, as described with reference to FIGS. 11A to 11D, the number and positions of the extraction regions are changed according to the lighting pattern. Furthermore, the CPU 100 extracts a color from the extraction areas 304a to 304f, and generates a display pattern by combining the extracted color and the lighting pattern (S115).

  Thereafter, in steps S110 and S111, the CPU 100 replaces the frame images 212a to 212d in FIG. 12B with the extraction regions 304a to 304f corresponding to the new lighting pattern as shown in FIG. The surrounding frame images 213a to 213f are displayed.

  In the example of FIG. 12C, illumination is displayed on the illumination unit 40 so that green, pink, green, and light blue light flow in the regions 304a to 304f, respectively.

  As described above, according to the configuration of this modified example, a color is designated for each predetermined area (area where light flows) of illumination.

  In addition, the extraction area is set according to the lighting pattern and arranged on the display surface 21b. In particular, when an operation for switching the lighting pattern is accepted, the lighting pattern is updated to another lighting pattern, and the number of extraction regions is changed accordingly. Illumination of the number of colors according to the lighting pattern can be displayed.

<Others>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications other than those described above can be made to the embodiments of the present invention. .

  In the above embodiment, the lighting patterns are illustrated in FIGS. 4A to 5B, but these are merely examples, and a predetermined color can be combined with various lighting patterns.

In the above embodiment, the display pattern is generated based on the captured image 202 captured using the camera module 23. However, the display pattern can be generated based on any still image or moving image such as an illustration or CG as well as a captured image. In addition, the still image and the moving image are not limited to those captured using the mobile phone 1, but may be acquired via wired communication or wireless communication. For example, the photographed image 202 may be an image of a subject that was previously photographed by another photographing device.

  In the above embodiment, the illumination unit 40 is arranged separately from the key operation unit 11. However, for example, when the key top of the key operation unit 11 is lit (in this case, a light source is disposed on the back of the key top), the key operation unit 11 also serves as the illumination unit 40. May be. In this case, the display pattern is a pattern for the key operation unit 11 to function as the illumination unit 40 and display the illumination.

  In the above embodiment, the illumination unit 40 is arranged on the front surface of the second cabinet 20. However, the illumination unit 40 may be disposed at any position on the outer surfaces (front surface, back surface, and outer peripheral surface) of the first and second cabinets 10 and 20.

  For example, the illumination unit 40 may be disposed on the back surface of the first cabinet (the surface exposed to the outer surface when the cellular phone 1 is folded (FIG. 1B)). In this case, for example, while the captured image 202 is displayed on the display surface 21b, a display pattern is generated as needed, and illumination according to the generated display pattern is displayed on the illumination unit 40. When the sub display is arranged on the back surface of the first cabinet 10, a configuration in which the sub display is the illumination unit 40 may be employed.

  In the above embodiment, the number of colors that can be displayed by the illumination unit 40 is less than the number that can be displayed by the display 21. However, the number of colors that can be displayed by the illumination unit 40 can be the same as or larger than the number that can be displayed by the display 21.

  In the above embodiment, the color (color information) of the pixel located at the center of the extraction region is extracted, and this color is combined with the lighting pattern for generating the display pattern. However, the color may be extracted by various methods based not only on the color of the pixel located at the center of the extraction region but also on a part or the whole of the extraction region. For example, a color obtained by averaging the colors of a plurality of pixels near the center of the extraction region can be extracted. Further, a configuration may be adopted in which the color of a portion having high brightness, saturation, luminance, and the like is extracted in the extraction region. Further, when a color is extracted, a process for correcting the brightness and hue of the color may be further executed.

  The arrangement positions of the extraction areas on the display surface 21b described in the above embodiment and Modification Examples 1 and 2 are merely examples, and the extraction areas can be arranged at various arrangement positions. Moreover, the structure which enables an extraction area to move by a user may be taken.

  In the above-described embodiment, the color of light emitted from the light sources 40a to 40n is set based on the pressing of the enter key once, but a configuration in which a desired color is set for each of the light sources 40a to 40n. May be taken. For example, it is possible to adopt a configuration in which colors corresponding to the light sources 40a to 40n are set in order each time the enter key is pressed. In this case, when the user presses the decision key by the number of the light sources 40a to 40n (14) while displaying the captured image 202 so that the desired color is on the extraction region, the light emitted from the light sources 40a to 40n. All the colors are determined, and a display pattern desired by the user is generated.

  In the above embodiment, the extraction region is provided at a certain position on the display surface 21b. However, for example, the extraction region may be set at a position based on an input operation performed by the user. For example, when a touch sensor (not shown) is arranged on the display surface 21b, a configuration in which an extraction region is provided at a touched position and a color is extracted therefrom can be adopted.

  In the above embodiment, the foldable mobile phone 1 is used, but a mobile phone 1 of a slide type or a straight type (including a so-called smartphone) can also be used. For example, a configuration in which the illumination unit 40 is arranged on a predetermined portion (for example, the outer peripheral portion) of the smartphone-type mobile phone 1 can be taken. In this case, as shown in FIG. 13, a color (light blue in the case of FIG. 13) is extracted from the image on the extracted area 301 of the display surface 21 b in the photographed image 202, and the extracted color and a predetermined lighting pattern are obtained. In combination, a display pattern is generated, and illumination according to the display pattern is displayed on the illumination unit 40.

  Further, the embodiment of the present invention can be applied to a mobile terminal device such as a PDA, a tablet PC, and an electronic book terminal instead of the mobile phone 1.

  In addition, the embodiment of the present invention can be variously modified as appropriate within the scope of the technical idea shown in the claims. For example, some or all of the above embodiments can be combined.

1 Mobile phone (mobile terminal device)
21 Display (Display section)
21b Display surface 23 Camera module (Camera part)
40 Illumination part 40a-40n Light source (light emitting part)
100 CPU (display control unit, generation unit, setting unit, extraction unit)
202 Captured image (image)

Claims (7)

A display unit for displaying an image on the display surface;
An illumination unit for displaying an illumination according to an illumination pattern composed of a combination of a predetermined lighting pattern and a predetermined color;
A setting unit for setting an extraction area of color information for the image displayed on the display surface;
An extraction unit for extracting color information included in the extraction region;
A generating unit that generates the illumination pattern based on the color information extracted by the extracting unit;
A portable terminal device comprising: The mobile terminal device according to claim 1,
The illumination unit displays the illumination based on the generated illumination pattern in response to the generation of the illumination pattern by the generation unit.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 1 or 2,
A camera unit;
A display control unit for controlling the display unit,
The display control unit performs control to display an image taken by the camera unit on the display unit,
The setting unit sets the extraction region of the color information for the image captured by the camera unit;
The portable terminal device characterized by the above-mentioned. In the portable terminal device according to any one of claims 1 to 3,
The setting unit sets a plurality of extraction regions for the image displayed on the display surface,
The extraction unit extracts the color information from each extraction region,
The generation unit generates the illumination pattern based on a plurality of the color information extracted by the extraction unit.
The portable terminal device characterized by the above-mentioned. The mobile terminal device according to claim 4,
The setting unit sets the number of the extraction regions in the image according to the lighting pattern.
The portable terminal device characterized by the above-mentioned. A display unit for displaying an image on the display surface;
A computer of a portable terminal device comprising: an illumination unit that displays an illumination according to an illumination pattern composed of a combination of a predetermined lighting pattern and a predetermined color;
A function of setting an extraction area of color information for the image displayed on the display surface;
A function of extracting color information included in the extraction region;
A function of generating the illumination pattern based on the color information extracted from the extraction region;
A program characterized by providing A display unit for displaying an image on the display surface;
In an illumination generation method of a portable terminal device comprising: an illumination unit that displays an illumination according to an illumination pattern composed of a combination of a predetermined lighting pattern and a predetermined color;
Extracting color information contained in a predetermined area of the image displayed on the display surface;
Generating the illumination pattern based on the extracted color information;
An illumination generation method comprising:

Images