JP2004194193A - Imaging device and method therefor, and image display device and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize an image pickup device, capable of enhancing entertainment properties and its method, and to provide an image display device and its method. <P>SOLUTION: When a character image string, corresponding to each unit movement, is successively read out, by successively designating the unit movements at random from among the character image strings for expressing each unit movement, which are respectively assigned in each unit movement of a plural kinds of the unit movements performed by a character, while the image is being displayed, each read-out character image string is superposed and is displayed on the displayed image as animation, after designating in the order of making transiting each unit movement so as to continuously connect each unit movement each other. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an imaging apparatus and method, and an image display apparatus and method, and is suitably applied to, for example, a digital camera that records an image captured by an imaging device on a recording medium.
[0002]
[Prior art]
Conventionally, in a digital camera of this type, a still image captured by a user at a desired timing is sequentially displayed while displaying a video image formed on an image sensor as a moving image on a display screen of a display screen provided in the camera body. The data is recorded on a memory card.
[0003]
In recent years, there has been proposed a type in which an animation of a character designated by a maker or the like is superimposed on a captured image as necessary on a display screen of a display screen provided in a digital camera.
(For example, see Patent Document 1).
[0004]
[Patent Document 1]
Japanese Patent Application Laid-Open No. 9-322024 (Pages 1 and 2, FIG. 1)
[0005]
[Problems to be solved by the invention]
However, such a method of superimposing an animation on a captured image merely changes the color of a character or the like in order to inform a user of an automatic adjustment function mounted on a digital camera, and is displayed on a display screen of an LCD monitor. The user has recognized that the determined animation is repeated when a certain action is performed or the action pattern of the animation is monotonous so as to be predictable, and the determined action is repeated.
[0006]
Furthermore, each animation of the character does not have continuity, and the character appears and disappears instantly as the device operation switches, so even if the character is displayed to get the user familiar with it, Since the presence of the character itself has no sense of presence, it is very difficult to give the user a familiarity.
[0007]
The present invention has been made in consideration of the above points, and has as its object to propose an imaging device and a method thereof, and an image display device and a method thereof capable of improving entertainment.
[0008]
[Means for Solving the Problems]
In order to solve such a problem, according to the present invention, an image pickup unit for picking up an image of a subject, a display unit for displaying an image picked up by the image pickup unit, and a plurality of types of unit actions performed by a character are used to represent each unit action. Character image sequences are assigned respectively, and all the character image sequences are stored in advance, and a unit operation is sequentially designated at random from a plurality of types of unit operations, and the character images correspond to the respective unit operations. Reading control means for sequentially reading out the character image sequence to be read from all the character image sequences stored in the storage means, and each character image sequence read by the reading control means is displayed on the display means as animation. Display control means for superimposing and displaying the captured image is provided, and the read control means connects each unit operation while maintaining continuity. The each unit operation was to be specified in order to transition to.
[0009]
As a result, with this imaging device, it is possible to give the user an impression as if the character is present on the display screen.
[0010]
Further, in the present invention, an imaging unit for imaging a subject, a display unit for displaying an image captured by the imaging unit, and a three-dimensional image representing a head character image of each unit operation for each of a plurality of types of unit operations performed by the character Data is assigned to each of the three-dimensional data, and all or part of the three-dimensional data is pre-stored, and a plurality of types of unit operations are sequentially designated at random to correspond to each unit operation. Reading control means for sequentially reading out the three-dimensional data to be read from the storage means or supplementing it with other three-dimensional data as necessary, and starting images based on the three-dimensional data read or complemented by the reading control means. While sequentially generating a character image representing the corresponding unit action, the character image sequence is displayed on the display means as an animation. And display control means for superimposing displayed on the captured image provided that, the read control means, and as specified in the order in which the unit operation to each other to shift the respective unit operation so as to be connected to maintain continuity.
[0011]
As a result, with this imaging device, it is possible to give the user an impression as if the character is present on the display screen.
[0012]
Further, in the present invention, a first step of displaying a captured image obtained by capturing an image of a subject, and a character image sequence for expressing each unit action assigned to each of a plurality of types of unit actions performed by the character A second step of sequentially designating the unit actions randomly from among them and sequentially reading out the character image strings corresponding to the respective unit actions, and a captured image displayed using the read out character image strings as an animation And a third step of superimposing the unit operations is provided. In the second step, the unit operations are designated in the order of transition so that the unit operations are connected while maintaining continuity.
[0013]
As a result, this imaging method can give the user an impression as if the character is present in the display screen.
[0014]
Further, in the present invention, a first step of displaying a captured image obtained by capturing an image of a subject, and a three-dimensional image representing a head character image of each unit operation assigned to each of a plurality of types of unit operations performed by the character A second step of sequentially designating unit operations randomly from all or a part of the data and sequentially reading out three-dimensional data corresponding to each of the unit operations, or supplementing with other three-dimensional data as necessary And sequentially generating a character image representing a corresponding unit action from a start image based on the read or complemented three-dimensional data, and superimposing the character image sequence on the displayed captured image as an animation. A third step of causing each unit operation to be connected while maintaining continuity. It was to be specified in the order in which to transition the work.
[0015]
As a result, this imaging method can give the user an impression as if the character is present in the display screen.
[0016]
Further, in the present invention, a display means for displaying an image, and a character image sequence for expressing each unit operation are assigned to each of a plurality of types of unit operations performed by the character. All of the character image sequences stored in the storage means, in which the storage means stored in advance and the unit actions are randomly designated sequentially from among the plurality of types of unit actions, and the character image strings corresponding to the respective unit actions are stored in the storage means. Read control means for sequentially reading from among the characters, and display control means for superimposing and displaying, as an animation, each character image sequence read by the read control means on the image displayed on the display means, wherein the read control means comprises: Each unit operation is specified in the order of transition so that the unit operations are connected while maintaining continuity.
[0017]
As a result, in this image display device, it is possible to give the user an impression as if the character is present on the display screen.
[0018]
Further, in the present invention, display means for displaying an image, and three-dimensional data representing a leading character image of each unit action are assigned to each of a plurality of types of unit actions performed by the character. A storage unit in which all or part of the unit operation is stored in advance, and a unit operation is sequentially designated at random from a plurality of types of unit operations, and three-dimensional data corresponding to each of the unit operations is sequentially read from the storage unit. A character image representing a corresponding unit action is sequentially read from a readout control unit that complements the image data using other three-dimensional data in accordance with the three-dimensional data and a start image based on the three-dimensional data read out or complemented by the readout control unit. Display control means for superimposing and displaying the character image sequence as an animation on the captured image displayed on the display means while generating. Only, read control means, and as specified in the order in which the unit operation to each other to shift the respective unit operation so as to be connected to maintain continuity.
[0019]
As a result, in this image display device, it is possible to give the user an impression as if the character is present on the display screen.
[0020]
Further, in the present invention, a first step of displaying an image and a random sequence of unit actions are sequentially performed from a character image sequence for expressing each unit action assigned to each of a plurality of types of unit actions performed by the character. A second step of sequentially reading the character image sequence corresponding to each unit operation, and a third step of superimposing and displaying each read character image sequence as an animation on the displayed image. In the second step, the unit operations are specified in the order of transition so that the unit operations are connected while maintaining continuity.
[0021]
As a result, with this image display method, it is possible to give the user an impression as if the character is present on the display screen.
[0022]
Further, in the present invention, a first step of displaying an image, and all or a part of three-dimensional data representing a leading character image of each unit action assigned to each of a plurality of types of unit actions performed by the character, A second step of sequentially designating unit operations randomly and sequentially reading out three-dimensional data corresponding to each unit operation or supplementing with other three-dimensional data as necessary; A third step of sequentially generating a character image representing a corresponding unit action from the start image based on each of the three-dimensional data and superimposing the character image sequence on the displayed captured image as an animation. In the second step, the unit operations are designated in the order in which the unit operations are transitioned so that the unit operations are connected while maintaining continuity. It was so.
[0023]
As a result, with this image display method, it is possible to give the user an impression as if the character is present on the display screen.
[0024]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0025]
(1) Overall configuration of digital camera
In FIG. 1, reference numeral 1 denotes a digital camera as a whole, and an image pickup signal S1 is obtained by photoelectrically converting an image pickup light L1 incident via a projection lens 2 on an image pickup surface of a charge coupled device (CCD) 3. This is sent to a CDS (Correlated Double Sampling) and an AGC (Auto Gain Control) unit 4.
[0026]
The CDS and AGC unit 4 reduces the noise component by clamping the signal level at a predetermined potential during a period in which reset noise is generated in the image signal S1, and automatically adjusts the amplitude of the image signal S1. By controlling the output to a predetermined value, a change in contrast is prevented. The imaging signal S2 obtained in this manner is sent to the subsequent analog / digital conversion (A / D) section 5, where it is converted to a digital imaging signal S3 and sent to the signal processing section 6.
[0027]
The signal processing unit 6 performs Y / C separation, white balance adjustment processing, γ (gamma) correction, and the like on the digital image pickup signal S3, and converts the digital image signal S3 into a video signal S4 by matrix processing. 7
[0028]
The CCD 3, CDS and AGC unit 4, A / D converter 5, signal processor 6, and memory controller 7 are connected to a CPU (Central Processing Unit) 9 that controls the entire digital camera 1 via a bus 8. Various operations are executed based on the control of the CPU 9.
[0029]
The memory controller 7 sequentially writes the video signal S4 into a RAM (Random Access Memory) 10 on a frame basis at the time of normal subject confirmation (that is, at the time of capturing a moving image), and performs a predetermined method (for example, After converting into a composite signal S5 of the NTSC (National Television System Committee) system or the like, this is transmitted to the liquid crystal monitor 12 and the external connection terminal 13.
[0030]
At this time, when a command such as a zoom process is received from the user, the memory controller 7 enlarges or reduces the video signal S4 to a designated magnification in frame units, and then sequentially writes the enlarged or reduced image in the RAM 10. At the same time, the image is displayed on the display screen 12A of the liquid crystal monitor 12 via the video encoder 11.
[0031]
As a result, a moving image based on the composite signal S5 (hereinafter referred to as a finder image) is displayed on the display screen 12A of the liquid crystal monitor 12 in real time, and a personal computer, a video monitor, or the like (shown in FIG. ) Is connected, the finder image based on the composite signal S5 supplied via the external connection terminal 13 is displayed in real time.
[0032]
The CPU 9 has a ROM (Read Only Memory) 14 in which an operation program and various constants are stored, and a non-volatile memory 15 which can continuously store various constants and various information relating to the operation of the digital camera 1 even when the power is turned off. Are connected via a bus 8 so that various data can be recorded and / or reproduced to and from these memories 14 and 15 as necessary.
[0033]
Further, an operation unit 16 composed of various keys and the like is connected to the CPU 9 via an I / O (Input / Output) port 17 and the following bus 8, and a memory card slot 18 is connected via a card interface 19 and the following bus 8. And receives a transmission signal according to the content of operation by the user through the operation unit 16 and insertion / removal of the memory card MC.
[0034]
Here, when the user presses a shutter button (not shown) of the operation unit 16 (that is, at the time of capturing a still image), the CPU 9 sends a recording command signal S10 to the memory controller 7 via the bus 8. Thus, the memory controller 7 reads the frame image captured at the time of shutter from the plurality of frame images based on the video signal S4 while writing the video signal S4 in the RAM 10, and then sends the data compression / decompression unit 20 via the bus 8. To send to.
[0035]
The data compression / expansion unit 20 performs compression encoding processing based on the JPEG (Joint Photographic Experts Groupe) standard on the frame image read from the RAM 10 to obtain a frame image in which image information is compressed. The compressed frame image thus obtained is written to the memory card MC inserted into the memory card slot 18 via the card I / F 19 under the control of the CPU 9.
[0036]
Here, when the user operates the mode changeover switch (not shown) of the operation unit 16 to switch from the imaging mode to the reproduction mode, the CPU 9 sends a read command signal S11 to the OSD (On Screen Display) 21 and After causing the OSD 21 to read a character pattern (for example, “index” or “print”) representing a playback mode setting menu (hereinafter, referred to as a playback menu) recorded in advance, via the video encoder 11. To superimpose on the display screen 12A of the liquid crystal monitor 12.
[0037]
Next, when the “index” display is selected and set from the playback menu in response to the operation of the cross key (not shown) of the operation unit 16 by the user, the CPU 9 sets the card I / F 19 and the memory card slot 18 to A selection setting signal S12 is transmitted to the memory card MC via the memory card MC, and a predetermined number (for example, six) of compressed frame images selected by the user are read out from the plurality of compressed frame images written in the memory card MC and read out. Are transmitted to the data compression / decompression unit 20 via the bus 8.
[0038]
Under the control of the CPU 9, the data compression / decompression unit 20 performs a decompression decoding process on a given number of given compressed frame images to restore them to the original frame images, and then restores these to the RAM 10 via the bus 8. , And sends it to the memory controller 7.
[0039]
The memory controller 7 reduces each of the predetermined number of frame images to a predetermined magnification under the control of the CPU 9, writes these as index images in the RAM 10, and displays the frame images on the display screen 12 A of the liquid crystal monitor 12 via the video encoder 11. Display.
[0040]
As described above, a desired index image is selected from the predetermined number of index images displayed on the display screen 12A of the liquid crystal monitor 12 in accordance with the operation of the cross key (not shown) of the operation unit 16 by the user. Then, the CPU 9 reads the original frame image corresponding to the index image from the RAM 10 and sends it to the memory controller 7. The memory controller 7 writes the frame image in the RAM 10 as it is, and displays the frame image on the display screen 12A of the liquid crystal monitor 12 via the video encoder 11.
[0041]
Incidentally, in addition to the character pattern representing the playback menu, a character pattern representing save or cancel, and a plurality of image frame patterns having various sizes and shapes are recorded in the OSD 21 in advance. When a read command signal S11 is given from the CPU 9 in response to the operation, the OSD 21 reads a character pattern or an image frame pattern based on the read command signal S11 and superimposes the character pattern or the image frame pattern on the display screen 12A of the liquid crystal monitor 12.
[0042]
On the other hand, the OSD 21 sends to the CPU 9 a display position signal S13 indicating where the character pattern or the image frame pattern is displayed on the display screen 12A of the liquid crystal monitor 12. Thereby, the CPU 9 can always determine at which position on the display screen 12A the character pattern or the image frame pattern currently displayed on the display screen 12A of the liquid crystal monitor 12 is displayed based on the display position signal S13. it can.
[0043]
Here, the RAM 10 has an internal structure as shown in FIG. 2A, and includes a program work (Program Work) area AR1, first and second finder areas AR2 and AR3, and a capture (Capture) area AR4. , A movie area AR5 and an image work area AR6.
[0044]
The program work area AR1 is a work area used by the memory controller 7 and the CPU 9 when executing a program. The first and second finder areas AR2 and AR3 are areas for providing a finder image in real time, and a series of video signals S4 supplied are written to one finder area and simultaneously read from the other. The operation is performed.
[0045]
The capture area AR4 is an area for recording a frame image based on the video signal S4 to be imaged at the time of capturing a still image, and the movie area AR5 is a first and second finder areas AR2, This is an area for temporarily recording the video signal S4 alternately reproduced from the AR3 when the video signal S4 is subjected to image size conversion, data compression, and the like and recorded on the memory card MC.
[0046]
Further, an image work area AR6 is an area for recording various image data and the like necessary for displaying an animation of a character according to the present embodiment. As shown in FIG. 2B, a JPEG file work (File Work) area AR6X, a JPEG file (Files) area AR6Y and a raw image work (Raw Image Work) area AR6Z.
[0047]
(2) Character animation display method according to the present embodiment
(2-1) Character animation generation
In the present embodiment, an animation of a character desired by the user is simultaneously superimposed and displayed while a finder image obtained as an imaging result is displayed on a display screen 12A of a liquid crystal monitor 12 mounted on the digital camera 1. Has been done.
[0048]
In the animation display method according to the present embodiment, a character image group that is a set of a plurality of types of character images with a desired character as a motif is prepared in advance, and for each of various unit operations performed by the character, the unit operation is performed. A plurality of character images necessary for expressing the character image are appropriately selected and continuously combined, and the character image sequence having continuity is displayed on the display screen 12 as an animation while being synchronized with the display timing of the finder image. The information is displayed on the screen 12A.
[0049]
The unit motion of such a character can be represented as a motion transition table as shown in FIG. 3, and includes "front facing", "please", "worry", "dancing", "jump", "slope", " Each unit operation ND such as "turn left", "turn right", "move left two steps", "move right two steps", etc. ₁ ~ ND ₁₀ Are able to transit with each other under predetermined conditions.
[0050]
For example, the unit operation ND of “front facing” ₁ Unit movement ND of "turn left" for the character performing ₇ Is selected, the unit operation ND as shown in FIG. ₇ Is continuously combined with a plurality of character images P1 to P8 corresponding to the unit operation ND. ₇ Has been realized.
[0051]
At that time, in order to provide continuity in the animation display of the character, it is necessary to set a transition condition between the unit actions so that the next unit action that transitions from the current unit action is naturally and smoothly connected. Further, it is necessary to set the transition condition between the unit operations so as to continuously transition at random which is unpredictable for the user.
[0052]
For this reason, for all unit actions that can be performed by the character, a transition probability table indicating the transition probability from the current unit action to the next unit action is set in advance, and when connecting each unit action of the character, By comparing the transition probability table with random numbers generated by, for example, a linear congruential method, the next unit motion of the character is determined.
[0053]
When the transition between the unit actions of the character is discontinuous, the transition probability between the unit actions is set to 0. In the operation transition table in FIG. 3 described above, arrows are omitted between unit probabilities with a transition probability of 0.
[0054]
In addition, in the animation display for each unit action of the character, the unit action is usually selected at random in a normal case. When there is, the normal random selection is stopped and a unit operation according to the request is selected. Even in this case, in order to maintain the continuity from the current unit operation to the unit operation corresponding to the request, the unit operation similar to the unit operation corresponding to the request is sequentially extracted and the optimal unit operation is selected. Has been done.
[0055]
(2-2) Reading of character image group
In the present embodiment, a group of character images used for displaying an animation of a character are all stored in the memory card MC as JPEG file data that has been subjected to a compression encoding process based on the JPEG standard.
[0056]
When the digital camera 1 is started, the CPU 9 takes in all the JPEG file data for animation display into the RAM 10 and records it. At that time, in the RAM 10, addresses are sequentially designated and recorded in the JPEG file area AR6Y (FIG. 2B) of the image work area AR6.
[0057]
(2-3) Animation display
After all the JPEG file data used for displaying the animation of the character are recorded in the RAM 10 as described above, a character image group corresponding to the determined unit operation is read out from the RAM 10 and the finder image is displayed on the display screen 12A of the display screen 12. Is superimposed and displayed as an animation.
[0058]
More specifically, first, the CPU 9 reads a character image group as JPEG file data from the RAM 10 and sends it to the data compression / decompression unit 20 to sequentially decompress the character image group to have the same image format as the finder image. Convert to a character image group consisting of raw images. Such decompression processing is appropriately performed for each unit operation of the character image. The character image group composed of the raw images decompressed by the data compression / decompression unit 20 is stored in the low image work area AR6Z constituting the image work area AR6 of the RAM 10.
[0059]
Here, the character image sequence for each unit operation is actually read from the low image work area AR6Z constituting the image work area AR6 of the RAM 10 and constructed by sequentially combining a plurality of character images as shown in FIG. It has been made.
[0060]
Specifically, first, each character image in the character image group has a corresponding number (hereinafter referred to as a character number) P ₁ ~ P _Z (Z is the total number), the storage positions of the character images in the low image work area AR6Z in the RAM 10 are allocated as address IDs (Identity), respectively, and the character number and the character number of each character image are assigned. A table (hereinafter, referred to as an image correspondence table) defining the correspondence with the address ID is stored in the program work area AR1 of the RAM 10.
[0061]
.., MX (X is a natural number), for example, when the unit operation M1 is designated, the CPU 9 causes the plurality of unit images M1 to form the character image sequence NA corresponding to the unit operation M1. Character image P _A1 , P _A2 , ……, P _NA After specifying the character numbers corresponding to the character numbers in the display order, the address ID corresponding to each character number is searched based on the above-described image correspondence table.
[0062]
Subsequently, the CPU 9 controls the memory controller 7 to sequentially read out the character images corresponding to the searched address ID from the low image work area AR6Z in the RAM 10, and then read out the plurality of character images and the plurality of character images read out from the RAM 10. Character image P _A1 , P _A2 , ..., P _NA Are combined in the display order described above to construct a character image sequence M1 corresponding to the specified unit action.
[0063]
Note that each character image sequence N1, N2,..., NX is the number of character images P for each unit operation M1, M2,. _A1 ~ P _NA , P _B1 ~ P _NB , P _C1 ~ P _NC , P _D1 ~ P _ND , ……, P _Y1 ~ P _YA Is determined, not all unit operations use the same number of character images.
[0064]
In this way, a character image group necessary for all the unit operations is stored in the RAM 10 by addressing one by one in frame units, and each time a unit operation is specified, a character image sequence corresponding to the unit operation is stored. Since the necessary character images are read from the RAM 10 to construct the character image sequence, the amount of data used in the RAM 10 is significantly reduced as compared with the case where moving image data is prepared in advance for each character image sequence. There is an advantage that can be.
[0065]
Thereafter, immediately before the memory controller 7 reads frame images based on the video signal S4 from the first or second finder areas AR2 and AR3 of the RAM 10 alternately and sequentially and transfers the frame images to the video encoder 11, the first or second frame image is read. The character images read from the image work area AR6 of the RAM 10 are superimposed one by one on the frame images (that is, the finder images) on the corresponding sides of the finder areas AR2 and AR3 so that the character images are displayed at the display timing of the fan finder image. Synchronously, it is superimposed on the display screen 12A of the display screen 12.
[0066]
When the character images are sequentially superimposed on the finder image one frame at a time on the display screen 12A of the display screen 12 in this manner, the frame image (finder image) from the first and second finder areas AR2 and AR3 of the RAM 10 is reduced by 1 /. Since the image is read alternately every 30 [seconds], the timing for reading out the character image, which is a raw image, from the image work area AR6 of the RAM 10 one frame at a time is also set every 1/30 [second].
[0067]
Further, when the memory controller 7 superimposes the character image sequence read from the RAM 10 on the display screen 12A of the liquid crystal monitor 12 as an animation via the video encoder 11, the display screen of the display screen 12 is synchronized with the unit operation of the character. The display position on 12A is set at random.
[0068]
Specifically, the memory controller 7 changes the coordinate position of each character image forming the character image sequence on the display screen 12A of the display screen 12 based on the control of the CPU 9 as needed on the display screen 12A. A menu bar (not shown) or the like to be displayed is randomly selected within a display range that does not obscure the menu bar. However, in this case as well, the coordinate position of each character image is selected as if the character naturally moves around in the display screen 12A so as to transition from the current unit operation to the next unit operation while maintaining continuity. Is done.
[0069]
(2-4) Background removal processing
The character image described above is formed from a character image portion representing the character itself (other than this item, this is simply called a character image) and a background image portion excluding the character image portion. The hue level of the image portion is set in advance so as to be substantially opposite to the hue level of the character image portion (for example, a complementary color relationship or a relationship similar thereto).
[0070]
On the display screen 12A of the display screen 12, when the character images are sequentially superimposed and displayed one frame at a time on the finder image, the image data based on the JPEG standard does not employ a transparent color as a compression target. It is necessary to extract the background image portion of the character image in advance without superimposing it on the finder image.
[0071]
For this reason, the CPU 9 determines whether or not each character image is within the range of the hue level corresponding to the background image portion while sequentially detecting the hue level for each pixel in the frame, and based on the determination result. By obtaining the range of the background image portion for each frame, the background image portion is extracted from the character image.
[0072]
(2-5) Brightness adjustment
When the brightness of the character image displayed on the display screen 12A of the display screen 12 is different from the brightness of the surrounding finder image, the brightness of the character image is made to match the brightness of the surrounding finder image, whereby the character image is displayed. Is adapted to the peripheral image so that expression with more realistic and realistic feeling can be performed.
[0073]
Specifically, the CPU 9 calculates the average value of the luminance level of the entire character image and the average value of the luminance level of the entire finder image, respectively, so that the luminance level of the entire character image matches the luminance level of the entire finder image. adjust.
[0074]
In addition, such adjustment of the brightness level of the character image and the entire finder image is performed not only automatically, but also when the user does not adjust the brightness level to a desired state, the brightness levels of both the brightness levels are manually adjusted by the user operation. It has been made to be able to.
[0075]
(3) Character animation display processing procedure RT1
In the digital camera 1, when the character mode is selected by a user operation in a state where the finder image is displayed on the display screen 12A of the display screen 12, the CPU 9 executes the animation display processing procedure RT1 shown in FIG. Starting from step SP0, in the following step SP2, the display screen 12A of the display screen 12 first displays an animation of the character corresponding to the unit motion representing "turns to the front".
[0076]
Subsequently, the CPU 9 determines whether or not a request based on the operation content of the user or the status of the device inside the digital camera has been received. If an affirmative result is obtained, the process proceeds to step SP3, where the unit operation corresponding to the request is performed. It is determined whether or not the transition is possible, that is, whether or not the transition probability is 0.
[0077]
If a positive result is obtained in step SP3, this means that it is possible to continuously transition from the current unit operation to the unit operation corresponding to the request. At this time, the CPU 9 proceeds to step SP4 to Is superimposed and displayed on the display screen 12A of the display screen 12.
[0078]
On the other hand, if a negative result is obtained in step SP3, this indicates that the transition to the unit operation corresponding to the request becomes discontinuous, and at this time, the CPU 9 approximates the unit operation corresponding to the request. After selecting an optimal unit operation capable of maintaining continuity from the unit operations to be performed, an animation of the unit operation is superimposed on the display screen 12A of the display screen 12.
[0079]
On the other hand, if a negative result is obtained in step SP2, it indicates that this is a normal time in which no request is received, and at this time, the CPU 9 proceeds to step SP6 and performs a unit operation that can transition from the current unit operation. Is randomly determined, and the animation of the determined unit operation is superimposed on the display screen 12A of the display screen 12.
[0080]
When the animation of the unit operation is displayed in steps SP4 to SP6, the CPU 9 proceeds to step SP7, determines whether the next request has been received by the end of the unit operation, and receives a positive result. In this case, the process returns to step SP3 and repeats the same processing as described above.
[0081]
On the other hand, if a negative result is obtained in step SP7, this means that no request has been received. At this time, the CPU 9 proceeds to step SP8 and determines whether or not the selection of the character mode has been canceled by the user's operation. Judge.
[0082]
If a negative result is obtained in step SP8, the CPU 9 returns to step SP6 again and superimposes the animation of the unit operation capable of maintaining the continuity selected at normal time on the display screen 12A of the display screen 12, While repeating each loop of steps SP7-SP3-SP4-SP7 or steps SP7-SP8-SP6-SP7, it waits until a negative result is obtained in step SP8.
[0083]
On the other hand, if a positive result is obtained in step SP8, this means that the animation display of the character has been terminated by the user's operation, and at this time, the CPU 9 proceeds directly to step SP9 to execute the animation display processing procedure RT1. To end.
[0084]
(4) Conversation display processing procedure RT2
The digital camera 1 has a function of communicating with the user using the character in addition to the animation display processing of the character.
[0085]
Specifically, in a state where the character is displayed as an animation on the display screen 12A of the display screen 12, a curve (hereinafter, referred to as a line) surrounding a character string of dialogue (line) drawn in a shape blowing out from a portion corresponding to the mouth of the character. This is referred to as ballooning) and is superimposed on the finder image to be displayed as an image, thereby allowing the user to visually confirm the contents of the dialogue included in the balloon image.
[0086]
The dialog content contained in the balloon image is a conversation content responding to a request such as a user operation content or a device status inside the digital camera, and the CPU 9 sequentially generates objects for controlling the conversation content. .
[0087]
At this time, based on the control of the CPU 9, the memory controller 7 displays the coordinate position of the balloon image on the display screen 12A of the display screen 12 as necessary on a menu bar (not shown) displayed on the display screen 12A. ), Etc., are selected at an optimum position within a display range that does not obscure.
[0088]
At this time, when it is necessary to move the animation of the character in order to display the balloon image at the optimum position on the display screen 12A of the display screen 12, the memory controller 7 controls each character forming the character image sequence to be animated. The coordinate position of the image is determined to be an optimal position on the premise that continuity between unit operations is maintained.
[0089]
In the digital camera 1, when the conversation display mode is selected by a user operation in a state where the finder image is displayed on the display screen 12A of the display screen 12, the CPU 9 executes the conversation display processing procedure RT2 shown in FIG. Is started from step SP10, and in the following step SP11, it is determined whether or not the above-described character mode has already been selected.
[0090]
If a positive result is obtained in step SP11, the CPU 9 proceeds to step SP12, and proceeds to step SP13 while displaying the animation of the character on the display screen 12A of the liquid crystal monitor 12 in the current unit operation.
[0091]
On the other hand, if a negative result is obtained in step SP11, the CPU 9 proceeds to step SP14 to cause the display screen 12A of the liquid crystal monitor 12 to first display an animation of the character corresponding to the unit operation indicating "turn to the front". After that, the process proceeds to step SP13.
[0092]
In this step SP13, the CPU 9 determines whether or not a request has been received based on the content of the user's operation or the status of the device inside the digital camera. It is determined whether the balloon image can be displayed.
[0093]
If a positive result is obtained in step SP15, the CPU 9 proceeds to step SP17 to display a balloon image in response to the request in accordance with the animation of the character, and then proceeds to step SP18.
[0094]
On the other hand, if a negative result is obtained in step SP15, this means that the balloon image cannot be displayed while maintaining continuity while the unit motion of the character is currently being performed or at the current animation display position of the character. At this time, the CPU 9 proceeds to step SP16, waits for the current unit operation to end, or moves the animation display position of the character to the optimum position. Thereafter, the CPU 9 proceeds to step SP17, displays a balloon image responding to the request in accordance with the animation of the character, and then proceeds to step SP18.
[0095]
In this step SP18, the CPU 9 determines whether or not the next request has been received during the display of the balloon image. If an affirmative result is obtained, the process proceeds to step SP15 again to repeat the same processing as described above. If a negative result is obtained, the process directly proceeds to step SP19.
[0096]
On the other hand, when a negative result indicating that the request is not received is obtained in step SP13, the CPU 9 proceeds directly to step SP19.
[0097]
In this step SP19, the CPU 9 determines whether or not the selection of the conversation display mode is canceled by the operation of the user, and if an affirmative result is obtained, proceeds directly to step SP20 to end the conversation display processing procedure RT2. .
[0098]
On the other hand, if a negative result is obtained in step SP19, the CPU 9 returns to step SP13 again and repeats the same processing as described above until a negative result is obtained in step SP19.
[0099]
(5) Imprinting characters on still images
In the present embodiment, in addition to the above-described animation display method of the character, in the digital camera 1, the finder image actually displayed on the display screen 12 </ b> A of the liquid crystal monitor 12 at the imaging timing according to the user operation is displayed. The finder image can be recorded as a still image on the memory card MC with the character image superimposed thereon.
[0100]
As a specific still image generation method, the memory controller 7 writes a frame image (finder image) based on a video signal S4 obtained as an imaging result in a capture area of the RAM 10 at an imaging timing according to a user operation. Before performing compression coding or image size conversion according to the JPEG standard, a character image is superimposed on a frame image written in the capture area and written, thereby realizing the printing of the character image on the finder image. ing.
[0101]
At this time, the coordinate position at which the character image is superimposed on the finder image as a still image is determined in correspondence with the coordinate position of the character image displayed on the display screen 12A of the liquid crystal monitor 12. The background removal processing and the brightness adjustment described above are performed in the same manner as in the animation display method.
[0102]
In the RAM 10 according to the present embodiment, since the capture area is larger than the first or second finder area, a larger character image is required than when displaying on the display screen 12A of the liquid crystal monitor 12.
[0103]
Therefore, two types of character images for animation (for example, 64 × 80 pixels) and corresponding character images for still images (for example, 408 × 512 pixels) are prepared and stored in the memory card MC. The image size ratio between the character image for animation and the character image for still image is the ratio between the capture area AR4 and the first or second finder area AR2, AR3.
[0104]
Actually, before the character image for the still image is written over the frame image (finder image) written in the capture area AR4 of the RAM 10, the character image is read from the memory card MC and the image work area of the RAM 10 is written. After recording in the JPEG file work area AR6Y constituting the AR6, the data is subsequently decompressed via the data compression / decompression unit 20 and recorded in the low image work area AR6Z constituting the image work area AR6.
[0105]
By creating a still image on which such a character image is imprinted, and printing out the still image as a photograph or displaying the still image on a display screen of a personal computer, the presence of the character further enhances the presence. It is possible to improve the amusement for the user as well as to provide.
[0106]
(6) Reflecting characters in moving images
In the present embodiment, in the same manner as in the above-described printing of a character on a still image, in digital camera 1, a plurality of finder images actually displayed on display screen 12A of liquid crystal monitor 12 in accordance with a user operation are displayed. A character image group can be recorded on the memory card MC as a moving image in a state where the character image group is superimposed on a frame-by-frame basis on the finder image group in real time.
[0107]
As a specific moving image generation method, the memory controller 7 writes a frame image group (finder image group) based on a video signal S4 obtained as an imaging result in a movie area of the RAM 10 during imaging according to a user operation. Thereafter, the character image group is superimposed and written for each frame on the frame image group written in the movie area, thereby realizing the reflection of the character image group on the finder image group.
[0108]
Note that, in the RAM 10 in the present embodiment, the movie area is the same as the first or second finder area, and therefore, a character image group for animation can be used for reflection in a moving image. The other processing is performed in the same manner as that for imprinting a still image, thereby obtaining an effect that the presence of the character can make the presence more realistic and improve the amusement for the user.
[0109]
(7) Operation and effect according to the present embodiment
With the above configuration, in the digital camera, for a character capable of expressing a plurality of types of unit actions, first, all character image groups necessary for constructing the character are read from the memory card MC and recorded in the RAM 10. .
[0110]
Subsequently, a finder image group obtained as an imaging result is displayed on the display screen 12A of the liquid crystal monitor 12 mounted on the digital camera 1 in real time, and at the same time in synchronization with the display timing, randomly or user operation contents or digital contents are displayed. An animation based on a character image sequence read from the RAM 10 is superimposed and displayed for each unit operation determined by a request such as a device status inside the camera.
[0111]
At this time, by setting the transition condition between unit actions so that the next unit action that transitions from the current unit action of the character is connected while maintaining continuity, the animation of the character naturally and smoothly in the display screen The character can be moved, and an impression as if the character is present in the display screen can be given to the user.
[0112]
In a digital camera, a character animated and displayed on the display screen 12A of the liquid crystal monitor 12 generates a balloon image based on a request such as a user's operation content or a device status inside the digital camera, as if the character is emitting the character. The superimposed display on the display screen 12A allows the user to visually confirm the dialog content included in the balloon image with amusement.
[0113]
Furthermore, in the digital camera 1, a still image or a moving image is displayed in a state in which a character image is superimposed at an imaging timing according to a user operation among finder images as a still image or a moving image actually displayed on the display screen 12A of the liquid crystal monitor 12. If the still image or the moving image is printed or displayed on a monitor by recording the image on the memory card MC, the mere imaging result can be made more realistic by the presence of the character, and the amusement for the user can be improved. Can be done.
[0114]
According to the above configuration, in this digital camera, in the state where the finder image obtained as the imaging result is displayed in real time on the display screen 12A of the liquid crystal monitor 12, the operation content of the user is randomly or in synchronization with the display timing. By simultaneously superimposing and displaying the animation of the character for each unit operation determined by the request such as the status of the device inside the digital camera, etc., and by giving continuity to the connection between the unit operations, it is as if the character It is possible to give the user an impression as if the digital camera 1 is present in the display screen 12A, and thus it is possible to realize the digital camera 1 capable of improving the entertainment property.
[0115]
(8) Other embodiments
In the above-described embodiment, a case has been described in which the digital camera 1 as shown in FIG. 1 is applied as the imaging device according to the present invention. However, the present invention is not limited to this, and various other configurations are available. And an image display device as a mobile device such as a wristwatch, a mobile phone, and a personal digital assistant (PDA).
[0116]
Further, in the above-described embodiment, a case has been described where the imaging means for photographing the subject is configured by the imaging unit 3, the CDS and AGC unit 4, the A / D conversion unit 5, and the signal processing unit 6. The present invention is not limited to this, and an imaging unit having various other configurations may be applied as long as the imaging result of the subject can be obtained as image data.
[0117]
Further, in the above-described embodiment, a case has been described in which the liquid crystal monitor 12 mounted on the digital camera 1 is applied as a display unit that displays a captured image as a captured result, but the present invention is not limited to this. However, the present invention can be widely applied to display means having various other configurations, and is separated from the main body of the digital camera 1 if it is electrically connected via a cable such as a cable or wirelessly via a wireless LAN or the like. May be.
[0118]
Further, in the above-described embodiment, a character image sequence for expressing each unit operation is assigned to each of a plurality of types of unit operations performed by the character, and all the character image sequences are stored in advance. As the storage means, a case has been described in which the RAM 10 in which all the character image strings read from the memory card MC are taken in is applied. However, the present invention is not limited to this, and the present invention is not limited to having an external memory at all. Alternatively, all character image strings may be recorded in the RAM 10 as an internal memory in advance.
[0119]
Further, the present invention is not limited to the case where a character image sequence for expressing each unit operation is stored in advance for each of a plurality of types of unit operations performed by the character. For each type of unit operation, all or a part of 3D data (three-dimensional data) representing the character image at the head of each unit operation is previously stored in the RAM (storage unit) 10, and a plurality of types of unit operations are stored. , The unit operation is sequentially designated at random, and the three-dimensional data corresponding to each unit operation is sequentially read out or supplemented by using other three-dimensional data as necessary, thereby starting based on each of the three-dimensional data. While sequentially generating a character image representing the corresponding unit action from the image, the character image sequence is displayed on the display means as an animation. It may be caused to superimposed on the captured image are.
[0120]
At this time, when processing the three-dimensional data using so-called 3D engine software, another three-dimensional data is smoothed using an interpolation method using a Bezier curve so that an image based on the three-dimensional data is smoothed. The case where three-dimensional data between data is complemented has been described. However, the present invention is not limited to this. In addition, a B-spline (Beta-Spline) curve, a Hermite (Hermite) curve, a rational Bezier ( The three-dimensional data may be complemented using a compound curve such as a Rational Bezier curve and a NURBS (Non Uniform Rational Beta-Spline) curve.
[0121]
Further, in the above-described embodiment, a unit operation is randomly designated in order from all the character image sequences stored in the RAM (storage means) 14, and the character image sequence corresponding to each of the unit operations is sequentially determined. The case where the reading control means for reading is constituted by the memory controller 7 and the CPU 9 has been described. However, the present invention is not limited to this. If the unit operations can be designated in the order in which the respective unit operations are transitioned so as to maintain the connection, the present invention can be widely applied to read control means having other various configurations.
[0122]
On the other hand, when the above-described three-dimensional data is used, the reading control unit randomly specifies a unit operation sequentially from all the three-dimensional data stored in the RAM (storage unit) 10 and What is necessary is just to sequentially read or complement the three-dimensional data corresponding to the unit operation.
[0123]
Further, the memory controller 7 and the CPU 9 constituting the reading control means store the data in the RAM (storage means) 14 when receiving a request based on various operation contents by the user or a request based on the state of the device inside the digital camera. A case has been described in which a unit operation corresponding to the request is designated from among all the character image sequences that have been performed, and the character image sequence corresponding to the unit operation is read out. Various situations can be freely set inside and outside the device (including inside as well as outside).
[0124]
For example, an arrow pointer that can be moved by a mouse operation or the like may be prepared as the user's operation content, and the arrow point may be specified at a position where the character wants to move according to the user's operation. When the designated position is determined in this way, the character may move to the designated position.
[0125]
When the user selects an item “tutorial” from a menu displayed on the display screen 12A of the liquid crystal monitor 12, the character may perform a tutorial on how to use the digital camera 1. Specifically, a description will be given of how the auto focus is fixed by half-pressing the shutter and how to use the zoom function. In the flow of explanation, a pattern is determined, and a request is sent to the object according to the flow, and it is also determined whether or not the button operation has been performed correctly.
[0126]
Further, as the content of the user's operation, when the user performs a specific operation in a certain state, the character may perform a reaction. Specifically, when the shutter is half-pressed, the character performs an operation of turning back, and after the flash fires, the character makes a gesture of “bright”.
[0127]
Further, as the user's operation content, after the user has taken a picture, the character may evaluate the picture and the state at the time of imaging. In this case, for those that should be retaken, that effect will be described. Specifically, he advises that it is better to take the picture again with the flash because it is too dark, or that it is better to take the picture again because there is a possibility of blurring and camera shake. Darkness can be judged from the image, and out-of-focus and camera shake are also judged based on the image (same method as judgment by contrast and auto focus function). May be determined.
[0128]
On the other hand, as a situation inside and outside the device, the character may be moved based on information obtained from an existing image recognition engine. Specifically, it is possible to change the operation or position of the animation by recognizing a two-dimensional pattern code, or to change the operation, position, or color of the character in response to a specific color.
[0129]
Alternatively, the character may report that the remaining battery power is low as a situation inside or outside the device. Since many digital cameras have information about the remaining battery level, such information is used to convey that the battery level is low through animation or conversation. For example, a state in which the character is in trouble is expressed by an animation, and a request to "charge the battery" is issued.
[0130]
Further, as a situation inside and outside the device, the character may notify the remaining amount of the memory card MC for storing images (remaining number of images that can be taken, and possible time of imaging). For example, when the number of remaining images is two or less, or when the available imaging time is 15 seconds or less, information is transmitted by the conversation function. In particular, a warning is issued when imaging is impossible due to insufficient remaining amount.
[0131]
Further, as a situation inside and outside the apparatus, when the digital camera communicates with peripheral equipment, the character may transmit the progress information. For example, if the communication speed becomes slow, the character is tired by an animation, or if there is a communication failure, the user is notified of that. In the case of wireless communication, a radio wave condition may be transmitted.
[0132]
Further, as the situation inside and outside the apparatus, the character may notify the user of the situation of the temperature and humidity around the camera based on the detection results of various sensors provided in the digital camera 1. A warning is issued especially when the camera exceeds the rated range. Furthermore, by measuring the acceleration and speed, the user is notified when a shock is applied to the camera, and encourages the user not to handle the camera unnecessarily.
[0133]
Further, as a situation inside and outside the apparatus, the character may transmit various warnings (cautions) set in the digital camera 1 to the user. For example, when the memory card MC is suddenly removed, or when writing access to the RAM 10 or the like cannot be performed, the character notifies the user as a warning.
[0134]
Further, in the above-described embodiment, a display in which each character image sequence read from the RAM 10 is superimposed on a finder image (captured image) displayed on the display screen 12A of the liquid crystal monitor (display means) 12 as an animation. Although the case where the control means is constituted by the memory controller 7, the CPU 9 and the video encoder 11 has been described, the present invention is not limited to this, and the OSD 21 may be added. The OSD 21 may be used. In short, as long as the character image sequence can be superimposed and displayed on the finder image as an animation, various configurations of the display control means may be applied.
[0135]
In the case where the above-described three-dimensional data is used, the display control unit is a character that represents a corresponding unit action from a start image based on each of the three-dimensional data read by the memory controller 7 and the CPU 9 (read control unit). While the images are sequentially generated, the character image sequence may be superimposed on the captured image displayed on the liquid crystal monitor (display unit) 12 as an animation.
[0136]
At this time, a case has been described in which the display control means has a function of generating a character image sequence from each three-dimensional data in advance, but the present invention is not limited to this, and the RAM (storage means) 10 So-called 3D engine software may be stored in advance as a program for processing three-dimensional data, and may be read from the RAM (storage means) 10 as needed.
[0137]
Further, in the above-described embodiment, when a request is received based on the operation contents of the user and / or the situation inside and outside the apparatus, the request is responded based on the animation composed of each character image sequence (or three-dimensional data). Although the case in which the balloon image is superimposed on the captured image has been described, the present invention is not limited to this, and various display contents such as not only balloon images but also meaningful marks are associated with the animation of the character. You may make it display simultaneously.
[0138]
Further, in the above-described embodiment, an animation composed of each character image sequence (or each three-dimensional data) is superimposed and displayed on a finder image (captured image) displayed on the display screen 12A of the liquid crystal monitor (display means) 12. In the state described above, the recording means for recording the display state of the display screen 12A as a still image or a moving image by the user's operation is configured from the CPU 9 and the memory card MC, but the present invention is not limited to this. Instead, the present invention may be widely applied to an external memory having various configurations other than the memory card MC, and may be applied not only to the external memory but also to an internal memory such as the RAM 10.
[0139]
In addition to recording an animation consisting of each character image sequence as a still image or a moving image in a state of being superimposed and displayed on a finder image (captured image), without recording the animation, the coordinates of the character at the time of shooting by a user's operation are recorded. After the position is stored and only the finder image is recorded as a still image or a moving image, the animation of the character may be superimposed and displayed on the finder image composed of the still image or the moving image as necessary. For example, character information may be added to a JPEG tag so that the character can be displayed or hidden as needed when viewing a photograph.
[0140]
Further, in the above-described embodiment, the size of the character displayed on the display screen 12A of the liquid crystal monitor 12 remains constant. However, the present invention is not limited to this. May be changed in size or position. That is, the size and position of the character may be changed in conjunction with the enlargement or reduction of the screen by zooming. As a result, an effect that the presence of the character can be made more realistic can be obtained. Since the target size and position can be calculated from the zoom information, the character size and position are superimposed and displayed based on the calculation result. When changing the size, it is necessary to superimpose and display the image recorded in the low image work area in the image work area of the RAM 10 once via the memory controller 7.
[0141]
Furthermore, in the above-described embodiment, the case where no speaker is mounted on the digital camera 1 has been described, but a speaker may be mounted. In this case, various sound effects may be produced in accordance with the contents of the animation operation of the character. For example, when the character performs a walking motion, footsteps are output according to the walking. The voice data may be written in a program in advance, and the voice data based on the voice data may be emitted at a predetermined timing by the animation object by sending the voice data to the sound chip in accordance with the animation.
[0142]
Furthermore, in the above-described embodiment, a case has been described where the animation of the character is superimposed and displayed on the real-time finder image, and is limited to a single character. However, the present invention is not limited to this, and a plurality of types of characters may be used. A character may be set so that the user can select a desired character.
[0143]
Furthermore, in the above-described embodiment, a case has been described in which image data is compression-encoded according to a data format based on the JPEG standard. However, in addition to this, TIFF (Tagged Image File Format) and DCF (Design Rule for Camera File System), Exif (Exchangeable Image File Format), CIFF (Camera Image File Format), GIF (Graphics Interchange Format), BMP (Bit Map), PICT (Quick Draw Picture Format) and FPX (Flash Pix) The image data D1 may be managed according to the image file format described above, and may be compressed and encoded as necessary.
[0144]
【The invention's effect】
As described above, according to the present invention, an imaging unit for imaging a subject, a display unit for displaying an image captured by the imaging unit, and a plurality of unit operations performed by a character for expressing each unit operation A character image sequence is assigned to each of the character image sequences, and a storage unit in which all of the character image sequences are stored in advance, and a unit operation is sequentially designated at random from a plurality of types of unit operations to correspond to each of the unit operations. Reading control means for sequentially reading out the character image sequence from all the character image sequences stored in the storage means; and image pickup displayed on the display means as an animation of each character image sequence read out by the reading control means Display control means for superimposing and displaying the image on the image, and the read control means controls each of the unit operations so that the unit operations are connected while maintaining continuity. By specifying the order in which the position motions are shifted, it is possible to give the user an impression as if the character is present in the display screen, and thus it is possible to realize an imaging device capable of improving the entertainment property. .
[0145]
Further, according to the present invention, imaging means for imaging a subject, display means for displaying an image captured by the imaging means, and, for each of a plurality of types of unit actions performed by the character, representing a leading character image of each unit action. Dimensional data is assigned to each of the three-dimensional data, and all or a part of the three-dimensional data is stored in advance. Reading control means for sequentially reading out the corresponding three-dimensional data from the storage means or supplementing with other three-dimensional data as necessary, and a start image based on each three-dimensional data read or complemented by the reading control means , While sequentially generating a character image representing the corresponding unit action, the character image sequence is displayed on the display means as an animation. Display control means for superimposed display on the captured image is provided, and the readout control means designates the order of transition of each unit operation so that each unit operation is connected while maintaining continuity. It is possible to give the user an impression as if the character is present in the display screen, and thus it is possible to realize an imaging device capable of improving entertainment.
[0146]
Further, according to the present invention, a first step of displaying a captured image obtained by capturing an image of a subject, and a character image sequence for expressing each unit action assigned to each of a plurality of types of unit actions performed by the character A second step of sequentially designating unit motions at random from among them and sequentially reading a character image sequence corresponding to each of the unit motions; and an imaging process in which each read character image sequence is displayed as an animation. A third step of superimposing and displaying the images on the image is provided. In the second step, the unit operations are specified in the order of transition so that the unit operations are connected while maintaining continuity. Gives the user the impression that the character is present in the display screen, thus improving the entertainment The imaging method can be realized that can.
[0147]
Further, according to the present invention, a first step of displaying a captured image obtained by capturing an image of a subject and a first character image representing a head character image of each unit action assigned to each of a plurality of types of unit actions performed by the character A second unit for sequentially designating unit operations at random from all or a part of the dimensional data and sequentially reading out three-dimensional data corresponding to the unit operations or using other three-dimensional data as necessary. While sequentially generating a character image representing a corresponding unit action from a step and a start image based on the read or complemented three-dimensional data, the character image sequence is superimposed on the displayed captured image as an animation. And a third step of displaying. In the second step, each of the unit operations is connected so as to maintain continuity between the unit operations. The by you to specify the order of transitioning, though the character can give the user the impression of being present in the display screen, thus can realize an imaging method capable of improving entertainment property.
[0148]
Further, according to the present invention, a display means for displaying an image and a character image sequence for expressing each unit operation are assigned to each of a plurality of types of unit operations performed by the character. Is stored in advance, and all the character images stored in the storage means are sequentially designated at random from among a plurality of types of unit operations, and a character image sequence corresponding to each unit operation is designated. Read control means for sequentially reading from the rows, and display control means for superimposing and displaying, as an animation, each character image sequence read by the read control means on an image displayed on the display means, wherein the read control means By specifying the order in which each unit operation is to be transitioned so that each unit operation is connected while maintaining continuity, Taka also can give the impression as if the character is present within the display screen to the user, thus the image display device can be realized capable of improving entertainment property.
[0149]
Further, according to the present invention, display means for displaying an image, and three-dimensional data representing a character image at the head of each unit action are assigned to each of a plurality of types of unit actions performed by the character. And a storage unit in which all or a part of the unit operation is stored in advance, and a unit operation is sequentially designated at random from a plurality of types of unit operations, and three-dimensional data corresponding to each unit operation is sequentially read from the storage unit or A readout control unit that supplements the image data with other three-dimensional data as needed, and a character image expressing a corresponding unit action from a start image based on the three-dimensional data read out or complemented by the readout control unit. Display control means for superimposing the character image sequence on the captured image displayed on the display means as animation while sequentially generating the character image sequence. The reading control means specifies the order in which the unit operations are to be transitioned so that the unit operations are connected with maintaining continuity, so that the character is present as if the character exists in the display screen. Thus, it is possible to provide an image display device that can give a user an unusual impression, and thus can improve entertainment.
[0150]
Furthermore, according to the present invention, the first step of displaying an image and the unit motion are sequentially performed from a character image sequence for expressing each unit motion assigned to each of a plurality of types of unit motions performed by the character. A second step of sequentially reading character image strings corresponding to the respective unit actions by designating them randomly, and a third step of superimposing and displaying each read character image string as an animation on the displayed image In the second step, in the second step, each unit operation is specified in the order in which the unit operations are transitioned so as to maintain continuity with each other, so that the character exists in the display screen. Can provide the user with the impression of being in the middle, thus realizing an image display method that can enhance entertainment. .
[0151]
Further, according to the present invention, the first step of displaying an image and the whole or part of the three-dimensional data representing the leading character image of each unit action assigned to each of a plurality of types of unit actions performed by the character A second step of sequentially designating the unit operations at random and sequentially reading out the three-dimensional data corresponding to the respective unit operations or supplementing with other three-dimensional data as necessary; A third step of superimposing and displaying, as an animation, the character image sequence as an animation on the displayed captured image while sequentially generating a character image representing the corresponding unit motion from the start image based on the obtained three-dimensional data. In the second step, the unit operations are designated in the order in which the unit operations are transitioned so that the unit operations are connected while maintaining continuity. By was Unishi, though the character can give the user the impression of being present in the display screen, thus possible to realize an image display method capable of improving entertainment property.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a digital camera according to an embodiment.
FIG. 2 is a schematic diagram for explaining a recording area which is an internal configuration of the RAM shown in FIG. 1;
FIG. 3 is a schematic diagram for explaining an action transition table that defines transition conditions for a unit action of a character;
FIG. 4 is a schematic plan view illustrating an example of a unit action of a character.
FIG. 5 is a schematic diagram for explaining a character image sequence for each unit operation.
FIG. 6 is a flowchart for explaining a character animation display processing procedure;
FIG. 7 is a flowchart for explaining a conversation display processing procedure;
[Explanation of symbols]
1 ... Digital camera, 6 ... Signal processing unit, 7 ... Memory controller, 9 ... CPU, 10 ... RAM, 11 ... Video concoder, 12 ... LCD monitor, 16 ... Operation unit, MC ... ... memory card, RT1 ... animation display processing procedure, RT2 ... conversation display processing procedure.

Claims (28)

Imaging means for imaging a subject;
Display means for displaying an image captured by the imaging means;
For each of a plurality of types of unit actions performed by the character, a character image sequence for expressing each unit motion is assigned, and storage means in which all of the character image sequences are stored in advance,
From among the plurality of types of unit motions, the unit motions are sequentially designated at random, and the character image sequence corresponding to each unit motion is sequentially selected from all the character image sequences stored in the storage means. Read control means for reading;
Display control means for superimposing and displaying each of the character image strings read by the read control means as an animation on the captured image displayed on the display means,
An image pickup apparatus according to claim 1, wherein said read control means designates the unit operations in an order of transition so that the unit operations are connected while maintaining continuity. The reading control means includes:
When receiving a request based on the content of a user operation and / or a situation inside or outside the device, the unit operation corresponding to the request is specified, and the character image sequence corresponding to the unit operation is read from the storage unit. The imaging device according to claim 1, wherein: The display control means includes:
When a request based on the user's operation content and / or a situation inside or outside the device is received, a balloon image corresponding to the request is superimposed on the captured image based on the animation including the character image sequence. The imaging device according to claim 1, wherein: In a state where the animation composed of each of the character image strings is superimposed and displayed on the captured image displayed on the display unit by the display control unit, the display state of the display unit is changed to a still image or a moving image by a user operation. 2. The image pickup apparatus according to claim 1, further comprising a recording unit that records the image as an image. Imaging means for imaging a subject;
Display means for displaying an image captured by the imaging means;
Storage means in which three-dimensional data representing a leading character image of each of the unit actions is assigned to each of a plurality of types of unit actions performed by the character, and all or a part of each of the three-dimensional data is stored in advance;
From among the plurality of types of unit operations, the unit operations are sequentially designated at random, and the three-dimensional data corresponding to each of the unit operations is sequentially read out from the storage means or the other three-dimensional data is read as necessary. Read-out control means complemented by using
While sequentially generating a character image representing the corresponding unit action from a start image based on each of the three-dimensional data read or complemented by the read control means, the character image sequence is used as an animation to display the character image sequence. Display control means for superimposing and displaying the captured image displayed on the
An image pickup apparatus according to claim 1, wherein said read control means designates the unit operations in an order of transition so that the unit operations are connected while maintaining continuity. The reading control means includes:
When receiving a request based on the content of the user's operation and / or the situation inside and outside the device, the unit operation corresponding to the request is designated, and the three-dimensional data corresponding to the unit operation is read out from the storage unit or The imaging device according to claim 5, wherein the imaging device is complemented. The display control means includes:
When a request based on the user's operation content and / or a situation inside and outside the device is received, a balloon image corresponding to the request is generated based on the animation including the character image sequence generated based on each of the three-dimensional data. The imaging apparatus according to claim 5, wherein the imaging apparatus is superimposed on a captured image. In a state where the animation composed of the character image sequence generated based on each of the three-dimensional data by the display control means is superimposed on the captured image displayed on the display means, the display means is operated by a user. The image pickup apparatus according to claim 5, further comprising a recording unit that records the display state of the image as a still image or a moving image. A first step of displaying a captured image obtained by imaging the subject;
From the character image sequence for expressing each of the unit actions assigned to each of a plurality of types of unit actions performed by the character, the unit actions are randomly designated in order, and the character corresponding to each of the unit actions is designated. A second step of sequentially reading the image sequence;
A third step of superimposing and displaying each of the read character image strings as an animation on the displayed captured image. In the second step, each of the unit actions establishes continuity. An imaging method characterized by designating an order in which each unit operation is transitioned so as to be maintained and connected. In the second step,
When receiving a request based on the content of a user operation and / or a situation inside or outside the device, the unit operation corresponding to the request is designated, and the character image sequence corresponding to the unit operation is read. The imaging method according to claim 9. In the third step,
When a request based on the user's operation content and / or a situation inside or outside the device is received, a balloon image corresponding to the request is superimposed on the captured image based on the animation including the character image sequence. The imaging method according to claim 9, wherein: Recording the display state as a still image or a moving image by a user operation in a state where the animation including the character image strings is superimposed on the displayed captured image in the third step. The method according to claim 9, further comprising the steps of: A first step of displaying a captured image obtained by imaging the subject;
From the whole or a part of the three-dimensional data representing the leading character image of each of the unit actions assigned to each of the plurality of types of unit actions performed by the character, the unit actions are sequentially designated at random, and the respective unit actions are designated. A second step of sequentially reading out the three-dimensional data corresponding to the above or supplementing the three-dimensional data with other three-dimensional data as necessary;
While sequentially generating a character image representing the corresponding unit action from a start image based on the read or complemented three-dimensional data, the displayed captured image is displayed using the character image sequence as an animation. And a third step of superimposing and displaying the unit operations. In the second step, the unit operations are specified in the order of transition so that the unit operations are connected while maintaining continuity. Imaging method. In the second step,
When receiving a request based on the content of the user's operation and / or the situation inside and outside the device, specify the unit operation corresponding to the request and read or supplement the three-dimensional data corresponding to the unit operation. The imaging method according to claim 13, wherein: In the third step,
When a request based on the user's operation content and / or a situation inside and outside the device is received, a balloon image corresponding to the request is generated based on the animation including the character image sequence generated based on each of the three-dimensional data. The imaging method according to claim 13, wherein the imaging method is superimposed on the captured image. In a state in which the animation composed of the character image sequence generated based on each of the three-dimensional data in the third step is superimposed on the displayed captured image, the display state is stopped by a user operation. The imaging method according to claim 13, further comprising a fourth step of recording as an image or a moving image. Display means for displaying an image;
For each of a plurality of types of unit actions performed by the character, a character image sequence for expressing each unit motion is assigned, and storage means in which all of the character image sequences are stored in advance,
From among the plurality of types of unit motions, the unit motions are sequentially designated at random, and the character image sequence corresponding to each unit motion is sequentially selected from all the character image sequences stored in the storage means. Read control means for reading;
Display control means for superimposing the character image sequence read by the read control means on the image displayed on the display means as animation.
The image display device, wherein the reading control means specifies the order in which the unit operations are changed so that the unit operations are connected while maintaining continuity. The reading control means includes:
When receiving a request based on the content of a user operation and / or a situation inside or outside the device, the unit operation corresponding to the request is specified, and the character image sequence corresponding to the unit operation is read from the storage unit. The image display device according to claim 17, wherein: The display control means includes:
When a request is received based on the operation content of the user and / or a situation inside and outside the device, a balloon image corresponding to the request is superimposed on the image based on the animation including the character image sequence. The image display device according to claim 17, wherein Display means for displaying an image;
Storage means in which three-dimensional data representing a leading character image of each of the unit actions is assigned to each of a plurality of types of unit actions performed by the character, and all or a part of each of the three-dimensional data is stored in advance;
From among the plurality of types of unit operations, the unit operations are sequentially designated at random, and the three-dimensional data corresponding to each of the unit operations is sequentially read out from the storage means or the other three-dimensional data is read as necessary. Read-out control means complemented by using
While sequentially generating a character image representing the corresponding unit action from a start image based on each of the three-dimensional data read or complemented by the read control means, the character image sequence is used as an animation to display the character image sequence. Display control means for superimposing and displaying the captured image displayed on the
The image display device, wherein the reading control means specifies the order in which the unit operations are changed so that the unit operations are connected while maintaining continuity. The reading control means includes:
When receiving a request based on the content of the user's operation and / or the situation inside and outside the device, the unit operation corresponding to the request is designated, and the three-dimensional data corresponding to the unit operation is read out from the storage unit or The image display device according to claim 23, wherein the image display device complements the image display device. The display control means includes:
When a request based on the user's operation content and / or a situation inside and outside the device is received, a balloon image corresponding to the request is generated based on the animation including the character image sequence generated based on each of the three-dimensional data. The image display device according to claim 23, wherein the image display device superimposes the image on the image. A first step of displaying an image;
From the character image sequence for expressing each of the unit actions assigned to each of a plurality of types of unit actions performed by the character, the unit actions are randomly designated in order, and the character corresponding to each of the unit actions is designated. A second step of sequentially reading the image sequence;
A third step of superimposing and displaying each of the read character image strings as an animation on the displayed image, and in the second step, each of the unit actions maintains continuity. An image display method in which the unit operations are specified in the order in which the unit operations are transited so as to be connected. In the second step,
When a request based on the operation content of the user and / or a situation inside or outside the device is received, the unit operation corresponding to the request is designated from all the character image strings, and the character corresponding to the unit operation is designated. The image display method according to claim 20, wherein an image sequence is read. In the third step,
When a request is received based on the operation content of the user and / or the situation inside and outside the device, a balloon image corresponding to the request is superimposed on the image based on the animation including the character image sequence. The image display method according to claim 20, wherein: A first step of displaying an image;
From the whole or a part of the three-dimensional data representing the leading character image of each of the unit actions assigned to each of the plurality of types of unit actions performed by the character, the unit actions are sequentially designated at random, and the respective unit actions are designated. A second step of sequentially reading out the three-dimensional data corresponding to the above or supplementing the three-dimensional data with other three-dimensional data as necessary;
While sequentially generating a character image representing the corresponding unit action from a start image based on the read or complemented three-dimensional data, the displayed captured image is displayed using the character image sequence as an animation. And a third step of superimposing and displaying the unit operations. In the second step, the unit operations are specified in the order of transition so that the unit operations are connected while maintaining continuity. Image display method. In the second step,
When receiving a request based on the content of the user's operation and / or the situation inside and outside the device, specify the unit operation corresponding to the request and read or supplement the three-dimensional data corresponding to the unit operation. The image display method according to claim 26, characterized in that: In the third step,
When a request based on the operation content of the user and / or a situation inside or outside the device is received, a balloon image corresponding to the request is generated based on the animation including the character image sequence generated based on the three-dimensional data. The image display method according to claim 26, wherein the image is superimposed on an image.

Images