JP5178462B2 - Image creation apparatus and image creation program - Google Patents

Description

  The present invention relates to an image creation apparatus capable of drawing an image by handwriting and an image creation program executed in the apparatus.

2. Description of the Related Art Conventionally, a sketch drawing automatic creation device that creates a sketch drawing that automatically represents an original image from image data has been considered (Patent Document 1). The sketch drawing automatic creation device performs edge detection, thinning, broken line approximation, and dust deletion processing on the original image data by processing according to the edge sensitivity parameter, broken line approximation parameter, and dust size parameter. To draw line figures.
JP 2004-78411 A

  As described above, the conventional sketch drawing automatic creation apparatus automatically draws a line figure (sketch) based on the image data of the original image, and the user does not sketch. When a user draws a sketch image, it is considered not only to directly view the object to be drawn, but also to display the image to be drawn on the display and refer to this image. It was.

  However, for users who have not mastered painting techniques, even when an image is displayed on the display when practicing sketching, the drawing action is the same as when directly viewing the object to be drawn. It was not something to help.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide an image creation apparatus and an image creation program that can assist the user in drawing an image.

The present invention includes a touch panel capable of inputting an image handwritten by a user, a display integrated with the touch panel, an input unit for inputting an instruction to select an image to be drawn, and the input unit Auxiliary information indicating a drawing position for drawing by the user corresponding to the image indicated by the instruction input by selecting one of a plurality of types of auxiliary information corresponding to the drawing target image is generated. An auxiliary information generation unit, an auxiliary information display unit that displays auxiliary information generated by the auxiliary information generation unit on the display, and an image that inputs image data for the auxiliary information displayed by the auxiliary information display unit from the touch panel Input means and image display means for displaying according to the image data input by the image input means Characterized by comprising a.

  According to the present invention, it is possible to assist the user in drawing an image.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a functional configuration of an image creating apparatus according to the present embodiment. The image creation device is assumed to be realized by a portable game device, for example. Hereinafter, the game device will be described.

  As shown in FIG. 1, a CPU 10 is mounted on the game apparatus (image creation apparatus) in the present embodiment. The CPU 10 is connected to the RAM 11, GPUs 12 and 13, display controller 16, displays 17 and 18, interface unit 19, connector 23, and communication unit 26 via a bus.

  The CPU 10 controls the entire game device, and executes various processes according to programs stored in the RAM 11. The CPU 10 can execute sketch processing based on the image creation program in the present embodiment, and can draw (sketch) an image by handwriting by the user. In sketch processing, auxiliary information representing the drawing position for drawing by the user is generated and displayed corresponding to the image to be drawn, so that even a user who has not mastered painting technology can use painting technology. Make it easy to use sketches.

  The RAM 11 temporarily stores programs executed by the CPU 10 and data.

  The GPUs 12 and 13 generate display image data based on data for generating a display image stored in the RAM 11 in response to an instruction from the CPU 10. The GPU 12 is connected to the VRAM 14 and connected to the GPU 13. Is connected to the VRAM 15. The GPU 12 generates display image data to be displayed on the display 17 and draws it on the VRAM 14. Similarly, the GPU 13 generates display image data to be displayed on the display 18 and draws it on the VRAM 15.

  The displays 17 and 18 display images under the control of the display controller 16. The displays 17 and 18 are provided in each of two housings coupled so as to be openable and closable by, for example, a hinge mechanism. One display 18 is formed integrally with a touch panel 20 to be described later, and displays auxiliary information (for example, auxiliary lines) indicating a drawing position for the user to draw when sketch processing is executed. Thus, image data corresponding to the display position indicated by the auxiliary information can be input from the touch panel 20.

  The display controller 16 controls screen display on the displays 17 and 18, displays an image on the display 17 based on the image data stored in the VRAM 14, and displays the image on the display 17 based on the image data stored in the VRAM 15. The image is displayed at.

  The interface unit 19 is for connecting various devices. For example, a touch panel 20, a speaker 21, and a button group 22 are connected. The touch panel 20 inputs image (drawing) data corresponding to a position instruction on the panel surface using the pen 20a.

  The touch panel 20 is configured to be laminated and integrated with the display 18, for example, and thus can easily input an image (sketch) using auxiliary information (auxiliary lines or the like) displayed on the display 18.

The speaker 21 outputs operation sounds and various sounds (message voice, music, sound effects, etc.) according to various applications under the control of the CPU 10.
The button group 22 is for inputting an instruction from the user, and includes, for example, a cross button, a start button, a select button, an A button, a B button, an X button, a Y button, a power button, an L button, and an R button. It is out.

The connector 23 is for detachably connecting an external device. For example, the memory device 24 and the image input device 25 can be connected.
The memory device 24 is equipped with, for example, a ROM that stores game programs and a RAM that stores backup data in a rewritable manner. The game device can be operated as an image creation device by attaching the memory device 24 having the image creation program recorded in the ROM to the game device and executing the image creation program. In addition to the program, the RAM of the memory device 24 appropriately stores temporary data obtained by the CPU 10 executing the program.

  The image input device 25 is used to capture an image and input image data. For example, the image input device 25 is a camera device that inputs image data of a still image. The image data input by the image input device 25 can be used in sketch processing by an image creation program.

  The communication unit 26 is for controlling communication with other game devices and other devices connected via a network, and performs transmission and reception of data used in sketch processing, for example.

  Note that the image creation program in the present embodiment may be supplied not only to the game apparatus through an external storage medium such as the memory device 24 but also through a wired or wireless communication line. The image creation program may be recorded in advance in a non-volatile storage device mounted on the game device. The information storage medium for storing the image creation program is not limited to a nonvolatile semiconductor memory, and may be a CD-ROM, a DVD, or an optical disk storage medium similar to them.

FIG. 2 shows an example of auxiliary information management data for controlling sketch processing in the present embodiment.
In the image creation apparatus (image creation program) of this embodiment, images to be sketched (drawn) by a user are prepared in advance, and assistance provided to the user when each image is selected as a drawing target. Information is available. In the example of auxiliary information management data shown in FIG. 2, it is assumed that data for displaying an auxiliary line is set as auxiliary information.

  In the auxiliary information management data, a plurality of data indicating images that can be selected as sketch targets by the user are set (image 1, image 2,...). The image data is stored separately from the auxiliary information management data and includes, for example, still images, CG (computer graphics) images, and the like. The still image may be prepared in advance in the image creation program, or may be input from the image input device 25 by shooting by a user operation.

  A plurality of auxiliary lines that can be displayed are prepared for each image. The auxiliary lines differ depending on the image (motif image) to be sketched. For example, when using perspective to determine composition as a painting technique, assisting sketches by the user by presenting multiple lines extending from the vanishing point set in the image as auxiliary lines. can do. Moreover, the sketch of a motif can be assisted by showing the outline (outline) of a motif as an auxiliary line for beginners. In addition, when sketching a person, the person by the user is presented by presenting an auxiliary line that represents the human skeleton and an auxiliary line or outline that suggests the layout of the eyes, nose, mouth, etc. according to the orientation of the face. Can help with sketching. In addition to the auxiliary lines described above, auxiliary lines that can assist various sketches can be used.

  In the auxiliary information management data, a user level is set for information on each auxiliary line. The user level indicates the skill of the user's sketch, and is specified by the user (may be set in advance) before starting the sketch process. In the sketch process, when the auxiliary line is displayed, only those corresponding to the level designated by the user set in the auxiliary information management data are displayed. For example, the auxiliary line 1 of the image 1 is displayed when the user level is level 1 to 3, and the auxiliary line 2 is displayed when it is level 1 or 2. Thereby, it is possible to prevent the auxiliary line from being presented more than necessary by presenting the auxiliary line suitable for the skill level of the sketch to the user.

  In the present embodiment, it is assumed that CG (computer graphics) can be targeted in addition to general still images as images targeted for drawing (sketching) by the user. In the case of CG, it is not necessary to prepare auxiliary line data in advance in the auxiliary information management data, and it can be generated based on CG data (FIG. 3, auxiliary line information generation process).

  Further, although auxiliary lines have been described as an example of auxiliary information, other auxiliary information may be set in auxiliary information management data so that it can be displayed when the user sketches. For example, it may be a point display suggesting a drawing position, a color-coded display, or a text explanation display. The auxiliary lines described above can be used as auxiliary information for determining the composition. For example, when sketching (inputting) an image using colors, any color can be used at any position. You may make it display the auxiliary information which suggests what should be done.

Next, the operation of the game device in this embodiment will be described.
FIG. 3 is a flowchart illustrating sketch processing executed in the game device according to the present embodiment. Here, it is assumed that the user level of the user who uses the game apparatus is set in advance. Data indicating the user level is stored in the RAM 11 (or the memory device 24) and can be referred to by the CPU 10 at an arbitrary timing.

  First, when the game device is instructed to start a sketch process (image creation program) by a user's predetermined operation on the button group 22, the game apparatus executes the image creation program loaded in the RAM 11 to start the sketch process (step S1). A1).

  Next, the user selects a motif image to be sketched. Here, for example, it is assumed that a CG image or another still image (photograph) can be selected.

  Here, when a CG image is selected as the motif image (step A3, Yes), the CPU 10 executes an auxiliary line information generation process on the CG image (step A4).

FIG. 4 is a flowchart showing auxiliary line information generation processing for a CG image.
First, the CPU 10 displays a list of a plurality of motif images (CG images) prepared in advance on the display 18 and allows the user to select one of them by an operation with the pen 20a or an operation with the button group 22 (step) B1).

  When the target motif image is selected, the CPU 10 displays an adjustment screen for allowing the user to adjust the orientation and size of the CG image to be sketched based on the CG data of the selected image. 18 is displayed.

FIG. 6 shows an example of a screen for adjusting a CG image.
In the example shown in FIG. 6, an image display area in which a CG image selected as a sketch target is displayed is provided at the center of the screen, and rotation operation knobs 31 and 32 are provided on the right and lower sides of the image display area, respectively. Yes. The rotation operation knob 31 on the right side inputs an instruction to rotate the displayed CG image in the vertical direction by moving the pen 20a in the vertical direction while being in contact with the display screen (the detection surface of the touch panel 20). Can do. The lower rotation operation knob 32 moves the pen 20a in the left-right direction while being in contact with the display screen (the detection surface of the touch panel 20), thereby giving an instruction to rotate the displayed CG image in the left-right direction. Can be entered. Further, an enlargement / reduction slider 33 is provided on the left side of the image display area. The enlargement / reduction slider 33 enlarges the CG image by moving the pen 20a upward while touching the display screen (detection surface of the touch panel 20), and reduces the CG image by moving the pen 20a downward. Instructions can be entered.

  When the CPU 10 inputs an adjustment instruction by an operation on the rotary operation knobs 31 and 32 and the enlargement / reduction slider 33, the CPU 10 changes a virtual camera position for photographing the motif and generates a CG image based on the CG data ( Step B2). In the case of a CG image, the size, orientation, etc. of the CG image can be freely adjusted by arithmetic processing on CG data in accordance with an instruction from the user.

  When the orientation and size of the CG image are adjusted as desired by the user and the adjustment completion is instructed, the CPU 10 causes the display 18 to display the motif image (CG image) after the adjustment is completed.

FIG. 7 shows an example of a display screen on the display 18 when the adjustment of the CG image is completed.
Next, the user selects an auxiliary line to be displayed corresponding to the adjusted motif image. For example, in the auxiliary information management data, a plurality of auxiliary lines are set as auxiliary line information for each image, and the plurality of auxiliary lines are displayed in a list and selected by the user (step B3).

  When an instruction for selecting any auxiliary line type is input from the auxiliary line list display, the CPU 10 generates auxiliary line information corresponding to the selected auxiliary line type based on the CG data. (Step B4). For CG data, information used in painting techniques, such as vanishing points in images, outlines of objects (motifs), human figures and their skeleton and face orientation, can be easily extracted by arithmetic processing ( By embedding information necessary for the calculation in advance in the CG data, the auxiliary information can be extracted more easily).

  When the auxiliary line is generated by the auxiliary line information generation processing in this way, the CPU 10 shifts to a state where sketch input is possible, displays the sketch input screen on the display 18, and displays the auxiliary line generated in this screen. Display (step A7). The display form of the auxiliary line may be displayed by a thin line, a broken line, or the like, or may be displayed lightly so as not to interfere with the display of an image input by a sketch, for example. Moreover, you may make it display using a predetermined color. In addition, comments using various display forms are possible.

  CPU10 inputs drawing data according to operation which inputs the sketch (image) with respect to touch panel 20 using pen 20a, and memorize | stores it in RAM11 (step A8). Further, the CPU 10 displays an image input on the display 18 based on the drawing data stored in the RAM 11.

FIG. 8 shows an example in which an auxiliary line 34 is displayed on the screen for sketch input, and a sketch is made using the auxiliary line 34.
FIG. 8 shows an example in which a contour line extracted based on the CG data of the motif adjusted as shown in FIG. 7 is displayed as an auxiliary line. As shown in FIG. 8, since the contour line is displayed as an auxiliary line, by operating the pen 20a so as to trace the contour line and inputting drawing data, Even a beginner can easily sketch a motif whose size has been adjusted.

  In the example shown in FIG. 8, the contour line is displayed as an auxiliary line. However, in the auxiliary line information generation process, the user arbitrarily selects the type of the auxiliary line and is selected based on the CG data. An auxiliary line is generated and displayed on the screen for sketch input.

  Further, in the auxiliary line information generation processing for the CG image, the type of the auxiliary line is selected by the user, but the auxiliary line set in the auxiliary information management data according to the preset user level. May be selected as a display target.

  Next, a case where a still image (other image) is selected as a motif image will be described.

  When a still image prepared in advance as a motif image is selected (step A5, Yes), the CPU 10 executes an auxiliary line information generation process for the still image (step A6).

FIG. 5 is a flowchart showing auxiliary line information generation processing for a still image.
First, the CPU 10 displays a list of a plurality of motif images (still images) prepared in advance on the display 18 and allows the user to select one of them by an operation with the pen 20a or an operation with the button group 22 (step). C1).

  When the target motif image is selected, the CPU 10 determines a preset user level by referring to data indicating the user level stored in the RAM 11 (or the memory device 24) (step C2). ).

  Then, the CPU 10 generates auxiliary information corresponding to the user level set in the auxiliary information management data based on the still image selected as the motif image (step C3).

  For example, the CPU 10 extracts auxiliary line information from the original still image using existing image processing corresponding to the auxiliary line to be generated. For example, lines existing in the image are extracted by performing spatial differentiation of the image on the color space value. If the motif image is a person, an auxiliary line representing a human skeleton model can be generated by image processing using a known human body posture estimation method based on the person image in the image. . In addition, information that can be used as auxiliary information can be extracted by using various types of image processing according to the types of auxiliary lines corresponding to the motif image.

FIG. 9 shows an example of a still image selected as a motif image.
The motif image (still image) shown in FIG. 9 is a suitable image when the perspective method is used as a painting technique. For example, the CPU 10 extracts auxiliary information as shown in FIG. 10 from the still image shown in FIG. 9 by image processing. That is, the vanishing point 36 set in the image and a plurality of auxiliary lines 37 extending from the vanishing point 36 are generated. By presenting the auxiliary line shown in FIG. 10 to the user, it is possible to assist the sketch using the perspective method by the user.

FIG. 11 shows an example of auxiliary lines extracted using a posture estimation method for a person.
FIG. 11A shows an example in which the contour line of a person is generated as the auxiliary line 38. FIG. 11B shows an example in which an auxiliary line 39 representing a skeleton line model of a person is generated. By presenting auxiliary lines 38 and 39 shown in FIG. 11 to the user, it is possible to assist in sketching a person.

FIG. 12 shows an example of auxiliary lines extracted for a human face.
For a human face image, for example, each position is detected based on the shape of eyes, nose, mouth, ears, and the like, and the face orientation is detected based on the arrangement of each position. Then, based on the positions of the eyes, nose, mouth, ears and the orientation of the face, as shown in FIG. A line that represents the line of the nose and mouth is generated.

  Note that the above-described auxiliary line is an example, and various auxiliary lines corresponding to the motif image can be generated and presented to the user.

  When the auxiliary line is generated by the auxiliary line information generation processing in this way, the CPU 10 shifts to a state where sketch input is possible, displays the sketch input screen on the display 18, and displays the auxiliary line generated in this screen. Display (step A7). CPU10 inputs drawing data according to operation which inputs the sketch (image) with respect to touch panel 20 using pen 20a, and memorize | stores it in RAM11 (step A8). Further, the CPU 10 displays an image input on the display 18 based on the drawing data stored in the RAM 11.

  Here, by referring to the auxiliary line displayed on the display 18, it is possible to easily grasp at which position the drawing should be performed, and by operating the pen 20 a on the detection surface of the touch panel 20, You can get a feeling of writing directly and enjoy sketching. Furthermore, since the auxiliary line is directly shown on the display 18, even a user who has not acquired the painting technique can expect to draw a picture at the user level.

  When the end of drawing (sketching) is instructed by a button operation or the like (step A9, Yes), the CPU 10 calculates an evaluation value by comparing the auxiliary line displayed on the display 18 with the input drawing data. (Step A10). That is, an evaluation value for determining whether the user is drawing correctly at the drawing position indicated by the auxiliary line is calculated.

  For example, the CPU 10 calculates the feature value represented by the auxiliary line and the feature value represented by the drawing data input by the user, and calculates the similarity of each feature value as the evaluation value. The evaluation function used as the evaluation value calculation method can be used by applying various existing image recognition methods.

  FIG. 13 shows an example of the auxiliary line 41 representing the outline of the motif displayed on the sketch input screen. FIG. 14 shows a state in which an image sketched using the pen 20a is displayed on the auxiliary line 41 shown in FIG.

  In the example shown in FIG. 14, since the auxiliary line 41 represents the outline of the motif, an image representing the motif can be input by drawing the auxiliary line 41 while tracing it. Therefore, the CPU 10 calculates an evaluation value indicating that the sketch is correctly performed as the shift between the position of the auxiliary line and the input drawing data is smaller.

  In the case of another auxiliary line different from the contour line as shown in FIG. 13, the drawing position for the user to draw does not match the position of the auxiliary line. For example, in the case of the auxiliary line 39 representing a human skeleton model as shown in FIG. 11B, the feature amount of the auxiliary line 39 itself is not set as a calculation target, but drawn by the user represented by the auxiliary line 39. The evaluation value may be calculated by comparing the feature amount of the drawing position (line) to be performed with the feature amount of the drawing data input by the user. Similarly, for other types of different auxiliary lines, evaluation values may be calculated using an evaluation function (calculation method) corresponding to the auxiliary lines.

  The CPU 10 displays the evaluation value calculated here on, for example, the display 17 (step A11). By referring to the evaluation value displayed on the display 17, the user can objectively grasp whether the motif image has been correctly sketched or the skill improvement status of the user.

  As described above, in the game device (image creation device) according to the present embodiment, by selecting the motif image to be sketched, an auxiliary line that assists the drawing of the motif image is presented on the display 18. Therefore, it is possible to easily draw a motif image. In addition, the evaluation value calculated based on the auxiliary line is output for the actually drawn image (drawing data), so you can know the progress of your painting technique with reference to this evaluation value. it can.

  In the above description, the auxiliary line is generated in the auxiliary line information generation process based on the CG image or the still image selected as the motif image by the user. (Or other auxiliary information) data may be set in the auxiliary information management data, and auxiliary lines (auxiliary information) may be displayed based on this data. As a result, the processing load on the CPU 10 during sketch processing can be reduced. As described above, the data for displaying the auxiliary line can be generated using various methods based on the motif image (CG image, still image, etc.).

  Further, when an image is captured using the image input device 25 and this image is used as a motif image, the auxiliary line information generation process described in step A6 may be executed.

  Further, in the above description, various lines (solid line, broken line, etc.) indicating the drawing position are displayed as auxiliary information for the user, but other forms of holding document information may be displayed. For example, it is possible to use other display forms such as displaying arrows, various marks, animations, etc. to indicate the drawing position, or making the display color different from others.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the method described in the embodiment uses, as a program (software means) that can be executed by a computer, for example, a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), and a semiconductor memory. It can also be stored in a recording medium such as (ROM, RAM, flash memory, etc.), or transmitted and distributed via a communication medium. The program stored on the medium side also includes a setting program that configures in the computer software means (including not only the execution program but also a table and data structure) that is executed by the computer. A computer that implements the present apparatus reads the program recorded on the recording medium, constructs software means by a setting program in some cases, and executes the above-described processing by controlling the operation by the software means. The recording medium referred to in this specification is not limited to distribution, but includes a storage medium such as a magnetic disk or a semiconductor memory provided in a computer or a device connected via a network.

FIG. 2 is a block diagram illustrating a functional configuration of the image creation apparatus according to the present embodiment. The figure which shows an example of the auxiliary information management data for controlling the sketch process in this embodiment. The flowchart explaining the sketch process performed in the game device in this embodiment. 6 is a flowchart showing auxiliary line information generation processing for a CG image. 9 is a flowchart showing auxiliary line information generation processing for a still image. The figure which shows an example of the screen for adjustment of a CG image. The figure which shows an example of the display screen in the display 18 at the time of adjustment of a CG image being completed. The figure which shows an example in which the auxiliary line 34 is displayed on the screen for sketch input, and is sketching using this auxiliary line 34. The figure which shows an example of the still image selected as a motif image. The figure which shows an example of the auxiliary line with respect to the still image shown in FIG. The figure which shows an example of the auxiliary line extracted using the attitude | position estimation method for the person. The figure which shows an example of the auxiliary line extracted for the face of a person. The figure which shows an example of the auxiliary line 41 showing the outline of the motif displayed on the screen for sketch input. The figure which shows the state by which the image sketched using the pen 20a was displayed with respect to the auxiliary line 41 shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... CPU, 11 ... RAM, 12, 13 ... GPU, 14,15 ... VRAM, 16 ... Display controller, 17, 18 ... Display, 19 ... Interface unit, 20 ... Touch panel, 20a ... Pen, 21 ... Speaker, 22 ... Button group, 23 ... connector, 24 ... memory device, 25 ... image input device, 26 ... communication unit.

Claims (5)

A touch panel capable of inputting an image handwritten by a user;
A display integrated with the touch panel;
An input means for inputting an instruction to select an image to be rendered;
Auxiliary information indicating a drawing position for drawing by the user corresponding to the image indicated by the instruction input by the input unit is selected from a plurality of types of auxiliary information corresponding to the image to be drawn. an auxiliary information generating means for generating Te,
Auxiliary information display means for displaying auxiliary information generated by the auxiliary information generation means on the display ;
Image input means for inputting image data for the auxiliary information displayed by the auxiliary information display means from the touch panel ;
An image creating apparatus comprising: an image display unit configured to display in accordance with image data input by the image input unit. Image storage means for storing CG (computer graphics) data of an image to be rendered;
The image creation apparatus according to claim 1 , wherein the auxiliary information generation unit generates auxiliary information based on the CG data. Image storage means for storing image data of an image to be rendered;
The auxiliary information generating means, wherein said generating an image representing the position of an object included in the image by image processing on the image data, and generates an auxiliary information image representing the position on the basis Item 2. The image creating apparatus according to Item 1 .   2. The image according to claim 1, further comprising an evaluation value calculating means for calculating an evaluation value for the image data input by the image input means based on the auxiliary information displayed by the auxiliary information display means. Creation device. A computer having a touch panel capable of inputting an image handwritten by a user and a display laminated and integrated with the touch panel ,
An input means for inputting an instruction to select an image to be rendered;
Auxiliary information indicating a drawing position for drawing by the user corresponding to the image indicated by the instruction input by the input unit is selected from a plurality of types of auxiliary information corresponding to the image to be drawn. an auxiliary information generating means for generating Te,
Auxiliary information display means for displaying auxiliary information generated by the auxiliary information generation means on the display ;
Image input means for inputting image data for the auxiliary information displayed by the auxiliary information display means from the touch panel ;
An image creation program for causing an image display unit to function in accordance with image data input by the image input unit.

Images