JP7278341B2 - Information provision system and information provision program - Google Patents

Description

The present invention relates to an information providing system and the like.

Techniques have been proposed for providing information using invisible codes that are invisible to the human eye.
For example, Patent Literature 1 discloses a technique of presenting information using a head-mounted display. The head-mounted display in this document has a half mirror on the optical path for the user to see the state of the outside world, and further includes a light receiving device (imaging device) capable of photographing the state of the outside world reflected by the half mirror. Installed. When a user wears this head-mounted display and sees a presentation in which an invisible code is embedded, an image or the like corresponding to the invisible code captured by an imaging device is read from the server system and mounted on the head-mounted display. The image is projected onto the half mirror by the image display device.

Japanese Patent Application Laid-Open No. 2003-242168

According to Patent Literature 1, a user wearing a head-mounted display can see an image that does not originally exist there superimposed on the image of the presentation object. Apart from the significance of providing information, the experience of seeing things through a screen that cannot be seen with the naked eye in the real world is very impressive even for adults. Moreover, it is not difficult to imagine that children will feel even more impressive, and the impact is considered to be highly valuable in the intellectual training of children.

An object of the present invention is to provide a more interesting information providing system and its program.

The information providing system of the present invention includes an information unit (for example, the painting unit 914 in FIG. 1) in which information is described, a photographing unit (for example, the camera module 955 in FIG. 2), a display unit (for example, the touch panel 942 in FIG. ), a storage unit (e.g., IC memory 952 in FIG. 2, semiconductor memory incorporated in SoC 951), and a control unit (e.g., control board 950, SoC 951 in FIG. 2). A loupe-type handy device 930), wherein when the information part is recognized by the photographing part, the information to be photographed in the information part is displayed based on the recognized image. An image whose position is identifiable is drawn, and the storage unit stores image data (for example, picture image data 522 in FIG. 10) for displaying a partial image of the information unit on the display unit. The control section identifies a position to be photographed in the information section based on the image recognized by the photographing section, and generates a partial image in the information section corresponding to the position to be photographed based on the image data. to control the display on the display unit (for example, step S10 in FIG. 12). Further, an information providing system of the present invention is an information providing system comprising an information section in which information is described, and a portable electronic device having an imaging section, a display section, a storage section, and a control section, wherein the information section , image content configured such that, when the information part is recognized by the photographing part, a position to be photographed in the information part can be identified based on the recognized image, and the storage part is and storing at least partial image data for displaying a partial image of the image content drawn in the information section on the display section, and the control section stores an image in the information section based on the image recognized by the photographing section. a position to be photographed is identified, a partial image in the information section corresponding to the position to be photographed is controlled to be displayed on the display section based on the partial image data, and the position to be photographed satisfies a predetermined position condition. a special image is displayed on the partial image displayed on the display unit, the information unit includes at least two code images, and when the code image is recognized by the photographing unit, the recognized code A position to be photographed in the information section can be identified based on the image.

Further, in the information providing system of the present invention, when the photographed position satisfies a positional condition for displaying a predetermined special image, the control unit displays the special image on the partial image displayed on the display unit. Preferably, the image (eg, the first image element W41 of the special image in FIG. 5) is displayed (eg, steps S30-S34 in FIG. 12).

Further, in the information providing system of the present invention, the portable electronic device further includes a clock unit (for example, the clock unit 953 in FIG. 2), and the control unit controls the special image based on the clock information by the clock unit. is variably controlled (for example, step S32 in FIG. 12).

Further, in the information providing system of the present invention, the control unit causes the storage unit to store a history of the special images displayed on the display unit (for example, special image display history data 560 in FIG. 9), and stores the history in the history. Based on this, it is preferable to have a record display control section (for example, the record display control section 222 in FIG. 8, steps S50 to S56 in FIG. 12) for displaying the special image displayed in the past on the display section.

Moreover, in the information providing system of the present invention, it is preferable that the information section is configured such that the photographed images obtained by the close-up photographing are different images depending on the position to be photographed.

Further, in the information providing system of the present invention, the information section may be composed of an image in which a code image (for example, the code image 20 in FIG. 3) that cannot be distinguished by the naked eye is embedded according to a predetermined arrangement rule. preferable.

Moreover, in the information providing system of the present invention, it is preferable that the portable electronic device has a loupe shape with a viewing window, and the display section is provided in the viewing window.

Further, according to the present invention, in a computer provided with a photographing unit and a display unit, when information described in an information unit is recognized by the photographing unit, based on the image recognized by the photographing unit, Photographed position identifying means for identifying the photographed position (for example, control unit 200, photographed position identifying unit 206 in FIG. 8, step S8 in FIG. 12), based on the photographed position identified by the photographed position identifying means , partial image display control means (for example, control unit 200 and partial image display control unit 208 in FIG. 8, step S10 in FIG. 12) for controlling the display of partial image data in the information section stored in the computer on the display section; ), the information providing program (for example, the handy device control program 502 in FIG. 9).

Further, in the information providing program of the present invention, the computer may be configured as imaging position condition identification means for determining a position condition of the imaging position, and when the position condition satisfies a position condition for displaying a predetermined special image. Further, it is preferable to function as special image display control means for controlling display of the special image on the partial image displayed on the display unit.

Further, in the information providing program of the present invention, the computer further comprises a clocking unit, and the computer functions as clocking information corresponding image selection means for variably controlling display of the special image based on clocking information from the clocking unit. It is preferable to let

Further, in the information providing program of the present invention, the special image display history storage means for causing the computer to store a history of the special images displayed on the display unit in the storage unit; It is preferable to function as record display control means for controlling display of the special image on the display section.

Further, in the information providing program of the present invention, it is preferable that the photographed position is identified by pattern matching of an image in the pictorial image in the photographed position identifying means.

Further, in the information providing program of the present invention, it is preferable that, in the photographed position identification means, the photographed position is identified by a code image in the pictorial image that cannot be distinguished by the naked eye.

The figure which shows the structural example of the toy system for children. The figure which shows the structural example of a loupe type handy apparatus. The figure which shows the structural example of a picture part. FIG. 4 is a diagram for explaining an example of image display in a loupe-type handy device; FIG. 10 is a diagram for explaining an example of image display in a loupe-type handy device when a captured image includes a specific range; FIG. 10 is a diagram for explaining an example of image display in a loupe-type handy device when a captured image includes a specific range; The figure which shows the example of a screen display in observation picture book mode. FIG. 2 is a functional block diagram showing a functional configuration example of a loupe-type handy device; FIG. 4 is a diagram showing an example of programs and data stored in the storage unit of the loupe-type handy device; FIG. 4 is a diagram showing a data configuration example of painting department definition data; FIG. 4 is a diagram showing a data configuration example of special image definition data; 4 is a flowchart for explaining the flow of processing in the loupe-type handy device;

An example of an embodiment to which the present invention is applied will be described below, but embodiments to which the present invention can be applied are not limited to the following examples.
FIG. 1 is a diagram showing a configuration example of a toy system 900 for children according to this embodiment. A toy system 900 of this embodiment has a main body 910 and a loupe-type handy device 930 .

The body portion 910 is a booklet having a plurality of pages 912 with a painting portion 914 . In this embodiment, a so-called "ring file" is used, in which a plurality of pages 912 are detachably bound to a binder frame 916 with a ring 918, but a glued or string-bound notebook or book type may also be used.
Note that the number of pages 912 can be set as appropriate, and one page 912 may be foldable. In the illustrated example, one page 912 is provided with one painting section 914 . A so-called "spread" configuration is also possible. The picture section 914 may be printed on each page 912 or pasted on each page 912 as a sticker sheet.

The loupe-type handy device 930 is an electronic device corresponding to a computer having an appearance that resembles a magnifying glass.

2A and 2B are diagrams showing a configuration example of the loupe-type handy device 930 according to the present embodiment, FIG. 2(1) being a front view facing the user, and FIG. 2(2) being a side view.
The appearance of the loupe-type handy device 930 has the shape of a magnifying glass. Specifically, a rod-shaped grip portion 936 extends from the side surface of the disk-shaped housing portion 932 via the operation portion 934, and the rear surface of the disk-shaped housing portion 932 is made of transparent resin or translucent resin. A resulting substantially cylindrical lighting skirt portion 938 is provided. In addition, it is preferable that the holding portion 936 is provided with such a length that when the lighting skirt portion 938 is placed on the painting portion 914 , it can stand on its own without falling down.

A viewing window portion 940 is provided in the center of the front side of the disk-shaped housing portion 932, and a touch panel 942 is fitted therein.

A control board 950 is incorporated inside the disk-shaped housing portion 932 . The control board 950 includes, for example, an SoC (System-on-Chip) 951 , an IC memory 952 , a clock section 953 , a touch panel driver IC 954 for operating the touch panel 942 , and a camera module 955 . Then, power is obtained from a built-in battery 937 built in the grip part 936 to operate.

The CPU built in the SoC 951 reads out the programs and data stored in the IC memory 952, performs arithmetic processing, and comprehensively controls the operation of the loupe-type handy device 930. FIG.
The IC memory 952 stores programs and various data. The various data include image data of the picture image of the picture portion 914 printed on each page 912 .

The clock unit 953 is a real-time clock IC incorporated in a personal computer or the like, and outputs the current date and time to the SoC 951 as a constant clock. Of course, the clock unit 953 may be implemented as a software clock that counts the date and time based on the clock signal by the SoC 951, and has a clock unit that clocks at least one of time, date, and season. is preferred.

The camera module 955 is a known image sensor equipped with an optical element such as a lens, a filter, a sensor chip that generates data of a captured image and outputs the data to the SoC 951, and the like. The camera module 955 of the present embodiment can capture not only visible light but also infrared or ultraviolet light in the invisible range. The camera module 955 is provided so as to photograph a subject on the back side of the disk-shaped casing 932 , and its optical axis L is set so as to pass through the screen center point of the touch panel 942 . The optical specifications, including the focal length, are set so that at least the subject pressed against the end surface of the lighting skirt portion 938 is in focus, enabling close-distance photography. It may be equipped with an autofocus function. Further, in this embodiment, an infrared transmission film is provided in front of the camera module 955 . In the present embodiment, the infrared transmission film is for more efficiently recognizing the infrared light reflected by the visible printed portion by the camera module 955, and absorbs visible light and transmits infrared light. It is not particularly limited as long as it is a member.

The operation unit 934 includes a plurality of operation switches 935 for user operation input.

FIG. 3 is a diagram showing a configuration example of the painting unit 914 in this embodiment.
In the present embodiment, the painting portion 914 is image content, and a painting image classified as an illustration is exemplified. , Typography, etc., regardless of genre. Alternatively, it may be text information (included in the “image” referred to in this embodiment). Moreover, it may be a two-dimensional image or a stereoscopic image.

The painting portion 914 is composed of a visible printed portion printed with normal visible ink that reflects infrared light and an invisible printed portion printed with invisible infrared absorbing ink that is invisible to the naked eye of the user. In the present embodiment, for example, the invisible printed portion is printed using a printing paint having a high absorbance of light in the near-infrared region. Fig. 3(1) shows only the visible printed portion. This is the image that the user can perceive with the naked eye, that is, the appearance of the painting portion 914 in the visible light range. In addition to this, when the invisible printed portion is schematically illustrated, it becomes as shown in FIG. 3(2).

The visible printed portion of the pictorial image of the painting portion 914 includes one or more observation target portions 30 (in the illustrated example, two locations, a right observation target portion 30a and a left observation target portion 30b). . Each observation target portion 30 is defined as a unique specific range 32 (in the illustrated example, two places, a right specific range 32a and a left specific range 32b). The specific range 32 of the present embodiment is a rectangular area surrounding the observation target portion 30, and is defined by the coordinates of the four corners in the pictorial image coordinate system.

The invisible printing unit is configured by arranging a plurality of code images 20 according to a predetermined arrangement rule so that when photographed by the camera module 955, the photographed position in the pictorial image can be identified based on the photographed image. ing. In other words, the pictorial image of the pictorial portion 914 is embedded with the code image 20 that cannot be discerned with the naked eye. Note that in the present embodiment, “shooting” includes not only the act of recording an image as a photograph or moving image, but also all acts of recognizing an image by the camera module 955 .

The code image 20 is printed with invisible infrared-absorbing ink, and the visible printed portion is printed with ink that reflects light in the infrared region. It becomes possible to recognize and identify the code image 20 as shown in FIG. 3(2). In the present embodiment, providing a light source for irradiating infrared rays to painting portion 914 is preferable because the identification rate of code image 20 can be further improved. For example, an infrared LED can be provided on the rear surface side of the disk-shaped housing part 932 so that the subject of the camera module 955 can be irradiated with infrared light.

Code images 20 are codes each carrying unique information. A plurality of codes are arranged according to a predetermined rule. Specifically, the code images 20 in the normal posture are arranged at predetermined intervals in the first row from the top. In the second row from the top, the code images 20 in the tilted posture rotated counterclockwise by 45° from the normal posture are arranged at predetermined intervals that are half the interval of the code images 20 in the first row. In the third row, the code images 20 in the first row are horizontally reversed, and in the fourth row, as in the second row, the code images 20 are arranged in an inclined posture rotated 45° counterclockwise from the normal posture. , are arranged at predetermined intervals that are shifted by half from the intervals of the code images 20 in the third row. The number of code images 20 per row and the number of rows can be set as appropriate. However, it is preferable to determine the layout rule (including the number of layouts) of the code images 20 so that at least one code image 20 is included in the photographed image when the lighting skirt 938 is applied to the painting portion 914 and close-up photography is performed. . This is for ascertaining the photographing position, photographing angle, etc. from the code image 20 included in the photographed image. It is preferable to capture the captured image so that at least two code images 20 are included in the captured image and determine the two or more code images so that the captured position can be calculated more accurately.

Since the code images 20 each carry unique information and are arranged in an inclined posture, by identifying and decoding the code images 20 appearing in the images taken by the camera module 955, the relevant information can be obtained. It is possible to know where in the painting image of the painting unit 914 the photographed image is photographed. In other words, it is possible to know the position to be photographed, and it is possible to know the center position coordinates of the photographed image in the coordinate system of the pictorial image.

The code pattern (code information) of the printed code image 20 includes information unique to each page 912, that is, each picture section 914, position information in the picture section 914, and the code image 20 corresponding to the picture section 914. and orientation information indicating which orientation is being printed with respect to the Also, the code image 20 itself has an orientation. Therefore, from the photographed code image 20, which picture part 914 is currently being photographed by the camera module 955, the rough position of which position in the picture part 914 is being photographed, and what angle (on the page 912). You can know whether you are shooting in clockwise/counterclockwise rotation angle). Also, from the distance between two or more code images that have been photographed, it is possible to know the detailed position of the photographed position in the picture portion 914 .

The purpose of the invisible printing unit is to make it possible to identify the photographed position in the pictorial image based on the photographed image when the pictorial image is photographed close-up by the camera module 955 . In other words, it is possible to identify the photographed position or range in the pictorial image, or a representative position for defining the range.
Therefore, the configuration of the invisible printing unit is not limited to the example of this embodiment, and for example, a configuration in which identification information is defined by dot distribution may be used. Alternatively, a code image or dot distribution drawn with normal visible ink may be used. Alternatively, by sequentially identifying the image contents of the visible print portion photographed by the camera module 955 by image pattern recognition, the photographed position or range, or a representative position for defining the range is identified. is also possible. In that case, the invisible printed portion itself, the infrared transmission film, the infrared light source, etc. can be omitted, which is preferable.

FIG. 4 is a diagram for explaining an example of image display on the loupe-type handy device 930. As shown in FIG.
The user grasps the grip portion 936 of the magnifying handy device 930 with a hand as if observing an insect with a magnifying glass, and holds the magnifying handy device 930 so that the end surface of the lighting skirt portion 938 (see FIG. 2) contacts the page 912 . Apparatus 930 is applied to painting section 914 . Then, the user looks through the viewing window portion 940 .

The loupe-type handy device 930 successively photographs the page 912 with the camera module 955, detects the code image 20 in the photographed image, and analyzes the code image 20. FIG. Then, from the code image 20 reflected in the photographed image, it is determined which picture part 914 is currently photographed, what position in the picture part 914 is photographed, and at what angle the photograph is taken.

As described above, the IC memory 952 of the loupe-type handy device 930 stores image data capable of displaying the same picture image as that of all the picture sections 914 on the display section. The loupe-type handy device 930 extracts from the IC memory 952 a partial image W2 of a range that matches the range to be captured in the painting image of the painting unit 914 that is currently being captured, and displays it on the touch panel 942 . In other words, the touch panel 942 of the loupe-type handy device 930 does not display the photographed image of the picture portion 914 being photographed. Therefore, the user looking into the viewing window portion 940 can see the partial image W2 of the painting portion 914 where the loupe-type handy device 930 is applied through the touch panel 942 . In the example of FIG. 4, since the loupe-type handy device 930 is placed on the forest on the upper left of the image, a partial image W2 of the portion where the state of the forest is drawn is displayed. If the loupe-type handy device 930 is shifted, the image displayed as the partial image W2 is also scrolled and changed accordingly, so that the user feels as if the touch panel 942 is a lens of a magnifying glass. .

In this embodiment, the partial image W2 extracted from the IC memory 952 may be an image whose contour matches the contour of the image of the range to be photographed, and the content of the image itself does not necessarily match. no need to let For example, an image that is colored differently from the image of the range to be photographed, or to which an image for performing a special effect is added may be displayed as a partial image. By making the image of the range to be photographed different from the partial image, it is possible to display more interesting images, and the loupe-type handy device 930 with improved interest can be provided.

It should be noted that, just to make sure, the description will be repeated, but what is displayed on the touch panel 942 of the loupe-type handy device 930 is not the photographed image itself. A partial image corresponding to the photographing range determined from the photographed image is displayed on the touch panel 942 among the image data of the pictorial image stored in the IC memory 952 . By displaying only the image data of the pictorial image stored in the IC memory 952 on the display unit as in the present embodiment, the code image that would normally be displayed on the display unit is not displayed on the display unit. , there is no need to worry about the code image being reflected in the display section, and the degree of freedom in designing the picture image in the picture section and the code image can be improved with a simple configuration.

5 and 6 are diagrams for explaining an example of image display on the loupe-type handy device 930 when the specific range 32 is included in the captured image.
When it is determined that part or all of the specific range 32 is included in the captured image captured by the camera module 955 , the loupe-type handy device 930 selects the observation target section 30 corresponding to the specific range 32 . Display the "special image" on the partial image. In other words, the special image is displayed on the partial image by judging that the position to be photographed satisfies the predetermined position condition.

A “special image” is composed of one or more image elements that represent the difference in appearance of the observation target part 30 in the photographed pictorial image. The loupe-type handy device 930 stores in advance the definition data of the special image associated with the specific range 32, and the definition data stores image element data for each “display condition”. One image element constituting the special image in this embodiment is an animation (moving image), but may be a still image. There are two types of special images in this embodiment, depending on how the "display conditions" are set.

The example of FIG. 5 is a type in which the elapsed time from the date and time when the specific range 32 was first photographed is set as the “display condition”.
In the example of FIG. 5, a potted plant and an eyebrow rabbit character growing it are drawn in the observation target portion 30a on the right side of the painting portion 914, but the potted plant has not sprouted. If the elapsed time from the date and time when the specific range 32a of the observation target portion 30a was first photographed is the first day or the second day, the first image element W41 that animates the rabbit watering is selected as the special image. is displayed on the touch panel 942. When the elapsed time is from the third day to the fifth day, the second image element W43 showing sprouts is selected and displayed. From the 6th to 8th days, the third image element W46 with buds is selected and displayed. Then, when the elapsed time reaches the ninth day or later, the fourth image element W49 showing the flowering state is selected and displayed. Note that four types of image elements forming the special image have been described here, but the intermediate progress of these and the state after the fourth image element W49 (for example, until the flower withers and seeds and the whole withers) ) may be provided to include more image elements.

The example in FIG. 6 is of a type that uses the current date and time of shooting as a condition for the 'display condition'. For example, the season, month, day of the week, date, time period, time, identification of holidays (for example, Children's Day), summer vacation, Christmas period, etc. at the time of photographing can be appropriately set as display conditions. Then, the image element of the special image associated with the display condition that matches the current date and time is selected and displayed.
In the example of FIG. 6, in the observation target portion 30b on the left side of the painting portion 914, a potted morning glory and a spotted rabbit character looking at it are drawn, but the potted flower is not blooming. If the time at which the user uses the loupe-type handy device 930 is from 6:00 p.m. (the same content as the However, if the time at which the loupe-type handy device 930 is being used is from 5:00 am to 6:00 pm, the second image element W62 (flower blooming) associated with the time period is displayed. be.
Here, two types of image elements are described as constituting the special image associated with the specific range 32b. More image elements may be prepared in association with the time zone so as to include the deflated state.

The loupe-type handy device 930 stores the image display history of the special images displayed as shown in FIGS. 5 and 6, and can reproduce and display the special images displayed by the user in the past at any timing. This is called a "recording and displaying function", but since the assumed users are children, it would be appropriate to call it an "observation picture book mode" for children.

FIG. 7 is a diagram showing a screen display example in the observation picture book mode.
When a predetermined record display start operation is input through the operation unit 934 or the touch panel 942, the loupe-type handy device 930 displays the record display screen W8. The record display screen W8 has a menu display portion 40, a designated position display portion 41, and a record display portion .

In the menu display section 40, thumbnails 43 of special images displayed in the past are displayed for each observation target section 30 in a scrollable manner. The user performs a slide operation so that the thumbnail 43 indicating the special image to be displayed again is aligned with the designated position display portion 41 .

The loupe-type handy device 930 displays the previously displayed special images associated with the thumbnails 43 aligned with the specified position display section 41 on the record display section 42 so as to be scrollable for each image element. When a touch operation to an image element on the recording display unit 42 is detected, the image element is enlarged and displayed as shown in a screen display example W10.

Note that the record display screen W8 includes a record display end operation icon 44 . When the icon is touched or a predetermined operation is performed on the operation unit 934, the display of the record display screen W8 can be terminated.

Moreover, in the present embodiment, the loupe-type handy device 930 may further include an audio output section. It is possible to provide a loupe-type handy device that further arouses the user's interest by outputting sound accompanying the display of an image on the display unit.

[Description of functional configuration]
FIG. 8 is a functional block diagram showing a functional configuration example of the loupe-type handy device 930. As shown in FIG.
A loupe-type handy device 930 of this embodiment includes an operation input unit 100 , an imaging unit 102 , a control unit 200 , an image display unit 372 , and a storage unit 500 .

The operation input section 100 outputs operation input signals to the control section 200 according to various operation inputs made by the player. For example, it can be realized by a push switch, joystick, touch pad, trackball, acceleration sensor, gyro, and the like. In the example of FIG. 2, the operation switch 935 and the touch panel 942 correspond to this.

The imaging unit 102 outputs image data of the captured image to the control unit 200 . In the example of FIG. 2, the camera module 955 corresponds to this.

The control unit 200 is realized by, for example, an SoC or a microcontroller IC in which a CPU, a GPU, a semiconductor memory, etc. are integrated on a single semiconductor device, and communicates with each functional unit including the operation input unit 100, the imaging unit 102, and the storage unit 500. data input/output control between Various arithmetic processing is executed based on predetermined programs and data, operation input signals from the operation input unit 100, image data from the imaging unit 102, etc., and the operation of the loupe-type handy device 930 is comprehensively controlled. do. It is included in the control board 950 in the example of FIG.

The control unit 200 in this embodiment includes an invisible code identification unit 202, a code identification unit 204, an imaged position identification unit 206, a partial image display control unit 208, a special image display control unit 210, a special image It has a display history storage control section 220 , a record display control section 222 , a clock section 226 and an image generation section 272 .

The invisible code identification unit 202 analyzes the image data of the image captured by the image capturing unit 102, detects the code image 20 in the image, and identifies the code pattern, the direction of the code image 20, and the like. do.

The photographed/picture discrimination unit 204 discriminates the type of the picture portion 914 from the code image 20 identified by the invisible code identification unit 202 (which one of the plurality of prepared picture portions 914).

The photographed position identification unit 206 identifies the relative photographed position in the image of the picture part 914 being photographed.
Specifically, the position in the picture portion 914 is determined from the code pattern identified by the invisible code identification portion 202, and the photographing direction with respect to the picture portion 914 is determined from the direction in which the code image 20 is captured. From the discriminated position and discriminated orientation, the photographed position (position to be photographed, which can also be called a photographed range) is determined. Further, in this embodiment, since it can be determined that the focal length of the photographing unit 102 is fixed by the lighting skirt portion 938, the photographed position can be obtained from the discriminated position and discriminated orientation.

The partial image display control unit 208 causes the image display unit 372 to display and control the partial image W2 (see FIG. 4) in the painting portion 914 corresponding to the shooting position identified by the shooting position identification unit 206 .

The special image display control unit 210 selects a partial image W2 to be displayed on the image display unit 372 when the shooting position identified by the shooting position identification unit 206 satisfies the positional conditions for displaying a predetermined special image. A special image associated with the painting portion 914 determined by the photographed/painting determination portion 204 is displayed above. In this embodiment, in order to variably control the special image based on the timekeeping information, the image selection section 212 corresponding to the current date and time and the image selection section 214 corresponding to the passage of time are provided as image selection sections corresponding to the timekeeping information.

In the present embodiment, when the specific range 32 surrounding the observation target part 30 is included in the captured image (paradoxically, when the central position coordinates of the captured image are included in the specific range 32), it is determined that the position condition is satisfied. judge. Then, the current date/time corresponding image selection unit 212 selects an image element to be a special image to be displayed according to the current date/time (see FIG. 6). The time passage corresponding image selection unit 214 selects an image element that will be a special image to be displayed according to the passage of time from the first date and time when the position condition is satisfied (the date and time when the observation target part 30 is first used as an object) to the present ( See Figure 5).

The special image display history storage control unit 220 causes the storage unit 500 to store the history of the special images displayed on the image display unit 372 .

The record display control unit 222 causes the image display unit 372 to display the record display screen W8 (see FIG. 7), and based on the history stored by the special image display history storage control unit 220, displays the previously displayed special images as images. Displayed on the display unit 372 .

The clock unit 226 clocks the date and time. The clock section 953 in FIG. 2 corresponds to this.

The image generation unit 272 is implemented by, for example, a processor such as a GPU or a digital signal processor (DSP), programs and circuits such as a video signal controller and video codec, semiconductor memory for drawing frames such as a frame buffer, and the like.
Then, the image generation unit 272 generates one image in a predetermined one-frame time (for example, 1/60 second) under the control of the partial image display control unit 208, the special image display control unit 210, and the record display control unit 222. An image signal of the generated image is output to the image display unit 372 .

The image display section 372 displays various images based on the image signal input from the image generation section 272 . The touch panel 942 in FIG. 2 corresponds to this.

The storage unit 500 stores programs, various data, and the like for realizing various functions for causing the control unit 200 to integrally control the loupe-type handy device 930 . It is also used as a work area for arithmetic processing of the control unit 200, and stores calculation results executed by the control unit 200 according to various programs, input data input from the operation input unit 100, image data input from the imaging unit 102, and the like. Store temporarily. Such functions are realized by, for example, semiconductor memories such as RAM and ROM, magnetic disks such as hard disks, and optical disks such as CD-ROMs and DVDs. In the example of FIG. 2, the IC memory 952 mounted on the control board 950 and the semiconductor memory incorporated in the SoC 951 correspond to this.

FIG. 9 is a diagram showing an example of programs and data stored in the storage unit 500 of this embodiment. The storage unit 500 of this embodiment stores a system program 501 , a handy device control program 502 , painting part identification definition data 510 , painting part definition data 520 , and special image definition data 530 . In addition, according to use, information such as an ID 550 of picture to be photographed, photographed position definition data 552, special image display history data 560, timers and counters for timing, and various flags are stored.

The system program 501 is a basic program for making the loupe-type handy device 930 realize basic input/output functions as a computer.

The handy device control program 502 is a program for causing the control unit 200 to implement each functional unit such as the invisible code identification unit 202 .

The painting part identification definition data 510 is prepared for each painting part 914 included in the main body part 910 . One piece of picture part identification definition data 510 includes information of each code image included in the picture part 914 (information such as code pattern, embedded (printed) position and orientation in the picture part) and the picture. A picture part ID for identifying the part 914 is stored. Of course, data other than these, such as information on the arrangement rule when the arrangement rule of the code image is changed for each picture section 914, can be stored as appropriate.

The painting section definition data 520 is prepared for each painting section 914 included in the main body section 910 and defines each painting section 914 . One piece of painting part definition data 520 includes, for example, painting part ID 521, painting image data 522, and special image ID 524 associated with specific range definition data 523, as shown in FIG. Of course, data other than these can be stored as appropriate.

The painting image data 522 has the same image coordinate system as the corresponding painting portion 914 and is data for displaying the same content. In this embodiment, it is two-dimensional image data.

The specific range definition data 523 defines the specific range 32 for each observation target portion 30 set in the corresponding painting portion 914 (see FIG. 3). Since the specific range 32 of this embodiment is a rectangular area, the coordinates of the four corners of the rectangular area in the image coordinate system of the painting image data 522 are stored.

The special image ID 524 indicates a special image associated with the observation target portion 30 and its specific range 32 .

Returning to FIG. 9, special image definition data 530 is prepared for each special image and defines the configuration of the special image. One special image definition data 530 includes, for example, as shown in FIG. 11, a special image ID 531, image element display origin coordinates 535, and an image element ID 541 and image element data 543 associated with a display condition 540. . Of course, data other than these can be stored as appropriate.

The display origin coordinates 535 for image elements define the reference position of where the special image is displayed in the image of the painting section 914 .

A display condition 540 is a condition for displaying the corresponding image element data 543 . In this embodiment, as described above, there is a type that sets the elapsed time from the date and time when the first image was taken (see FIG. 5), and a type that sets the date and time conditions for comparison with the current date and time when the image was taken (see FIG. 6). There are two types of

The image element data 543 is data defining an image element having the same size as the specific range 32 . The first image element W41 to fourth image element W49 in FIG. 5 and the first image element W61 and second image element W62 in FIG. 6 are also defined by this data. When displayed based on the image element display origin coordinates 535 , the image element is displayed at a position that covers the specific range 32 .

Returning to FIG. 9, the photographed painting part ID 550 stores the painting part ID of the painting part 914 currently photographed.

The photographed position definition data 552 stores information for defining a relative photographed position, that is, a photographed portion in the currently photographed image of the painting portion 914 . In the present embodiment, a captured image center position coordinate 554, a rotation angle 556, and a range four corner position coordinate list 558 indicating the position coordinates of the four corners of the range to be captured are included.
In this embodiment, the loupe-type handy device 930 is used with the lighting skirt portion 938 in contact with the page 912, so the optical axis L of the camera module 955 is perpendicular to the printing surface of the painting portion 914. Alternatively, it is maintained in a predetermined relative posture that is approximately vertical. Therefore, the rotation angle 556 is only the rotation component about the vertical axis (the roll angle about the optical axis L). However, in a configuration in which the lighting skirt portion 938 is not provided, the relative posture with respect to the page 912 is not always constant.

The special image display history data 560 stores a special image ID 563 of the displayed special image and an image element ID 565 of the special image in association with the display date and time 561 each time the special image is displayed. It should be noted that all the history of the special image display history data 560 is deleted when a predetermined history deletion operation is performed.

[Description of process flow]
FIG. 12 is a flowchart for explaining the flow of processing in the loupe-type handy device 930. As shown in FIG.
The control unit 200 determines whether or not the captured image captured by the camera module 955 includes the code image 20 (step S2).

In the affirmative case (YES in step S2), the control section 200 identifies the type of the painting section 914 that is being photographed from the code pattern of the code image 20 in the photographed image (step S4). Then, the painting part ID of the identified painting part 914 is set in the photographed painting part ID 550 (see FIG. 9).

Next, the control unit 200 specifies the photographed position in the image of the identified painting portion 914 (step S8). Specifically, based on the code pattern and the position and orientation of the captured image in which the code image 20 is captured, the location of the painting portion 914 being captured and the angle of capture (roll angle with respect to the direction of capture: clockwise / counterclockwise angle) are specified, and the coordinates of the four corners of the range to be photographed are calculated.

Then, based on the painting image data 522 of the painting part definition data 520 having the painting part ID 521 (see FIG. 10) that matches the photographing object painting part ID 550, the partial image corresponding to the position to be photographed is displayed on the touch panel 942 (step S10).

If all or part of the specific range 32 is included in the captured image (captured range), it is determined that the positional condition for displaying the specific image is satisfied (YES in step S30), The control unit 200 selects the special image corresponding to the relevant specific range 32 (step S32), and displays the special image superimposed on the partial image displayed in step S10 (step S34).

Specifically, for each specific range 32 defined by the specific range definition data 523 (see FIG. 10), the specific range is partly or wholly included in the shooting range indicated by the shooting position definition data 552. determine whether there is
If included, the special image definition data 530 matching the special image ID 524 associated with the specific range definition data 523 is referred to (see FIG. 11). Then, a suitable condition is retrieved from the display conditions 540 of the special image definition data 530 referred to, and the image element data 543 corresponding to the suitable display condition 540 is selected and read out, based on the image element display origin coordinates 535. Then, it is displayed on the partial image whose screen display was started in step S10. As described above, the image element defined by the image element data 543 has the same size as the specific range 32 surrounding the observation target portion 30 and is superimposed and displayed directly above it. Therefore, on the screen display displayed on the touch panel 942, it looks as if the state of the observation target part 30 has changed.

Next, the control unit 200 stores the display history of special images (step S36).
That is, the current date and time is set as a new display date and time 561 in the special image display history data 560 (see FIG. 9), and the image of the image element data 543 displayed in step S34 is stored in the image element ID 565 associated with this. The element ID 541 (see FIG. 11) is stored, and the special image ID 531 (see FIG. 11) is set as the special image ID 563 .

Then, when a predetermined record display start operation is detected (YES in step S50), the control unit 200 causes the touch panel 942 to display the record display screen W8 (see FIG. 7) (step S52).

Then, the special image displayed in the past is displayed on the record display section 42 according to the selection operation on the menu display section 40 (step S54). Specifically, the image element ID 565 corresponding to the special image ID 563 having the same ID as the image element data 543 corresponding to the thumbnail 43 at a predetermined position (above the designated position display section 41) is read out and displayed on the record display section 42. Let

When a predetermined recording display end operation is detected (NO in step S56), the display of the recording display screen W8 is canceled (step S58).

On the other hand, if the code image 20 is not shown in the photographed image in step S2 (NO in step S2), the photographing is not performed correctly (the photographing is not performed with the lighting skirt portion 938 applied to the painting portion 914). ), the photographed image is erased, and a predetermined display indicating that the photographing is not performed correctly, such as nothing is displayed on the screen (step S60).

In other words, the information providing program according to the present invention converts a computer having a photographing unit (eg, camera module 955) and a display unit (eg, touch panel 942) into display image data (eg, painting image data). 522), and when the photographing unit recognizes an image described in the information unit (for example, the painting unit 914), the information unit stores the information unit based on the image recognized by the photographing unit. a photographed position identification means for identifying a photographed position in the display section from display image data stored in the display image data storage means based on the photographed position identified by the photographed position identification means; an information providing program functioning as partial image display control means for generating partial image data to be displayed on the display unit and controlling display of a partial image based on the partial image data on the display unit, wherein the outline of the partial image is the It is an information providing program in which at least a part of the contour of an image recognized by a photographing unit matches.

According to the present embodiment, when a pictorial image of a pictorial part is observed (captured) with a handy device, a partial image corresponding to the captured range of the pictorial image is displayed on the display part of the handy device. For example, if the painting image is a photograph of an insect, the user can experience a pseudo-observation action as if observing the insect with a handy device. In addition, by displaying only the image data stored in the storage unit on the display unit, the code image that would normally be displayed on the display unit is not displayed on the display unit. There is no longer a need to worry about , and the degree of freedom in designing the painting image and the code image in the painting department can be improved.

According to this embodiment, it is possible to display a special image that is not drawn on the picture section as the image displayed on the display section of the handy device. In other words, it is possible to provide an experience that "things that cannot be seen with the naked eye can be seen through the screen". As I said before, these experiences are very impressive. Therefore, although it is a pseudo bodily sensation, the enjoyment of the "observation action" can be impressed, and a good effect can be brought about in the intellectual training of the child.

Further, according to the present embodiment, it is possible to change the content of the special image according to the date and time when the picture portion was observed (photographed) by the handy device, the elapsed time from the start of observation, and the like. For example, if a special image with contents that can be seen only in a predetermined time period (for example, early morning) is prepared and the child can see it well, the importance of observation time can be impressed.

Further, according to the present embodiment, it is possible to simulate an observation record and the act of revisiting it later.

In addition, according to the present embodiment, the content displayed on the display section changes depending on the position from which the painting section is observed (photographed), so that it is possible to motivate the child to observe every corner of the painting section.

Further, according to the present embodiment, it is possible to determine the photographed position from the layout relationship of the code images. The calculation load can be much less than the configuration in which the image recognition technology is used to determine each part of the pictorial image to determine the position to be photographed. Since it is not necessary to use an arithmetic unit with such a high processing capacity to realize the control unit, the manufacturing cost of the handy device can be reduced.

In addition, according to the present embodiment, by designing the handy device to imitate a “magnifying glass”, which is a representative item of observation behavior, it is possible to enhance the atmosphere of observation behavior even in a simulated experience.

As described above, according to the present embodiment, if the user puts the loupe-type handy device 930 against the painting part 914 and looks into the viewing window part 940 as if observing an object with a magnifying glass, the observation object part can be viewed depending on the passage of time and the occasion. You can see how 30 has changed. Therefore, it is possible to provide a pseudo-experience in which new discoveries are made by observing over time or by changing the time of observation. As a matter of course, the content of the observation target section 30 can be set in a variety of ways, such as those in the field of natural science, the process of home cooking, and the like, so that the user can be given a simulated experience in a wide variety of fields.

[Modification]
Note that the application form of the present invention is not limited to the above-described embodiment, and constituent elements can be changed, added, or omitted as appropriate.

For example, the magnifier-type handy device 930 is not limited to a specially designed device as in the above embodiment. A handy device control program 502 (see FIG. 9), painting section identification definition data 510, painting section definition data 520, and special image definition data 530 are stored in a portable electronic device such as a smartphone or a tablet device, It may be realized by executing the handy device control program 502 as an application. However, in the case of this configuration, it is not possible to prepare a part corresponding to the lighting skirt portion 938, so the image of the painting portion 914 and the special image are displayed only during a predetermined close-up photographing (when the photographing distance is appropriate). It can be realized by controlling to Specifically, only when the code image 20 can be correctly recognized in step S2, the processes after step S6 may be executed. Of course, a part corresponding to the lighting skirt portion 938 that can be attached to and detached from the portable electronic device may be prepared separately and attached to the portable electronic device when used.

Also, in the above embodiments, a toy system for children has been described as an example, but of course the present invention is not limited to this. For example, it can be applied to an information providing system that executes content for adults.

20... Code image 30... Observation target part 32... Specific range 40... Menu display part 41... Designated position display part 42... Record display part 43... Thumbnail 44... Record display end operation icon 102 Photographing unit 200 Control unit 202 Invisible code identification unit 204 Photographed picture identification unit 206 Photographed position identification unit 208 Partial image display control Section 210 Special image display control section 212 Current date and time image selection section 214 Elapsed time image selection section 220 Special image display history storage control section 222 Record display control section 226 Clock section 372 Image display section 500 Storage section 502 Handy device control program 510 Painting section identification definition data 520 Painting section definition data 522 Painting image data 523...Specific range definition data 530...Special image definition data 540...Display condition 543...Image element data 550...Picture part ID to be photographed
552 Shooting range definition data 554 Captured image center position coordinates 556 Rotation angle 558 Range four corner position coordinate list 560 Special image display history data 561 Display date and time 563. ..Special image ID
565: image element ID
900 Toy system 910 Main unit 914 Painting unit 930 Loupe-type handy device 938 Lighting skirt unit 940 Peeping window unit 942 Touch panel 950 Control Substrate 951... SoC
952... IC memory 953... Clock unit 955... Camera module W2... Partial image W41... First image element W43... Second image element W46... Third image Element W49: Fourth image element W61: First image element W62: Second image element W8: Recording display screen

Claims (13)

An information providing system comprising an information unit in which information is described, and a portable electronic device comprising an imaging unit, a display unit, a storage unit, and a control unit,
In the information part, when the information part is recognized by the photographing part, image content is drawn so that a position to be photographed in the information part can be identified based on the recognized image,
the storage unit stores at least partial image data for displaying a partial image of the image content drawn in the information unit on the display unit;
The control section identifies a position to be photographed in the information section based on the image recognized by the photographing section, and selects a partial image in the information section corresponding to the position to be photographed based on the partial image data. Let the display control the display,
displaying a special image on the partial image displayed on the display unit when the position to be photographed satisfies a predetermined position condition;
The information section includes at least two code images, and when the code image is recognized by the imaging section, the position to be captured in the information section can be identified based on the recognized code image.
information system. In claim 1 ,
one of the code images is arranged in an inclined state with respect to the other of the code images;
information system. In claim 1 or claim 2 ,
The portable electronic device further comprises a timer,
The information providing system, wherein the control unit variably controls the special image based on time information from the time measuring unit. In any one of claims 1 to 3 ,
The control unit has a record display control unit that stores a history of the special images displayed on the display unit in the storage unit and displays the special images that were displayed in the past on the display unit based on the history. information system. In any one of claims 1 to 4 ,
The information providing system in which the information section is configured such that the photographed images obtained by the close-up photographing are composed of different images depending on the positions to be photographed. In any one of claims 1 to 4 ,
An information providing system in which the information part is composed of an image in which a code image that cannot be distinguished by the naked eye is embedded according to a predetermined arrangement rule. In any one of claims 1 to 6 ,
The portable electronic device has a loupe shape with a viewing window,
The information providing system, wherein the display section is provided in the viewing window section. A portable electronic device having an imaging unit and a display unit,
photographed position identification means for identifying a photographed position in the information portion based on the image recognized by the photographing unit when an information portion having image content is recognized by the photographing unit;
partial image display control means for controlling display of partial image data in the information section stored in the portable electronic device on the display section based on the photographed position identified by the photographed position identification section;
Photographed position condition identification means for identifying a position condition of the photographed position;
special image display control means for controlling display of a special image on the partial image displayed on the display unit when the positional condition satisfies a predetermined positional condition;
An informational program that functions as a In claim 8 ,
The portable electronic device further comprises a timer,
the portable electronic device,
An information providing program functioning as timekeeping information corresponding image selection means for variably displaying and controlling the special image based on the timekeeping information by the timekeeping unit. In claim 8 or claim 9 ,
the portable electronic device,
special image display history storage means for storing a history of the special images displayed on the display unit in the storage unit;
An information providing program that functions as record display control means for controlling display of the special image displayed in the past on the display unit based on the history. In any one of claims 8 to 10 ,
An information providing program in which the photographed position is identified by pattern matching of images in the information section in the photographed position identifying means. In any one of claims 8 to 11 ,
In the photographed position identifying means, the photographed position is defined as at least one code image in the information section having at least two code images and the other arranged in a state inclined with respect to the one code image. and an informational program identified by a code image of. In any one of claims 8 to 12 ,
An information providing program wherein, in the photographed position identifying means, the photographed position is identified by a code image in the pictorial image which cannot be distinguished by the naked eye.

Images