JP2022088329A - Voice reading system, voice reading apparatus, information processing apparatus, imaging apparatus, and computer program - Google Patents

Abstract

To provide a voice reading system, a voice reading apparatus, an information apparatus, an imaging apparatus, and a computer program that children and their families can enjoy.SOLUTION: A voice reading system according to the present invention has an imaging apparatus, and a main body apparatus having a mount surface on which an object with description of an imaging object can be mounted. The imaging apparatus has a holding part, a window part through which a user can view an imaging target, an imaging unit which can capture the imaging object while the imaging object is being view by a user through the window part, and a transmission unit for transmitting the image data obtained by capturing the object by the imaging unit. The main body apparatus has a reception unit for receiving the image data, an analysis unit for analyzing the image data received by the reception unit, and an output unit for outputting a sound based on an analysis result of the analyzing unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a voice reading system, a voice reading device, an information processing device, an image pickup device, and a computer program.

Patent Document 1 discloses application software capable of recognizing small characters by recognizing characters in a printed matter using a camera function of a smartphone and reading the recognized characters aloud.

Japanese Unexamined Patent Publication No. 2014-127197

However, the application software of Patent Document 1 is useful for users with myopia and presbyopia because the printed matter is easy to read, but it lacks playfulness because it simply reads out the text, and is not of interest to children or their families. rice field.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a voice reading system, a voice reading device, an information processing device, an image pickup device, and a computer program that can be enjoyed by children and their families. ..

The present application includes a plurality of means for solving the above problems, and to give an example thereof, the voice reading system is a main body having an image pickup device and a mounting surface on which an object on which an image pickup target is described can be placed. The image pickup device includes a grip portion, a window portion for looking into the image pickup target, and an image pickup unit capable of taking an image of the image pickup target with the image pickup target being looked into through the window portion. The main unit includes a transmitting unit that transmits image data obtained by imaging with the imaging unit to the main unit, and the main unit receives a receiving unit that receives the image data and an image data received by the receiving unit. It includes an analysis unit for analysis and an output unit for outputting voice based on the analysis result of the analysis unit.

According to the present invention, children and their families can enjoy themselves.

It is an external perspective diagram which shows an example of the structure of the voice reading system of 1st Embodiment. It is a block diagram which shows an example of the structure of the voice reading system of 1st Embodiment. It is a schematic diagram which shows an example of the structure of a BGM list. It is explanatory drawing which shows an example of a sound (sound). It is a schematic diagram which shows an example of the correction method of the image pickup range by a correction part. It is a flowchart which shows an example of the processing procedure of the voice reading system of 1st Embodiment. It is a block diagram which shows an example of the structure of the voice reading apparatus. It is a block diagram which shows an example of the structure of the voice reading system of 3rd Embodiment. It is a flowchart which shows an example of the processing procedure of the voice reading system of 3rd Embodiment. It is a figure which shows an example of the structure of the information processing system of 4th Embodiment. It is a figure which shows an example of the processing procedure of the information processing system of 4th Embodiment. It is a figure which shows an example of the processing procedure of an interest analysis function. It is a figure which shows an example of the analysis result of the interest interest analysis function. It is a figure which shows an example of the processing procedure of the interest type analysis function. It is a figure which shows an example of the analysis result of the interest type analysis function. It is a figure which shows an example of the processing procedure of an activity type analysis function. It is a figure which shows an example of the analysis result of the activity type analysis function. It is a figure which shows an example of the processing procedure of a favorite color analysis function. It is a figure which shows an example of the analysis result of the favorite color analysis function. It is a figure which shows an example of the structure of the information processing system of 5th Embodiment. It is a figure which shows an example of the processing procedure of the information processing system of 5th Embodiment. It is a figure which shows an example of the structure of the information processing system of 6th Embodiment. It is a figure which shows an example of the processing procedure of the image pickup apparatus 50 of 6th Embodiment. It is a figure which shows an example of the transition of the interest analysis result. It is a figure which shows an example of the interest analysis result by age group, region, and time series.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an external perspective view showing an example of the configuration of the voice reading system 100 of the first embodiment. The voice reading system 100 includes an image pickup device 50 and a main body device 10. The image pickup apparatus 50 includes a grip portion 62 and a window portion 61 provided on one end side of the grip portion 62. The grip portion 62 is a portion held by a user (for example, an infant, a child, a family member, etc.) when holding the image pickup device 50 in his / her hand. A lens (magnifying glass), transparent resin or glass is provided in the window portion 61, or an opening is simply formed, and a user can use the window portion 61 to make an object (for example, a picture book, a picture book, a children's book, etc.) through the window portion 61. It is possible to look into the described imaging target (for example, a character string such as a sentence, a figure including a photograph, etc.).

Further, the image pickup apparatus 50 is started by the image pickup unit 51 capable of taking an image of the image pickup target in a state where the image pickup target is looked into through the window portion 61, the distance sensor 52 for detecting the distance to the object, and the image pickup unit 51. A button (shutter button) 63 for receiving an operation is provided. The image pickup unit 51 and the distance sensor 52 are provided on one side of the grip portion 62 (the image pickup target side when looking into the image pickup target through the window portion 61), and the button 63 is provided on the other side of the grip portion 62 (window portion). It is provided on the user's face side when looking into the image pickup target via 61). The image pickup unit 51 can be composed of at least one camera. The distance sensor 52 may be any sensor that can detect the distance. Instead of the distance sensor 52, the distance may be measured according to the parallax of a plurality of cameras.

The main body device 10 has a mounting surface 21 on which an object can be mounted. The mounting surface 21 has a rectangular shape in a plan view, and is inclined so that the height decreases from each of the pair of edge portions 22 sandwiching the central portion 23 of the mounting surface 21 toward the central portion 23. There is. As a result, picture books, pictorial books, children's books, etc. can be placed on the mounting surface 21 in a spread state, and the pages of both pages can be placed at an angle smaller than 180 degrees, making it easier to see documents and figures. The object can be placed as such.

The main body device 10 includes a regulating unit 24 that regulates the movement of an object on another pair of edge portions of the mounting surface 21 along the direction in which the mounting surface 21 is inclined. The regulating portion 24 is provided so as to protrude from the mounting surface 21. This makes it possible to prevent picture books, pictorial books, children's books, etc. placed on the mounting surface 21 from slipping off the mounting surface 21.

The main body device 10 has an accommodating portion 25 for accommodating the image pickup apparatus 50 on the mounting surface 21. The shape of the accommodating portion 25 in a plan view can be the same as the shape of the image pickup device 50 in a plan view. The image pickup device 50 may be fitted into the accommodating portion 25, or both may be attracted to each other by using a magnet or the like. This prevents the risk of losing the image pickup device 50 and makes it possible for infants and children to get into the habit of cleaning up afterwards.

The main body device 10 may be provided with an indicator light (for example, an LED or the like) 26 for displaying the state of the main body device 10. The indicator lamp 26 can display states such as power supply drive, battery drive, operation, charging, and abnormality. Although not shown, a display panel capable of touch operation may be provided. The required setting operation may be performed via the display panel.

FIG. 2 is a block diagram showing an example of the configuration of the voice reading system 100 of the first embodiment. The image pickup apparatus 50 includes a correction section 53, a memory 54, a processor 55, and a communication section 56 in addition to the image pickup section 51 and the distance sensor 52 described above. The processor 55 can control the entire image pickup apparatus 50. The memory 54 is composed of a semiconductor memory or the like, and can store image data obtained by imaging with the image pickup unit 51.

The communication unit 56 realizes a communication function with the main body device 10 via a home network 1 such as a wireless LAN. The image pickup device 50 (for example, the processor 55) transmits the image data (including the image data temporarily stored in the memory 54) obtained by the image pickup by the image pickup unit 51 to the main body device 10 via the communication unit 56. Can be done.

The correction unit 53 can correct the imaging range of the imaging unit 51 so that the imaging target in the field of view through the window unit 61 can be imaged according to the distance detected by the distance sensor 52. The details of the correction unit 53 will be described later.

The main body device 10 includes a control unit 11, a communication unit 12, an analysis unit 13, a voice synthesis unit 14, an order estimation unit 15, a storage unit 16, a microphone 17, a speaker 18, and an emotion index calculation unit 19. The analysis unit 13 includes a character string analysis unit 131 and a figure analysis unit 132. The storage unit 16 is composed of, for example, a semiconductor memory, and can store the BGM list 161 and the voice data list 162. The control unit 11 can be configured by a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

The communication unit 12 realizes a communication function with the image pickup device 50 via the home network 1. The communication unit 12 can receive the image data transmitted by the image pickup apparatus 50.

The analysis unit 13 analyzes the image data received via the communication unit 12. Specifically, the analysis unit 13 can perform image recognition on the image data received via the communication unit 12 and analyze whether the image pickup target is a character string or a figure. For image recognition, for example, a known method can be used, and processing such as preprocessing, feature extraction, collation / classification, and the like may be performed.

The character string analysis unit 131 is equipped with an image processing engine and a language processing engine, and can analyze image data and output a character string (text). For the process of extracting the character string from the image data, for example, a known method may be used.

The figure analysis unit 132 is equipped with an image processing engine, and can analyze image data to analyze what a figure (including a photograph) included in an image represents. For example, the imaging target includes, but is not limited to, vehicles such as trains and automobiles, animals, insects, musical instruments, and the like.

The control unit 11 can output voice through the speaker 18 based on the analysis result of the analysis unit 13.

With the above configuration, infants and children can use the window 61 of the image pickup device 50 to look into the things they are interested in in picture books, pictorial books, and children's books and press the button 63 to teach them by voice. Infants and children will be able to experience using voice, and they will be interested in things and have a fun experience. In addition to simply looking at things and playing, when a baby or child looks into something that they are wondering or interested in, it provides a voice-based experience that leads to new discoveries. Can be provided, and "fun to know" can be provided. In addition, families (for example, parents) can have a fun experience with infants and children.

Next, the first to third methods will be described with respect to how to prepare the sound to be output.

In the first method, a text (sentence) described in a picture book, a picture book, a children's book, or the like is recorded in advance, and the recorded voice is stored in the storage unit 16 as a voice data list 162. In the voice data list 162, the information indicating the text is associated with the voice data of the text for each book such as a picture book, a picture book, or a children's book. By outputting the voice data corresponding to the character string (text) analyzed by the character string analysis unit 131 from the speaker 18, "speech reading" can be performed. When playing back the recorded voice, the attributes of the speaker may be changed. For example, the voices of men or women, the voices of young people or the elderly, and the voices of anime voice actors may be set according to preference. Such a setting may use a display panel capable of touch operation. Alternatively, the microphone 17 may be used to record the voice of the parent and reproduce the recorded voice.

The second and third methods can be performed by the voice synthesis unit 14. First, in the second method, voices are synthesized by concatenating pre-recorded voice elements. Specifically, recorded characters (for example, "a", "ka", etc.), words, and phrases can be concatenated to synthesize speech. In this case, the speech speed, the pitch of the voice, the intonation (tone, intonation), and the like can be adjusted to adjust the voice so that it sounds natural. In addition, corpus-based speech synthesis may be used. Corpus-based speech synthesis predicts the basic frequency and phoneme duration based on language features such as text sentences, phrases, accent phrases, morphology, phonemes, and accents, and is most suitable for the predicted basic frequency and phoneme duration. This is a method of selecting and concatenating matching phoneme pieces from a voice database prepared in advance.

The third method synthesizes voice using the voice features of the voice recorded in advance. Specifically, the voice synthesis unit 14 includes a trained model in which voice features are learned from the recorded voice. The voice synthesis unit 14 can convert into a voice waveform based on the voice features output by the trained model. The voice feature amount includes, for example, a mel frequency cepstrum coefficient (MFCC), a line spectrum pair (LSP), a fundamental frequency, and the like.

The control unit 11 can output BGM (also referred to as background music or background music) via the speaker 18 based on the analysis result of the analysis unit 13. Hereinafter, a specific description will be given.

The emotion index calculation unit 19 can calculate the emotion index by performing semantic analysis on the character string extracted by the character string analysis unit 131. For example, the emotion index calculation unit 19 can extract a word expressing emotion from a character string and calculate an emotion index based on the extracted word. Emotions are divided into positive emotions and negative emotions, and it is decided in advance whether the words expressing emotions are positive or negative. Furthermore, for each word expressing emotion, a value indicating the strength of emotion is determined. The emotion index calculation unit 19 can calculate an emotion index for the extracted character string based on whether each of the extracted words is positive or negative, or the value of the emotion intensity. The unit for calculating the emotional index may be, for example, one paragraph or a group of paragraphs in which a plurality of paragraphs are grouped together.

The control unit 11 can output the BGM corresponding to the emotion index calculated by the emotion index calculation unit 19 from the speaker 18 by using the BGM list 161 stored in the storage unit 16.

FIG. 3 is a schematic diagram showing an example of the configuration of the BGM list 161. The BGM list 161 defines the correspondence between the emotional index and the BGM. The emotional index can be, for example, positive levels 1 to 3 and negative levels 1 to 3. The higher the level value, the stronger the emotional strength. A plurality of BGMs are associated with each level. For example, as shown in the figure, BGM1a, BGM1b, BGM1c, and BGM1d are associated with positive level 1. The same applies to other emotional indicators. Here, the reference numerals a to d identify different scenes in stories and sentences described in picture books and the like. For example, when the emotion index is positive level 1, when the scene changes, the BGM can also change according to the scene. As a result, it is possible to produce a sense of realism, make the experience using voice more enjoyable, and make the emotions of excitement.

Further, the control unit 11 can output a sound (also referred to as a sound) via the speaker 18 based on the analysis result of the analysis unit 13. Specifically, the control unit 11 has a sound related to an analysis result (indicating what the figure (including a photograph) included in the image represents) analyzed by the figure analysis unit 132 via the speaker 18. Can be output.

FIG. 4 is an explanatory diagram showing an example of sound. As shown in the figure, the image pickup target (the figure included in the image) can be, for example, a train, a car, an animal, an insect, a musical instrument, or the like. When the figure is a train, the running sound of the train is output. If the figure is an animal, the animal's bark is output. If the figure is a musical instrument, the sound of the musical instrument is output. As a result, an infant or a child opens a picture book or the like and looks into the picture of interest through the window portion 61, so that a sound related to the picture seen by the eyes is reproduced. By looking into an object that has doubts or interests, infants and children can know what kind of sound the object makes, and can lead infants and children to new discoveries.

The order estimation unit 15 can estimate the reading order of the character strings based on the array of character strings (layout, etc.) extracted by the character string analysis unit 131. For example, if the layout of the document is vertical writing, the reading order of the character strings may be from top to bottom, and if the layout of the document is horizontal writing, the reading order of the character strings may be from left to right. As a result, the voice can be read aloud regardless of whether the object is written vertically or horizontally.

Next, the correction unit 53 will be described.

FIG. 5 is a schematic diagram showing an example of a method of correcting an imaging range by the correction unit 53. The correction unit 53 sets the image pickup range so that the image pickup target within the visible range (in the field of view) within the frame of the window unit 61 can be imaged according to the distance between the window unit 61 and the object such as a picture book. It can be corrected. As shown in the figure, when the position of the window portion 61 with respect to the surface of the picture book is P1, the range that can be seen through the window portion 61 is defined as S1. The imaging range S1 has a rectangular shape, whereas the window portion 61 has a circular shape. Therefore, the field of view is circular to be exact, but for convenience, a rectangular shape is captured in which the circular shape is an inscribed circle. The range is S1. When the window portion 61 is slightly away from the picture book and the surface of the picture book is looked at at the position P2, the distance between the window portion 61 and the picture book becomes long, and the range that can be seen through the window portion 61 becomes large. , It is corrected so as to be larger than the imaging range S1.

Next, the operation of the voice reading system 100 will be described.

FIG. 6 is a flowchart showing an example of the processing procedure of the voice reading system 100 of the first embodiment. The image pickup apparatus 50 receives a shutter button operation (S11), detects the distance to the object (S12), images the image pickup target (S13), and corrects the image data captured according to the distance to the object. (S14). The image pickup apparatus 50 transmits the image data obtained by imaging to the main body apparatus 10 (S15), and ends the process. In addition, the image data may be transmitted together with the later analysis range without correcting the image data.

The main unit device 10 receives the image data (S16) and determines whether or not the image pickup target is a character string (S17). When the image pickup target is a character string (YES in S17), the main unit 10 reads out the character string (S18). The main unit device 10 calculates an emotional index based on the character string (S19), and outputs BGM according to the emotional index (S20).

The main unit 10 determines whether or not the scene has changed based on the character string (S21), and if the scene has changed (YES in S21), changes the BGM and outputs it (S22), and ends the process. If the scene has not changed (NO in S21), the main unit 10 ends the process.

In step S17, when the image pickup target is not a character string (NO in S17), the main body apparatus 10 determines whether or not the image pickup target is a figure (S23). When the image pickup target is a figure (YES in S23), the main body apparatus 10 outputs a sound related to the content of the figure to be an image pickup target (S24) as illustrated in FIG. 4, and ends the process. When the image pickup target is not a figure (NO in S23), the main unit 10 ends the process.

(Second Embodiment)
In the first embodiment, the voice reading system 100 is configured to include the image pickup device 50 and the main body device 10, but in the second embodiment, the function of the main body device 10 is incorporated into the image pickup device 50 to form a voice reading device. ..

FIG. 7 is a block diagram showing an example of the configuration of the voice reading device 200. The appearance of the voice reading device 200 is the same as that of the image pickup device 50 of the first embodiment, and includes a grip portion, a shutter button, and a window portion. The voice reading device 200 includes a control unit 201, an imaging unit 202, a distance sensor 203, a correction unit 204, an analysis unit 205 having a character string analysis unit 206 and a figure analysis unit 207, a voice synthesis unit 208, an order estimation unit 209, and a BGM list. It includes a storage unit 210 for storing the 211 and the voice data list 212, a microphone 213, a speaker 214, and an emotion index calculation unit 215. Control unit 201, imaging unit 202, distance sensor 203, correction unit 204, analysis unit 205 having character string analysis unit 206 and figure analysis unit 207, voice synthesis unit 208, order estimation unit 209, BGM list 211 and voice data list 212. Since each function of the storage unit 210, the microphone 213, the speaker 214, and the emotion index calculation unit 215 for storing the above is the same as in the case of the first embodiment, the description thereof will be omitted.

(Third Embodiment)
The third embodiment includes an image pickup device 50 and a main body device 30 as in the case of the first embodiment, further includes a server 300, and incorporates the main functions of the main body device 10 of the first embodiment into the server 300. Is.

FIG. 8 is a block diagram showing an example of the configuration of the voice reading system of the third embodiment. The image pickup device 50 has the same configuration as the image pickup device 50 of the first embodiment. The main body device 30 includes a control unit 36, a first communication unit 31, a second communication unit 32, a storage unit 33 for storing the BGM list 331, a microphone 34, and a speaker 35.

The first communication unit 31 realizes a communication function with the image pickup device 50 via the home network 1.

The second communication unit 32 realizes a communication function with the server 300 via a communication network 2 such as the Internet. Since the BGM list 331, the microphone 34, and the speaker 35 are the same as those in the first embodiment, the description thereof will be omitted.

The server 300 includes a control unit 301, a communication unit 302, an analysis unit 303 having a character string analysis unit 304 and a figure analysis unit 305, a voice synthesis unit 306, an order estimation unit 307, a storage unit 308 for storing a voice data list 309, and a storage unit 308. The emotion index calculation unit 310 is provided.

The communication unit 302 realizes a communication function with the main body device 30 via the communication network 2. Each function of the control unit 301, the analysis unit 303, the character string analysis unit 304, the figure analysis unit 305, the voice synthesis unit 306, the order estimation unit 307, the voice data list 309, and the emotion index calculation unit 310 is the case of the first embodiment. Since it is the same as the above, the description thereof will be omitted.

FIG. 9 is a flowchart showing an example of the processing procedure of the voice reading system of the third embodiment. Since the processing of the image pickup apparatus 50 is the same as that of the first embodiment, it will be omitted. The main body device 30 receives the image data transmitted by the image pickup device 50 (S31), and transmits the received image data to the server 300 (S32).

The server 300 receives the image data (S33) and determines whether or not the image pickup target is a character string (S34). When the image pickup target is a character string (YES in S34), the server 300 converts the character string into voice data and transmits the converted voice data to the main unit 30 (S35). The main unit 30 outputs audio (S36). This makes it possible to read out the character string.

The server 300 calculates an emotion index based on the character string (S37), and transmits the calculated emotion index to the main unit device 30 (S38). The main body device 30 receives the emotion index (S39) and outputs the BGM corresponding to the received emotion index (S40).

The server 300 determines whether or not the scene has changed based on the character string (S41), and if the scene has changed (YES in S41), notifies the main unit 30 of the change in the scene (S42). The main unit 30 changes the BGM and outputs it (S43). If the scene has not changed (NO in S41), the server 300 ends the process.

In step S34, when the image pickup target is not a character string (NO in S34), the server 300 determines whether or not the image pickup target is a figure (S44). When the image pickup target is a figure (YES in S44), the server 300 transmits sound data related to the contents of the figure to the main body device 30 (S45), and ends the process. The main unit device 30 receives the sound data, outputs the sound based on the received sound data (S46), and ends the process. When the image pickup target is not a figure (NO in S44), the server 300 ends the process.

The voice reading system and the voice reading device of the present embodiment are for infants and children to play at home, play in facilities such as nursery schools and kindergartens, and temporary storage (for example, shopping malls and supermarkets). , Beauty salon, etc.) can be used for play. Further, the user is not limited to infants and children, and elderly people, foreigners, and persons with handicap may use the voice reading system and voice reading device of the present embodiment.

The image pickup device, the main body device, and the voice reading device of the voice reading system of the present embodiment can be provided to the user by sale or rental as toys. It is also possible to provide the server function on the cloud and obtain a usage fee as a consideration.

(Fourth Embodiment)
In the fourth embodiment, the same function as the above-mentioned voice reading system 100 is used to allow a guardian such as a parent to have an interest in a child such as an infant (particularly the latest interest or interest due to growth or environmental change). Information processing device systems, information processing devices, image pickup devices, etc. that can grasp the transition of information processing devices, etc. will be described.

In recent years, the number of double-income families has increased, and the number of children entrusted to nursery centers has increased, making it difficult for parents to observe their children's daytime behavior, and what interests and interests their children have. It's hard to tell if you've started or have one. Parents do not properly capture the child's growth and changes that occur little by little every day.

In addition, in the approach to develop intellectual curiosity in early childhood education, there are many uniform approaches and rough approaches to adapt to many children, such as individual intellectual curiosity, interests, and changes in taste. Is not taken into account. In the following embodiments, in order to solve these problems, support for developing interests and interests suitable for each individual will be described in consideration of real-time interest / preference states, child growth, environmental changes, and the like.

FIG. 10 is a diagram showing an example of the configuration of the information processing system of the fourth embodiment. The information processing system includes an image pickup device 50, a terminal device 100, and a server 400 as an information processing device. The image pickup apparatus 50 has the same configuration as that of the first embodiment, except that the speaker 57 is incorporated in the image pickup apparatus 50. The image pickup device 50 can take an image of an image pickup target in the reading mode or the sound effect reproduction mode. The reading mode or the sound effect reproduction mode may be manually switched by a switch, a button, or the like, or may be automatically switched according to the analysis result by the analysis unit 403 of the server 400. The image pickup device 50 is a device like a magnifying glass that infants and children hold in their hands and play, and the terminal device 100 is a terminal carried by a guardian.

The terminal device 100 has the same configuration as the main body device 30 of the third embodiment, but is different in that it includes a display unit 104 and an operation unit 105. The control unit 101, the first communication unit 102, the second communication unit 103, and the storage unit 106 are the control unit 36, the first communication unit 31, the second communication unit 32, and the storage unit of the main unit 30 of the third embodiment. It is the same as 33. The terminal device 100 can be configured by, for example, a smartphone, a tablet terminal, or the like. The display unit 104 can be configured as a liquid crystal display or an organic EL (Electro Luminescence) display. The operation unit 105 may be composed of a touch panel or the like, and may perform a character input operation on the display unit 104 and an operation for an icon, an image, a character, or the like displayed on the display unit 104.

The server 400 includes a control unit 401, a communication unit 402, an analysis unit 403, an order estimation unit 406, a voice synthesis unit 407, a sound reproduction unit 408, an analysis unit 409, and a storage unit 410 that control the entire server 400. The analysis unit 403 includes a character string analysis unit 404 and a figure analysis unit 405. The storage unit 410 stores the voice data list 411. The control unit 401, the communication unit 402, the analysis unit 403, the character string analysis unit 404, the figure analysis unit 405, the order estimation unit 406, the voice synthesis unit 407, and the storage unit 410 are the control unit 301 of the server 300 of the third embodiment. , Communication unit 302, analysis unit 303, character string analysis unit 304, figure analysis unit 305, order estimation unit 307, voice synthesis unit 306, and storage unit 308. Depending on whether the analysis result of the analysis unit 403 is a figure or a character string, the sound effect reproduction mode or the reading mode may be automatically switched. Further, the sound reproduction unit 408 is the same as the sound reproduction function of the control units 11, 301 and the like. The details of the analysis unit 409 will be described later.

FIG. 11 is a diagram showing an example of a processing procedure of the information processing system of the fourth embodiment. The image pickup apparatus 50 images an image pickup target (S101), and transmits the image data obtained by the image pickup to the terminal device 100 (S102). The terminal device 100 receives the image data (S103) and transmits the received image data to the server 400 (S104).

The server 400 receives the image data (S105) and analyzes the image pickup target based on the received image data (S106). The server 400 generates a sound or a voice that reads out a character string related to the content of the figure, depending on whether the image pickup target is a figure or a character string (S107). The sounds related to the contents of the figure and the voices that read out the character strings may be collectively referred to as "voices".

The server 400 transmits the generated sound or voice to the terminal device 100 (S108). The terminal device 100 receives the sound or voice (S109), and transmits the received sound or voice to the image pickup device 50 (S110). The image pickup apparatus 50 receives sound or sound (S111) and outputs the received sound or sound (S112).

The server 400 records the received image data and the analysis result in the storage unit 410 (S113), and updates the number of times of sound reproduction or voice reading (S114). Each time a user such as an infant or a child holds an image pickup device 50 and images an image pickup target, the process shown in FIG. 11 is repeated to collect information such as image data, analysis results, sound reproduction, or voice reading frequency. be able to.

The control unit 401 has a function as a collection unit, and collects image data obtained by imaging an image pickup target via the communication unit 402. The analysis unit 409 analyzes the interests of the user (infant or child) who has taken an image of the image pickup target based on the collected image data. The control unit 401 has a function as a providing unit and can provide the analysis result of the analysis unit 409.

In this way, the image data of the image to be imaged taken by the infant or child with interest is collected as one of the life logs, and the daily interests of the infant or child are analyzed based on the collected life log. By feeding back the analysis results to the parents, the parents can follow and support according to the interests of infants and children. For example, parents can buy goods related to areas where their children are interested or interested, or take them to places or facilities where their children are interested or interested.

Next, the details of the analysis process by the analysis unit 409 will be described. The analysis unit 409 has each function such as (1) interest-interest analysis function, (2) interest-interest type analysis function, (3) activity type analysis function, and (4) favorite color analysis function. The life log to be analyzed can be the image data captured and collected by the image pickup device 50, and the number of times of use for each of the reading mode and the sound effect reproduction mode. Hereinafter, each analysis function will be described.

FIG. 12 is a diagram showing an example of a processing procedure of the interest analysis function. Hereinafter, for convenience, the subject of processing will be described as the control unit 401. The control unit 401 collects image data (S121) and determines whether or not the image data has been collected over the first predetermined period (S122). The first predetermined period can be, for example, one week, but is not limited to this. If the collection has not been performed for the first predetermined period (NO in S122), the control unit 401 continues the process of step S121.

When the collection is performed over the first predetermined period (YES in S122), the control unit 401 classifies the imaging target by field (S123). Specifically, the analysis unit 409 classifies the imaging target by field. The analysis unit 409 may include a learning model for object detection. The learning model is, for example, HOG (Histogram of Oriented Gradients), R-CNN (Region-based CNN), Fast R-CNN, RPN (Region Proposal Network), YOLO (You Only Look Once), SSD (Single Shot Detector). , Transformer, etc. are included. Objects (objects) detected by the analysis unit 409 may be classified by field (category). The field can be appropriately determined, for example, a train, a car, an airplane, a flower, an animal, a food, a musical instrument, a fish, an insect, a doll, or the like.

The control unit 401 calculates the total number of imaging targets for each field (S124). For example, if a child images 20 objects classified as "trains" in one week, the number of "trains" is 20. The control unit 401 registers a large number of images as an "interesting" field (S125). For example, the number of images taken is calculated for each field, and the top five fields are registered as "interested" fields in descending order of the number of images taken. The number of "interested" fields is not limited to five.

The control unit 401 has a field in which the number of images taken in each field in the first predetermined period (for example, this week) is increasing as compared with the number of images taken in each field in the latest first predetermined period (for example, last week). Is registered as a "surge" (S126). For example, it is possible to calculate the difference in the number of images taken for each field between last week and this week, and register the field in which the calculated difference is equal to or higher than the predetermined difference threshold value as the field of "rapid rise". Alternatively, the field having the largest calculated difference may be registered as the field of "rapid rise".

The control unit 401 registers the field with the largest number of appearances as "my boom" among the number of images taken for each field (S127). For example, the field in which the number of objects imaged in one week is the largest can be referred to as "my boom".

The control unit 401 transmits the analysis result (“interested”, “rapid rise”, “my boom”) to the terminal device 100, and the terminal device 100 displays the analysis result. As a result, the control unit 401 provides the analysis result (S128) and ends the process. The words "interested", "surge", and "my boom" are examples, and are not limited to these words.

As described above, the analysis unit 409 has a function as a first analysis unit, and can analyze the field to be imaged based on the collected image data. The analysis unit 409 may analyze the interests of infants and children (users) based on the analysis results collected each time during the first predetermined period.

The analysis unit 409 may analyze the "interest" (first index) regarding the user's interests based on the number of images to be imaged for each field analyzed. This allows parents to provide their children with appropriate follow-up and support without overlooking the child's "beginning of interest."

The analysis unit 409 may analyze the "rapid rise" (second index) regarding the user's interests based on the change in the number of images to be imaged for each field in the first predetermined period. Further, even if the analysis unit 409 analyzes "my boom" (third index) regarding the user's interests based on the field having the largest number of image pickup targets among the number of image pickup targets imaged for each field. good.

FIG. 13 is a diagram showing an example of the analysis result of the interest analysis function. The “OO-chan's interest analysis result” screen 501 shown in FIG. 13 can be displayed on the display unit 104 of the terminal device 100. The "OO-chan's interest analysis result" screen 501 displays, for example, a message display area 502 that displays an error message such as "The date is incorrect." It has a display area 503, a display area 505 that displays the field of "interesting", a display area 506 that displays the field of "my boom", and a display area 507 that displays the field of "rapid rise". In addition, the "view all logs" icon 504 for viewing all logs is displayed.

In the example of FIG. 13, "train", "flower", "car", "food", and "animal" are displayed as "interested" fields, and 20 and 18 shots are taken in each field. , 15, 10 and 9 are displayed. In "My Boom", for example, from the field of "Train" with the largest number of shots, the object with the highest number of shots (in the example of FIG. 13, the image of "Shinkansen") is displayed, and "Current My" is displayed. The wording is displayed like "Boom is the Shinkansen!" As a result, the guardian can easily grasp the real-time interests and preferences of the child. In "rapid rise", the wording such as "It seems that you have recently become interested in" food "" is displayed. As a result, parents can appropriately grasp changes in interests and tastes, which can be useful for daily communication and life (purchasing activities, etc.) with children.

FIG. 14 is a diagram showing an example of a processing procedure of the interest type analysis function. The control unit 401 compares the ranking of "interested" in the most recent first predetermined period (for example, last week) with the ranking of "interested" in the current first predetermined period (for example, this week) (S131). , It is determined whether or not there is a change in the higher ranking (S132). For example, as illustrated in FIG. 13, when the ranking of "interested" is displayed from the 1st to the 5th place, the higher ranking can be the 1st and 2nd place rankings. Not limited. The ranking of the top two last week is, for example, "animal" in the first place and "flower" in the second place, and the ranking of the top two this week is, for example, "train" in the first place and "train" in the second place. In the case of "flower", the first place in the ranking has changed from "animal" to "train" from last week to this week, so in this case, it can be judged that there is a change in the higher ranking.

When there is a change in the higher ranking (YES in S132), the control unit 401 classifies the user's interest type into the "curiosity" type (S133), and transfers the analysis result (interest type) to the terminal device 100. Provided (S134), and the process is terminated.

When there is no change in the higher ranking (NO in S132), the control unit 401 determines whether or not there is a change in the ranking (S135). If there is no change in the ranking (YES in S135), that is, if there is no change in the ranking from 1st to 5th between last week and this week, the control unit 401 sets the user's interest type to "contemplation type doctor" type. (S136), and the process of step S134 is performed.

When there is a change in the ranking (NO in S135), that is, when there is a change in the lower ranking excluding the upper rank (for example, the ranking from the 3rd place to the 5th place), the control unit 401 sets the user's interest type to ". It is classified into the "intermediate" type (S137), and the process of step S134 is performed. Further, the change in the ranking may be determined not only by the replacement of the upper and lower ranks but also by the change of the entire ranking. For example, the weekly ranking in descending order of the number of detections is determined according to what percentage of the rankings (for example, TOP 20th place, etc.) from the highest rank to the predetermined number (the predetermined number is variable) have been replaced. May be good. For example, if the rankings are changed to N% or more, it is judged to be "curiosity", and if the rankings are not changed to N% or less, it is judged to be "contemplation type doctor". In the case of, it may be determined as "intermediate". The numerical value of N can be set as appropriate. The words "curiosity", "contemplation type doctor", and "intermediate" are examples, and are not limited to these words.

As described above, the analysis unit 409 has a function as a specific unit, identifies changes in the "interested" (first index) analyzed at each first predetermined period, and identifies the changed "interested". Depending on the user's interests, the type of interest may be analyzed.

FIG. 15 is a diagram showing an example of the analysis result of the interest type analysis function. The “OO-chan's interest analysis result” screen 511 shown in FIG. 15 can be displayed on the display unit 104 of the terminal device 100. The "OO-chan's interest analysis result" screen 511 has, for example, a message display area 502 for displaying an error message, a display area 512 for displaying an interest type, and a display area 514 for displaying this week's log.

In the example of FIG. 15, as an interest type, a word such as "OO-chan is interested in various things" is displayed. As a result, the guardian can grasp the interest type of the child and can follow or support the child's interest type. By operating the "select photo" icon 513, the guardian can display the desired photo from the child's photo recorded on the terminal device 100 and the child's photo uploaded from another smartphone or PC. Can be done.

As a log of this week, you can display the ranking of "interested" fields, the ratio of objects taken this week by field, "surge" and so on. By operating the "see details" icon 515, the ranking of the "interested" field can be displayed in more detail. Also, by operating the "View last week's log" icon 516, the last week's log can be displayed in place of this week's log or together with this week's log.

As a result, the guardian can appropriately grasp the interest type of the child and can use it for daily communication and life (purchasing activity, etc.) with the child.

FIG. 16 is a diagram showing an example of a processing procedure of the activity type analysis function. The control unit 401 records the number of times the sound effect reproduction function and the reading function have been used (S141), and determines whether or not the recording has been performed over a third predetermined period (S142). The number of times the sound effect reproduction function and the reading function are used is the number of times the sound effect reproduction mode and the reading mode are used. The sound effect reproduction mode and the reading mode can be set manually or automatically. The third predetermined period can be, for example, one month, but is not limited thereto.

If recording has not been performed for a third predetermined period (NO in S142), the control unit 401 continues the process of step S141. When recording is performed over a third predetermined period (YES in S142), the control unit 401 determines whether or not the ratio of the number of times the sound effect reproduction function is used is M% or more of the whole (S143). The whole is the total number of times the sound effect reproduction function is used and the number of times the reading function is used. Further, the numerical value of M is variable and can be changed as appropriate. When the ratio of the number of times the sound effect reproduction function is used is M% or more of the whole (YES in S143), the control unit 401 classifies the user's activity type into "explorer type" (S144), and the analysis result (activity). Type) is provided to the terminal device 100 (S145), and the process is terminated.

When the ratio of the number of times the sound effect reproduction function is used is not M% or more of the whole (NO in S143), the control unit 401 determines whether or not the ratio of the number of times the reading function is used is M% or more of the whole (NO). S146). When the ratio of the number of times the reading function is used is M% or more of the whole (YES in S146), the control unit 401 classifies the user's activity type into the "reader type" (S147), and performs the process of step S145. .. When the ratio of the number of times the reading function is used is not M% or more of the whole (NO in S146), the control unit 401 classifies the user's activity type into the "interesting type" (S148), and performs the process of step S145.

As described above, the analysis unit 403 has a function as a third analysis unit, and can analyze whether the image pickup target is a character string or a figure based on the collected image data. The voice synthesis unit 407 has a function as a reading unit, and when it is analyzed that the image pickup target is a character string, the character string can be read aloud. The sound reproduction unit 408 has a function as a reproduction unit, and when it is analyzed that the image pickup target is a figure, the sound reproduction unit 408 can reproduce the sound related to the content of the figure. The analysis unit 409 analyzes the activity type of the user based on the number of times the voice synthesis unit 407 reads aloud and the number of times the sound reproduction unit 408 reproduces over a third predetermined period (for example, one month). May be good.

FIG. 17 is a diagram showing an example of the analysis result of the activity type analysis function. The “OO-chan's interest analysis result” screen 521 shown in FIG. 17 can be displayed on the display unit 104 of the terminal device 100. The "OO-chan's interest analysis result" screen 521 has, for example, a message display area 502 for displaying an error message, a display area 522 for displaying an activity type, and a display area 514 for displaying this week's log.

In the example of FIG. 17, as an activity type, a word such as "OO-chan likes to search for and play various things" is displayed. As a result, the guardian can grasp the activity type of the child, and can follow or support according to the activity type of the child. By operating the "select photo" icon 513, the guardian can display the desired photo from the child's photo recorded on the terminal device 100 and the child's photo uploaded from another smartphone or PC. Can be done.

This week's log is the same as in FIG. 15, so the explanation is omitted. In this way, the guardian can properly grasp the activity type of the child and can be useful for daily communication and life (purchasing activity, etc.) with the child.

FIG. 18 is a diagram showing an example of a processing procedure of a favorite color analysis function. The control unit 401 collects image data (S151) and determines whether or not the image data has been collected over a second predetermined period (S152). The second predetermined period can be, for example, one week, but is not limited to this. If the collection has not been performed for the second predetermined period (NO in S152), the control unit 401 continues the process of step S151.

When the collection is performed over the second predetermined period (YES in S152), the control unit 401 classifies the colors included in the image pickup target (S153). Specifically, the analysis unit 409 includes a learning model having a color analysis function, and can detect the color of the region in the image and the color used in the image. The analysis unit 409 can classify colors according to frequent colors.

The control unit 401 calculates the total number of imaging targets for each color (S154). That is, the total number of detected objects for each classified color may be calculated. For example, if a child images 20 objects classified as yellow in one week, the number of "yellow" objects is 20. The control unit 401 registers a color with a large number of images as a "favorite color" (S155).

The control unit 401 has a color in which the number of images taken for each color in the second predetermined period (for example, this week) is increasing as compared with the number of images taken for each color in the latest second predetermined period (for example, last week). Is registered as a "surge" (S156). For example, the difference between the number of images taken for each color last week and this week can be calculated, and a color whose calculated difference is equal to or greater than a predetermined difference threshold value can be registered as a "rapid rise" color. Alternatively, the color with the largest calculated difference may be registered as the "rapid rise" color.

The control unit 401 provides the analysis result (“favorite color”, “rapid rise”) to the terminal device 100 (S157), and ends the process. The words "favorite color" and "rapid rise" are examples, and are not limited to these words.

As described above, the analysis unit 409 has a function as a second analysis unit, and analyzes the color included in the image pickup target based on the collected image data. The analysis unit 409 may analyze the user's interest in color based on the analysis result collected each time during the second predetermined period.

FIG. 19 is a diagram showing an example of the analysis result of the favorite color analysis function. The “OO-chan color preference analysis result” screen 531 shown in FIG. 19 can be displayed on the display unit 104 of the terminal device 100. The "OO-chan color preference analysis result" screen 531 is, for example, a message display area 502 for displaying an error message, a display area 532 for displaying an image of a favorite color, a display area 533 for displaying a favorite color, and a "surge". It has a display area 534 for displaying the color of.

In the display area 532, "OO-chan seems to like" yellow "especially! ], And images 1 to 4 drawn in a favorite color (yellow) are displayed. Images 1 to 4 may be displayed by switching to another image from the life log each time a certain period of time elapses. The order of favorite colors (in the example of FIG. 19, the first place is "kiiro", the second place is "red", and the third place is "blue") is displayed.

In the display area 533, the number of objects detected for each color is displayed in the order of favorite colors. In the example of FIG. 19, the numerical values of "kiiro" are 20 cases, "red" is 18 cases, "blue" is 15 cases, "orange" is 10 cases, and "green" is 7 cases. "Green" is a color in which the number of shots taken this week is increasing compared to last week's analysis results and this week's analysis results, and the wording "surge!" Is given.

In the display area 534, words such as "It seems that you have recently become interested in" green "" are displayed, and images 1 and 2 drawn in green are displayed. This makes it easy for parents to grasp in real time the colors that their children are beginning to be interested in and the colors they like. In addition, parents can appropriately grasp the change in color they are interested in, which can be useful for daily communication and life (purchasing activities, etc.) with their children. In the above example, the interest type shown in FIG. 15, the activity type shown in FIG. 17, and the favorite color analysis shown in FIG. 19 have been described using individual figures for convenience, but these are examples. Therefore, the interest type, activity type, and favorite color analysis can be displayed simultaneously on the same screen.

(Fifth Embodiment)
In the fourth embodiment, the server 400 is provided with an analysis function, but the present invention is not limited to this. In the fifth embodiment, a configuration in which the terminal device is provided with an analysis function will be described.

FIG. 20 is a diagram showing an example of the configuration of the information processing system of the fifth embodiment. The information processing system includes an image pickup device 50 and a terminal device 150 as an information processing device. The image pickup apparatus 50 is the same as in the case of the fourth embodiment. Compared with the case of the fourth embodiment, the terminal device 150 has an analysis unit 155, a character string analysis unit 156, a figure analysis unit 157, an order estimation unit 158, a voice synthesis unit 159, a sound reproduction unit 160, and an analysis unit 161. It differs in that it includes an audio data list 163 and a computer program 164. The analysis unit 155, the character string analysis unit 156, the figure analysis unit 157, the order estimation unit 158, the voice synthesis unit 159, the sound reproduction unit 160, the analysis unit 161 and the voice data list 163 are each provided with the server 400 of the fourth embodiment. This is the same as the analysis unit 403, the character string analysis unit 404, the figure analysis unit 405, the order estimation unit 406, the voice synthesis unit 407, the sound reproduction unit 408, the analysis unit 409, and the voice data list 411. The computer program 164 is executed by the control unit 151, so that the analysis unit 155, the character string analysis unit 156, the figure analysis unit 157, the order estimation unit 158, the speech synthesis unit 159, the sound reproduction unit 160, and the analysis unit 161 All or part of the functions can be realized.

FIG. 21 is a diagram showing an example of a processing procedure of the information processing system of the fifth embodiment. The image pickup apparatus 50 images an image pickup target (S161), and transmits the image data obtained by the image pickup to the terminal device 150 (S162). The terminal device 150 receives the image data (S163) and analyzes the image pickup target based on the received image data (S164). The terminal device 150 generates a sound or a voice that reads out a character string related to the content of the figure, depending on whether the image pickup target is a figure or a character string (S165).

The terminal device 150 transmits the generated sound or voice to the image pickup device 50 (S166). The image pickup apparatus 50 receives sound or sound (S167) and outputs the received sound or sound (S168). The terminal device 150 records the received image data and the analysis result in the storage unit 162 (S169), and updates the number of times of sound reproduction or voice reading (S170). Each time a user such as an infant or a child holds an image pickup device 50 and images an image pickup target, the process shown in FIG. 21 is repeated to collect information such as image data, analysis results, sound reproduction, or voice reading frequency. be able to. In addition, since the same processing as in the case of FIGS. 12 to 19 is performed in the fifth embodiment, the description thereof will be omitted.

The control unit 151 has a function as a collection unit, and collects image data obtained by imaging an image pickup target via the communication unit 152. The analysis unit 161 analyzes the interests of the user (infant or child) who has taken an image of the image pickup target based on the collected image data. The control unit 151 has a function as a providing unit and can provide the analysis result of the analysis unit 161.

Further, the computer program 164 operating on the terminal device 150 collects image data obtained by imaging the image pickup target on the computer, and based on the collected image data, interests the user who imaged the image pickup target. Analyze, provide analysis results, and execute processing.

In this way, the image data of the image to be imaged taken by the infant or child with interest is collected as one of the life logs, and the daily interests of the infant or child are analyzed based on the collected life log. By feeding back the analysis results to the parents, the parents can follow and support according to the interests of infants and children. For example, parents can buy goods related to areas where their children are interested or interested, or take them to places or facilities where their children are interested or interested.

(Sixth Embodiment)
In the sixth embodiment, a configuration in which the image pickup apparatus is provided with an analysis function will be described.

FIG. 22 is a diagram showing an example of the configuration of the information processing system of the sixth embodiment. The information processing system includes an image pickup device 50 and a terminal device 100. The terminal device 100 is the same as in the case of the fourth embodiment. Compared with the case of the fourth embodiment, the image pickup apparatus 50 includes an analysis unit 71, a character string analysis unit 72, a figure analysis unit 73, an order estimation unit 65, a voice synthesis unit 66, a sound reproduction unit 67, and an analysis unit 68. The difference is that the voice data list 70 is provided. The analysis unit 71, the character string analysis unit 72, the figure analysis unit 73, the order estimation unit 65, the voice synthesis unit 66, the sound reproduction unit 67, the analysis unit 68, and the voice data list 70 are each provided with the server 400 of the fourth embodiment. This is the same as the analysis unit 403, the character string analysis unit 404, the figure analysis unit 405, the order estimation unit 406, the voice synthesis unit 407, the sound reproduction unit 408, the analysis unit 409, and the voice data list 411.

FIG. 23 is a diagram showing an example of the processing procedure of the image pickup apparatus 50 of the sixth embodiment. The image pickup apparatus 50 takes an image of the image pickup target (S181) and analyzes the image pickup target based on the image data obtained by the image pickup (S182). The image pickup apparatus 50 generates a sound or a voice that reads out a character string related to the content of the figure, depending on whether the image pickup target is a figure or a character string (S183).

The image pickup apparatus 50 outputs the generated sound or sound (S184). The image pickup apparatus 50 records the image data and the analysis result in the storage unit 69 (S185), and updates the number of times of sound reproduction or voice reading (S186). Each time a user such as an infant or a child holds an image pickup device 50 and images an image pickup target, the process shown in FIG. 23 is repeated to collect information such as image data, analysis results, sound reproduction, or voice reading frequency. be able to. In the sixth embodiment as well, the same processing as in the case of FIGS. 12 to 19 is performed, so the description thereof will be omitted.

The image pickup apparatus 50 is imaged by a grip portion 62, a window portion 61 for looking into the image pickup target, an image pickup unit 51 capable of capturing an image pickup target in a state where the image pickup target is viewed through the window portion 61, and an image pickup unit 51. An analysis unit 68 that analyzes the interests of the user who has imaged the imaged object based on the image data collected in the above process, and a control unit 11 as a providing unit that provides the analysis results of the analysis unit 68 are provided.

In this way, the image data of the image to be imaged taken by the infant or child with interest is collected as one of the life logs, and the daily interests of the infant or child are analyzed based on the collected life log. By feeding back the analysis results to the parents, the parents can follow and support according to the interests of infants and children. For example, parents can buy goods related to areas where their children are interested or interested, or take them to places or facilities where their children are interested or interested.

In the fourth to sixth embodiments, the analysis function is provided in any one of the server 400, the terminal device 150, or the image pickup device 50, but the analysis function is provided in the server 400, the terminal device 150, and the image pickup device. It may be configured to be distributed and provided by at least two of 50.

FIG. 24 is a diagram showing an example of changes in the results of interest analysis. By collecting the interest-interest analysis results of users such as infants and children according to the growth of the users, it is possible to grasp the interest-interest analysis results of individual users in transition. In the example of FIG. 24, the transition of the “my boom” field (for example, fields A1 to A5) and the transition of the “rapid rise” field (for example, fields B1 to B5) are displayed together with the age of the user. The fields A1 to A5 and B1 to B5 may include shared fields. By collecting the data shown in FIG. 24 for a large number of users, it becomes possible to utilize it as big data. In addition, it may be possible to predict how the interests of individual users will change / change in the future.

FIG. 25 is a diagram showing an example of the results of interest analysis by age group, region, and time series. By collecting the analysis results of a large number of children, it is possible to grasp the difference in interest trends by age group, region, and time series. This makes it possible to grasp the time-series transition and future forecast for each arbitrary group (user group) such as by age group, region, and nursery school. FIG. 25A shows the results of analyzing the fields of interest by age groups A1, A2, A3, and so on. For convenience, the fields of interest are C1 to C5, but the number of fields is not limited to this. Areas of interest in each age group may be represented by a diagram such as a radar chart. FIG. 25B shows the results of analysis of areas of interest by regions L1, L2, L3, and so on. For convenience, the fields of interest are C1 to C5, but the number of fields is not limited to this. Areas of interest in each region may be represented by a diagram such as a radar chart. FIG. 25C shows the result of analyzing the field of interest according to the elapsed time from a predetermined reference time point (for example, a specific age). For convenience, the fields of interest are C1 to C5, but the number of fields is not limited to this. Areas of interest at each transition point may be represented by a diagram such as a radar chart. As described above, the analysis unit 409 uses the collected image data to determine the temporal transition of interests of the user group classified by at least one by age group and region (for example, a time series for each arbitrary group). (Including transitions or future forecasts) may be analyzed.

The voice reading system of the present embodiment includes an image pickup device and a main body device having a mounting surface on which an object on which an image pickup target is described can be placed, and the image pickup device includes a grip portion and an image pickup target. A window unit for looking into, an image pickup unit capable of capturing an image pickup target in a state where the image pickup target is looked into through the window portion, and an image data obtained by imaging with the image pickup unit are used as the main body device. The main unit includes a transmitting unit for transmitting to, a receiving unit for receiving the image data, an analysis unit for analyzing the image data received by the receiving unit, and a voice based on the analysis result of the analysis unit. It has an output unit to output.

In the voice reading system of the present embodiment, the front-described mounting surface has a rectangular shape in a plan view, and the height from each of the pair of edge portions sandwiching the central portion of the front-stated placement surface toward the central portion. Is tilted so that

In the voice reading system of the present embodiment, the main body apparatus has a regulating unit that regulates the movement of the object on another pair of edge portions of the previously described mounting surface along the tilting direction of the previously described mounting surface. Be prepared.

In the voice reading system of the present embodiment, the main body apparatus forms an accommodating portion for accommodating the image pickup apparatus on the above-mentioned mounting surface.

In the voice reading system of the present embodiment, in the image pickup device, the detection unit that detects the distance to the object and the image pickup target in the field of view through the window unit according to the distance detected by the detection unit. It is provided with a correction unit that corrects the imaging range of the imaging unit so that imaging is possible.

In the voice reading system of the present embodiment, the analysis unit analyzes whether the image pickup target is a character string or a figure based on the image data received by the reception unit, and the output unit is the analysis unit. When it is analyzed that the image pickup target is a character string, the voice that reads out the character string is output.

In the voice reading system of the present embodiment, when the analysis unit analyzes that the image pickup target is a figure, the output unit outputs a sound related to the content of the figure.

The voice reading system of the present embodiment includes an emotion index calculation unit that calculates an emotion index by performing semantic analysis on the character string when the analysis unit analyzes that the image pickup target is a character string. The output unit outputs background music according to the emotion index calculated by the emotion index calculation unit.

The voice reading system of the present embodiment includes a reading order estimation unit that estimates the reading order of the character string based on the arrangement of the character string when the analysis unit analyzes that the image pickup target is a character string.

The voice reading system of the present embodiment includes a voice synthesis unit that synthesizes voice using a voice feature amount of a voice recorded in advance, and the output unit analyzes that the image pickup target is a character string by the analysis unit. If so, the voice synthesized by the voice synthesizer based on the character string is output.

The voice reading system of the present embodiment includes a voice synthesis unit that concatenates pre-recorded voice elements to synthesize voice, and the output unit analyzes that the image pickup target is a character string by the analysis unit. If so, the voice synthesized by the voice synthesizer based on the character string is output.

The voice reading device of the present embodiment includes a grip portion, a window portion for looking into the image pickup target, and an image pickup unit capable of taking an image of the image pickup target in a state where the image pickup target is looked into through the window portion. It is provided with an analysis unit that analyzes image data obtained by imaging with the image pickup unit, and an output unit that outputs audio based on the analysis result of the analysis unit.

In the voice reading device of the present embodiment, the detection unit that detects the distance to the object on which the image pickup target is described and the image pickup target in the field of view through the window portion according to the distance detected by the detection unit. It is provided with a correction unit that corrects the imaging range of the imaging unit so that imaging is possible.

The information processing apparatus of the present embodiment has a collection unit that collects image data obtained by imaging the image pickup target, and a user who has imaged the image pickup target based on the image data collected by the collection unit. It is provided with an analysis unit for analyzing the above and a providing unit for providing the analysis result of the analysis unit.

The information processing apparatus of the present embodiment includes a first analysis unit that analyzes a field to be imaged based on the image data collected by the collection unit, and the analysis unit collects the first analysis unit each time during the first predetermined period. 1 Based on the analysis result of the analysis unit, the interest of the user is analyzed.

In the information processing apparatus of the present embodiment, the analysis unit analyzes the first index related to the user's interests based on the number of images to be imaged for each field analyzed by the first analysis unit.

The information processing apparatus of the present embodiment includes a specific unit that specifies a change in the first index analyzed by the analysis unit at each first predetermined period, and the analysis unit includes the first unit specified by the specific unit. 1 The type of interest of the user is analyzed according to the change of the index.

In the information processing apparatus of the present embodiment, the analysis unit is interested in the user based on the change in the number of images to be imaged for each field analyzed by the first analysis unit for each first predetermined period. Analyze a second indicator of interest.

In the information processing apparatus of the present embodiment, the analysis unit is based on the field in which the number of image pickup targets is the largest among the number of image pickup targets imaged in each field analyzed by the first analysis unit. Interest The third index of interest is analyzed.

The information processing apparatus of the present embodiment includes a second analysis unit that analyzes colors included in the image pickup target based on the image data collected by the collection unit, and the analysis unit collects each time during the second predetermined period. Based on the analysis result of the second analysis unit, the user's interest in color is analyzed.

The information processing apparatus of the present embodiment has a third analysis unit that analyzes whether the image pickup target is a character string or a figure based on the image data collected by the collection unit, and a third analysis unit that analyzes the image pickup target. When it is analyzed that is a character string, a reading unit which reads out the character string, and when the third analysis unit analyzes that the image pickup target is a figure, a reproduction unit which reproduces a sound related to the contents of the figure. The analysis unit analyzes the activity type of the user based on the number of times the reading unit reads aloud and the number of times the reproduction unit reproduces the data over a third predetermined period.

The information processing apparatus of the present embodiment includes an output unit that outputs voice based on the analysis result of the third analysis unit.

In the information processing apparatus of the present embodiment, the analysis unit determines the temporal transition of the interests of the user group classified by at least one by age group and region based on the image data collected by the collection unit. analyse.

The image pickup apparatus of the present embodiment includes a grip portion, a window portion for looking into the image pickup target, and an image pickup unit capable of taking an image of the image pickup target with the image pickup target being looked into through the window portion. It includes an analysis unit that analyzes the interests of the user who has imaged the image pickup target based on the image data captured and collected by the image pickup unit, and a providing unit that provides the analysis result of the analysis unit.

The computer program of the present embodiment collects image data obtained by imaging an image pickup target on a computer, and analyzes and analyzes the interests of the user who imaged the image pickup target based on the collected image data. Perform processing that provides results.

1 Home network 2 Communication network 10 Main unit 11, 201 Control unit 12 Communication unit 13, 205 Analysis unit 131, 206 Character string analysis unit 132, 207 Figure analysis unit 14, 208 Voice synthesis unit 15, 209 Order estimation unit 16, 210 Storage unit 161, 211 BGM list 162, 212 Voice data list 17, 213 Microphone 18, 214 Speaker 19, 215 Emotion index calculation unit 21 Mounting surface 22 Edge part 23 Central part 24 Restriction part 25 Containment part 26 Indicator light 30 Main unit 31 1st communication unit 32 2nd communication unit 33 Storage unit 331 BGM list 34 Microphone 35 Speaker 36 Control unit 50 Imaging device 51, 202 Imaging unit 52, 203 Distance sensor 53, 204 Correction unit 54 Memory 55 Processor 56 Communication unit 57 Speaker 69 Storage unit 70 Voice data list 100, 150 Terminal device 101, 151 Control unit 102 First communication unit 103 Second communication unit 152 Communication unit 104, 153 Display unit 105, 154 Operation unit 106, 162 Storage unit 163 Voice data list 164 Computer program 300 Server 301 Control unit 302 Communication unit 303 Analysis unit 304 Character string analysis unit 305 Figure analysis unit 306 Voice synthesis unit 307 Order estimation unit 308 Storage unit 309 Voice data list 310 Emotion index calculation unit 400 Server 401 Control unit 402 Communication unit 403, 155, 71 Analysis unit 404, 156, 72 Character string analysis unit 405, 157, 73 Figure analysis unit 406, 158, 65 Order estimation unit 407, 159, 66 Voice synthesis unit 408, 160, 67 Sound reproduction unit 409, 161 and 68 Analysis unit 410 Storage unit 411 Voice data list

Claims (25)

It is provided with an image pickup device and a main body device having a mounting surface on which an object on which an image pickup target is described can be placed.
The image pickup device
With the grip
A window for looking into the object to be imaged, and
An image pickup unit capable of taking an image of the image pickup target while the image pickup target is being looked into through the window portion.
It is provided with a transmission unit that transmits image data obtained by imaging with the image pickup unit to the main unit.
The main body device is
A receiving unit that receives the image data and
An analysis unit that analyzes the image data received by the reception unit, and an analysis unit.
It is provided with an output unit that outputs voice based on the analysis result of the analysis unit.
Voice reading system. The above-mentioned surface is
It has a rectangular shape in a plan view,
The height is inclined from each of the pair of edge portions sandwiching the central portion of the above-mentioned mounting surface toward the central portion.
The voice reading system according to claim 1. The main body device is
The other pair of edge portions of the previously described mounting surface are provided with a regulating portion for restricting the movement of the object along the inclined direction of the previously described mounting surface.
The voice reading system according to claim 1 or 2. The main body device is
An accommodating portion for accommodating the image pickup device is formed on the above-mentioned mounting surface.
The voice reading system according to any one of claims 1 to 3. The image pickup device
A detector that detects the distance to the object,
It is provided with a correction unit that corrects the image pickup range of the image pickup unit so that the image pickup target in the field of view through the window unit can be imaged according to the distance detected by the detection unit.
The voice reading system according to any one of claims 1 to 4. The analysis unit
Based on the image data received by the receiving unit, it is analyzed whether the image pickup target is a character string or a figure.
The output unit is
When the analysis unit analyzes that the image pickup target is a character string, it outputs a voice that reads out the character string.
The voice reading system according to any one of claims 1 to 5. The output unit is
When the analysis unit analyzes that the image pickup target is a figure, the sound related to the contents of the figure is output.
The voice reading system according to claim 6. When the analysis unit analyzes that the image pickup target is a character string, the analysis unit includes an emotion index calculation unit that performs semantic analysis on the character string and calculates an emotion index.
The output unit is
Outputs background music according to the emotion index calculated by the emotion index calculation unit.
The voice reading system according to claim 6 or 7. When the analysis unit analyzes that the image pickup target is a character string, the analysis unit includes a reading order estimation unit that estimates the reading order of the character string based on the arrangement of the character strings.
The voice reading system according to any one of claims 6 to 8. It is equipped with a voice synthesizer that synthesizes voice using the voice features of pre-recorded voice.
The output unit is
When the analysis unit analyzes that the image pickup target is a character string, the voice synthesized unit outputs the voice synthesized based on the character string.
The voice reading system according to any one of claims 6 to 9. Equipped with a voice synthesizer that synthesizes voice by concatenating pre-recorded voice elements
The output unit is
When the analysis unit analyzes that the image pickup target is a character string, the voice synthesized unit outputs the voice synthesized based on the character string.
The voice reading system according to any one of claims 6 to 9. With the grip
A window for looking into the object to be imaged, and
An image pickup unit capable of taking an image of the image pickup target while the image pickup target is being looked into through the window portion.
An analysis unit that analyzes image data obtained by imaging with the imaging unit, and an analysis unit.
It is provided with an output unit that outputs voice based on the analysis result of the analysis unit.
Voice reading device. A detector that detects the distance to the object on which the imaging target is described, and
It is provided with a correction unit that corrects the image pickup range of the image pickup unit so that the image pickup target in the field of view through the window unit can be imaged according to the distance detected by the detection unit.
The voice reading device according to claim 12. A collection unit that collects image data obtained by imaging an image target, and
An analysis unit that analyzes the interests of the user who captured the image pickup target based on the image data collected by the collection unit, and an analysis unit.
It is provided with a providing unit that provides the analysis result of the analysis unit.
Information processing equipment. A first analysis unit that analyzes the field to be imaged based on the image data collected by the collection unit is provided.
The analysis unit
Based on the analysis result of the first analysis unit collected each time in the first predetermined period, the interest of the user is analyzed.
The information processing apparatus according to claim 14. The analysis unit
The first index relating to the interest of the user is analyzed based on the number of images to be imaged for each field analyzed by the first analysis unit.
The information processing apparatus according to claim 15. A specific unit for specifying a change in the first index analyzed by the analysis unit is provided for each first predetermined period.
The analysis unit
The type of interest of the user is analyzed according to the change of the first index specified by the specific unit.
The information processing apparatus according to claim 16. The analysis unit
The second index regarding the interest of the user is analyzed based on the change in the number of images to be imaged for each field analyzed by the first analysis unit for each first predetermined period.
The information processing apparatus according to any one of claims 15 to 17. The analysis unit
Among the number of images to be imaged for each field analyzed by the first analysis unit, the third index relating to the interest of the user is analyzed based on the field having the largest number of images to be imaged.
The information processing apparatus according to any one of claims 15 to 18. It is provided with a second analysis unit that analyzes the colors included in the image pickup target based on the image data collected by the collection unit.
The analysis unit
Based on the analysis result of the second analysis unit collected each time in the second predetermined period, the user's interest in color is analyzed.
The information processing apparatus according to any one of claims 15 to 19. A third analysis unit that analyzes whether the image pickup target is a character string or a figure based on the image data collected by the collection unit, and
When the third analysis unit analyzes that the image pickup target is a character string, the reading unit that reads out the character string and the reading unit
When the third analysis unit analyzes that the image pickup target is a figure, the third analysis unit includes a reproduction unit that reproduces the sound related to the contents of the figure.
The analysis unit
3. The activity type of the user is analyzed based on the number of times the reading section reads aloud and the number of times the reading section plays the song over a predetermined period.
The information processing apparatus according to any one of claims 15 to 20. An output unit that outputs voice based on the analysis result of the third analysis unit is provided.
The information processing apparatus according to claim 21. The analysis unit
Based on the image data collected by the collection unit, the temporal transition of interests of the user group classified by at least one by age group and region is analyzed.
The information processing apparatus according to any one of claims 1 to 22. With the grip
A window for looking into the object to be imaged, and
An image pickup unit capable of taking an image of the image pickup target while the image pickup target is being looked into through the window portion.
An analysis unit that analyzes the interests of the user who has imaged the image pickup target based on the image data captured and collected by the image pickup unit.
It is provided with a providing unit that provides the analysis result of the analysis unit.
Imaging device. On the computer
Collect the image data obtained by imaging the image to be imaged,
Based on the collected image data, the interests of the user who imaged the image pickup target are analyzed.
Providing analysis results,
A computer program that executes processing.

Images