JP6618068B2 - Substance learning support device, server device, substance learning support method, and program - Google Patents

Description

  The present invention relates to a substance learning support device that supports learning of fine substances that cannot be visually recognized.

  Conventionally, learning is performed by reading books, attending lectures, and the like in academic fields where invisible fine substances such as pharmacy and medicine are learned (see, for example, Non-Patent Document 1).

Masafumi Tamura, “Drug and biological three-dimensional structure chemistry-Did you see the medicine and target?”, [Online], [Search January 3, 2015], Internet [URL: http: // www .kyoto-hirokawa.co.jp / info / detail.php? id = 13796392434]

  However, in the past, it has been difficult to efficiently and effectively learn in the academic field of learning invisible fine substances such as pharmacy and medicine.

  The substance learning support apparatus according to the first aspect of the invention includes a reception unit that receives a book-captured video that is an image of a book on which an invisible fine object that is an object related to a fine substance that cannot be visually recognized is printed, and a reception unit Transmitting unit for transmitting 3D moving image specifying information, which is an object identifier for identifying an invisible fine object included in the received book shooting image or the book shooting image received by the receiving unit, and transmission of the 3D moving image specifying information Accordingly, the server device is a three-dimensional moving image specified by the three-dimensional moving image specifying information, and a receiving unit that receives the three-dimensional moving image of the invisible fine object and the book photographing image received by the receiving unit, A substance learning support device including an output unit that outputs a three-dimensional video received by a receiving unit.

  Such a configuration enables efficient and effective learning in a study area where invisible fine substances such as pharmacy and medicine are learned.

  In addition, the substance learning support device according to the second aspect of the present invention accumulates the storage unit that stores the three-dimensional moving image of the invisible fine object and the three-dimensional moving image received by the receiving unit in the storage unit, as compared with the first invention. A storage unit, and the output unit is stored in the storage unit so as to be superimposed on the book shooting video received by the reception unit while the reception unit receives the book shooting video including the same invisible fine object. It is a substance learning support device that repeatedly outputs 3D moving images.

  Such a configuration enables efficient and effective learning in a study area where invisible fine substances such as pharmacy and medicine are learned.

  Further, in the substance learning support device according to the third aspect of the invention, in response to the transmission of the three-dimensional moving image specifying information, the receiving unit uses the three-dimensional moving image specifying information from the server device in response to the transmission of the three-dimensional moving image specifying information. Two or more three-dimensional moving images that are specified, and two or more three-dimensional moving images of invisible fine objects are sequentially received, and the output unit superimposes the book photographed video received by the receiving unit, and the receiving unit receives it. This is a substance learning support apparatus that sequentially outputs two or more three-dimensional moving images.

  With such a configuration, more effective learning can be performed in an academic area in which fine substances that are invisible such as pharmacy and medicine are learned.

  In addition, the substance learning support device according to the fourth aspect of the invention detects whether the invisible fine object included in the book photographed image received by the receiving unit has changed with respect to any one of the first to third aspects of the invention. The transmission unit further includes a book shooting video received by the reception unit when the detection unit detects a change in an invisible fine object included in the book shooting video received by the reception unit, or the reception unit. Is a substance learning support device that transmits three-dimensional moving image specifying information that is an object identifier for identifying an invisible fine object included in a book photographed video received by the server to a server device.

  With such a configuration, learning can be performed more efficiently in an academic field in which fine substances invisible such as pharmacy and medicine are learned.

  Further, the substance learning support device according to the fifth aspect of the present invention is a case where the book photographed video includes two or more invisible fine objects with respect to any one of the first to fourth aspects, and the three-dimensional moving image specifying information Are each an object identifier for identifying each of two or more invisible fine objects included in the book-captured video received by the reception unit or the book-captured video received by the reception unit, and the reception unit transmits the three-dimensional video specifying information In response, the server device receives a three-dimensional moving image of each of the two or more invisible fine objects, and the output unit superimposes the book photographed image received by the receiving unit and receives the two or more three-dimensional images received by the receiving unit. It is a substance learning support device that outputs moving images on the same screen.

  With such a configuration, it is possible to more efficiently and effectively learn in an academic area in which fine substances that are invisible such as pharmacy and medicine are learned.

  Further, in the substance learning support device of the sixth invention, in contrast to the fifth invention, the output unit is superimposed on the book photographed image received by the receiving unit, and each of the two or more three-dimensional images received by the receiving unit. It is a substance learning support device that outputs a moving image to a position where a corresponding invisible fine object is included.

  With this configuration, it is possible to more effectively and effectively learn in a study area where invisible fine substances such as pharmacy and medicine are learned.

  In addition, the substance learning support device according to the seventh aspect of the invention determines whether a three-dimensional moving image corresponding to the three-dimensional moving image specifying information exists in the storage unit with respect to any one of the second to sixth inventions. And a transmission unit, when the determination unit determines that the 3D video corresponding to the 3D video identification information is present in the storage unit, does not transmit the 3D video identification information to the server device, The output unit is a substance learning support device that outputs a three-dimensional moving image stored in the storage unit in a manner superimposed on the book photographing video received by the reception unit.

  With this configuration, it is possible to reduce the communication load and output a three-dimensional moving image at high speed, thereby enabling more efficient learning.

  The server device of the eighth invention is a server device that transmits one or more three-dimensional moving images to the substance learning support device, according to any one of the first to seventh inventions. A server storage unit storing one or two or more 3D moving images in association with each object identifier, a server receiving unit receiving 3D moving image specifying information, and 3D moving image specifying information received by the server receiving unit A server processing unit that acquires one or more three-dimensional moving images corresponding to the server, and a server transmission unit that transmits the one or more three-dimensional moving images acquired by the server processing unit to the substance learning support device Device.

  Such a configuration enables efficient and effective learning in a study area where invisible fine substances such as pharmacy and medicine are learned.

  According to the substance learning support system according to the present invention, it is possible to efficiently and effectively learn in a study area where a fine substance that is invisible such as pharmacy or medicine is learned.

Conceptual diagram of substance learning support system according to Embodiment 1 Block diagram of the same substance learning support system The flowchart explaining operation | movement of the same substance learning assistance apparatus 1 A flowchart for explaining the operation of the server device 2 Figure showing the 3D video management table The figure which shows the example of the book Diagram explaining the same shooting Figure showing the same output example Figure showing the same output example Diagram explaining the same shooting Figure showing the same output example Overview of the computer system Block diagram of the computer system

  Hereinafter, embodiments of a substance learning support system and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
In the present embodiment, a substance learning support system that enables efficient and effective learning in an academic area in which fine substances that are invisible such as pharmacy and medicine are learned will be described.

  Specifically, in the present embodiment, a substance that receives and outputs a three-dimensional moving image corresponding to a photographed object when the invisible minute object is photographed with respect to a book on which the invisible minute object is printed. A learning support system will be described.

  Further, in the present embodiment, a material learning support system will be described that, when a book on which an invisible fine object is printed is photographed, a three-dimensional moving image is repeatedly reproduced and the reproduction is terminated when it is detected that the photographing has been completed.

  In addition, in this embodiment, a substance learning support system capable of sequentially receiving and outputting two or more moving images for photographing one invisible fine object will be described.

  In this embodiment, when a change in an object to be imaged is detected during shooting, the server device is inquired and receives and outputs a three-dimensional video following the change in the object being shot. The system will be described.

  In the present embodiment, a substance learning support system that receives two or more three-dimensional moving images of the objects and outputs them on the same screen when two or more objects are photographed will be described.

  Further, in the present embodiment, a substance learning support system that outputs a position of an invisible fine object in a book and a three-dimensional moving image of two or more invisible fine objects will be described.

  In the present embodiment, a substance learning support system that determines whether or not a three-dimensional moving image has already been stored and stores the three-dimensional moving image when it is stored will be described.

  FIG. 1 is a conceptual diagram of a substance learning support system in the present embodiment. The substance learning support system includes one or more substance learning support apparatuses 1 and a server apparatus 2. The substance learning support device 1 is a terminal used by a user. The substance learning support device 1 is, for example, a so-called smartphone, tablet terminal, mobile phone, or the like. The user is usually a learner who learns a fine substance.

  FIG. 2 is a block diagram of the substance learning support system in the present embodiment.

  The substance learning support device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, a transmission unit 14, a reception unit 15, and an output unit 16.

    The processing unit 13 includes a detection unit 131, a determination unit 132, a storage and product unit 133.

  The server device 2 includes a server storage unit 21, a server reception unit 22, a server reception unit 23, a server processing unit 24, and a server transmission unit 25.

  The storage unit 11 constituting the substance learning support device 1 can store one or two or more three-dimensional moving images. The three-dimensional video stored in the storage unit 11 is a three-dimensional video of an invisible fine object. An invisible fine object is an object related to a fine substance that cannot be visually recognized. Examples of the fine substance include proteins, cells, DNA, and compounds. The invisible fine object is, for example, the structure of a substance such as protein, the structural formula of the substance, or the substance itself. The 3D moving image is not limited to CG, video, or the like. A three-dimensional video is usually a video that represents the structure and outline of a substance in three dimensions. The three-dimensional moving image is, for example, a moving image obtained by rotating a diagram showing the structure of a substance.

  The accepting unit 12 accepts a book photographing video that is a video obtained by photographing a book on which an invisible fine object is printed. The book here is usually a book that contains invisible fine objects such as pharmacy and medicine as learning targets.

  The accepting unit 12 usually accepts a book photographing video from a camera, a CCD, or the like. The book-captured video is usually a video captured by a user held by the substance learning support apparatus 1. The video is usually a moving image, but may be a still image. Note that the substance learning support device 1 preferably includes a photographing unit including a CCD or the like.

  The processing unit 13 performs various processes. The various processes are processes performed by the detection unit 131, the determination unit 132, and the storage unit 133, for example.

  The detecting unit 131 detects that the invisible fine object included in the book photographed image received by the receiving unit 12 has changed. When the invisible fine object is identified by a code, the detection unit 131 performs, for example, a code recognition process. The code recognition process is a process performed by a code reader such as a barcode reader, and is a process for acquiring an object identifier for identifying an invisible fine object. The code is, for example, a barcode or a two-dimensional barcode such as a QR code (registered trademark). However, the type of code does not matter.

Here, the change is, for example, the following case (1) or (2).
(1) A case where an invisible fine object included in a book photographed image is changed to a different invisible fine object. For example, when the invisible fine object included in the book photographed video changes from the invisible fine object A to the invisible fine object B.
(2) A case where a new invisible fine object is included in the invisible fine object included in the book photographed video. For example, when the invisible minute object included in the book photographed video changes from the invisible minute object A to the invisible minute objects A and B.

In the case of (3) below, the detection unit 131 does not need to detect.
(3) When the invisible fine object included in the book photographing video is not included. For example, when the invisible fine object included in the book photographing video changes from the invisible fine objects A and B to the invisible fine object A.

  The detection unit 131 may recognize an invisible fine object and acquire an object identifier. In such a case, for example, the storage unit 11 stores one or more sets of object identifiers and images of invisible fine objects. And the detection part 131 cuts out the area | region of an invisible fine object from a book photography image | video, and acquires an image. Then, the detection unit 131 calculates the degree of similarity between the image and the image of one or more invisible fine objects in the storage unit 11, and acquires an object identifier that is paired with the image of the invisible fine object having the highest degree of similarity. . For example, the detection unit 131 cuts out a frame area having a predetermined shape (for example, a rectangle) and acquires an image. Further, since the technique for calculating the similarity between two images is a known technique, detailed description thereof is omitted.

  The determining unit 132 determines whether or not a 3D moving image corresponding to the 3D moving image specifying information exists in the storage unit 11. The 3D moving image specifying information is information that can specify a 3D moving image. The three-dimensional moving image specifying information is, for example, a book shooting video received by the receiving unit 12. Also, the 3D moving image specifying information is an object identifier for identifying an invisible fine object included in the book photographed video received by the receiving unit 12, for example.

  The accumulation unit 133 accumulates the three-dimensional moving image received by the reception unit 15 in the storage unit 11. The accumulation unit 133 preferably accumulates the three-dimensional moving image in the storage unit 11 in pairs with the object identifier for identifying the invisible fine object. That is, it is preferable that the receiving unit 15 described later does not receive a three-dimensional moving image that has been received once.

  The transmission unit 14 transmits one or more three-dimensional moving image specifying information to the server device 2. As described above, the three-dimensional moving image specifying information is an object identifier that identifies a book-captured video received by the reception unit 12 or an invisible fine object included in the book-captured video received by the reception unit 12.

  Further, when the detection unit 131 detects that the invisible fine object included in the book photographed video received by the reception unit 12 has changed, the transmission unit 14 transmits the three-dimensional moving image specifying information to the server device 2. Is preferred. In addition, the case where it changes here is the above-mentioned "(1) When the invisible fine object contained in a book photography image changes into a different invisible fine object. (2) Among the invisible fine objects contained in a book photography image. It is preferable that “(3) When the invisible fine object included in the book-captured video is no longer included” is not included.

  In addition, when the determination unit 132 determines that the 3D moving image corresponding to the 3D moving image specifying information exists in the storage unit 11, it is preferable that the transmitting unit 14 does not transmit the 3D moving image specifying information to the server device 2. is there.

  The receiving unit 15 is a 3D moving image specified by the 3D moving image specifying information from the server device 2 in response to the transmission of the 3D moving image specifying information, and receives the 3D moving image of the invisible fine object.

  The receiving unit 15 receives two or more three-dimensional moving images of invisible fine objects, which are two or more three-dimensional moving images specified by the three-dimensional moving image specifying information from the server device 2 in response to transmission of the three-dimensional moving image specifying information. It is preferable to receive sequentially.

  It is preferable that the receiving unit 15 receives a three-dimensional moving image of each of two or more invisible fine objects from the server device 2 in response to transmission of the three-dimensional moving image specifying information.

  It is preferable that the receiving unit 15 does not receive a three-dimensional moving image that has been received once.

  The output unit 16 outputs the 3D moving image received by the receiving unit 15. It is preferable that the output unit 16 outputs the 3D moving image received by the receiving unit 15 by superimposing it on the book photographed image received by the receiving unit 12.

  The output unit 16 repeatedly outputs (reproduces) the same three-dimensional video while superimposing the book photographed video received by the accepting unit 12 while the accepting unit 12 accepts the book photographed video including the same invisible fine object. It is preferable to do. Note that the same 3D moving image to be played back is usually a 3D moving image stored in the storage unit 11. Here, when reception of the book photographing video by the reception unit 12 is stopped, it is preferable that the reception unit 15 stops reception of the three-dimensional video. In addition, when the reception of the book photographed video by the reception unit 12 is stopped, it is preferable that the output unit 16 stops the output of the three-dimensional video.

  It is preferable that the output unit 16 sequentially outputs two or more three-dimensional moving images received by the receiving unit 15 in a superimposed manner on the book photographing video received by the receiving unit 12.

  It is preferable that the output unit 16 outputs two or more three-dimensional moving images received by the receiving unit 15 on the same screen while being superimposed on the book photographed video received by the receiving unit 12. In such a case, the output unit 16 normally outputs two or more three-dimensional moving images to the same screen at the same time.

  The output unit 16 outputs the two or more three-dimensional moving images received by the receiving unit 15 to the position where the corresponding invisible fine object is included, superimposed on the book photographing video received by the receiving unit 12. The position including the invisible fine object is a position in the book being photographed.

  Here, “output” usually means outputting a three-dimensional moving image to a position where an invisible fine object is printed. In such a case, the invisible fine object in the book is usually hidden behind the three-dimensional moving image and cannot be seen.

  However, the output unit 16 may output both the invisible fine object and the three-dimensional moving image.

  The output is usually a display on a display, but projection using a projector, printing with a printer, sound output, transmission to an external device, accumulation in a recording medium, other processing devices and other It is a concept that includes delivery of processing results to a program or the like.

  The server storage unit 21 constituting the server device 2 stores one or more three-dimensional moving images in association with one or more object identifiers.

  The server reception unit 22 receives a set of an object identifier and a three-dimensional video. The input means such as the object identifier may be anything such as a keyboard, mouse, or menu screen. The server reception unit 22 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.

  The server receiving unit 23 receives one or more three-dimensional moving image specifying information from the substance learning support device 1.

  The server processing unit 24 acquires one or more 3D moving images corresponding to the 3D moving image specifying information received by the server receiving unit 23.

  The server processing unit 24 may acquire one or two or more three-dimensional moving images that are paired with the received object identifier from the server storage unit 21.

  Further, the server processing unit 24 acquires an object identifier for identifying an invisible fine object included in the book photographed video from the received book photographed video, and displays one or more three-dimensional moving images paired with the object identifier. It may be acquired from the server storage unit 21.

  The server processing unit 24 recognizes an invisible fine object included in the book-captured video, and displays one or more three-dimensional moving images paired with the invisible fine object most similar to the invisible fine object from the server storage unit 21. You may get it. In such a case, the server storage unit 21 stores one or more sets of invisible fine objects and three-dimensional moving images. The invisible fine object may be considered as an object identifier. Since the image recognition process and the process of calculating the similarity between two images (may be a process of determining whether or not they are similar) are well-known techniques, detailed description thereof is omitted.

  The server processing unit 24 accumulates the set of the object identifier and the three-dimensional moving image received by the server receiving unit 22 in the server storage unit 21.

  The server transmission unit 25 transmits one or more three-dimensional moving images acquired by the server processing unit 24 to the substance learning support device 1. The server transmission unit 25 may transmit a pair of the 3D moving image acquired by the server processing unit 24 and an object identifier to the substance learning support device 1.

  When the server processing unit 24 acquires two or more three-dimensional moving images, it is preferable that the server transmission unit 25 sequentially transmits two or more three-dimensional moving images.

  The storage unit 11 and the server storage unit 21 are preferably non-volatile recording media, but can also be realized by volatile recording media.

  The process in which information is stored in the storage unit 11 and the server storage unit 21 does not matter. For example, information may be stored in the storage unit 11 or the like via a recording medium, information transmitted via a communication line or the like may be stored in the storage unit 11 or the like, or The information input via the input device may be stored in the storage unit 11 or the like.

  The processing unit 13, the detection unit 131, the determination unit 132, the storage unit 133, and the server processing unit 24 can be usually realized by an MPU, a memory, or the like. The processing procedure of the processing unit 13 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The transmission unit 14 and the server transmission unit 25 are usually realized by wireless or wired communication means, but may be realized by broadcasting means.

  The receiving unit 15 and the server receiving unit 23 are usually realized by wireless or wired communication means, but may be realized by means for receiving a broadcast.

  The output unit 16 may be considered as including or not including an output device such as a display or a speaker. The output unit 16 can be realized by driver software for an output device or driver software for an output device and an output device.

  The server reception unit 22 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.

  Next, the operation of the substance learning support system will be described. First, operation | movement of the substance learning assistance apparatus 1 is demonstrated using the flowchart of FIG.

  (Step S301) The receiving unit 12 determines whether or not a book-captured video that is a video of a book on which an invisible fine object is printed has been received. If a book-captured video is accepted, the process goes to step S302. If a book-captured video is not accepted, the process returns to step S301.

  (Step S302) The processing unit 13 acquires one or more three-dimensional moving image specifying information using the book photographed video received in step S301. For example, the processing unit 13 reads one or more codes from a book photographed video and acquires one or more object identifiers. Here, the object identifier is three-dimensional moving image specifying information.

  (Step S303) The detection unit 131 determines whether or not the invisible fine object included in the book photographed video received by the reception unit 12 has changed. If it is determined that the change has occurred, the process goes to step S304. If it is determined that the change has not occurred, the process goes to step S306.

  (Step S304) The determination unit 132 determines whether or not a 3D moving image corresponding to one or more 3D moving image specifying information acquired in Step S302 exists in the storage unit 11. If it exists, go to step S309, otherwise go to step S305.

  (Step S305) The transmission unit 14 transmits one or more three-dimensional moving image specifying information acquired in Step S302 to the server device 2.

  (Step S306) The receiving unit 15 determines whether one or more three-dimensional moving images have been received from the server device 2 in response to the transmission of the three-dimensional moving image specifying information in Step S305. If a 3D video is received, the process goes to step S307, and if no 3D video is received, the process returns to step S306.

  (Step S307) The output unit 16 sequentially outputs the one or more three-dimensional moving images received in step S306. The output unit 16 may output two or more three-dimensional moving images simultaneously. Here, the output unit 16 normally outputs the received three-dimensional moving image superimposed on the book photographed video received by the receiving unit 12.

  (Step S308) The accumulation unit 133 accumulates one or more three-dimensional moving images received in step S306 in the storage unit 11. It is preferable that the storage unit 133 stores one or more three-dimensional moving images in association with object identifiers or the like. The process returns to step S301.

  (Step S309) The processing unit 13 reads one or more three-dimensional moving images corresponding to one or more three-dimensional moving image specifying information acquired in step S302 from the storage unit 11.

  (Step S310) The output unit 16 sequentially outputs the one or more three-dimensional moving images read in Step S309. The process returns to step S301.

  In the flowchart of FIG. 3, when the receiving unit 12 determines that the book shooting video has not been received (shooting has ended) while receiving the three-dimensional video, the receiving unit 15 receives the three-dimensional video. It is preferable to end the process.

  In the flowchart of FIG. 3, when the receiving unit 12 determines that the book shooting video is not received (shooting is finished) while outputting the three-dimensional video, the output unit 16 outputs the three-dimensional video. It is preferable to end the process.

  In the flowchart of FIG. 3, for example, among the two or more 3D moving image specifying information acquired in step S302, only the 3D moving images corresponding to some of the 3D moving image specifying information are not present in the storage unit 11. The transmission unit 14 transmits one or more three-dimensional moving image specifying information corresponding to the three-dimensional moving image that does not exist in the storage unit 11 to the server device 2. And the output part 16 outputs the three-dimensional moving image which the receiving part 15 received about the three-dimensional moving image which does not exist in the storage part 11. FIG. When a 3D video corresponding to a part of the 3D video specifying information exists in the storage unit 11, the processing unit 13 transmits a 3D video corresponding to the part of the 3D video specifying information from the storage unit 11. The reading and output unit 16 outputs the three-dimensional moving image.

  Further, in the flowchart of FIG. 3, the process is ended by power-off or a process end interrupt.

  Next, operation | movement of the server apparatus 2 is demonstrated using the flowchart of FIG.

  (Step S401) The server reception unit 23 determines whether or not one or more three-dimensional moving image specifying information has been received from the substance learning support device 1. When one or more three-dimensional moving image specifying information is received, the process goes to step S402, and when not received, the process goes to step S404.

  (Step S402) The server processing unit 24 acquires one or more three-dimensional moving images corresponding to the three-dimensional moving image specifying information received in step S401 from the server storage unit 21.

  (Step S403) The server transmission unit 25 transmits one or more three-dimensional moving images acquired in step S402 to the substance learning support device 1. The process returns to step S401.

  (Step S404) The server reception unit 22 determines whether a set of an object identifier and a three-dimensional moving image has been received. If a 3D moving image or the like is received, the process goes to step S405. If a 3D moving image or the like is not received, the process returns to step S401.

  (Step S405) The server processing unit 24 accumulates in the server storage unit 21 the set of the object identifier and the three-dimensional moving image received in Step S404. The process returns to step S401.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Hereinafter, a specific operation of the substance learning support system in the present embodiment will be described. A conceptual diagram of the substance learning support system is shown in FIG.

  Now, the server storage unit 21 of the server device 2 stores the three-dimensional moving image management table shown in FIG. The three-dimensional moving image management table stores one or more records having “ID”, “object identifier”, “three-dimensional moving image”, and “position information”. In the attribute of “three-dimensional moving image”, one or two or more three-dimensional moving images are stored in association with the object identifier. The attribute value of “position information” is information indicating a default position at which a 3D moving image is output. The default position is the default position within the page. The default position is information indicating a relative position from a code (for example, QR code), for example. However, the data structure of “position information” is not limited.

  In such a situation, it is assumed that the user is learning a pharmaceutical book. Then, it is assumed that the user opens the book page shown in FIG. Both pages of FIG. 6 are printed with invisible fine objects that are objects relating to fine substances that cannot be visually recognized. Also, a two-dimensional barcode in which an object identifier for identifying the invisible fine object is embedded is printed near the invisible fine object.

  The user holds the substance learning support device 1 having a camera function. Then, it is assumed that the user is shooting the upper side of the right page of the book as shown in FIG. 7 using the camera function.

  Next, the reception unit 12 receives a book-captured video that is a video of a book on which an invisible fine object is printed.

  Next, the processing unit 13 acquires 3D moving image specifying information using the accepted book photographed video. Here, the processing unit 13 recognizes the two-dimensional barcode of the book-captured video, and acquires the object identifier “obj003”. Here, the object identifier is three-dimensional moving image specifying information.

  Next, the detection unit 131 determines whether or not the invisible fine object included in the book photographing video received by the reception unit 12 has changed. More specifically, the detection unit 131 tries to acquire the object identifier stored in the storage unit 11 but cannot acquire it (acquires “NULL”). Then, the detection unit 131 compares the acquired object identifiers “obj003” and “NULL”, and determines that the invisible fine object included in the book photographed video has changed.

  Next, the determining unit 132 determines that there is no 3D moving image corresponding to the acquired 3D moving image specifying information in the storage unit 11.

  Next, the transmission unit 14 transmits the acquired 3D moving image specifying information “obj003” to the server device 2.

  Next, the server receiving unit 23 of the server device 2 receives the three-dimensional moving image specifying information “obj003” from the substance learning support device 1.

Next, the server processing unit 24 stores the three-dimensional moving image and position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )” paired with the three-dimensional moving image specifying information “obj003” in the three-dimensional moving image management table ( Obtained from FIG.

Next, the server transmission unit 25 uses the acquired one-dimensional moving image, the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )”, and the object identifier “obj003” to the substance learning support apparatus 1. Send to.

Next, the receiving unit 15 of the substance learning support device 1 receives from the server device 2 one 3D video, position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )”, and object identifier “obj003”. Receive.

Next, the processing unit 13 detects the position and size of the two-dimensional barcode. Then, the processing unit 13 uses the position and size of the two-dimensional barcode and the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )” to position and size of the window for outputting the three-dimensional video. To decide. Note that the processing unit 13 determines only the position of the window for outputting the three-dimensional video by using the position of the two-dimensional barcode and the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )”. May be. In such a case, the size of the window is usually fixed.

  Next, the output unit 16 outputs the received one three-dimensional moving image to the determined window.

Next, the storage unit 133 pairs the received one-dimensional moving image with the object identifier “obj003” and the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )” in the storage unit 11. accumulate.

  And while the user is photographing the same position of the page of a book with a camera, the process part 13 reads the three-dimensional moving image of the storage part 11, and the output part 16 outputs the said three-dimensional moving image repeatedly. An example of such output is shown in FIG. In FIG. 8, reference numeral 801 denotes an area for outputting a moving image.

  Next, it is assumed that the user has stopped shooting. Then, the reception unit 12 determines that no book-captured video is received.

  Then, the processing unit 13 determines that a 3D moving image is being output. The output unit 16 ends the output of the three-dimensional video.

  As shown in FIG. 9, it is preferable that the output unit 16 outputs the three-dimensional moving image to an area different from the area of the invisible fine object 901 so that the output section 16 can be compared with the invisible fine object 901. Since the determined window area is the area of the invisible fine object 901, the output unit 16 outputs a three-dimensional moving image to an area other than the determined window area.

  Next, for example, it is assumed that the user has taken the book page by further separating the camera from the book so that two two-dimensional barcodes are included (see FIG. 10). Then, the accepting unit 12 accepts a book photographing video that is a video obtained by photographing a book on which two or more invisible fine objects are printed.

  Next, it is assumed that the processing unit 13 has acquired two pieces of three-dimensional moving image specifying information “obj002” and “obj003” using the received book photographing video. Note that the processing unit 13 recognizes the two-dimensional barcodes 1001 and 1002 and acquires two pieces of three-dimensional moving image specifying information “obj002” and “obj003”.

  Next, the detection unit 131 determines that the invisible fine object included in the book photographed video received by the reception unit 12 has changed. Specifically, for example, the detection unit 131 reads the 3D moving image specifying information “obj003” in the storage unit 11. Then, the detection unit 131 compares the acquired 3D moving image specifying information “obj002” and “obj003” with the read 3D moving image specifying information “obj003”. Then, since the object identifier of the invisible fine object has increased, the detection unit 131 determines that the invisible fine object included in the book photographed video has changed.

  Next, the determination unit 132 determines whether a three-dimensional moving image corresponding to the acquired two pieces of three-dimensional moving image specifying information “obj002” and “obj003” exists in the storage unit 11. Here, the determination unit 132 determines that there is a 3D moving image corresponding to the 3D moving image specifying information “obj003”, but there is no 3D moving image corresponding to the 3D moving image specifying information “obj002”.

  Next, the transmission unit 14 transmits the 3D moving image specifying information “obj002” corresponding to the nonexistent 3D moving image to the server device 2.

  Next, the server receiving unit 23 of the server device 2 receives the three-dimensional moving image specifying information “obj002” from the substance learning support device 1.

Next, the server processing unit 24 converts the two three-dimensional moving images corresponding to the received three-dimensional moving image specifying information “obj002” and the position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )” into the third order. Obtained from the original moving image management table (FIG. 5).

Next, the server transmission unit 25 uses the acquired two-dimensional moving image, the positional information “(x ₃ , y ₃ ) (x ₄ , y ₄ )”, and the object identifier “obj002” to the substance learning support apparatus 1. Send to.

Next, the receiving unit 15 of the substance learning support device 1 receives two three-dimensional moving images, position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )”, and an object identifier “obj002” from the server device 2. Receive.

Then, the output unit 16 outputs the two 3D moving images corresponding to the 3D moving image specifying information “obj002” to the position specified by the position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )”. . The method for calculating the position specified by the position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )” is as described above. The output unit 16 sequentially outputs two three-dimensional moving images to the position specified by the position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )”. During shooting, the output unit 16 repeatedly outputs two three-dimensional moving images sequentially.

Next, the storage unit 133 stores the two three-dimensional moving images in the storage unit 11 in association with the object identifier “obj002” and the position information “(x ₃ , y ₃ ) (x ₄ , y ₄ )”. .

Next, the processing unit 13 reads the three-dimensional moving image corresponding to the object identifier “obj003” and the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )” from the storage unit 11.

Next, the output unit 16 outputs the read three-dimensional moving image to the position specified by the position information “(x ₅ , y ₅ ) (x ₆ , y ₆ )”.

  An example of such output is shown in FIG. In FIG. 11, two three-dimensional moving images 1101 and 1102 are simultaneously output.

  As described above, according to the present embodiment, it is possible to efficiently and effectively learn in a study area in which a fine substance that is invisible such as pharmacy or medicine is learned.

  Further, according to the present embodiment, it is determined whether or not a 3D moving image corresponding to the 3D moving image specifying information exists in the storage unit, and if it exists, the 3D moving image specifying information is transmitted to the server device. By not doing so, it is possible to reduce the communication load and output a three-dimensional moving image at high speed, thereby enabling more efficient learning.

  Note that the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. In addition, the software which implement | achieves the substance learning assistance apparatus 1 in this Embodiment is the following programs. In other words, the program receives a computer that accepts a book-captured video that is a video of a book printed with an invisible fine object that is an object related to a fine substance that cannot be visually recognized, and the reception unit A transmission unit that transmits 3D moving image specifying information, which is an object identifier for identifying an invisible fine object included in a book shooting image or a book shooting image received by the receiving unit, to the server device; and transmission of the 3D moving image specifying information In response to the book image received by the receiving unit and the receiving unit that receives the three-dimensional moving image of the invisible fine object that is a three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device. In order to function as an output unit that superimposes and outputs the 3D video received by the receiving unit Is a program.

  In the above program, the computer-readable recording medium includes a storage unit that stores a three-dimensional moving image of an invisible fine object, and a computer serving as a storage unit that stores the three-dimensional moving image received by the receiving unit in the storage unit. The output unit is stored in the storage unit so as to be superimposed on the book photographed video received by the accepting unit while the accepting unit accepts the book photographed video including the same invisible fine object. It is preferable that the program is for causing a computer to function as repeatedly outputting a three-dimensional moving image.

  In the above program, the receiving unit is two or more three-dimensional moving images specified by the three-dimensional moving image specifying information from the server device in response to transmission of the three-dimensional moving image specifying information, and the invisible minute Two or more 3D moving images of the object are sequentially received, and the output unit superimposes the book photographing video received by the receiving unit, and sequentially receives the two or more 3D moving images received by the receiving unit. A program for causing a computer to function as an output is suitable.

  Further, in the above program, there is provided a program for further functioning as a detection unit that detects a change in an invisible fine object included in a book photographed video received by the reception unit, wherein the transmission unit is the detection unit. However, when it is detected that the invisible fine object included in the book shooting video received by the reception unit has changed, the book shooting video received by the reception unit or included in the book shooting video received by the reception unit A program for causing a computer to function as transmitting three-dimensional moving image specifying information, which is an object identifier for identifying an invisible fine object, to a server device is preferable.

  In the above program, the book photographed video includes two or more invisible fine objects, and the three-dimensional moving image specifying information is the book photographed video received by the reception unit or the book received by the reception unit. Each object identifier for identifying two or more invisible fine objects included in a captured video, and the receiving unit receives each of the two or more invisible fine objects from the server device in response to transmission of the three-dimensional moving image specifying information. The computer receives a three-dimensional moving image of an object, and the output unit superimposes the book photographing video received by the receiving unit and outputs two or more three-dimensional moving images received by the receiving unit on the same screen. It is preferable that the program is for functioning.

  In the above program, the output unit superimposes the book photographed video received by the receiving unit on each of the two or more three-dimensional videos received by the receiving unit at a position where the corresponding invisible fine object is included. A program for causing a computer to function as an output is suitable.

  Further, in the above program, a program for further functioning as a determination unit that determines whether or not a three-dimensional video corresponding to the three-dimensional video specification information exists in the storage unit, wherein the transmission unit includes: If the determination unit determines that a 3D moving image corresponding to the 3D moving image specifying information exists in the storage unit, the 3D moving image specifying information is not transmitted to the server device, and the output unit receives the receiving unit Is a program for causing a computer to function as one that outputs the three-dimensional moving image stored in the storage unit by superimposing it on the book photographed video received.

  FIG. 12 shows the external appearance of a computer that executes the program described in this specification to realize the substance learning support apparatus 1 and the like according to the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 12 is an overview diagram of the computer system 300, and FIG. 13 is a block diagram of the system 300.

  In FIG. 12, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

  In FIG. 13, in addition to the CD-ROM drive 3012, the computer 301 includes an MPU 3013, a bus 3014 connected to the MPU 3013 and the CD-ROM drive 3012, a ROM 3015 for storing a program such as a bootup program, and an MPU 3013. And a RAM 3016 for temporarily storing instructions of the application program and providing a temporary storage space, and a hard disk 3017 for storing the application program, the system program, and data. Although not shown here, the computer 301 may further include a network card that provides connection to a LAN.

  A program that causes the computer system 300 to execute the functions of the substance learning support apparatus 1 according to the above-described embodiment is stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. good. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 at the time of execution. The program may be loaded directly from the CD-ROM 3101 or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 301 to execute the functions of the substance learning support apparatus 1 according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 300 operates is well known and will not be described in detail.

  In the above program, in the step of transmitting information, the step of receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware) Processing) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  In each of the above embodiments, it is needless to say that two or more communication means (terminal information transmission unit, terminal information reception unit, etc.) existing in one device may be physically realized by one medium. .

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the substance learning support system according to the present invention has an effect that it enables efficient and effective learning in a study area where invisible fine substances such as pharmacy and medicine are learned. It is useful as a material learning support system.

DESCRIPTION OF SYMBOLS 1 Substance learning support apparatus 2 Server apparatus 11 Storage part 12 Reception part 13 Processing part 14 Transmission part 15 Reception part 16 Output part 21 Server storage part 22 Server reception part 23 Server reception part 24 Server processing part 25 Server transmission part 131 Detection part 132 Judgment unit 133 Accumulation unit 133 Product unit

Claims (10)

A reception unit that accepts a book-captured video that is a video of a book on which an invisible micro-object that is an object related to a fine substance that cannot be visually recognized is printed;
A detection unit that detects that an invisible fine object included in the book photographing video received by the reception unit has changed;
When the detection unit detects that the invisible fine object included in the book photographed video received by the reception unit has changed, the three-dimensional moving image specifying information that is the book photographed video received by the reception unit, or the reception A transmitting unit that transmits to the server device 3D moving image specifying information that is an object identifier for identifying an invisible fine object included in the book-captured video received by the unit;
The three-dimensional moving image of the invisible fine object, which is a three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of the book photographed video or the three-dimensional moving image specifying information that is the object identifier. A receiving unit for receiving
An output unit that outputs the three-dimensional video received by the reception unit, superimposed on the book-captured video received by the reception unit ;
The change detected by the detection unit is that a new invisible fine object is included in the invisible fine object included in the book photographing video,
The receiver is
Do not receive the 3D video once received, receive the 3D video corresponding to the new invisible fine object detected by the detection unit,
The output unit is
A substance learning support device that outputs a three-dimensional moving image corresponding to the new invisible fine object received by the receiving unit. A storage unit for storing a three-dimensional video of an invisible fine object;
A reception unit that accepts a book-captured video that is a video of a book on which an invisible micro-object that is an object related to a fine substance that cannot be visually recognized is printed;
A detection unit that detects that an invisible fine object included in the book photographing video received by the reception unit has changed;
When the detection unit detects that the invisible fine object included in the book photographed video received by the reception unit has changed, the three-dimensional moving image specifying information that is the book photographed video received by the reception unit, or the reception A transmitting unit that transmits to the server device 3D moving image specifying information that is an object identifier for identifying an invisible fine object included in the book-captured video received by the unit;
The three-dimensional moving image of the invisible fine object, which is a three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of the book photographed video or the three-dimensional moving image specifying information that is the object identifier. A receiving unit for receiving
An output unit that outputs the three-dimensional video received by the reception unit, superimposed on the book-captured video received by the reception unit;
Comprising a storage unit for storing a three-dimensional videos received by the receiver in the storage unit,
The output unit is
While the accepting unit accepts the book-captured video including the same invisible fine object, it superimposes on the book-captured video accepted by the accepting unit, and repeatedly outputs the three-dimensional video stored in the storage unit , when the receiving unit is no longer accepting of the book shot video, to that substance learning support device stops the output of the three-dimensional video. A judgment unit for judging whether or not a three-dimensional video corresponding to the three-dimensional video specification information exists in the storage unit;
The transmitter is
If the determination unit determines that a 3D moving image corresponding to the 3D moving image specifying information exists in the storage unit, the 3D moving image specifying information is not transmitted to the server device,
The output unit is
The substance learning support device according to claim 2, wherein the three-dimensional moving image stored in the storage unit is output by being superimposed on a book photographed image received by the reception unit. The receiver is
In response to the transmission of the 3D moving image specifying information, two or more 3D moving images specified by the 3D moving image specifying information are sequentially transmitted from the server device, and two or more 3D moving images of the invisible fine object are sequentially Receive,
The output unit is
The substance learning support device according to any one of claims 1 to 3, wherein two or more three-dimensional moving images received by the receiving unit are sequentially output in a superimposed manner on the book photographing video received by the receiving unit. . A reception unit that accepts a book-captured video that is a video of a book on which two or more invisible micro-objects that are objects relating to fine substances that cannot be visually recognized are printed;
Two or more identifying three-dimensional moving image specifying information, which is a book photographed image including two or more invisible fine objects received by the reception unit, or two or more invisible fine objects included in the book photographed image received by the reception unit A transmitting unit that transmits the three-dimensional moving image specifying information that is the object identifier of the server device,
A three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of a book photographed video including the two or more invisible fine objects or three-dimensional moving image specifying information that is the two or more object identifiers. A receiving unit that receives a three-dimensional moving image of each of the two or more invisible fine objects;
A substance learning support apparatus comprising: an output unit that outputs two or more three-dimensional moving images received by the receiving unit, superimposed on the book photographing video received by the receiving unit. The output unit is
6. The substance learning support apparatus according to claim 5, wherein two or more three-dimensional moving images received by the receiving unit are output to a position where a corresponding invisible fine object is included, superimposed on the book photographing video received by the receiving unit. . A substance learning support method realized by a reception unit, a detection unit, a transmission unit, a reception unit, and an output unit,
A reception step of receiving a book-captured video that is a video of a book on which an invisible fine object that is an object related to a fine substance that cannot be visually recognized is printed;
A detection step in which the detection unit detects that an invisible fine object included in the book photographing video received in the reception step has changed;
When the transmitting unit detects that the invisible fine object included in the book photographed image accepted in the accepting step has changed in the detecting step, the three-dimensional image that is the book photographed image accepted in the accepting step. Transmitting step of transmitting to the server device moving image specifying information or 3D moving image specifying information that is an object identifier for identifying an invisible fine object included in the book-captured video received in the receiving step;
The receiving unit is a three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of the book photographed video or the three-dimensional moving image specifying information that is the object identifier, and the invisible minute A receiving step for receiving a three-dimensional video of the object;
The output unit includes an output step of outputting the 3D video received in the reception step in a manner superimposed on the book photographed video received in the reception step ;
The change detected in the detection step is that a new invisible fine object is included in the invisible fine object included in the book photographing video,
In the receiving step,
Do not receive the 3D video once received, receive the 3D video corresponding to the new invisible fine object detected in the detection step,
In the output step,
A substance learning support method for outputting a three-dimensional moving image corresponding to the new invisible fine object received in the receiving step . A substance learning support method realized by a reception unit, a transmission unit, a reception unit, and an output unit,
An accepting step for accepting a book-captured video that is an image of a book on which two or more invisible fine objects are printed, wherein the accepting unit is an object related to a fine substance that cannot be visually recognized;
The transmission unit is 3D moving image specifying information that is a book photographed image including two or more invisible fine objects accepted in the accepting step, or each of two or more invisible items included in the book photographed image accepted in the accepting step. A transmitting step of transmitting three-dimensional moving image specifying information, which is two or more object identifiers for identifying a fine object, to the server device;
In response to transmission of the book photographed video including the two or more invisible fine objects or the three-dimensional video specifying information that is the two or more object identifiers, the receiving unit is specified by the three-dimensional video specifying information from the server device A receiving step of receiving a three-dimensional moving image of each of the two or more invisible fine objects,
A substance learning support method comprising: an output step in which the output unit outputs two or more three-dimensional moving images received in the receiving step, superimposed on the book photographed video received in the receiving step. Computer
A reception unit that accepts a book-captured video that is a video of a book on which an invisible fine object that is an object related to a fine substance that cannot be visually recognized is printed;
A detection unit for detecting that an invisible fine object included in the book photographing video received by the reception unit has changed;
When the detection unit detects that the invisible fine object included in the book photographed video received by the reception unit has changed, the three-dimensional moving image specifying information that is the book photographed video received by the reception unit, or the reception A transmitting unit that transmits to the server device 3D moving image specifying information that is an object identifier for identifying an invisible fine object included in the book-captured video received by the unit;
The three-dimensional moving image of the invisible fine object, which is a three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of the book photographed video or the three-dimensional moving image specifying information that is the object identifier. A receiving unit for receiving
It is a program for superimposing on the book photographing video received by the reception unit to function as an output unit that outputs the three-dimensional video received by the reception unit ,
The change detected by the detection unit is that a new invisible fine object is included in the invisible fine object included in the book photographing video,
The receiver is
Do not receive the 3D video once received, receive the 3D video corresponding to the new invisible fine object detected by the detection unit,
The output unit is
A program for outputting a three-dimensional moving image corresponding to the new invisible fine object received by the receiving unit . Computer
A reception unit that accepts a book-captured video that is a video of a book on which two or more invisible micro-objects that are objects relating to fine substances that cannot be visually recognized are printed;
Two or more identifying three-dimensional moving image specifying information, which is a book-captured video including two or more invisible fine objects received by the reception unit, or two or more invisible fine objects included in the book-captured video received by the reception unit A transmitting unit that transmits the three-dimensional moving image specifying information that is the object identifier of the server device,
A three-dimensional moving image specified by the three-dimensional moving image specifying information from the server device in response to transmission of a book photographed video including the two or more invisible fine objects or three-dimensional moving image specifying information that is the two or more object identifiers. A receiving unit that receives a three-dimensional moving image of each of the two or more invisible fine objects;
A program for functioning as an output unit that outputs two or more three-dimensional moving images received by the receiving unit, superimposed on a book photographed video received by the receiving unit.