JP5838179B2 - Print preview program and electronic device - Google Patents

Description

  The present invention relates to a print preview program and an electronic apparatus that realizes an AR (Augmented Reality) technique for displaying on a display device a synthesized image obtained by synthesizing additional information with an image captured by an imaging device.

  Conventionally, printing in which the scheduled print image to be printed on the recording medium by the image forming apparatus is displayed on the recording medium in a pseudo manner before the scheduled printing image is actually printed on the recording medium. A preview is known (see Patent Document 1).

JP 2012-256180 A

  However, in the conventional print preview, the color of the recording medium displayed in a pseudo manner is, for example, a certain color such as white and different from the color of the actual recording medium. Since the appearance is different from the actual appearance of the recording medium, the preview image that displays the print target image printed on the recording medium in a pseudo manner and the printed material on which the print planned image is actually printed on the recording medium are completely different. There is a problem that it may be different.

  SUMMARY An advantage of some aspects of the invention is that it provides a print preview program and an electronic apparatus that can generate a preview image close to an actual printed matter.

  The print preview program of the present invention is a print preview program executed by an electronic apparatus including an imaging device and a display device, and acquires a print scheduled image to be printed on a recording medium by an image forming apparatus. When the recording medium is included in the image captured by the imaging device and the imaging device, a composite image obtained by combining the scheduled print image acquired by the planned print image acquisition unit on the recording medium in the captured image is obtained. The electronic device is caused to function as display control means for displaying on the display device.

  With this configuration, the print preview program of the present invention causes the display device to display a composite image obtained by synthesizing the print-scheduled image on the recording medium in the captured image by the imaging device on the display device. As a result, a preview image close to the actual printed matter can be generated.

  In the print preview program of the present invention, the display control unit recognizes that the recording medium is included in the captured image by recognizing the recording medium itself on the captured image by an image recognition technique. Also good.

  With this configuration, the print preview program of the present invention is scheduled to print on the recording medium in the photographic image even if a special marker for recognizing that the photographic image includes the recording medium is not formed on the recording medium. The composite image obtained by combining the images can be displayed on the display device as a preview image.

  The print preview program of the present invention causes the electronic device to function as an apparatus designation receiving unit that receives the designation of the image forming apparatus, and the display control unit is configured to receive the designation of the image forming apparatus. The appearance of the scheduled print image in the composite image may be changed according to the output characteristics.

  With this configuration, the print preview program of the present invention specifies the appearance of the scheduled print image in the composite image according to the output characteristics of the image forming apparatus by designating the image forming apparatus that prints the scheduled print image on the recording medium. Therefore, the preview image suitable for the image forming apparatus that prints the scheduled print image can be easily displayed on the display device.

  An electronic apparatus according to an aspect of the invention includes an imaging device, a display device, a printing scheduled image acquisition unit that acquires a scheduled printing image that is scheduled to be printed on a recording medium by the image forming apparatus, and an image captured by the imaging device. Display control means for causing the display device to display a synthesized image obtained by synthesizing the scheduled print image acquired by the planned print image acquiring means on the recording medium in the captured image. Features.

  With this configuration, the electronic apparatus of the present invention displays a composite image obtained by combining the scheduled print image on the recording medium in the image captured by the imaging device on the display device, so that the appearance of the actual recording medium is displayed as a preview image. As a result, a preview image close to the actual printed matter can be generated.

  The print preview program and the electronic device of the present invention can generate a preview image close to an actual printed matter.

1 is a block diagram of an image forming system according to an embodiment of the present invention. FIG. 2 is a block diagram of the MFP shown in FIG. 1. It is a block diagram of the portable apparatus shown in FIG. It is a flowchart of operation | movement of the portable apparatus shown in FIG. 3 in the case of displaying a preview image. It is a figure which shows an example of the top position of the recording medium in the picked-up image displayed on the display part shown in FIG. It is a figure which shows an example of the preview image displayed on the display part shown in FIG. FIG. 7 is a diagram illustrating an example of a preview image displayed on the display unit illustrated in FIG. 3, and is a diagram illustrating an example different from the example illustrated in FIG. 6. It is a figure which shows an example of the preview image displayed on the display part shown in FIG. 3, Comprising: It is a figure which shows an example different from the example shown in FIG. 6 and FIG. It is a figure which shows an example of the preview image displayed on the display part shown in FIG. 3, Comprising: It is a figure which shows an example different from the example shown in FIGS. It is a figure which shows an example of the preview image displayed on the display part shown in FIG. 3, Comprising: It is a figure which shows an example different from the example shown in FIGS.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  First, the configuration of the image forming system 10 according to the present embodiment will be described.

  FIG. 1 is a block diagram of an image forming system 10 according to the present embodiment.

  As shown in FIG. 1, an image forming system 10 includes a plurality of MFPs including an MFP (Multifunction Peripheral) 20, a management server 30 such as a PC (Personal Computer) that manages the plurality of MFPs, and a smartphone such as a smartphone. Mobile device 40. A plurality of MFPs of the image forming system 10, the management server 30, and the portable device 40 are communicably connected to each other via a network 11 such as a LAN (Local Area Network) or the Internet. The portable device 40 constitutes the electronic device of the present invention.

  The configuration of each of the plurality of MFPs in the image forming system 10 is the same as the configuration of the MFP 20. Therefore, in the following, the MFP 20 will be described on behalf of a plurality of MFPs of the image forming system 10.

  FIG. 2 is a block diagram of the MFP 20.

  As shown in FIG. 2, the MFP 20 is a display device such as an operation unit 21 that is an input device such as buttons for inputting various operations by the user and an LCD (Liquid Crystal Display) that displays various information. A display unit 22, a scanner 23 that is a reading device that reads an image from a document, a printer 24 that is a printing device that executes printing on a recording medium such as paper, an external facsimile apparatus (not shown), a public telephone line, and the like A fax communication unit 25 that is a fax device that performs fax communication via a communication line, a network communication unit 26 that is a network communication device that communicates with an external device via the network 11 (see FIG. 1), and various data. Stored EEPROM (Electrically rasable Programmable Read Only Memory), an HDD (Hard Disk Drive) storage unit 27 is a nonvolatile storage device such as, and a control unit 28 which controls the entire MFP 20.

  The operation unit 21 may include an input device that forms a touch panel together with the display unit 22.

  The control unit 28 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores programs and various data in advance, and a RAM (Random Access Memory) used as a work area of the CPU. ing. The CPU is configured to execute a program stored in the ROM or the storage unit 27.

  FIG. 3 is a block diagram of the mobile device 40.

  As shown in FIG. 3, the mobile device 40 includes an operation unit 41 that is an input device such as buttons for inputting various operations, a display unit 42 that is a display device such as an LCD that displays various information, An image capturing unit 43 that is an image capturing device that captures an image or a moving image, a network communication unit 44 that is a network communication device that communicates with an external device via the network 11 (see FIG. 1), a program, and various data A storage unit 45 that is a non-volatile storage device such as an HDD, and a control unit 46 that controls the entire portable device 40.

  The operation unit 41 may include an input device that forms a touch panel together with the display unit 42.

  The storage unit 45 stores a print preview program 45 a executed by the control unit 46.

  The control unit 46 includes, for example, a CPU, a ROM that stores programs and various data in advance, and a RAM that is used as a work area of the CPU. The CPU is configured to execute a program stored in the ROM or the storage unit 45.

  The control unit 46 executes a print preview program 45 a stored in the storage unit 45, thereby executing a print scheduled image acquisition unit 46 a that acquires a scheduled print image scheduled to be printed on a recording medium by the MFP, and the imaging unit 43. When the recording image is included in the captured image, the display control unit 46b causes the display unit 42 to display a composite image obtained by combining the scheduled print image acquired by the planned print image acquisition unit 46a on the recording medium in the captured image. , And functions as a device designation accepting unit 46c that accepts designation of an MFP for printing a scheduled print image.

  Next, the operation of the image forming system 10 will be described.

  First, the operation of the image forming system 10 when an image is read from a document and a print scheduled image is generated will be described.

  In response to an instruction via the operation unit 21, the control unit 28 of the MFP 20 reads an image from a document by the scanner 23 to generate a print scheduled image, and the generated print planned image is transmitted to the management server 30 via the network communication unit 26. Send to.

  Therefore, the management server 30 receives and stores the scheduled print image transmitted from the MFP 20.

  Next, the operation of the portable device 40 when displaying a preview image will be described.

  When the user of the portable device 40 desires to display a preview image on the portable device 40, the user activates the print preview program 45a to cause the control unit 46 of the portable device 40 to execute the operation shown in FIG. .

  FIG. 4 is a flowchart of the operation of the portable device 40 when displaying a preview image.

  As illustrated in FIG. 4, the scheduled print image acquisition unit 46 a of the mobile device 40 acquires a planned print image scheduled to be printed on a recording medium by the MFP via the network communication unit 44 (S <b> 101). Here, the scheduled print image acquisition unit 46a may acquire the scheduled print image generated from the document by the MFP and stored by the management server 30 from the management server 30, or from the document by the MFP. A print schedule image other than the print schedule image generated and stored by the management server 30 may be acquired. For example, the scheduled print image acquisition unit 46a may acquire a planned print image generated from a document by the MFP 20 and directly transmitted from the MFP 20 to the portable device 40, or may be generated by a PC (not shown) and the portable device. The scheduled print image transmitted to 40 may be acquired.

  Next, the device designation accepting unit 46c of the portable device 40 accepts designation of the MFP via the operation unit 41 by causing the display unit 42 to display an MFP designation screen for accepting designation of the MFP for printing the scheduled print image (S102). ).

  Next, the display control unit 46b of the portable device 40 causes the display unit 42 to display the image captured by the imaging unit 43 (S103).

  Next, the display control unit 46b determines whether or not a recording medium is included in the captured image by the imaging unit 43 until it is determined that the recording image is included in the captured image by the imaging unit 43 (S104). Here, the display control means 46b recognizes that the recording medium is included in the captured image by recognizing the recording medium itself on the captured image by the image recognition technique. The user of the mobile device 40 adjusts the position and direction of the mobile device 40 so that the captured image displayed on the display unit 42 is included in the captured image. Can be included.

  If the display control means 46b determines in S104 that the recording medium is included in the captured image by the imaging unit 43, it determines whether or not the top position of the recording medium in the captured image is designated (S105). The user of the portable device 40 can designate the top position of the recording medium on the captured image displayed on the display unit 42 by designating the top position via the operation unit 41.

  FIG. 5 is a diagram illustrating an example of the top position of the recording medium in the captured image displayed on the display unit 42.

  In FIG. 5, the captured image 50 displayed on the display unit 42 includes a recording medium 51. On the recording medium 51 on the photographed image 50, the top position 60 designated by the user is highlighted with a bold line.

  As shown in FIG. 4, when the display control means 46b determines in S105 that the top position is not designated, the display control means 46b returns to the process of S104 again.

  If the display control means 46b determines in S105 that the top position has been designated, the display control means 46b acquires the size of the recording medium on the photographed image with reference to the top position designated in S105 (S106). Specifically, the display control unit 46b acquires the vertical length of the recording medium in the captured image and the horizontal length of the recording medium in the captured image.

  Next, the display control unit 46b takes the captured image based on the size acquired in S106 and the magnification of the scheduled print image when the planned print image is printed on the recording medium (hereinafter referred to as “scheduled print magnification”). The size of the image scheduled to be printed is calculated (S107). For example, when the vertical length and the horizontal length of the recording medium in the captured image are 52 and 37, respectively, and the scheduled print magnification is 1, the display control unit 46b displays the print scheduled image in the captured image. When the vertical length and horizontal length are calculated as 52 and 37, respectively, and the planned printing magnification is double, the vertical length and horizontal length of the planned printing image in the captured image are 104 and 74, respectively. calculate. The initial value of the scheduled print magnification is 1, for example.

  Next, the display control unit 46b generates a preview print image (hereinafter referred to as “preview print schedule image”) in S107 based on the print schedule image acquired in S101 and the size calculated in S107. It generates with the calculated size (S108). Here, the display control unit 46b changes the appearance of the preview print scheduled image according to the output characteristics of the MFP received in S102. For example, when the MFP received in S102 is a monochrome-only machine, the display control unit 46b changes the preview print scheduled image to a monochrome image. Further, even when the MFP received in S102 is a color machine, the display control means 46b may change the preview print scheduled image according to the color reproducibility of the MFP. .

  Next, the display control unit 46b corresponds to the position of the origin of the print scheduled image with respect to the recording medium (hereinafter referred to as “print planned position”) when the print scheduled image is printed on the recording medium in the recording medium on the photographed image. A composite image is generated by synthesizing the preview print scheduled image generated in S108 with the captured image at the position (S109). Note that the initial value of the scheduled print position is, for example, the leftmost position of the upper end of the recording medium when the top position is the top.

  Next, the display control unit 46b displays the composite image generated in S109, that is, the preview image on the display unit 42 (S110). That is, the display control unit 46b displays a preview image as shown in FIG. 6 on the display unit 42 by the AR technique, for example.

  A preview image 70 shown in FIG. 6 is a composite image obtained by combining the photographed image 50 with the preview print scheduled image 71.

  As shown in FIG. 4, the display control means 46b determines whether or not the magnification of the preview print-scheduled image has been changed via the operation unit 41 after the process of S110 (S111). For example, the user of the mobile device 40 can change the magnification of the preview print scheduled image by directly operating the preview print planned image displayed on the display unit 42 via the operation unit 41. It is also possible to change the magnification of the preview print scheduled image by specifically designating a numerical value representing the magnification via the operation unit 41.

  When the display control unit 46b determines in S111 that the magnification of the preview print scheduled image has not been changed, whether or not the position of the preview print planned image with respect to the recording medium on the photographed image has been changed via the operation unit 41 is determined. Judgment is made (S112). For example, the user of the mobile device 40 can change the position of the preview print planned image by directly operating the preview print planned image displayed on the display unit 42 via the operation unit 41. It is also possible to change the position of the preview print scheduled image by specifically designating a numerical value representing the position via the operation unit 41.

  If the display control unit 46b determines in S112 that the position of the preview print scheduled image with respect to the recording medium on the captured image has not been changed, the display control unit 46b returns to the process of S104 again.

  If the display control means 46b determines in S111 that the magnification of the preview print-scheduled image has been changed, the display control unit 46b changes the print-scheduled magnification in accordance with this change (S113), and returns to the process of S104 again. That is, the display control unit 46b displays a preview image as shown in FIG. 7 on the display unit 42 by the AR technique, for example.

  In the preview image 70 shown in FIG. 7, the size of the preview print scheduled image 71 is ½ times that of the preview image 70 shown in FIG. That is, in the state shown in FIG. 7, the scheduled print magnification is set to 1/2.

  As shown in FIG. 4, when the display control unit 46b determines in S112 that the position of the preview print scheduled image with respect to the recording medium on the photographed image has been changed, the display control unit 46b changes the planned print position in accordance with this change (S114). The process returns to S104 again. That is, the display control unit 46b displays a preview image as shown in FIG. 8 on the display unit 42 by the AR technique, for example.

  The preview image 70 shown in FIG. 8 is different from the preview image 70 shown in FIG. 7 in the position of the preview print scheduled image 71 with respect to the recording medium 51 on the captured image 50.

  In the above description, the vertical and horizontal dimensions of the recording medium in the captured image are the same as the vertical and horizontal dimensions of the captured image. However, as shown in FIG. It may be different from the side.

  Further, even if the shape of the recording medium in the captured image has changed from the rectangular shape, the display control unit 46b may change the shape of the scheduled print image in the composite image in accordance with the change. For example, even if the shape of the recording medium in the photographed image changes to the shape shown in FIG. 10, the display control unit 46b shows the shape of the scheduled print image in the composite image in accordance with the shape of the recording medium in the photographed image. Can be changed. With this configuration, the portable device 40 changes not only the preview image from the front of the recording medium but also the preview images from various angles with respect to the recording medium by changing the shooting angle of the imaging unit 43 with respect to the recording medium. Can be confirmed.

  In the above description, processing in the case where the size of the scheduled print image is the same as the size of the recording medium on which the planned print image is printed has been described. Since the display control means 46b can acquire the size of the scheduled print image from the planned print image itself, the size of the print medium can be controlled even if the size of the print planned image is different from the size of the print medium. By specifically specifying via the unit 41, the size of the preview print scheduled image can be generated with an appropriate size based on the size of the print scheduled image and the size of the recording medium.

  Next, the operation of the image forming system 10 when printing a scheduled print image on the MFP will be described.

  When the control unit 46 of the portable device 40 is instructed to print on the recording medium in accordance with the preview image displayed on the display unit 42 via the operation unit 41, the print scheduled image acquired in S101 is printed. Along with the planned magnification and the planned print position, the data is transmitted to the MFP received in S102 via the network communication unit 44.

  Therefore, the control unit of the MFP uses the scheduled print magnification and the planned print position transmitted from the portable device 40 together with the scheduled print image to the print scheduled image transmitted from the portable device 40 by the printer. Print on.

  As described above, the portable device 40 causes the display unit 42 to display a composite image obtained by combining the print-scheduled image on the recording medium in the image captured by the imaging unit 43 as a preview image (S110). Can be reflected in the preview image, and as a result, a preview image close to the actual printed matter can be generated.

  The portable device 40 recognizes that the recording medium is included in the captured image by recognizing the recording medium itself on the captured image by the image recognition technique (S104), and thus recognizes that the recording image is included in the captured image. Even if a special marker for this purpose is not formed on the recording medium, a composite image obtained by combining the scheduled print image on the recording medium in the captured image can be displayed on the display unit 42 as a preview image.

  Note that the portable device 40 may be configured to recognize that a photographic image includes a recording medium by reading a special marker for recognizing that the photographic image includes the recording medium from the recording medium. good.

  Since the portable device 40 changes the appearance of the scheduled print image in the composite image in accordance with the output characteristics of the MFP by designating the MFP that prints the scheduled print image on the recording medium, the MFP that prints the planned print image Can be easily displayed on a display device.

  Note that the image forming system in the present embodiment includes a portable device as the electronic device of the present invention, but an electronic device other than the portable device such as a PC may be included as the electronic device of the present invention.

  The image forming system according to the present embodiment includes an MFP as the image forming apparatus of the present invention. However, an image forming apparatus other than the MFP, such as a printer dedicated machine, a copy dedicated machine, or a FAX dedicated machine, can be used. It may be provided as a forming device.

20 MFP (image forming apparatus)
40 Mobile devices (electronic devices)
42 Display unit (display device)
43 Imaging unit (imaging device)
45a Print preview program 46a Print scheduled image acquisition means 46b Display control means 46c Device designation acceptance means 50 Captured image 51 Recording medium 70 Preview image (composite image)
71 Preview print image (print image)

Claims (3)

A print preview program executed by an electronic apparatus including an imaging device and a display device,
A print-scheduled image acquisition means for acquiring a print-scheduled image scheduled to be printed on a recording medium by the image forming apparatus ;
When the recording medium is included in the captured image by the imaging device, a composite image obtained by combining the scheduled print image acquired by the planned print image acquisition unit on the recording medium in the captured image is displayed on the display device. Display control means for displaying , and
Causing the electronic device to function as a device designation accepting unit that accepts designation of the image forming device ;
The print control program , wherein the display control means changes the appearance of the scheduled print image in the composite image according to the output characteristics of the image forming apparatus received by the apparatus designation receiving means .   2. The display control unit according to claim 1, wherein the display control unit recognizes that the recording medium is included in the captured image by recognizing the recording medium itself on the captured image by an image recognition technique. Print preview program.   An imaging device;
  A display device;
  A print schedule image acquisition means for acquiring a print schedule image scheduled to be printed on a recording medium by the image forming apparatus;
  When the recording medium is included in the captured image by the imaging device, a composite image obtained by combining the scheduled print image acquired by the planned print image acquisition unit on the recording medium in the captured image is displayed on the display device. Display control means for displaying;
  An apparatus designation accepting unit for accepting designation of the image forming apparatus,
  The electronic apparatus according to claim 1, wherein the display control unit changes an appearance of the scheduled print image in the composite image in accordance with an output characteristic of the image forming apparatus received by the apparatus designation receiving unit.

Images