JP4309954B2 - Camera device and method with orientation detection - Google Patents

Abstract

A camera system comprising at least one area image sensor for imaging a scen e; a camera processor means for processing the imaged scene in accordance with a predetermined scene transformation requirement; and a printer for printing out the processed imaged scene on print media, utilizing printing ink stored in a single detachable module inside the camera system; the camera system comprising a portable hand held unit for the imagi ng of scenes by the area image sensor and printing the scenes directly out of the camera system via the printer.

Description

  The present invention relates to an image processing method and apparatus, and in particular, discloses a digital instant camera having an image processing function.

  The invention further relates to the field of digital camera technology, and in particular, discloses a digital camera with an integrated color printer.

  Conventional camera technology has relied on equipment for optical processing systems that have relied on the negative of the image projected on the photosensitive film for many years. This photosensitive film is subsequently "fixed" and subsequently chemically processed to allow the creation of a positive print that reproduces the original image. Although such image processing techniques have become standard, full color processing of images can be overly complex due to the need for expensive and difficult techniques. In recent years, digital cameras have become available. These cameras typically rely on the use of charge coupled devices (CCDs) to detect prominent images. The camera typically has a storage medium for storage of the detected scene, in addition to a connector for transferring the image to a computer device for subsequent image manipulation and printing.

  Such a device is inconvenient in that all images are stored by the camera and can only be printed after a while. In addition, the camera must have a sufficient storage capacity for storing a large number of images, and in addition, the user of the camera can use a computer system for downloading images or printing with a computer printer. Must have access to.

  In addition, Polaroid® type instant cameras that allow instant image generation have been used for some time. However, this type of camera is limited to producing only limited-size images, and there are many problems with the use of chemicals, especially photos made with these types of cameras. to degrade.

  When using such a device or another image capture device, it is desirable to appropriately capture audio and other environmental information when taking a picture.

  Furthermore, it is well known to generate stereoscopic vision in which an illusion of a three-dimensional surface is caused between a first image given to the left eye and a second image given to the right eye. However, the conventional system requires complicated preparation, and an image very close to the actual product (hi-fi image) is usually impossible. Furthermore, the general selection of images is limited to only specially prepared images.

  It is generally necessary to create a 3D image (Hi-Fi 3D image) that is very close to the actual product on demand, and in particular to create an image with a portable camera device that can shoot 3D images freely.

  In addition, if such a camera photo image creation system could create an automatically customized postcard with an image captured by the camera device on one side and a postage-paid stamp and address on the other side, Would be useful.

  In recent years, photo playback techniques for creating “panoramic” views of long images have become increasingly common. These images are created on photo paper or the like. Also, to create additional “panoramic” type views, the composition of the image is lengthened relative to the width. Unfortunately, problems arise when the photographic paper on which the image is formed is stored in a small diameter roll.

  In recent years, it has become fairly common to place filters to create an effect in the image that resembles a popular artistic picture style. These filters are designed to take an image and give the first image an artistic presentation about one of the artistic styles to obtain a second image. One of the most popular artists in modern times is Vincent van Gogh. One characteristic of the artist's work of art is that the direction of the brush stroke in the flat area of his painting actively follows the direction of the edges of the main objects in the painting. For example, his works entitled “Roads with cypresses and stars”, “Starry Night” and “Doctor Gouache” are examples that help explain this process. It would be desirable to provide a computer algorithm that automatically creates a “Fan Van Gogh” effect on any input image and outputs it from a portable camera system.

  Unfortunately, warping systems are commonly used on high-end computers, and it is generally inconvenient to scan and print out images into the computer. When acquiring images using a portable camera, etc., it is necessary to transfer the acquired images to a computer system for image processing later, and then process those images as required. Is particularly inconvenient.

  Furthermore, it is often considered that the effect of imitating various drawing methods has great value. Proceeding further, if these effects could be incorporated into one simple form, they would be appropriate to be combined with a portable camera device having digital shooting capability, thereby allowing the camera device to The desired filtered image could be generated for the scene being filmed.

  Unfortunately, digital image conversion technology and filter replacement technology, which is a demanding system requirement for cameras manufactured today, clearly cannot use technologies that are not yet available, and so far, creation has also been conceived. You can't use a filter that isn't.

  One very common method of camera technology is traditional negative film and positive print photography. In this case, a camera is used to capture the scene on the negative film, which is processed to fix the negative. A series of prints is then created from the negative film. In addition, a series of prints can be made immediately from the negative at any time. Unfortunately, digital cameras (including those proposed in this application) require that captured and printed photos be permanently stored in digital form if a later copy of the image is required. Let's go. It may generally be inconvenient that a “photo” copy simply requires an initial print. Of course, instead, copying can be done using a high quality color photographic copying machine. Unfortunately, such devices have limited copy capabilities, and signal degradation often occurs when such copy forms are performed. Obviously, a more suitable form for making a copy of a camera print is desired.

In addition, the color is limited due to the choice of media in the undercoat, and almost all artistic paintings about the landscape use a limited gamut. Such restrictions are often used by artists to create various artistic effects. Classic examples of this process include the following well-known works of art:
-Camille Pissarro "L'le Lacroix-Rouen, the effect of the fog" 1888, Philadelphia Museum of Art-Charles Engran "The River Seine, Morning"-1889, Petit Palais Collection, Geneva-Henri van de Verde “Crepuscule”-1892, Klerer-Müller National Museum, Otterlo-Giorgio Sula “The Sands of Bas-Butane, Honfleur” 1886—The Trune Museum

  It is desirable to create an output image having effects and characteristics similar to those in the above list from an arbitrary input image.

  Many creative judgments are made when some kind of clothing is created. First is the shape and style of the clothing, plus the color and type of the fabric. For the time being, fashion designers try many different alternatives to draw the final fashion product before completing the garment.

  Such a process is generally unsatisfactory for rapid and flexible changes in clothing and the opportunity to quickly determine the final appearance of a fashion product.

  Binoculars and telescopes are well known. In particular, taking binoculars as an example, the device provides a telephoto extension of the landscape to enhance the user's visual ability. In addition, devices such as night binoculars are also used to enhance the visual acuity of the user. Unfortunately, these systems rely on real-time, analog optics, and it is difficult to permanently record the visible scene. One method that may be suitable for recording a permanent copy of a scene is to attach a sensor device, such as a CCD, that captures the scene and store it in a storage device for subsequent printing. Unfortunately, such a structure is particularly disturbing when it is desired to use binoculars outdoors and in a portable state.

  Many forms of storage of information in a compressed state are well known. For example, in the field of computer devices, the use of a fixed or portable magnetic disk device is common. For disks that can be carried, "floppy disks" and "zip disks" and other methods of portable magnetic storage media are widely accepted in the market. Another form of portable storage is a compact disc “CD” that utilizes a series of elongated pits (placed along a spiral track and read by a laser beam device). The use of a CD provides a very low cost form of storage. However, the required technology is quite complex and the use of rewritable CD type devices is very limited.

  Another type of storage is a magnetic card, often used for credit cards and the like. These cards typically have a magnetic strip on the back to record information related to the card user. In recent years, the convenience of magnetic cards has increased in the form of smart card technology in which integrated circuit type devices are incorporated into cards. Unfortunately, the cost of such devices is often high and the complexity of the technology used is also significant.

  Conventional silver halide camera processing systems have resulted in the use of the well-known “negative” for the production of multiple prints. The negative is typically the source of print production, and its practice has grown for care and protection for continuous use.

  In an encoding system that is detected by a fault tolerant method, there is an important concern as to how encoding that enables efficient decoding is possible. Therefore, it is desired to provide an efficient encoding system.

The present invention relates to the provision of an alternative form of a camera system comprising a digital camera with an integrated color printer. In addition, the camera includes hardware and software to increase the apparent resolution of the image sensing system and convert the image to various “artistic styles” and graphic enhancements.

  According to a further aspect of the present invention, at least one area image sensor for imaging a scene, a camera processor means for processing the imaged scene according to a predetermined scene conversion request, and the processed There is provided a camera system comprising a printer for printing a scene of a printed image on a print medium, and a print medium and printing ink stored in one removable module inside the camera system. The camera system includes a handheld unit for capturing an image of the scene by the area image sensor and outputting a print of the scene by the printer directly from the camera system.

  The camera system preferably includes a print roll for storing print media and printing ink used by the printer. The print roll is removable from the camera system. In addition, the print roll includes an authentication chip that includes authentication information so that authentication is performed when the print roll is inserted into the camera system, and the camera processing device interrogates the authentication chip. Is adapted to be.

  Furthermore, the printer includes a drop-on-demand ink printer and a cutting device for separating printed photographs.

  According to a first aspect of the present invention, at least one area image sensor for capturing a scene, a camera processing device for processing the captured scene in accordance with a predetermined scene conversion request, and a scene of the processed image There is provided a camera system having a printer for printing the image on a print medium, and a print medium and printing ink housed in one removable module inside the camera system. The camera system includes a handheld unit for capturing an image of the scene by the area image sensor and outputting a print of the scene by the printer directly from the camera system.

  Preferably, the camera system includes a print roll for storing print media and printing ink used by the printer. The print roll can be removed from the camera system. In addition, the print roll includes an authentication chip that includes authentication information so that authentication is performed when the print roll is inserted into the camera system, and the camera processing device interrogates the authentication chip. Is adapted to be.

  Furthermore, the printer includes a drop-on-demand ink printer and a cutting device for separating printed photographs.

  In accordance with a further aspect of the invention, a user interface for operating the device is provided. The user interface has a card that is inserted into the device and the surface of the card contains a visual description of the effect the card has on the output of the device.

According to a further aspect of the invention,
At least one area image sensor for imaging a scene;
A camera processing device for processing the imaged scene according to a predetermined scene conversion request;
A printer for printing the processed image scene on a print medium;
A removable module for housing the print media and printing ink for the printer;
A camera system is provided. The camera system includes a handheld unit for capturing an image of the scene by the area image sensor and outputting a print of the scene by the printer directly from the camera system.

According to a further aspect of the invention,
A sensor means for detecting images;
A processing device for processing the detected image in accordance with a predetermined processing request;
An audio recording device for matching the audio signal with the detected image, if any;
A printing device for printing the detected image processed in the first area of the print medium supplied by the camera system and printing the encoded version of the audio signal in the second area of the print medium;
A camera system is provided. Preferably, the detected image is printed on the first side of the print media and the encoded version of the audio signal is printed on the second side of the print media.

In accordance with the present invention,
An area image sensor device for detecting images;
An image storage device for storing the detected image;
A localization device that detects the orientation of the camera during image detection;
A processing device for processing the detected image using a detected camera orientation;
A camera system is provided.

According to a further aspect of the invention,
Capture the image using focusing technology;
Use the focus setting as an indicator of the position of the composition in the image;
Processing the image using the focus setting to generate an effect specific to the focus setting;
Printing the image;
A digital image processing method is provided.

  Preferably, the image is captured using a zoom technique, and the zoom setting is used in a heuristic way to process portions of the image.

  According to a further aspect of the present invention, a method for processing an image taken with a digital camera with an eye position sensing device is provided. The method includes using the eye position information in the sensed image to process the image with a spatially varying sense (which depends on the eye position information).

  The step includes using eye position information to locate the area of interest in the sensed image. The process can include placement of speech balloons on the image and application of specific distortion to the image. Instead, the process also includes applying a brush stroke filter to the eye position information area for a finer image. Ideally, the camera can print the result of the process almost immediately.

  According to a further aspect of the present invention, there is provided a method for processing an image taken with a digital camera having an automatic exposure setting, the processing method comprising using said information to process a sensed image.

  The information using step described above uses the automatic exposure setting to determine remapping of the colors in the image to generate a corrected image in which the colors in the image are converted to the calculation of the automatic exposure setting. Including doing. The process is such that when the exposure setting shows a dark state, the color of the remapped image is deeper and more vivid, and when the exposure setting shows a bright state, the color of the remapped image Make it brighter and more saturated.

  Recently, digital cameras have become increasingly popular. These cameras typically acquire desired images using a charge-coupled device (CCD) array and save the captured scene to an electronic storage medium (for subsequent image processing and printing). , For download to be performed later on the computer system). Typically, when using a computer system to print an image, sophisticated software is used to process the image according to requirements.

  Unfortunately, such systems require significant post-processing of the captured image, usually presenting the image in the direction it was taken and performing the necessary or required correction of the captured image. Therefore, it depends on the post-processing process. In addition, a lot of environmental information is lost when the picture is taken.

In accordance with a further aspect of the present invention, there is provided a method of processing an image captured using a digital camera and flash, the method comprising the following steps.
(a) Discovery (identification) of distortion of the captured image resulting from the use of the flash
(b) Modification of the image to locally reduce the effect of the distortion

In accordance with a second aspect of the present invention, a digital camera that reduces the effects of flash distortion in captured images, comprising:
(a) a digital image capture device that captures images for image capture;
(b) a distortion position determining device for finding a position in the captured image of a flash-induced color distortion;
(c) an image distortion correction device connected to the distortion position determination device and the digital image capture device and adapted to process the captured image to reduce the effects of the distortion;
(d) a display device connected to the image distortion correction device for display;
A digital camera is provided.

In accordance with a further aspect of the present invention, there is provided a method for creating a stereographic image, the method comprising:
(a) using a camera device to image the scene as a stereoscopic image;
(b) The lens system prints a stereoscopic image such as an integrally formed image at a predetermined position on the first surface portion of the transparent print medium formed on the second surface, and the printed three-dimensional image Make the left and right eyes of the viewer of the captured image see the scene as a stereoscopic image;
Including that.

  In accordance with further aspects of the present invention, print media and ink supply devices are provided. The supply device is adapted to supply ink and the print media for ink deposition to the printing device. The supply device includes a roll-shaped medium wound on a media former. The supply device is integrally formed in the medium and the ink supply device, and is adapted to be connected to the printing device for supplying ink and print media to the printing device. Having at least one ink reservoir;

  According to a further aspect of the present invention, there is provided a print roll for use in a camera imaging system, wherein a large number of formatted postcard information is printed on the back surface at predetermined intervals.

  In accordance with a second aspect of the present invention, a method for creating a customized postcard is provided. The production method includes the following steps. That is, the use of a camera device comprising a print roll with a back side having a number of formatted postcard information areas in place, taking a customized image on the corresponding image surface of the print roll, and the camera device Use of the print roll to create a postcard.

  According to a third aspect of the present invention, a method is provided for sending a postcard with a camera image through a postal system, the method comprising the following steps. That is, selling a print roll that includes prepaid postage, using the print roll in a camera system to create a postcard, and sending the prepaid postcard by mail.

  The present invention is further directed to a camera system with an integrated printer device. The print media is removable from the camera system and includes a device for storing important information about the print media.

  The present invention is further directed to a camera system with an integrated printer device. The print media is removable from the camera system and provides authenticated access to the print media.

  In accordance with a further aspect of the present invention, a flat print media is provided that is less curled in use. The print medium has anisotropic rigidity in the direction of the plane. In accordance with a second aspect of the present invention, a method is provided for reducing curl in an image printed on a flat print media having anisotropic stiffness. The method applies local pressure to the print media portion.

  The present invention is further directed to a camera system having an integrated printer device. The camera system has an indicator of the number of images remaining in the printer. The indicator can display the number of prints remaining in many different modes.

  In accordance with a further aspect of the invention, an automatic image processing method is provided. The processing method includes locating features with high spatial variation in the image and drawing a series of brush strokes radiating from those regions with high spatial variation in the image. Preferably, the brush stroke is small in size near what is important to characterize. In addition, the position of the predetermined part of the brush stroke is under jittering.

In accordance with a further aspect of the invention, a warping method is provided that distorts an input image. The method
A warp map for an arbitrary output image having a predetermined dimension A × B, wherein each element of the warp map represents a corresponding region in the theoretical input image corresponding to the coordinate position of the element. Inputting a warp map that maps to pixel locations in the output image;
Reducing the warp map to warp image dimensions to form a reduced warp map;
For each element of the reduced warp map, calculating a contribution region in the input image from the value of the element and the value of an adjacent element;
Determining the color of the output image for the pixels of the warped image corresponding to the element from the contribution region;
including.

In accordance with a further aspect of the invention, a method is provided for increasing the resiliency of data stored on media for reading by a sensor device. The method
(a) modulating data stored in a recoverable manner with a modulated signal having a high frequency component;
(b) storing the data in a modulated format on the medium;
(c) detecting the modulated and stored data by the sensor device;
(d) neutralizing the modulation of the modulated and stored data in order to detect the separation of the position of the modulated and stored data in the medium;
(e) retrieving the unmodulated portion of the stored data from the modulated one of the stored data;
including.

  The preferred embodiment of the present invention will be described with reference to a card reading system for reading data via a CCD type device, which is provided in a camera system for later decoding. . Furthermore, the description of the preferred embodiment is highly dependent on the field of error control systems. Therefore, those who are guided by this specification should become experts in error control systems. In particular, "Error control system for digital communication and storage" Stefan. B. You should be well versed in standard texts and other standard texts, such as Wicker, Pretis Hall Publishing, 1995, especially the Reed-Solomon Code consideration.

An object of the present invention is to provide a method for converting a scanned image containing scanned pixels into a corresponding bitmap image, which for each bit in the bitmap image: It has the following steps. That is,
a. Determining the expected position in the scanned image at the current bit from the position of surrounding bits in the scanned image;
b. Determining an appropriate value for the bit from the value at the expected corresponding pixel location in the scanned image;
c. Determining the size of the center of gravity of the center of expected intensity at the expected location;
d. Determining the size of the center of gravity for pixels close to the periphery of the current bit in the scanned image;
e. If the dimension of the center of gravity extends in relation to the expected position, adjusting the expected position to a pixel having an extended center of gravity dimension;
And have.

According to a further aspect of the invention,
A digital camera device capable of detecting images;
An image processing data input card that is inserted into the digital camera device to process the image and is adapted to give the digital camera device an image processing instruction (an instruction to add text to the image);
A text input device having a series of non-Roman font characters used by the digital camera device in association with the image processing instructions to create new text characters to be added to the image, the image input device A text input device connected to the digital camera device for inputting the character to be added;
An apparatus for text editing, comprising:

  Preferably, font characters are sent to the digital camera device when requested by the device in accordance with the image processing instructions to create the new text character. The image processing data input card includes a set of Roman font characters and non-Roman characters including at least one of Hebrew, Cyrillic, Arabic, Chinese characters, and Chinese characters.

According to a further aspect of the invention,
A digital camera device capable of detecting images;
An image processing data input card that is inserted into the digital camera device to process the image and is adapted to give the digital camera device an image processing instruction (an instruction to add text to the image);
A text input device having a series of non-Roman font characters used by the digital camera device in association with the image processing instructions to create new text characters to be added to the image, the image input device A text input device connected to the digital camera device for inputting the character to be added;
An apparatus for text editing, comprising:

  The object of the present invention is to take advantage of the updated technology in addition to the advantages of the new filter created and of the flexible method for processing the digital images to be printed out.

According to a further aspect of the present invention, there is provided an image copying device for reproduction of an input image consisting of a series of ink dots, the device comprising:
(a) an imaging array device that captures the input image at a sampling rate higher than the frequency of the dots to create a corresponding sample image;
(b) a processing device for processing the image to determine the position of the printed dots in the sample image;
(c) a printing apparatus that prints ink dots at positions on a print medium corresponding to the positions of the print dots;
including.

2. The image copying apparatus according to claim 1, wherein the copying apparatus prints a full color copy of the input image.

In accordance with a further aspect of the present invention, a camera system for outputting a blur corrected image is provided. The camera system
An image sensor for detecting images;
Speed detection means for detecting any movement of the image associated with the external environment and generating a speed output indicative of the movement;
Processor means connected to the image sensors and speed detection means and adapted to perform blur correction of the image and output a blur correction image;
It has.

  Preferably, the camera system is a portable handheld unit that is connected to a printing apparatus for outputting the blur correction image immediately. The speed detection means may include an accelerometer such as a micro electromechanical (MEMS) device.

In accordance with a further aspect of the present invention, a photosensor reading article is provided, the article comprising:
(a) a series of light emitter housings for insertion of light emitting elements;
(b) a light emitting element focusing device for focusing the light emitted from the series of light emitting elements on the surface of the object to be imaged;
(c) a photo sensor housing for inserting a series of photo sensor rows;
(d) a focusing device that focuses reflected light from an object to be imaged on a characteristic part of the CCD array;
Is provided.

  In accordance with an aspect of the present invention, a printer device connected to a computer system is provided. The printer apparatus includes a print head unit including an ink jet print head for printing an image on a print medium, and further includes a cavity for inserting a print roll unit which is a consumable. The print roll unit includes print media and ink that are consumables inserted into the cavity, and the ink is used by the inkjet print head to print an image on the print media.

According to a further aspect of the invention,
A sensing means for detecting images;
A correction device for correcting the detected image in accordance with a correction instruction input to the camera;
An output device for outputting the modified image;
A digital camera system is provided, wherein the correction device is a series of processing elements (PEs) arranged around a central crossbar switch. Preferably, the PE has an arithmetic logic unit (ALU) that operates under the control of a microcode store having a writable control store. The PE can include an internal input / output FIFO for storing pixel data used by the element, and the correction device can read and write image pixel data to and from the device. It is connected to the.

  Each PE can be arranged in a ring, and each PE is connected alone to the nearest neighboring PE. The ALU receives a series of inputs connected to a series of central processing units within the ALU via an internal crossbar switch and has a number of internal registers for temporary data storage. The central processing unit has at least one of a multiplier, an adder, and a barrel shifter.

  The PE is further connected to a common data bus for transferring pixel data to the PE. The data bus is connected to a data cache, and the data cache functions as an intermediate cache between the PE and a memory store for storing images.

According to a further aspect of the present invention, there is provided a method for decoding detected image data stored on a card at a high pitch rate in real time in a short time, the method comprising:
Detecting an initial position of the image data;
Decoding the image data to determine a corresponding bit pattern of the image data;
including.

According to a further aspect of the present invention, there is provided a method for quickly decoding detected image data stored at a high pitch rate on a card and rotated, warped or marked by a fault tolerant method, the method comprising:
Determining an initial position of the start of the image data;
Detecting the image data at a sampling rate higher than the pitch rate;
Process the detected image data in the column by column processing, keep the expected position of the center (centroid) of each dot in the next column, and update each column to update the expected next center of gravity position. Performing a fine adjustment of the center of gravity when processing;
Have

According to a further aspect of the present invention, there is provided a method for accurately detecting dot values of sensed image data consisting of a dot array and sampled at a rate higher than the pitch frequency of the dot array to form a pixel array, The method
Determining an expected central pixel in the pixel array corresponding to an expected centroid position;
Using the detection value of the central pixel and the detection values of many adjacent pixels as an index of a look-up table that outputs a dot value centered on the pixel position;
Consists of.

According to a further aspect of the present invention, there is provided a method for accurately determining the position of dots in detected image data stored at a high pitch rate on a card and premised on the effects of rotation, warping and marking, the method comprising:
Processing the image data in a column in a column format;
Recording a dot pattern of the processed column of pixels;
Setting the expected dot pattern from the recorded dot pattern of the processed pixels to the latest column position;
Comparing the expected dot pattern with the actual dot pattern of the sensed image data at the latest column position;
If the comparison is within a predetermined error, the latest dot position is returned to the latest column position to adapt to the expected dot pattern to generate a new actual dot position. Using column positions;
Using the actual dot position at the latest column position when determining the dot position of the next column;
including.

In accordance with a further aspect of the present invention, there is provided a method of combining an image bump map to mimic the effect of a painting on an image, the method comprising:
Define an image canvas bump map that gives rise to a surface to be painted,
Define the paint bump map of the object drawn on the surface,
Combining the image canvas bump map and the paint bump map to create a final combined bump map using a stiffness factor that determines the degree of modification of the paint bump map by the image canvas bump map. ,
including.

According to a further aspect of the present invention, there is provided a method for automatically processing an input image to produce an artistic effect, the method comprising:
Determine the mapping of the input gamut to a given output gamut to create the desired artistic effect,
Using a mapping that maps the input image to an output image having a predetermined output gamut;
Including that.

  Preferably, the method further comprises the step of post-processing the output image using a brush stroke filter.

  Further preferably, the output gamut is formed by mapping a predetermined number of input gamut values to corresponding output color gamut values and inserting a maintenance of the mapping from input gamut values to output color gamut values. . The insertion process includes a weighted sum of a predetermined number of mappings of input gamut values to output color gamut values.

According to a second aspect of the present invention, there is provided a method for compressing an input color gamut to enter an output color gamut, the method comprising:
Determining a zero chrominance value with the current input color strength;
Determining a source distance that is a distance from the zero chrominance value to an edge of the input color gamut;
Determining a suitable edge of the output color gamut;
Determining a target distance that is a distance from the zero chrominance value to an edge of the output color gamut;
Measuring the current input color strength by a factor derived from the ratio of the source distance to the target distance;
Is provided.

  Preferably, the current input color strength is measured by a factor based on the distance of the current input color from the zero chrominance value point.

In accordance with a further aspect of the present invention, a portable camera for the output of an image detected by a camera is provided, the camera comprising:
A sensing means for detecting the image;
A tile device that imparts a tile effect to the detected image to create a tiled image;
A display device for displaying the tiled image;
It has.

In accordance with a further aspect of the present invention, a portable camera for the output of an image detected by a camera is provided, the camera comprising:
A sensing means for detecting the image;
A texture mapping device for applying a texture effect to the detected image to create an image with a texture added;
A display device for displaying an image to which the texture is added;
Is provided.

According to a further aspect of the present invention, a portable camera is provided for outputting an image detected by a camera, the camera comprising:
Sensor means for detecting images;
An illumination device for providing an illumination effect to the detected image so as to create an illumination image that mimics the effect of a light source that emits light in the detected image;
A display device for displaying the illumination image;
It has.

According to a further aspect of the invention,
A series of input token signals input to the camera for processing of the sensed image for output to a display device depicting fabric apparel having the sensed image characteristics;
A camera device adapted to read the input token signal, detect the image, and process the image according to the read input token signal to form the output image;
A display device adapted to display the output image;
A clothing creation system is provided.

  According to a further aspect of the present invention, there is provided a clothing creation system that includes an image sensor device and an image display device, maps a portion of an arbitrary image detected by the image sensor device to clothing, and displays the image on the image display device. Provided.

  In accordance with a further aspect of the invention, a method is provided for forming a processed image, the method comprising connecting a series of camera manipulation units, each camera processing unit being processed. In order to form an output image, the input image is subjected to image processing. The first one of the camera processing units senses an image from the surroundings, and at least the last one of the camera processing units forms an output image that does not fade.

According to a further aspect of the present invention, there is provided a portable imaging device for viewing a subject at a distant location, the imaging device comprising:
An optical lens system to magnify a subject that is visible far away;
A detection system for simultaneously detecting the subject;
A processing device connected to the detection system for processing the detected image and sending it to a printer device;
A printer device connected to the processing device for printing an image detected by the portable imaging device on a print medium upon request;
It has.

  Preferably, the system further comprises a removable print media supply device arranged in a removable module connected to the printer device for supplying roll-shaped print media and ink to the printer device. Have

  The printer apparatus may include an inkjet printing apparatus that provides a full-color printer for outputting a detected image.

  Furthermore, the preferred embodiment is implemented as a binocular system with a beam splitter device for projecting a distant subject onto the detection system.

In accordance with a further aspect of the present invention, a system is provided for playing recorded audio that has been encoded by a fault tolerant method as a series of dots printed on a card. The system
An optical scanner device for the visual scanning of recorded audio;
A processing device connected to the optical scanner device to decode the scanned audio code to generate an audio signal;
An audio emitter device connected to the processing device to reproduce the audio signal on demand;
Is provided.

  The encoding may include Reed-Solomon encoding of recorded speech and may include a series of ink dots that are high frequency modulated to aid scanning.

  The system includes a rod-like arm having an elongated hole for inserting the card.

In accordance with a further aspect of the present invention, there is provided a method for carrying information on a printed card, the method comprising:
Dividing the surface of the card into a number of predetermined areas;
Printing first data to be stored in a first area of the predetermined area;
Using the printed first predetermined area when reading information stored on the card;
Determining a second predetermined area that is not used for printing data, when further information stored on the card is printed when updating the information stored on the card;
Have

  Preferably, these areas are selected in a predetermined order. It is also printed using a high resolution ink dot printer, for example, to print data with fault tolerance resulting from Reed-Solomon encoding of the data. A border area is provided that is printed to delineate the border of the area, and a number of border target markers are provided to help indicate the position of the border area. These border targets consist of a large area of the first color and a small area of the second color arranged in the center of the area.

  Preferably, the data is printed using a high frequency modulation signal such as a checkered pattern. The print is a dot array with a resolution greater than about 1200 dots per inch, and preferably a dot array with a resolution greater than at least 1600 dots per inch. The predetermined areas can be arranged in a standard array on the surface of the card, and the card can be sized and shaped like a typical rectangular credit card.

In accordance with a further aspect of the invention, a method is provided for creating a series of instructions for image processing. The method
Displaying an initial array of sample images for selection by the user;
Accepting a user to select at least one of the sample images;
Using the selection result to create a further array of sample images;
Applying the above steps repeatedly until the user selects at least one final optimal image;
Creating a sample image according to the series of instructions;
Outputting the series of instructions;
including.

  Further, the method includes scanning a user's photograph and using the scanned photograph as an initial image for creation of each sample image. In addition to being printed for visual display on the second side of the card, the instructions can be printed in encoded form on the first side. In addition, the processed image itself is printed.

  Various techniques can be used to form the image, including genetic algorithms and genetic programming techniques, to form the sequence. Further, in further image formation, the 'best so far' image is saved for use.

  Preferably, the method is performed on a computer system incorporated in a vending machine that sells cards and photos.

According to a further aspect of the present invention, there is provided an information storage device for storing information on an inserted card, the device comprising:
A sensor device for detecting a pattern printed on the surface of the card and disposed in a predetermined number of possible active areas on the card;
A decoding device for decoding the detected pattern into corresponding data;
A printing device for printing a dot pattern on the card surface and in one of the active areas;
A positioning device for placing a detected card at a known relative position of the sensor device and the printing device;
With
The detection device is adapted to detect a pattern printed in the latest operational print area of the card;
The decoding device is adapted to decode the detected pattern into corresponding latest data;
If the printing device is requested to update the latest data, the printing device is updated to the latest one of the active areas after the positioning device is driven for accurate card positioning. Is adapted to print.

  The printing apparatus preferably includes an inkjet printer apparatus including a card width print head capable of printing a card width at a time. The positioning device can comprise a series of pinch rollers for controlling the movement of the card across the card. The printed pattern is arranged in a fault tolerant manner, for example using Reed-Solomon decoding, and the decoding device has a decoder suitable for the fault tolerant pattern.

According to a further aspect of the invention,
An image sensor for detecting images;
A storage device for storing detected images and related system configurations;
A data input device for inputting an image correction data module to correct the detected image;
A processing device connected to the image sensor, the storage device, and the data input device for controlling the camera system in addition to processing of the detected image;
A printing device for printing the detected image on a supplied print medium as required;
A digital camera system comprising:
A method is provided for providing an image correction data module adapted to cause the processing device to modify the operation of the digital camera system based on the insertion of a further image correction module.

  Preferably, the image correction data module includes a card having encoded data on the surface thereof, the encoding of the data is performed by printing, and the data input device is an optical scanner for scanning the card surface Have The correction of the operation is repeated, and the images are corrected in the order in which a series of image correction modules are inserted into the same image.

According to a further aspect of the invention,
An image sensor for detecting images;
A storage device for storing detected images and related system configurations;
A data input device for inputting an image correction data module to correct the detected image;
A processing device connected to the image sensor, the storage device, and the data input device for controlling the camera system in addition to processing of the detected image;
A printing device for printing the detected image on a supplied print medium as required;
And the image correction data module is provided with a digital camera system adapted to perform a series of diagnostic tests in the digital camera system and to print the results through the printing device.

  Preferably, the image correction data module includes a card having encoded instruction data on one side thereof, and the processing device includes a device for reading the instruction data encoded on the card. The diagnostic test can also have a cleaning cycle for the printer device to improve the operation of the printer device, such as by printing all continuous black bands. Alternatively, the diagnostic test may include nozzle adjustments to improve the operation of the ink jet printer. In addition, various internal operating parameters of the camera system can be printed. If a gravitational impact sensor is installed in the camera system, the diagnostic test can also print the locality of the sensor.

  In accordance with a further aspect of the present invention, a camera system for generating an image is provided, the camera system comprising: a sensor for detecting the image; a processing device for processing the detected image according to some predetermined processing request; A printing apparatus for printing the detected image on the surface of the medium; and a magnetic recording apparatus for recording related information on the magnetic sensitive surface formed on the print medium. The related information includes audio information related to a detected image, and the printing apparatus preferably prints the detected image on the first surface of the print medium, and the magnetic recording device on the second surface of the print medium. The related information is recorded. The print media can be stored in a removable roll inside the camera system. In one embodiment, the magnetically sensitive surface can include a strip attached to the back surface of the print media.

  In accordance with a further aspect of the invention, a method is provided for making a non-fading copy of an image captured by an image sensor of a portable camera device. The portable camera device includes an integrated computer device and an integrated printer device for printing on a print medium stored in the camera device, and the method detects an image with the image sensor. Converting the image into an encoding format having fault-tolerant encoding properties; and printing the image encoding format as a non-fading record of the image using the integrated printer device. Become.

  Preferably, the integrated printer device includes devices for printing on a first surface and a second surface of a print medium, and the detected image and its visual processing are printed and encoded on the first surface. The form is printed on the second side. The thumbnail of the detected image can be printed beside the image encoding format. Its fault-tolerant coding also allows Reed-Solomon coding of images in addition to applying a checkered high-frequency modulation signal to a coding format in which permanent recordings have repeatable high-frequency spectral components. Also includes forming a version.

According to a first aspect of the present invention, a sales system for sales of image processing cards used in camera devices is provided. The camera device has a card processing interface for inserting an image processing card. The image processing card is for processing an image in the camera device. The sales system
A plurality of printing devices for outputting an image processing card;
In order to store a series of image processing card data necessary for creating the image processing card, each printing apparatus is connected to a corresponding computer system.

  Preferably, the computer system stores a series of image processing card data in a manner that is cached via a computer network. The card distribution computer distributes the new image processing card to the computer system.

  If the image processing card includes an event card (a seasonal card distributed to the computer system for printing out the card for use in seasonal events), the present invention has a specific application.

According to a further aspect of the present invention, a data structure is provided having a series of block data areas encoded on the surface of an object, each block data area comprising:
An encoded data area containing data to be decoded into an encoded format;
A series of clock mark configurations arranged to surround the first peripheral portion of the encoded data region;
A series of easily identifiable target configurations arranged to surround the second peripheral portion of the encoded data region;
Consists of.

  The block data area further includes an orientation data structure disposed around a third peripheral portion of the encoded data area. The directional data structure may include a line of equal data points along the peripheral edge.

  The clock mark configuration has a first line of equal data points in addition to a substantially adjacent second line of data points located along the edge of the encoded data region. Can do. The clock mark configuration can be located on the opposite side of the encoded data area.

  The target configuration has a set of data points blocks arranged so as to have a gap, and the data points have a constant value of a first magnitude except at the center. And having a size opposite to the first size at the central portion. The set of blocks further includes a target number display configuration having the values of the inverse continuous group.

  The data structure is ideally utilized with a series of dots printed on the substrate surface.

According to a second aspect of the present invention, there is provided a method for decoding a data structure encoded on the surface of an object, the data structure having a series of block data areas, each block data area comprising:
An encoded data area containing data to be decoded into an encoded format;
A series of clock mark configurations arranged to surround the first peripheral portion of the encoded data region;
A series of easily identifiable target configurations arranged to surround the second peripheral portion of the encoded data region;
The method comprises:
(a) scanning the data structure;
(b) searching for the starting position of the data structure;
(c) determining the latest direction of the target configuration and searching for the position of the target configuration;
(d) searching for the position of the clock mark configuration from the position of the target configuration;
(e) using the clock mark configuration to determine an expected position of bit data in the encoded data region;
(f) determining an expected data value for each bit data;
Have

  The clock mark configuration has a first line of equal data points in addition to a substantially adjacent second line of data points located along the edge of the encoded data region. Can do. The step of using (e) may also run along the second line of data points using a pseudo-phase locked loop type algorithm to maintain the current position in the clock mark configuration. Including.

Further, determining step (f)
Is
Dividing the sensed bit value into three adjacent regions (i.e., a central region, a first lower extreme region, and a second upper extreme region);
Those values are included in a first region and a bit value corresponding to the first region is determined;
Those values are included in a second region, and a bit value corresponding to the second region is determined;
Those values are included in the central region, and spatially enclosed values are used to determine whether the value is the first value or the second value.

In accordance with a further aspect of the invention, a method is provided for determining an output data value of sensing data, the method comprising:
(a) dividing the detected data value into three adjacent regions (ie, a central region, a first lower extreme region, and a second upper extreme region);
Those values are included in a first region and a bit value corresponding to the first region is determined;
Those values are included in a second region, and a bit value corresponding to the second region is determined;
Those values are included in the central region, and spatially enclosed values are used to determine whether the value is the first value or the second value.

In accordance with a further aspect of the invention, a fluid supply apparatus is provided for supplying a plurality of different fluids to a plurality of different supply slots. The supply slots are arranged at periodic intervals in an alternating manner, and the fluid supply device comprises:
A fluid inlet device for each of a plurality of different fluids;
A flow path for each of the different fluids, and a main channel flow path connected to each of the supply slots;
A sub-channel channel connecting each of the supply slots to a corresponding main channel channel;
It has. The number of fluids is more than 2, and at least two main channel fluid devices are formed on the first surface of the molded fluid supply unit, and the other main channel fluid devices have through holes penetrating the surface of the molded product. A sub-channel flow path connected to the slot is formed on the upper surface of the molded product.

  Preferably, the feeding device is plastic injection molded and the pitch rate of the slots is equal to or less than 1000 per inch. In addition, a collection of slots is formed in the width of the photograph. Preferably, the fluid supply device includes a plurality of roller slot devices for receiving one or more pinch rollers, the fluid is made of ink, and the rollers are connected to the slots. Is controlled so that the print media crosses the print media. The slot is divided into a series of color slots arranged in a column.

  Preferably, at least one channel of the fluid supply device is exposed during manufacture, and the exposed surface is sealed with a sealing tape. Advantageously, the fluid supply device is provided by a TAB slot to obtain a self-bonded wire by tape.

According to a further aspect of the present invention, there is provided a printer device for image printing that uses at least one ink ejection device that supplies via an ink supply path, the device comprising:
A series of ink supply ports, one for each output color, adapted to work with an ink supply device for supplying ink to the printer;
A series of conductive connector pads along the outer surface of the printer device;
A page width printhead having a series of ink ejection devices for ejecting ink;
An ink supply system for supplying ink from the ink supply port to the ink ejection device of the page width print head;
A plurality of interconnect wires connecting the page width printhead to the conductive connector pads;
With
The printer device is adapted to be removably inserted into a housing device, the housing comprising an interconnect for interconnection with the conductive connector pad, and ink supply by the ink supply device Therefore, an ink supply connector for interconnection with the ink supply port is provided.

Preferably, the plurality of interconnect wires wrap around the outer surface of the printer device and form an automatic tape bonding sheet connected to a conductive connector pad. The interconnected wires are
A first set of wires interconnected to the conductive connector pads and disposed generally parallel to each other along the entire length of the printhead;
A second set of wires arranged substantially parallel to each other from the surface of the printhead;
have. Each of the first set of wires is connected to many of the second set of wires. The ink supply port has a thin diaphragm portion. The diaphragm portion is perforated based on the ink supply connector being inserted into the housing device.

  The page width print head may include a number of substantially identical repeatable units, each having a predetermined number of ink ejection devices. The unit has a standard interface with a predetermined number of connection wires, and each standard interface device is connected to the conductive connector pads in groups. The print head itself can be made from a silicon wafer and separated into page width strips.

  In accordance with a further aspect of the present invention, there is provided a method of providing resistance to integrated circuit monitoring by a method of monitoring current fluctuations, the method comprising forming a spurious noise generating circuit that is part of the integrated circuit. Including.

  The noise generation circuit may include a random number generator such as LFSR (Linear Feedback Shift Register).

  In accordance with a further aspect of the present invention, a low power CMOS circuit is provided. The circuit comprises a p-type transistor having a gate connected to a first clock and an input, and an n-type transistor connected to a second clock and an input, the CMOS circuit comprising an overlap The first and second clocks are switched by a method that does not.

  The circuit can be substantially adjacent to a second circuit having high power switching characteristics. The second circuit can include a noise generation circuit.

  In accordance with a further aspect of the present invention, there is provided a method for providing resistance to memory circuit monitoring having a plurality of level states corresponding to different output states, the method utilizing an intermediate state only for valid output states. Including that. The memory can be a flash memory and can further have one or more parity bits.

  In accordance with a further aspect of the present invention, a resistance to tampering with the integrated circuit is used using a circuit path attached to a random noise generator to monitor attacks that tamper with the integrated circuit. A method of giving is provided.

Circuit
path) is opposite to each other and has a first path and a second path that are connected to various circuits and serve as exclusive OR circuits for generating a reset output signal.

  In accordance with a second aspect of the present invention, there is provided a tamper detection line having one end connected to a grounded large resistor and the other end connected to a second large resistor connected to a power source. The tamper detection line is further connected to a comparator that compares whether the expected voltage is within a predetermined tolerance. In the middle of the resistors connected to the series of tests, a large resistor is output so that the comparator outputs a reset signal when tampering is detected by one of the tests.

According to a further aspect of the invention, an authentication system is provided for determining the validity of an attached unit to be authenticated, the system comprising:
A central system unit for verification of the first and second secure keys;
An object holding first and second secure keys attached to the central system unit;
With
The object holding the second secure key is attached in such a way that it cannot be removed,
The central system unit is
Querying the object holding the first secure key to determine a first response;
Using the first response to query the object holding the second secure key to determine a second response;
further,
The first response and the second response are adapted to be compared to determine whether an object holding the second secure key is attached to a valid attachment unit.

  The object holding the second secure key further includes a response having a magnitude factor that effectively decreases monotonically so that the unit ceases operation after a predetermined use of the mounting unit. Thus, the attached unit can include consumables.

  In addition, the central system unit can query the object that holds the first secure key for the number of received first responses, at which time the central system unit can determine the second response. Using the first response when inquiring to the object holding the second secure key for the central system unit to determine the valid range of the second response The second response is used to make an inquiry to the unit that holds

  The system is ideally used to authenticate printer consumables, such as ink in inkjet printers.

  Indeed, in a camera system having a built-in inkjet printer, the preferred embodiment will be discussed in connection with consumable ink, but the invention is not so limited.

Description of other preferred embodiments

A digital image processing camera system constructed in accordance with the preferred embodiment is shown in FIG. The camera unit 1 includes means for inserting an integral print roll (not shown). The camera unit 1 includes an area image sensor 2 that detects an image 3 photographed by the camera. As an option, a second area image sensor may be mounted to visualize the scene 3 and optionally obtain a 3D image output product.

The camera 1 includes an optional color display 5 for displaying an image detected by the sensor 2. When a simple image is displayed on the display 5, the button 6 can be pressed, so that a printed image 8 is output by the camera unit 1. In the following, a series of cards called Artcards (Art Cards) 9 will contain information encoded on one side, and on the other side will contain images that have been distorted by a particular effect generated by Artcard 9. The Artcard 9 is inserted into the Artcard reader 10 on the side of the camera 1, and when inserted, an output image 8 that is distorted in the same manner as the distortion that appears on the surface of the Artcard 9 is obtained. Therefore, by using this simple user interface, a user who wants to produce a specific effect inserts one of a number of Artcards 9 into the Artcard reader 10 and takes a button to take a picture of the image 3 19 can be used, resulting in a corresponding distorted output image 8.

The camera unit 1 may include a number of other control buttons 13, 14 in addition to a simple LCD output display 15 for displaying explanatory information including the number of printouts remaining on the internal print roll of the camera unit. . In addition, various output formats can be controlled by the CHP switch 17.

Referring now to FIG. 2, a schematic diagram of the internal hardware of the camera 1 is shown. The internal hardware is centered on an Artcam central processor unit (ACP) 31.

Artcam central processor 31
The Artcam central processor 31 provides a number of functions that form the heart of the system. The ACP 31 is preferably implemented as a complex high speed CMOS system on chip. It is recommended to use a standard cell design with some full custom area. Fabrication on a 0.25 micron CMOS process achieves the required density and speed with a reasonably small die area.

The functions provided by ACP 31 include the following functions:
1. Control and digitization of the area image sensor 2. The 3D stereoscopic version of ACP requires a two area image sensor interface in which a second optional image sensor 4 is provided for the stereoscopic effect;
2. Area image sensor compensation, reformatting, and image enhancement;
3. Memory interface and management for storage device 33;
4). Interface, control and analog / digital conversion of the linear image sensor 34 of the Artcard reader provided for reading data from the Artcard 9;
5. Extraction of raw Artcard data from a digitized encoded Artcard image;
6). Reed-Solomon error detection and correction of Artcard encoded data. Coded surface of Artcard9 includes information on how to handle how image to generate an image distortion face the displayed effects of Artcard9. This information is in the form of a script called “Vark script” below. The Vark script is used by an interpreter executed within the ACP 31 to produce the desired effect;
7). Interpretation of Vark script on Artcard9;
8). Execution of image processing operations as specified by the Vark script;
9. Control of various motors for paper transport 36, zoom lens 38, autofocus 39, and Artcard driver 37;
10. Control of the cutter actuator 40 for operation of the cutter 41 that cuts the photograph 8 from the print roll 42;
11. Halftoning of image data for printing;
12 Supply of print data to the print head 44 at an appropriate time;
13. Control of the print head 44;
14 Control of ink pumping to the print head 44;
15. Control of optional flash unit 56;
16. Readout of various sensors in the camera, including orientation sensor 46, autofocus 47 and Artcard insertion sensor 49, and treatment based thereon;
17. Reading of the user interface buttons 6, 13, 14 and treatment based thereon.
18. Control of the status display 15;
19. Supply of a viewfinder and a preview image to the color display 5;
20. Control of system power consumption including ACP power consumption by the power management circuit 51;
21. Provision of external communication 52 to a general-purpose computer (using part USB);
22. Reading and storing information of the print roll authentication chip 53;
23. Reading and storing information of the camera authentication chip 54;
24. Communication with optional mini keyboard 57 for text correction.

Crystal resonator 58
The crystal resonator 58 is used as a frequency reference for the system clock. Since the system clock is very high, the ACP 31 includes a phase locked loop clock circuit to increase the frequency obtained from the crystal 58.

Image sensing area Image sensor 2
The area image sensor 2 converts an image passing through the lens into an electric signal. The area image sensor may be a charge coupled device (CCD) or an active pixel sensor (APS) CMOS image sensor. At present, available CCDs usually have a fairly high image quality, but the development of CMOS imagers is currently very active. It is expected that a CMOS imager will eventually be substantially less expensive than a CCD with a small number of pixels and can incorporate drive circuitry and signal processing. CMOS imagers can be fabricated with CMOS manufacturing technology that is moving to 12-inch wafers. Therefore, the manufacturing cost difference between the CCD and the CMOS imager increases, and the CMOS imager is likely to be gradually supported. However, at present, CCD is the best option.

The Artcam unit will produce adequate results with a 1500 × 1000 type area image sensor. However, small sensors such as, for example, a 750 × 700 type may also be appropriate in various markets. Artcam is less sensitive to the resolution of the image sensor than conventional digital cameras. The reason is that most styles in Artcard 9 process images so as to obscure the lack of resolution. For example, if the image is distorted to simulate the effect of being converted to an impressionist painting, the low resolution of the original image can be used with minimal impact. Further examples where low resolution input images are typically not noticed, such as image warping that results in very distorted images, multiple reductions of images (eg passport photos), bump mapping for base relief metal look Includes texture processing and photo composition into structured scenes.

Allowing low-resolution image sensors in this way is an important factor that reduces the manufacturing cost of the Artcam unit 1 camera. Artcams using low cost 750 × 500 type image sensors often give better results than conventional digital cameras using very expensive 1500 × 1000 type image sensors.

Optional stereoscopic 3D image sensor 4
The three-dimensional version of the Artcam unit 1 includes an auxiliary image sensor 4 for stereoscopic operation. This image sensor is identical to the main image sensor. A circuit for driving an optional image sensor may be accommodated as a standard part of the ACP chip 31 in order to reduce an increase in design cost. Alternatively, a separate 3D Artcam
An ACP may be designed. This option reduces the manufacturing cost of the mainstream single sensor Artcam.

Print roll authentication chip 53
The small chip 53 is accommodated in each print roll 42. This chip replaces the function of bar codes, optical sensors and wheels, and ISO / ASA sensors on other forms of camera film units such as APS (New Photo System) film cartridges.

The authentication chip also provides the following other functions:
1. Storage of data, not data detected mechanically and optically from the APS roll;
2. Display the remaining length of the medium with high resolution accuracy;
3. Authentication information to prevent bad imitation print roll copying.

The authentication chip 53 includes a 1024-bit flash memory, and 128 bits of the 1024 bits are an authentication key, and 512 bits are authentication information. In addition, an encryption circuit is accommodated to ensure that the authentication key is not directly accessed.

Print head 44
The Artcam unit 1 is sufficiently small, sufficiently low power, sufficiently fast, sufficiently high quality, and sufficiently low in cost, and can be used with any color printing technology adapted to the print roll. Hereinafter, the print head will be specifically described.

The specifications of the inkjet head are as follows.

オプションのインク圧コントローラ（図示せず）
インク圧コントローラの機能は、Ａｒｔｃａｍに組み込まれたインクジェットプリントヘッド４４のタイプに依存する。一部のタイプのインクジェットでは、インク圧は単純に大気圧であるため、インク圧コントローラを使用しなくても済む。他のタイプのプリントヘッドは、安定化した正のインク圧を必要とする。この場合、インプレッシャ・コントローラは、ポンプ及び圧力トランスデューサを含む。

Optional ink pressure controller (not shown)
The function of the ink pressure controller depends on the type of ink jet print head 44 incorporated in Artcam. In some types of inkjets, the ink pressure is simply atmospheric pressure, so there is no need to use an ink pressure controller. Other types of printheads require a stabilized positive ink pressure. In this case, the pressure controller includes a pump and a pressure transducer.

Other printheads require an ultrasonic transducer to cause regular oscillations in the ink pressure, typically about 100 KHz. In this case, the ACP 31 controls the frequency phase and amplitude of these vibrations.

Paper transport motor 36
The paper transport motor 36 moves the paper at a constant speed from inside the print roll 42 through the side of the print head. The motor 36 is a small motor geared down to an appropriate speed to drive a roller that moves the paper. Since mechanical play or other vibrations affect the printed dot row spacing, high quality motors and mechanical gears are required to achieve high image quality.

Paper transport motor driver 60
The motor driver 60 is a small circuit that amplifies the digital motor control signal from the APC 31 to a level appropriate for driving the motor 36.

Paper Pull Sensor The paper pull sensor 50 detects the pulling of the photograph from the camera unit by the user during the printing process. The APC 31 reads the sensor 50 and operates the cutting machine 41 when detecting this situation. The paper pull sensor 50 is incorporated to make the camera more absolutely reliable during operation. If the user attempts to pull out the paper forcibly during printing, the print mechanism 44 or print roll 42 will be damaged (in the extreme case). Since the pod is allowed to be pulled out of the Polaroid camera before it is completely ejected, the general public is accustomed to performing such an operation. Thus, the general public is likely not to pay attention to the printed instructions that the paper should not be pulled out.

Artcam preferably restarts the photo printing process after the cutter 41 cuts the paper after detecting the pull.

The pull sensor may be realized as a strain gauge sensor, or may be realized as an optical sensor that detects a small plastic flag deflected by a torque generated in the paper drive roller when the paper is pulled. The latter implementation is recommended because of its low cost.

Paper cutter actuator 40
The paper cutter actuator 40 is a small actuator that causes the cutter 41 to cut the paper at the end of the photo or when the paper pull sensor 50 is activated.

The paper cutter actuator 40 is a small circuit that amplifies the cutter control signal from the APC to a level required by the actuator 41.

Artcard9
Artcard 9 is a program storage medium for the Artcam unit. As described above, the program is in the form of a Vark script. Vark is a powerful image processing language specifically developed for the Artcam unit. Each Artcard 9 stores one Var script, thereby defining one image processing style.

Preferably, the VARK language is highly dedicated to image processing. By being highly dedicated to image processing, the storage capacity required to store details on the card is substantially reduced. Furthermore, the convenience with which new programs including enhanced functions can be created is substantially improved. Preferably, this language is warp map image warping, convolution, color look-up table, image posterization, image noise addition, image enhancement filter, painting algorithm, brush jittering and manipulation edge detection filter, tiling Illuminating with light sources, bump maps, text, face detection and object detection attributes, fonts including 3D fonts, and the ability to handle multiple image processing functions including any composite pre-rendering icons. More detailed contents of the operation of the Vark language interpreter are described below.

Thus, by utilizing a language structure as defined by the language created, a new impact on any image is created and built for an inexpensive storage device on Artcard, after which the camera owner It is possible to distribute. Further, on one side of the card, an example is provided for explaining the influence of a specific VARK script stored on the other side of the card on an arbitrary photographed image.

By using such a system, if the VARK interpreter is built into the camera device, a device-independent scenario is given, and as a result, the underlying technology can be completely changed in the long term. In this respect, camera technology can be disseminated without the risk of becoming obsolete. Furthermore, it is possible to update a Vark script when a new filter is created and distributed in an inexpensive manner, such as using a simple card for card reading.

Artcard 9 is a thin white plastic piece in the same format as a credit card (length 86 mm, width 54 mm). Artcard is printed on both sides using a high resolution inkjet printer. This inkjet printer technology is considered to be the same technology used for Artcam with a resolution of 1600 dpi (63 dpi). The main feature of Artcard 9 is its low manufacturing cost. Artcard can be manufactured at high speeds as well as a wide web of plastic films. The plastic web is covered on both sides with a hydrophilic dye fixing layer. The web is printed simultaneously on both sides using a page width color inkjet printer. The web is then cut and punched into individual cards. One side of the card is printed with a human-readable representation of the effect of Artcard 9 on the detected image. This may be a simple standard image processed using a Vark script stored on the back of the card.

On the back side of the card is printed an array of dots that can be decoded into a Vark script that defines the image processing sequence. This printing area is 80 mm × 50 mm, and there are a total of 15876000 dots. This dot arrangement can represent at least 1.89 MBytes of data. Extensive error detection and correction is built into the dot array to achieve high reliability. This ensures that even if a substantial portion of the card is soiled, worn, wrinkled or soiled, the data integrity is not affected. The data encoding method used is Reed-Solomon encoding, and half of the data is used for error correction. As a result, each Artcard 9 can store 967 Kbytes of error-corrected data.

Linear image sensor 34
The Artcard linear sensor 34 converts the Artcard data image into an electrical signal. As in the case of the area image sensors 2 and 4, the linear image sensor can be a CCD or APS.
It can be manufactured using CMOS technology. The effective length of the image sensor 34 is 50 mm, which matches the width of the data array on the Artcard 9. In order to satisfy the Nyquist sampling theorem, the resolution of the linear image sensor 34 must be at least twice the highest spatial frequency of the Artcard optical image reaching the image sensor. In practice, data detection is simpler when the image sensor resolution is substantially higher than this. If a resolution of 4900 dpi (180 dpi) is selected, a total of 9450 pixels are provided. This resolution requires a pixel sensor pitch of 5.3 μm. This can be easily achieved by using four rows of 20 μm pixel sensors that are staggered.

The linear image sensor is mounted in a special package that includes an LED 65 for illuminating Artcard 9 by a light pipe (not shown).

The Artcard reader light pipe may be a molded light pipe with several functions:
1. The light pipe uses all internally reflecting facets to diffuse the light from the LED across the width of the card.
2. The light pipe collects light on a 16 μm wide strip of Artcard 9, using an integrated cylindrical lens.
3. The light pipe uses a shaped array of microlenses to collect the light reflected from the Artcard onto the pixels of the linear image sensor.

Next, the operation of the Artcard reader will be described.

Artcard reader motor 37
The Artcard reader motor advances the Artcard at a relatively constant speed and passes the linear image sensor 34. Since it is not cost effective to house the precision precision mechanical components in the Artcard reader, the motor 37 is a standard small motor geared down to the proper speed to drive a pair of rollers that move the Artcard 9 . Since speed fluctuations, backlash, and other vibrations affect the raw image data, the circuitry within the APC 31 provides extensive compensation for these effects to reliably read the Artcard data.

The motor 37 is driven in the reverse direction when taking out the Artcard.

Artcard motor driver 61
The Artcard motor driver 61 is a small circuit that amplifies the digital motor control signal from the APC 31 to an appropriate level for driving the motor 37.

Card insertion sensor 49
The card insertion sensor 49 is an optical sensor that detects the presence or absence of a card inserted in the card reader 34. After the signal from this sensor 49, the APC 31 starts the card reading process, which includes the operation of the Artcard reader motor 37.

Card eject button 16
The card eject button 16 (FIG. 1) is used by the user to retrieve the current Artcard, allowing another Artcard to be inserted. The APC 31 detects the press of the button and reverses the Artcard reader motor 37 in order to take out the card.

Card status indicator 66
A card status indicator 66 is provided to inform the user of the status of the Artcard read process. This may be a standard bi-color (red / green) LED. When the card is read correctly and the integrity of the data is verified, the LED will continuously emit green light. If the card is defective, the LED emits red light.

When the camera is powered from a 1.5V battery rather than a 3V battery, the power supply voltage is less than the forward voltage drop of the green LED and the LED does not emit light. In this case, a red LED is used, or the LED is powered from a voltage pump that feeds other circuits in the Artcam that require higher voltages.

64Mbit DRAM33
The camera uses an 8 Mbyte memory 33 to perform a wide variety of image processing effects. This can be provided by a single 64 Mbit memory chip. Of course, it may be replaced with a larger DRAM storage capacity size as the memory technology changes.

High speed access to the memory chip is required. This is Rambus
This can be achieved by using DRAM (burst access speed of 500 megabytes per second), or double data rate (DDR) SDRAM or SyncLink DRAM chip.

Camera authentication chip The camera authentication chip 54 is the same as the print roll authentication chip 53 except that it stores different information. The camera authentication chip 54 has three main purposes:
1. Providing a secure means of comparing the authentication code with the print roll authentication chip;
2. Providing a storage device for manufacturing information such as the camera serial number;
3. Provide a small-capacity nonvolatile memory for storing user information.

The display Artcam includes an optional color display 5 and a small state display 15. The lowest cost consumer camera is a small TFT as found in some digital and video cameras
A color image display such as LCD 5 is included. The color display 5 is the main costly element of these versions of Artcam, and the display 5 with the backlight is the main power consumer.

Status display 15
The status display 15 is an LCD based on a small passive segment, similar to the LCDs found in current silver halide and digital cameras. Its primary function is to display the number of prints remaining on the print roll 42 and icons for various standard camera functions such as flash and battery status.

Color display 5
The color display is a complete moving image display and operates as a viewfinder, as a confirmation of the image to be printed, and as a user interface display. Since the cost of the display 5 is approximately proportional to its area, large display (eg, 4 inch diagonal) units are limited to expensive versions of Artcam units. For example, smaller displays, such as about 1 inch viewfinder TFTs in color video cameras, are effective for moderate Artcam.

Zoom lens (not shown)
Artcam can include a zoom lens. This may be a standard electronically controlled zoom lens that is identical to the zoom lens used in a standard electronic camera and is similar to the zoom lens of a pocket camera. The preferred version of the Artcam unit includes a standard interchangeable 35mm SLR lens.

Autofocus motor 39
The autofocus motor 39 changes the focus of the zoom lens. The motor is a small motor geared down to an appropriate speed to drive the autofocus mechanism.

Autofocus motor driver 63
The autofocus motor driver 63 is a small circuit that amplifies the digital motor control signal from the APC 31 to an appropriate level for driving the motor 39.

Zoom motor 38
The zoom motor 38 moves the zoom front lens in and out. The motor is a small motor geared down to an appropriate speed to drive the zoom mechanism.

Zoom motor driver 62
The zoom motor driver 62 is a small circuit that amplifies the digital motor control signal from the APC 31 to an appropriate level for driving the motor.

The communication ACP 31 includes a universal serial bus (USB) interface 52 for communication with a personal computer. Not all Artcam models are scheduled to incorporate a USB connector. However, since the silicon area required for the USB circuit 52 is small, the interface can be accommodated in a standard ACP.

Optional keyboard 57
The Artcam unit includes an optional mini-keyboard 57 that customizes the text specified by Artcard. Any text that appears in an Artcard image can be edited even when it is in the form of a complex metallic 3D font. The mini-keyboard includes a single line alphanumeric LCD to display the original text and the edited text. This keyboard may be standard equipment.

The ACP 31 includes a serial communication circuit that transfers data to and from a small keyboard.

The power supply Artcam unit uses a battery 48. Depending on the Artcam option, this battery can be either a 3V lithium battery, a 1.5V AA alkaline battery, or other battery device.

Power management unit 51
Power consumption is an important design constraint in Artcam. Desirably, either a standard camera battery (eg, a 3V lithium battery) or a standard AA or AAA alkaline battery can be used. Although the degree of complexity of the electronic circuit of the Artcam unit is significantly higher than that of a 35 mm photographic camera, the power consumption is not necessarily high. The Artcam power supply is carefully managed so that all units are powered off when not in use.

The most important power waste sources are APC 31, area image sensors 2 and 4, various motors of printer 44, flash unit 56, and optional color display 5. Then handle each part separately:
1. ACP: If manufactured using 0.25 μm CMOS and operate at 1.5V, ACP power consumption can be very small. The clock to the various parts of the ACP chip can also be very small. The clocks to the various parts of the ACP chip can be turned off when not in use, eliminating most of the standby current consumption. ACP is fully used for about 4 seconds per photo to be printed;
2. Area image sensor: power is supplied to the area image sensor only when the user places his finger on the button;
3. Printer power is supplied to the printer only when it is actually printing. This is about 2 seconds per picture. Even so, moderately low power consumption printing should be used;
4). The motors required for Artcam are all low-power small motors and are typically run for a few seconds per photo;
5. The flash unit 45 is used only for some photographs. Its power consumption is easily supplied by a 3V lithium battery with reasonable battery life.
6). The optional color display 5 is a major current wasting source for the following two reasons. The color display needs to be turned on for all the time that the camera is in use, and if a liquid crystal display is used, a backlight is required. Cameras incorporating color displays require larger batteries to achieve acceptable battery life.

Flash unit 56
The flash unit 56 is a standard small electronic flash for consumer cameras.

Overview of ACP 31 FIG. 3 shows the details of the Artcam central processor (ACP) 31 in detail. The Artcam central processor provides all of Artcam's processing power. The ACP is designed for a 0.25 micron CMOS process and has about 1.5 million transistors and an area of about 50 mm ² . Although the design of ACP 31 is complex, the design effort can be reduced by using data path compilation technology, macrocells, and IP cores. ACP31
RISC type CPU core 72;
A 4-way VLIW vector processor 74;
A direct RAMbus (Rambus) interface 81;
A CMOS image sensor interface 83;
A CMOS linear image sensor interface 88;
USB serial interface 52;
An infrared keyboard interface 55;
A numeric LCD interface 84;
Color TFT LCD interface 88;
A 4 Mbyte flash memory 70 for the program storage device 70;
including. The RISC type CPU, direct RAMbus interface 81, CMOS sensor interface 83, and USB serial interface 52 may be cores supplied by vendors. The ACP 31 is intended to operate at a clock rate of 200 MHz based on external 3V and internal 1.5V to minimize power consumption. The CPU core only needs to operate at 100 MHz. The following two block diagrams:
An exemplary Artcam that provides a single ACP 31 illustration and a high-level view of ACP 31 connected to the remaining hardware of Artcam,
Gives two views of ACP31.

Image Access As described above, the DRAM interface 81 serves to connect the other client unit of the ACP chip and the Rambus DRAM. In practice, each module in the DRAM interface is an address generator.

There are three logical types of images manipulated by ACP. They are,
A CCD image that is an input image taken from the CCD;
An internal image format that is an image format used internally by the Artcam device;
A print image that is an output image format printed by Artcam;
It is.

These images typically differ in color space, resolution, and output and input color space that can vary between cameras. For example, the CCD image of a low-price camera shakes a different resolution or different color characteristic than that used in a high-price camera. However, all internal image formats are the same format in terms of color space across all cameras.

In addition, the three image types can differ with respect to which direction is “upward”. The physical orientation of the camera gives rise to the idea of whether it is a person image or a landscape image, and this idea should be maintained throughout the process. For this reason, the internal image is always oriented correctly and rotation is performed on the image acquired from the CCD during the printing operation.

CPU core (CPU) 72
The ACP 31 incorporates a 32-bit RISC type CPU 72 to operate the Vark image processing language interpreter and execute the role of Artcam's general-purpose operating system. A wide variety of CPU cores are appropriate, and any processor core that has sufficient processing power to execute the required core calculation and control functions fast enough to satisfy consumer expectations. Absent. An example of a suitable core is LSI Logic's MIPS.
R4000, Strong ARM core included. There is no need to maintain instruction set continuity between different Artcam models. Artcard compatibility is maintained regardless of future processor advances and changes. This is because the Vark interpreter only needs to be recompiled for each new instruction set. Therefore, the architecture of ACP31 can be freely evolved. Various manufacturers can manufacture various ACP31 chip designs without licensing or porting the CPU core. This device independence avoids chip vendor lock-in as Intel does in the PC market. The CPU operates at 100 MHz and the single cycle time is 10 ns. This CPU needs to be sufficiently fast to execute the Vark interpreter even if the VLIW vector processor 74 is responsible for time-critical operations.

Program cache 72
Even if the program code is stored in the on-chip flash memory 70, it is unlikely that the well-packed flash memory 70 can operate in the 10 ns cycle time required by the CPU. Therefore, a small cache is required for superior performance. Each of the 16 cache lines of 32 bytes is 512 bytes in total, and is sufficiently large. The program cache 72 is defined in a chapter named program cache 72.

Data cache 76
A small data cache 76 is required for superior performance. This requirement generally arises due to the use of Rambus DRAM. Rambus DRAMs can provide high speed data in bursts, but are insufficient for single byte access. The CPU accesses a memory caching system that allows the size of the CPU data cache 76 to be flexibly manipulated. A minimum of 16 cache lines (512 bytes) are recommended for good performance.

The CPU memory model Artcam has a 32 MB area. In the basic model of Artcam, the CPU memory model is composed of 8 MB of physical RDRAM off-chip, and a maximum of 16 MB of off-chip memory is provided. The ACP 31 has a 4 MB flash memory 70 for storing programs. Finally, a 4 MB address space is allocated to various registers and controls of the ACP 31. Therefore, the memory map for Artcam is as follows.

アドレスをデコードする簡単な方法は、アドレスビット２３−２４を使用することである。即ち、
ビット２４がクリアされている場合、アドレスは下位側１６ＭＢのレンジに含まれるので、ＤＲＡＭ及びデータキャッシュ７６から充たされる。殆どの場合に、ＤＲＡＭは８ＭＢに過ぎないが、より上位のメモリモデルＡｒｔｃａｍを考慮に入れるため１６ＭＢが割り当てられる。
ビット２４がセットされ、ビット２３がクリアされている場合、アドレスは、フラッシュメモリ７０の４Ｍバイトレンジを表現し、プログラムキャッシュ７２によって充たされる。
ビット２４＝１及びビット２３＝１である場合、アドレスは、ＣＰＵメモリデコーダ６８によって、低速バス経由によるＡＣ内の要求されたコンポーネントへのアクセスに変換される。

A simple way to decode the address is to use address bits 23-24. That is,
If bit 24 is cleared, the address is included in the lower 16 MB range and is filled from the DRAM and data cache 76. In most cases, the DRAM is only 8 MB, but 16 MB is allocated to take into account the higher memory model Artcam.
If bit 24 is set and bit 23 is cleared, the address represents the 4 Mbyte range of flash memory 70 and is filled by program cache 72.
If bit 24 = 1 and bit 23 = 1, the address is converted by the CPU memory decoder 68 to access the requested component in the AC via the low speed bus.

Flash memory 70
The ACP 31 includes a 4 Mbyte flash memory 70 that stores the Artcam program. It is envisioned that the flash memory 70 has a higher fill factor than the mask ROM and allows higher flexibility for testing the camera program code. The weak point of the flash memory 70 is the access time, and this access time cannot be said to be sufficiently high with respect to the CPU operating speed of 100 MHz (10 ns cycle time). Therefore, the high-speed program instruction cache 77 acts as an interface between the CPU and the low-speed flash memory 70.

Program cache 72
A small cache is required for excellent CPU performance. This request is caused by the low speed of the flash memory 70 that stores the program code. Each of the 16 cache lines of 32 bytes is 512 bytes in total, and is sufficiently large. The program cache 72 is a read-only cache. Data used by the CPU program passes through the CPU memory decoder 68, and through the general data cache 76 if the address is in DRAM. The separation allows the CPU to operate independently of the VLIW vector processor 74. If the amount of data required for a given process is small, as a result, the process can run entirely on the cache.

Finally, the program cache 72 can be read by the CPU as data rather than as purely program instructions. This makes it possible to load a table, that is, microcode for the VLIW from the flash memory 70. The address where bit 24 is set and bit 23 is cleared is satisfied from program cache 72.

CPU memory decoder 68
The CPU memory decoder 68 is a simple decoder along with CPU data access. The decoder converts the data address to internal ACP register access via the internal low speed bus, thereby enabling memory mapped I / O of the ACP register. The CPU memory decoder 68 interprets only addresses where bit 24 is set and bit 23 is cleared. Caching is not performed in the CPU memory decoder 68.

DRAM interface 81
The DRAM used by Artcam is a single channel 64 Mbit (8 Mbyte) RAMbus RDRAM operating at 1.6 GB / s. RDRAM access is by a single channel (16 bit data path) controller. RDRAM has several effective modes of operation for low power operation. The Rambus specification describes a system with random 32-byte transfers that can achieve efficiencies greater than 95%, but this is true if only some of the 32 bytes are used. Absent. An efficiency of 86% or more can be obtained by performing two writes followed by two writes to the same device. Basic latency is necessary for bus turnarounds that change from write to read, and a delayed write mechanism exists, which can further improve efficiency. For writing, the write mask allows writing to a specific subset of bytes. These write masks will be set by "dirty bits" in the internal cache. In summary, Rambus Direct RDRAM can easily achieve throughputs in excess of 1 GB / s, combined with intelligent algorithms that make full use of 32 bytes of transfer knowledge (in most processes) multiple reads per write Therefore, a transfer rate exceeding 1.3 GB / second is expected. Every 10 ns, 16 bytes can be transferred to and from the core.

The DRAM configuration for Artcam of the DRAM configuration basic model (8MB RDRAM) is shown below.

注：非圧縮形式の印刷画像は４．５ＭＢ（１チャンネル当たり１．５ＭＢ）を必要とする。他の対象を８ＭＢモデルに収容するため、印刷画像は圧縮されるべきである。クロミナンスチャンネルが４：１に圧縮された場合、各クロミナンスチャンネルは０．３７５ＭＢしか必要としない。

Note: An uncompressed print image requires 4.5 MB (1.5 MB per channel). The print image should be compressed to accommodate other objects in the 8MB model. If the chrominance channels are compressed 4: 1, each chrominance channel requires only 0.375 MB.

The memory model described here assumes a single 8 MB RDRAM. Some other Artcam models have more memory and do not need to compress the printed image. In addition, as the amount of memory increases, the majority of the final image can be manipulated simultaneously, possibly increasing speed.

It should be noted that the removal or insertion of Artcard invalidates the 5.5 MB area that holds the printed image, one channel of the enlarged photographic image, and the image pyramid. This space is securely used by the Artcard interface to decrypt Artcard data.

Data cache 76
The ACP 31 includes a dedicated CPU instruction cache 77 and a general-purpose data cache 76. The data cache 76 handles all DRAM requests (data reads and writes) from the CPU, VLIW vector processor 74 and display controller 88. These requirements have very different aspects in terms of memory usage and algorithmic timing requirements. For example, the VLIW process processes an image in linear memory and looks up a table value for each value in the image. It is desirable to cache the entire look-up table so that there is little need to cache most of the image, but no actual memory access is required. Because the requirements are different in this way, the data cache 76 can intelligently define caching.

The Rambus DRAM interface 81 has very fast memory access capability (average throughput is 32 bytes at 25 ns), but is insufficient to handle single byte requests. To reduce the effective memory latency, ACP 31 includes 128 cache lines. Each cache line is 32 bytes wide. In this way, the total capacity of the data cache 76 is 4096 bytes (4 KB). The 128 cache lines are organized into groups of 16 programmable sizes. Each of the 16 groups must be a contiguous set of cache lines. The CPU plays a role of determining the number of cache lines to be allocated to each group. Within each group, the cache lines are packed according to a simple LRU (least recently used) algorithm. For CPU data requests, data cache 76 handles memory access requests with address bits 24 cleared. If bit 24 is cleared, the address falls within the lower 16 MB range and can be served from the DRAM and data cache 76. In most cases, the DRAM has only 8 MB, but 16 MB is allocated taking into account the Artcam of the model with more memory. If bit 24 is set, the address is ignored by data cache 76.

All CPU data requests are served from cache group 0. Although 16 minimum cache lines are recommended for good CPU performance, the CPU can assign any number of cache lines (other than 0) to cache group 0. The remaining cache groups (1-15) are allocated according to current requirements. This may mean an assignment to the VLIW vector processor 74 program or display controller 88. For example, a 256 byte lookup table that is required to be available at all times would require 8 cache lines. To write out images sequentially, only 2 to 4 cache lines are required (depending on the size of the generated record and whether the write request has been delayed for multiple cycles). Let ’s go. Associated with each cache line byte is a dirty bit that is used to create a write mask when the memory is written to the DRAM. Each cache line has a separate dirty bit associated with it, and whether this dirty bit has been written to any of the cache line's bytes (so the cache line must be written back to the DRAM before the DRAM is reused). Or not). Note that two different cache groups may access the same address in memory and get out of sync. The VLIW program writer is responsible for ensuring that this is not a problem. For example, it is perfectly reasonable to give one cache group the role of reading an image and giving another cache group the role of writing the changed image back into memory. It is advantageous to allocate cache lines in this way when images are read or written sequentially. A total of eight buses 182 connect the VLIW vector processor 74 to the data cache 76. Each bus is connected to an I / O address generator. (There are two I / O address generators 189 and 190 per processing unit 178 and there are four processing units in the VLIW vector processor 74. Therefore, the total number of buses is eight. .)
In a given cycle, in addition to a single 32-bit (4-byte) access to the CPU's cache group (group 0), there are 8 simultaneous 16-bit (2-byte) accesses to the remaining cache group. Permitted on the VLIW processor 74 bus. The data cache 76 is responsible for processing requests fairly. In a given cycle, only one request for a particular cache group is processed. If there are eight address generators 189, 190 in the VLIW vector processor 74, each of the address generators may reference a separate cache group. However, two or more address generators 189, 190 may access the same cache group, and in some cases it is more reasonable. The CPU is responsible for ensuring that the correct number of cache lines are allocated to the cache group and for ensuring that the various address generators 189, 190 of the VLIW vector processor 74 accurately reference a particular cache group. . As described above, the data cache 76 allows the display controller 88 and the VLIW vector processor 74 to be active at the same time. If the actions of these two components are never expected to occur at the same time, a total of nine cache groups is sufficient. The CPU uses cache group 0, the VLIW vector processor 74 and the display controller 88 share the remaining 8 cache groups, and can define a cache group that responds to a specific request with only 3 bits (not 4 bits). it can.

JTAG interface 85
A standard JTAG (Joint Test Group) interface is housed in the ACP 31 for testing purposes. Due to chip complexity, various test techniques are required, including BIST (built-in self test) and functional block isolation. An overhead of 10% of the chip area is assumed for all chip test circuits. Test circuits are outside the scope of this document.

Serial interface USB serial port interface 52
This is a standard USB serial port and is connected to the low speed bus of the internal chip so that the CPU can control it.

Keyboard interface 65
This is a standard low speed serial port and is connected to the internal chip low speed bus so that the CPU can control it. The keyboard interface is designed to optionally connect to a keyboard to allow simple data entry to customize the print.

Authentication chip serial interface 64
These are two standard low speed serial ports that are connected to the internal chip low speed bus so that the CPU can control them. The reason for having two ports is to use separate lines to connect to both the on-camera authentication chip and the print roll authentication chip. If only one line is used, the manufacturer of the imitation print roll may design a chip that causes the camera to malfunction so that it uses the code generated by the camera's authentication chip rather than generating an authentication code. it can.

Parallel interface 67
The parallel interface connects the ACP 31 to individual static electrical signals. The CPU can control each of these wirings as a memory mapped I / O via a low speed bus. The following table lists the wiring to the parallel interface.

ＶＬＩＷ入力及び出力ＦＩＦＯ７８、７９
ＶＬＩＷ入出力ＦＩＦＯは、プロセスとＶＬＩＷベクトルプロセッサ７４との間を接続するため使用される８ビット幅のＦＩＦＯである。両方のＦＩＦＯはＶＬＩＷベクトルプロセッサ７４によって制御されるが、ＣＰＵによってクリアしたり、（例えば、状態について）問い合わせをしたりすることができる。

VLIW input and output FIFO 78, 79
The VLIW input / output FIFO is an 8-bit wide FIFO used to connect between the process and the VLIW vector processor 74. Both FIFOs are controlled by the VLIW vector processor 74, but can be cleared or queried (eg, for status) by the CPU.

VLIW input FIFO78
The client writes 8-bit data to the VLIW input FIFO 78 to cause the VLIW vector processor 74 to process the data. The client includes an image sensor interface, an Artcard interface, and a CPU. Each of these processes is freed from processing by simply writing data to the FIFO and leaving the VLIW vector processor 74 to do all the heavy work. One example using the use of a VLIW input FIFO 78 by a client is the image sensor interface (ISI 83). The image sensor interface 83 acquires data from the image sensor and writes the data to the FIFO. The VLIW process gets data from the FIFO, converts the data to the correct image data format, and writes it to DRAM. As a result, the image sensor interface 83 is greatly simplified.

VLIW output FIFO79
The VLIW vector processor 74 writes 8-bit data to the VLIW output FIFO 79, and the client can read the data with the VLIW output FIFO. The client includes a printhead interface and a CPU. Both of these clients are freed from processing by simply reading the processed data from the FIFO and leaving the VLIW vector processor 74 to do all the heavy work. The CPU can be interrupted whenever data is contained in the VLIW output FIFO 79, and can process the data only when the data is available, instead of constantly polling the FIFO. One example using the use of the VLIW output FIFO 79 by the client is the printhead interface (PHI 62). The VLIW process acquires an image, rotates the image in the correct orientation, performs color conversion, and dithers the resulting image according to printhead requirements. The print head interface 62 only needs to read the dithered 8-bit data after formatting from the VLIW output FIFO 79 and pass the 8-bit data to the print head outside the ACP 31. As a result, the printhead interface 62 is greatly simplified.

VLIW vector processor 74
In order to realize high processing demands of Artcam, the ACP 31 includes a VLIW (very long instruction word) vector processor. The VLIW processor is a set of four identical processing units (for example, PU178) connected by a crossbar switch 183 and operating in parallel. Each processing unit, eg, PU 178, can perform four 8-bit multiplications, eight 8-bit additions, three 32-bit additions, I / O processing, and various logical operations in each cycle. . A processing unit, eg, PU 178, is microcoded and each comprises two address generators 189, 190, fully using the available cycles for data processing. Four processing units, eg, PU 178, are typically synchronized to implement a closely interacting VLIW processor. With a 200 MHz clock, the VLIW vector processor 74 operates at 12 Gop (12 billion operations per second). The instructions are tailored for image processing functions such as warping, artistic brushing, complex composite lighting, color conversion, image filtering, and compositing. These are two orders of magnitude faster than desktop computers.

As shown in detail in FIG. 3 (a), the VLIW vector processor 74 is four processing units, eg, PU178, connected by a crossbar switch 183, and each processing unit, eg, PU178, is a crossbar. Two inputs are supplied to the switch 183, and two outputs from the crossbar switch 183 are acquired. The two common registers form a control and synchronization mechanism for the processing unit, eg, PU 178. Eight cache buses 182 allow connection to DRAM via data cache 76, and the two buses lead to each processing unit, eg, PU 178 (one bus for each I / O address generator) ). Each processing unit, for example, PU 178, has an ALU (arithmetic logic unit) 188 (which stores a number of registers and some arithmetic logic for processing data), some microcode RAM 196, and the outside world (others). Wiring to ALU (including ALU). The local PU state machine runs in microcode and by this means the processing unit, eg PU 178, is controlled. Each processing unit, eg, PU 178, includes two I / O address generators 189, 190 that control the data flow between DRAM (via data cache 76) and ALU 188 (via input FIFO and output FIFO). The address generator can read and write data (especially images of various formats), as well as tables and simulated FIFOs in DRAM. The format is customizable under software control but is not microcoded. Data acquired from the data cache 76 is transferred to the ALU 188 via a 16-bit input FIFO. Output data is written to a 16-bit wide output FIFO and from there to the data cache 76. Finally, all processing units, eg, PU 178, share a single 8-bit wide VLIW input FIFO 78 and a single 8-bit wide VLIW output FIFO 79. With the low speed data bus connection, the CPU reads and writes the registers of the processing unit, eg, PU178, updates the microcode, and reads and writes the common registers shared by all the processing units of the VLIW vector processor 74, eg, PU178. Is possible. Referring now to FIG. 4, more details within a single processing unit, eg, PU 178, are shown, and the components and control signals are detailed below.

Each microcode processing unit, eg, PU 178, includes a microcode RAM 196 that holds a program for that particular processing unit, eg, PU 178. Rather than storing the microcode in the ROM, the microcode is stored in the RAM, and the CPU plays a role of uploading the microcode. Since this is the same space on the chip, this trade-off reduces the maximum size of one function to the size of RAM 196, but the number of functions written to microcode is unlimited. Functions implemented using microcode include Vark acceleration, Artcard reading, and printing. The VLIW vector processor 74 scheme has several advantages in the ACP31 case:
Hardware design complexity is reduced;
Hardware risk is reduced by reducing complexity;
Hardware design time does not depend on all Vark functions built into dedicated silicon;
The overall space on the chip is reduced (since many processes can be incorporated as microcode);
Functions can be added to Vark (by microcode) without affecting hardware design time.

Size and Content The microcode RAM 196 loaded by each processing unit, eg, the CPU controlling the PU 178, is 128 words and each word is 96 bits wide. A summary of the microcode sizes for controlling the various units of the processing unit, eg, PU 178, is listed in the table below.

１２８個の命令ワードによって、処理ユニット、例えば、ＰＵ１７８毎の全マイクロコードＲＡＭ１９６は、１２２８８ビット、即ち、正確に１．５ＫＢである。ＶＬＩＷベクトルプロセッサ７４は、４個の同一の処理ユニット、例えば、ＰＵ１７８により構成されるので、これは、６１４４バイト、即ち、正確に６ＫＢと一致する。マイクロコードワード内の一部のビットは、制御ビットとしてそのまま使用され、一方、他のビットはデコードされる。マイクロコードワードのビットの各々の解釈を詳細に記述する種々のユニット説明を参照せよ。

With 128 instruction words, the total microcode RAM 196 per processing unit, eg, PU 178, is 12288 bits, or exactly 1.5 KB. Since the VLIW vector processor 74 is composed of four identical processing units, eg, PU 178, this corresponds to 6144 bytes, ie exactly 6 KB. Some bits in the microcode word are used as they are as control bits, while other bits are decoded. See the various unit descriptions that describe in detail the interpretation of each bit of the microcode word.

Synchronization between processing units, eg, PU 178 Each processing unit, eg, PU 178, includes a 4-bit synchronization register 197. It is a mask that is used to determine the processing units that work together, eg, PU 178, and a bit is set for each of the corresponding processing units, eg, PU 178, that function as a single process. For example, if all processing units, eg, PU 178, function as a single process, each of the four synchronization registers 197 will all have 4 bits set. If there are two synchronization processes per two processing units, eg, PU 178, two processing units, eg, PU 178, are set with 2 bits in the synchronization register (corresponding to the PU itself), The other two processing units are set with another two bits in the synchronization register (corresponding to the PU itself):
The synchronization register 197 is used in two basic ways:
Stop and start a given synchronized process;
Pause execution within a process.

The stop and start process CPU is responsible for loading into the microcode RAM 196 and loading the execution address (usually 0) of the first instruction. When the CPU starts executing microcode, the microcode begins at the specified address.

The microcode is executed only when all the bits of the synchronization register 197 are also set in the common synchronization register 197. Thus, the CPU sets up all processing units, eg, PU 178, and then starts or stops the process with a single write to the common synchronization register 197.

This synchronization scheme allows multiple processes to run asynchronously on a processing unit, eg, PU 178, and is stopped and started as a process at the same time rather than as a single processing unit, eg, PU 178. ,
Suspend execution in process In a given cycle, a processing unit, eg, PU 178, is required for reading from or writing to the FIFO (based on the operation code of the current microcode instruction). . If the FIFO is empty on a read request or full on a write request, the FIFO request cannot be completed. The processing unit, eg, PU 178, therefore asserts the process suspension control signal 198. Process suspension signals from all processing units, eg, PU 178, are fed back to all processing units, eg, PU 178. The synchronization register 197 is ANDed with the four process pause bits, and if the result is non-zero, the write enable of the processing unit, eg, the PU 178 register, or the FIFO strobe is not set. Therefore, there is no processing unit whose own register or FIFO was updated during the cycle in a processing unit that could not finish the task, for example, a processing unit that forms the same process group as PU 178, for example, PU 178. . This simple technique keeps a given process group in sync. In each subsequent cycle, the processing unit, eg, PU 178 state machine, will attempt to re-execute this microcode instruction at the same address and will continue to do so until successful. Of course, the common synchronization register 197 can be written by the CPU to stop the entire process if necessary. This synchronization scheme allows any processing unit, eg, a combination of PU 178, to work together, with each group regarding pauses due to not being ready to read and write data. Only affect the processing units working together.

During each cycle of control and branching, each of the four basic input and calculation units (read, adder / logic, multiplication / interpolation, and barrel shifter) in the processing unit, eg, ALU 188 of PU178, It generates two status bits, a zero flag and a negative flag that indicate zero or negative. During each cycle, one of these four status bits is selected by a microcode instruction output from a processing unit, eg, PU178. Four status bits (one for a processing unit, eg, ALU 188 of PU 178) are combined into the 4-bit common status register 200. During the next cycle, the microcode program of each processing unit, eg, PU178, selects one of the bits from the common status register 200 and branches to another microcode address depending on the value of the status bit. Is possible.

The status bit each processing unit, eg, ALU 188 of PU 178, includes a number of input and calculation units. Each unit generates two status bits, a negative flag and a zero flag. One of these status bits is output from the processing unit, eg, PU 178, when the special unit asserts a value on the 1-bit 3-state status bit bus. One status bit is output from the processing unit, eg, PU 178, and then combined with the status bit of another processing unit, eg, PU 178, to update the common status register 200. The microcode that determines the output status bits takes the following form:

ＡＬＵ１８８内で、２ビット選択プロセッサブロック値は、４個の１ビットイネーブルビットにデコードされ、異なるイネーブルビットが各プロセッサユニットブロックへ送信される。状態選択ビット（零又は負を選択する）は、状態ビットバスへ出力されるべきビットを決定するため全てのユニットへ渡される。

Within ALU 188, the 2-bit selected processor block value is decoded into four 1-bit enable bits and different enable bits are sent to each processor unit block. The status selection bit (selecting zero or negative) is passed to all units to determine the bit to be output to the status bit bus.

Each branch processing unit in microcode, eg, PU 178, includes a 7-bit program counter (PC) that holds the current microcode address being executed. Normal program execution is linear, i.e., going from address N in one cycle to address N + 1 in the next cycle. However, from cycle to cycle, the microcode program has the ability to branch to a different location or to test and branch the status bits from the common status register 200. The microcode that determines the next execution address takes the following form:

ＡＬＵ１８８
図５はＡＬＵ１８８を詳細に示す図である。ＡＬＵ１８８内部には、多数の専用処理ブロックが存在し、マイクロコードプログラムによって制御される。専用処理ブロックには、
入力ＦＩＦＯからのデータを受け取る読み出しブロック２０２と、
出力ＦＩＦＯを介してデータを送り出す書き込みブロック２０３と、
加算及び減算と、比較及び論理演算のためのアダー／ロジックブロック２０４と、
乗算タイプの補間及び乗算／累算のための乗算／補間ブロック２０５と、
要求に応じてデータをシフトするバレルシフトブロック２０６と、
外部クロスバースイッチ１８３からのデータを受け取る入力ブロック２０７と、
外部クロスバースイッチ１８３へデータを送り出す出力ブロック２０８と、
一次記憶装置にデータを保持するレジスタブロック２１５と、
が含まれる。

ALU188
FIG. 5 shows the ALU 188 in detail. A large number of dedicated processing blocks exist within the ALU 188 and are controlled by a microcode program. In the dedicated processing block,
A read block 202 for receiving data from the input FIFO;
A write block 203 for sending data through an output FIFO;
An adder / logic block 204 for addition and subtraction, comparison and logic operations;
A multiplication / interpolation block 205 for multiplication type interpolation and multiplication / accumulation;
A barrel shift block 206 for shifting data on demand;
An input block 207 for receiving data from the external crossbar switch 183;
An output block 208 for sending data to the external crossbar switch 183;
A register block 215 for holding data in a primary storage device;
Is included.

Four dedicated 32-bit registers hold the results of the four main processing blocks:
M register 209 holds the result of the multiply / interpolate block;
L register 209 holds the result of the adder / logic block;
S register 209 holds the result of the barrel shifter block;
The R register 209 holds the result of the read block 202.

In addition, there are two internal crossbar switches 213 and 214 for data transfer. The various processing blocks are further expanded in subsequent sections with the microcode definitions belonging to each block. The microcode is decoded in the block to supply control signals to various internal units.

Data Transfer Between Processing Units, eg, PU 178 Each processing unit, eg, PU 178, can exchange data via an external crossbar. A processing unit, eg, PU 178, takes two inputs and outputs two values to the external crossbar. In this way, two operands for processing can be obtained in a single cycle, but in practice they are not used in the operation until the next cycle.

Input 207
This block is shown in FIG. 6 and includes two registers In1 and In2 that receive data from the external crossbar. The registers can be loaded every cycle or can remain intact. A selection bit for selecting from 8 inputs is output to the external crossbar switch 183. Microcode takes the form:

出力２０８
入力は出力２０８によって補完される。出力ブロックは図７に詳細に示されている。出力は、Ｏｕｔ₁及びＯｕｔ₂の２個のレジスタを含み、両方のレジスタは、他の処理ユニット、例えば、ＰＵ１７８によって使用するためサイクル毎に外部クロスバーへ出力される。また、書き込みユニットは、Ｏｕｔ₁とＯｕｔ₂のうちの一方を、ＡＬＵ１８８に取り付けられた出力ＦＩＦＯの一つに書き込むことが可能である。最終的に、両方のレジスタは、第１クロスバー２１３への入力として利用可能であり、第１クロスバー２１３は、そのレジスタ値がＡＬＵ１８８内の他のユニットへの入力として利用できるようにする。各サイクルで、２個のレジスタのうち何れか一方は、マイクロコード選択に応じて更新することができる。指定されたレジスタにロードされたデータは、（第１クロスバー２１３から選択された）Ｄ₀−Ｄ₃のうちの一つ、（第２クロスバー２１４から選択された）Ｍ、Ｌ、Ｓ及びＲのうちの一つ、２個のプログラマブル定数のうちの一つ、又は固定値０若しくは１である。出力用のマイクロコードは以下の形式をとる。

Output 208
Input is complemented by output 208. The output block is shown in detail in FIG. The output includes two registers, Out ₁ and Out ₂ , both of which are output to the external crossbar every cycle for use by other processing units, eg, PU 178. The writing unit can write _one of Out ₁ and Out ₂ to one of the output FIFOs attached to the ALU 188. Eventually, both registers are available as inputs to the first crossbar 213, which makes the register values available as inputs to other units in the ALU 188. In each cycle, either one of the two registers can be updated in response to microcode selection. The data loaded into the designated register is one of D ₀ -D ₃ (selected from the first crossbar 213), M, L, S and (selected from the second crossbar 214). One of R, one of two programmable constants, or a fixed value of 0 or 1. The output microcode takes the following form:

ＡＬＵ１８８内のローカルレジスタ及びデータ転送
前述の通り、ＡＬＵ１８８は、以下の４個のブロックの結果を保持するため、４個の専用３２ビットレジスタを含む：
Ｍレジスタ２０９は乗算／補間ブロックの結果を保持する；
Ｌレジスタ２０９はアダー／ロジックブロックの結果を保持する；
Ｓレジスタ２０９はバレルシフタブロックの結果を保持する；
Ｒレジスタ２０９は読み出しブロック２０２の結果を保持する。

Local Registers and Data Transfer in ALU 188 As described above, ALU 188 includes four dedicated 32-bit registers to hold the results of the following four blocks:
M register 209 holds the result of the multiply / interpolate block;
L register 209 holds the result of the adder / logic block;
S register 209 holds the result of the barrel shifter block;
The R register 209 holds the result of the read block 202.

The CPU has direct access to these registers and other units can select them as inputs via the second crossbar 214. In some cases, the operation needs to be delayed by one cycle or more. The register block includes four 32-bit registers D ₀ -D ₃ and holds temporary variables being processed. Each cycle is updatable one of the registers, while all registers, other unit (likewise _includes an In 1, an In _2, Out ₁ and Out ₂₎ the first cross bar 213 Is output for use through. The CPU directly accesses these registers. The data loaded into the designated register is one of D ₀ -D ₃ (selected from the first crossbar 213), M, L, S and (selected from the second crossbar 214). One of R, one of two programmable constants, or a fixed value of 0 or 1. Register block 215 is shown in detail in FIG. The microcode for registers takes the form:

第１クロスバー２１３
第１クロスバー２１３は図９に詳細に示されている。第１クロスバー２１３は、入力Ｉｎ_１、Ｉｎ₂、Ｏｕｔ₁、Ｏｕｔ₂、Ｄ₀−Ｄ₃から選択するため使用される。７個の出力が第１クロスバー２１３から発生され、３個が乗算／補間ユニットへ、２個がアダーユニットへ、１個がレジスタユニットへ、１個が出力ユニットへ向けられる。第１クロスバー２１３用の制御信号は、クロスバー入力を使用する様々なユニットから到来する。第１クロスバー２１３のため分離された特定のマイクロコードは存在しない。

First crossbar 213
The first crossbar 213 is shown in detail in FIG. First cross bar 213 is used to select the input _{In 1, In 2, Out 1} , Out 2, D 0 -D 3. Seven outputs are generated from the first crossbar 213, three for the multiplication / interpolation unit, two for the adder unit, one for the register unit and one for the output unit. The control signal for the first crossbar 213 comes from various units that use the crossbar input. There is no specific microcode separated for the first crossbar 213.

Second crossbar 214
The second crossbar 214 is shown in detail in FIG. The second crossbar 214 is used to select from the general purpose ALU 188 registers M, L, S and R. Six outputs are generated from the first crossbar 213, two to the multiply / interpolate unit, two to the adder unit, one to the register unit and one to the output unit. The control signal for the second crossbar 214 comes from various units that use the crossbar input. There is no specific microcode separated for the second crossbar 214.

Data Transfer Between Processing Units such as PU 178 and DRAM or External Process Referring to FIG. 4, processing units such as PU 178 directly share data with each other via an external crossbar. The processing unit also transfers data to and from external processes and DRAMs. Each processing unit, eg, PU 178, includes two I / O address generators 189 and 190 to transfer data to and from the DRAM. The processing unit, eg, PU 178, sends data to the DRAM via an I / O address generator output FIFO, eg, 186, or receives data from the DRAM via the I / O address generator input FIFO 187. . These FIFOs are localized in the processing unit, eg, PU178. There is also a mechanism for sending and receiving data to and from external processes in the form of a common VLIW input FIFO 78 and a common VLIW output FIFO 79 shared by all ALUs. The VLIW input and output FIFOs are only 8 bits wide and are used for printing, Artcard reading, and data transfer to the CPU. The local input and output FIFO is 16 bits wide.

Read The read process block 202 of FIG. 5 is responsible for updating the R register 209 of the ALU 188, which represents the external input data to the VLIW microcoding process. For each cycle, the read unit can read from either the common VLIW input FIFO 78 (8 bits) or two local input FIFOs (16 bits). A 32-bit value is generated and all or part of the data is transferred to the R register 209. The process is shown in FIG. Read microcodes are listed in the table below. The interpretation of some bit patterns is deliberately selected to aid in decoding.

書き込み
書き込みプロセスブロックは、各サイクルに、共通ＶＬＩＷ出力ＦＩＦＯ７９、又は２個のローカル出力ＦＩＦＯの一方の何れかへの書き込みを行うことができる。所与のサイクル中に書き込まれるＦＩＦＯは１個しかないので、１個の１６ビット値が全てのＦＩＦＯへ出力され、下位８ビットがＶＬＩＷ出力ＦＩＦＯ７９へ達することに注意する必要がある。マイクロコードは、どのＦＩＦＯがその値を受け付けるかを制御する。データ選択のプロセスは図１２により詳細に示されている。ソース値Ｏｕｔ₁及びＯｕｔ₂は出力ブロックから到来する。それらは、単に２個のレジスタである。書き込み用のマイクロコードは以下の形式をとる。

The write-write process block can write to either the common VLIW output FIFO 79 or one of the two local output FIFOs in each cycle. Note that since there is only one FIFO written during a given cycle, one 16-bit value is output to all FIFOs and the lower 8 bits reach the VLIW output FIFO 79. The microcode controls which FIFO accepts the value. The process of data selection is shown in more detail in FIG. Source values Out ₁ and Out ₂ come from the output block. They are simply two registers. The microcode for writing takes the following form:

計算ブロック
各ＡＬＵ１８８は、２個の計算プロセスブロック、即ち、アダー／ロジックプロセスブロック２０４と、乗算／補間プロセスブロック２０５と、を含む。更に、これらの計算ブロックを補助するためバレルシフタブロックが設けられる。レジスタブロック２１５からのレジスタは、パイプライン演算中に一時記憶装置として使用することも可能である。

Calculation Block Each ALU 188 includes two calculation process blocks: an adder / logic process block 204 and a multiplication / interpolation process block 205. In addition, a barrel shifter block is provided to assist these calculation blocks. Registers from register block 215 can also be used as temporary storage during pipeline operations.

The barrel shifter barrel shifter process block 206 is shown in detail in FIG. 13 and its input is derived from the output of the adder / logic, or the output of the multiply / interpolate process block, or the previous cycle of those blocks (ALU registers L and M). can get. The selected 32 bits are barrel-shifted by an arbitrary number of bits in any direction (with the sign extended as necessary) and output to the S register 209 of the ALU 188. The microcode for the barrel shift process block is listed in the table below. It should be noted that some bit pattern interpretations are intentionally chosen to aid in decoding.

アダー／ロジック２０４
アダー／ロジックプロセスブロックは、図１４により詳細に示され、簡単な３２ビット加算／減算、比較、及び論理演算用に設計されている。単一サイクルで、１回の加算、比較、又は論理演算を実行することができ、その結果はＡＬＵ１８８のＬレジスタ２０９に保存される。二つの主オペランドＡ及びＢが存在し、これらは、二つのクロスバーの何れか、又は４個の定数レジスタから選択される。一方のクロスバーの選択によって、前のサイクルの算術演算の結果を使用することが可能になり、もう一方は、このＡＬＵ又は他のＡＬＵ１８８によって前に計算されたオペランドへアクセスする。ＣＰＵは、４個の定数（Ｋ₁−Ｋ₄）にアクセスする唯一のユニットである。例えば、（Ａ＋Ｂ）×４のような演算が要求される場合、アダーからの直接出力は、バレルシフタへの入力として使用可能であり、先にＬレジスタ２０９にラッチしなくても２個分だけ左シフトさせることができる。アダーからの出力は、乗算−累算演算のため乗算ユニットで利用することができる。アダー／ロジックプロセスブロック用のマイクロコードは以下の表に掲載されている。一部のビットパターンの解釈はデコーディングを補助するため意図的に選択される。
アダー／ロジックユニット用のマイクロコードビット解釈

Adder / Logic 204
The adder / logic process block is shown in more detail in FIG. 14 and is designed for simple 32-bit addition / subtraction, comparison, and logic operations. A single addition, comparison, or logical operation can be performed in a single cycle, and the result is stored in the ALU 188 L register 209. There are two main operands A and B, which are selected from either of the two crossbars or from four constant registers. Selection of one crossbar makes it possible to use the result of the previous cycle's arithmetic operation, while the other accesses an operand previously calculated by this ALU or another ALU 188. The CPU is the only unit that accesses the _four constants (K ₁ -K ₄ ). For example, when an operation such as (A + B) × 4 is required, the direct output from the adder can be used as an input to the barrel shifter, and left by two without latching in the L register 209 first. Can be shifted. The output from the adder can be used by the multiply unit for multiply-accumulate operations. The microcode for the adder / logic process block is listed in the table below. The interpretation of some bit patterns is deliberately selected to aid in decoding.
Microcode bit interpretation for adder / logic units

乗算／補間２０５
乗算／補間プロセスブロックは、図１５に詳細に示され、４個の８×８形補間ユニットのセットであり、４個の８×８形補間ユニットは、１サイクル毎に４回の別個の８×８形補間を実行する能力を備えているか、又は１回の１６×１６形乗算を実行するため組み合わせることができる。これは、単一サイクル中に最大で４回のリニア補間、１回のバイリニア補間、又はトライリニア補間の半分を実行可能であることを示す。補間又は乗算の結果はＡＬＵ１８８のＭレジスタ２０９に保存される。二つの主オペランドＡ及びＢは、ＡＬＵ１８８の汎用レジスタの何れかから選択されるか、又は乗算／補間プロセスブロックの内部にある４個のプログラマブル定数から選択される。各補間ブロックは、簡単な８ビット補間器［結果＝Ａ＋（Ｂ−Ａ）ｆ］として機能するか、又は簡単な８×８形乗算［結果＝Ａ＊Ｂ］として機能する。演算が補間である場合、Ａ及びＢは、４個の８ビット数値Ａ₀からＡ₃（Ａ0は下位バイトである）、及びＢ₀からＢ₃として取り扱われる。ＡＧＥＮ、ＢＧＥＮ及びＦＧＥＮは、実行される演算に適合するように補間ユニットへの入力を並べる役目を果たす。例えば、バイリニア補間を実行するため、４個の値の各々は、異なる係数によって乗算され、その結果が加算され、一方、１６×１６形の乗算は、係数を０にする必要がある。アダー／ロジックプロセスブロック用のマイクロコードは、以下の表に掲載されている。尚、一部のビットパターンの解釈はデコーディングを補助するため意図的に選択される。

Multiplication / interpolation 205
The multiply / interpolate process block is shown in detail in FIG. 15 and is a set of four 8 × 8 interpolating units, where four 8 × 8 interpolating units are four separate 8 per cycle. It has the ability to perform x8 interpolation or can be combined to perform a single 16x16 multiplication. This indicates that a maximum of four linear interpolations, one bilinear interpolation, or half of trilinear interpolation can be performed in a single cycle. The result of interpolation or multiplication is stored in the M register 209 of the ALU 188. The two main operands A and B are selected from any of the general purpose registers of ALU 188, or selected from four programmable constants internal to the multiply / interpolate process block. Each interpolation block functions as a simple 8-bit interpolator [Result = A + (B−A) f] or as a simple 8 × 8 multiplication [Result = A * B]. If the operation is interpolation, A and B are treated as four 8-bit numbers A ₀ to A ₃ (A ₀ is the lower byte) and B ₀ to B ₃ . AGEN, BGEN and FGEN serve to line up the inputs to the interpolation unit to suit the operation being performed. For example, to perform bilinear interpolation, each of the four values is multiplied by a different coefficient and the results are added, while a 16 × 16 type multiplication requires the coefficient to be zero. The microcode for the adder / logic process block is listed in the table below. Note that the interpretation of some bit patterns is intentionally selected to assist in decoding.

同じ４ビットがＶ及びｆの選択のために使用されるが、Ｖの直前の４個の選択肢は、一般的にｆ値として意味をなさない。１又は０の係数を用いる補間は無意味であり、前の乗算若しくは現在の結果がｆに対する有意な値である可能性は少ない。

The same 4 bits are used for the selection of V and f, but the 4 choices immediately before V generally do not make sense as an f value. Interpolation using a coefficient of 1 or 0 is meaningless and the previous multiplication or current result is unlikely to be a significant value for f.

I / O address generators 189 and 190
The I / O address generator is shown in detail in FIG. VLIW processes do not access DRAM directly. Access is through two I / O address generators 189, 190, each I / O address generator including a unique input FIFO and output FIFO. A processing unit, eg, PU 178, reads data from one of the two local input FIFOs and writes data to one of the two local output FIFOs. Each I / O address generator is responsible for reading data from the DRAM and placing the data in the input FIFO, where it can be read by a processing unit, eg, PU 178. Each I / O address generator is responsible for taking data (put in by a processing unit, eg, PU 178) from its output FIFO and writing it to DRAM. The I / O address generator is a state machine that generates and controls an address for acquiring and storing DRAM data via the data cache 76. It can be customized under CPU software control but cannot be microcoded. The address generator
An image iterator used to repeat (read, write, or both) from the beginning to the end of an image pixel in a wide variety of ways;
Table I / O used to randomly access image pixels, table data, and simulate a DRAM FIFO;
Addresses that are roughly divided into two.

Each of the I / O address generators 189 and 190 has its own bus wiring to the data cache 76, and there are two bus wirings for one processing unit, eg, PU 178, throughout the VLIW vector processor 74. There are a total of 8 buses. The data cache 76 can serve for 4 of a maximum of 8 requests from 4 processing units, eg, PU 178, per cycle. The input FIFO and the output FIFO are FIFOs having a depth of 8 entries and a width of 16 bits. A wide variety of address generation (image iterator and table I / O) is described in a later section.

The register I / O address generator has a set of registers that are used to control address generation. The addressing mode also determines how data is formatted and sent to the local input FIFO and how the data is interpreted from the output FIFO. The CPU can access the registers of the I / O address generator via the low speed bus. The first set of registers defines housekeeping parameters for the I / O generator.

状態レジスタは以下の値をとる。

The status register takes the following values:

キャッシュ処理
複数のレジスタがキャッシュ処理機構を制御するため使用され、入力、出力等に使用するキャッシュグループを指定する。キャッシュグループのより詳細な情報についてはデータキャッシュ７６に関するセクションを参照のこと。

Cache processing Multiple registers are used to control the cache processing mechanism, and specify the cache group to be used for input, output, etc. See the section on data cache 76 for more information on cache groups.

画像イタレータ＝順次自動画素アクセス
ソフトウェア及びハードウェアアルゴリズム用の主な画像絵素アクセス方法は画像イタレータによる。画像イタレータは、画像チャンネル内の画素のキャッシュのアドレッシング及びアクセスの全てを実行し、それらのクライアントのため画素を読み出し、書き込み、或いは、読み書きする。読み出しイタレータは、そのクライアントのための特定の順序で画素を読み出し、書き込みイタレータはそのクライアントのための特定の順序で画素を書き込む。イタレータのクライアントは、ローカル入力ＦＩＦＯから画素を読み出し、又はローカル出力ＦＩＦＯを介して画素を書き込む。

Image Iterator = Sequential automatic pixel access software and main image pixel access method for hardware algorithm is image iterator. The image iterator performs all of the addressing and access of the pixels in the image channel and reads, writes, or reads and writes pixels for those clients. A read iterator reads pixels in a particular order for that client, and a write iterator writes pixels in a particular order for that client. The iterator client reads a pixel from the local input FIFO or writes a pixel through the local output FIFO.

The read image iterator reads through the image in a specific order and puts the pixel data into the local input FIFO. Each time the client reads a pixel from the input FIFO, the read iterator puts the next pixel from the image (via the data cache 76) into the FIFO.

The write image iterator writes the pixels in a specific order to write out the entire image. The client writes the pixel to the output buffer, which is then read by the write image iterator and written to the DRAM via the data cache 76.

Typically, a VLIW processor has an input coupled to a read iterator and an output coupled to a corresponding write iterator. From the perspective of a processing unit, eg, a microprogram of the PU 178, the FIFO is an efficient interface to the DRAM. The way in which the storage is actually performed (rather than the logical order of the data) is irrelevant. FIFOs are considered to be substantially limited in length, but in practice, FIFOs are finite in length, especially when multiple memory accesses compete, there is a delay in sorting and retrieving data . There are various image iterators that address the most common addressing requirements of image processing algorithms. In most cases, each read iterator has a corresponding write iterator. Various iterators are listed in the table below.

４ビットアドレスモードレジスタがイタレータタイプを決定するため使用される。

A 4-bit address mode register is used to determine the iterator type.

アクセス専用レジスタは以下のように使用される。

The access-only register is used as follows.

フラグレジスタ（アクセス専用１）は、データの読み出し及び書き込みに影響を与える要因を決定する多数のフラグを含む。フラグレジスタは以下のように構成される。

The flag register (access only 1) includes a number of flags that determine factors that affect the reading and writing of data. The flag register is configured as follows.

読み出しイネーブル及び書き込みイネーブルに関する注意：
読み出しイネーブルがセットされたとき、Ｉ／Ｏアドレス生成器は読み出しイタレータとして動作するので、特定の順序で画像を読み出し、画素を入力ＦＩＦＯに収容する。

Notes on read enable and write enable:
When read enable is set, the I / O address generator operates as a read iterator so that images are read in a specific order and the pixels are accommodated in the input FIFO.

When write enable is set, the I / O address generator operates as a write iterator so that images are written in a specific order and pixels are extracted from the output FIFO.

When both read enable and write enable are set, the I / O address generator operates as a read and write iterator, reads pixels in the input FIFO, and writes pixels from the output FIFO. Pixels are written only after they are read out. That is, the write iterator does not advance beyond the read iterator. Care should be taken to ensure a balance between input processing and output processing by the VLIW microcode when this mode is used. It is also possible to specify different cache groups for reading and writing by loading different values into CashGroup1 and CacheGroup2.

Notes on PassX and PassY:
If both PassX and PassY are set, the Y ordinate is accommodated in the input FIFO before the X ordinate.

PassX and PassY are set only when the read enable bit is cleared. Rather than passing the ordinate to the address generator, the ordinate is received directly in the input FIFO. The ordinate advances when it is removed from the FIFO.

If the write enable bit is set, the VLIW program must reliably balance ordinate reading from the input FIFO with writing to the output FIFO. This is because writing occurs only up to the ordinate (see note above regarding read enable and write enable).
A note about loops:
When the loop bit is set, reading resumes at [Start Pixel, Start Row] once [Last Pixel, Last Row] is reached. This is ideal when processing structures such as convolution kernels or dither cell matrices that require repeated reading of data.

Looping with read enable and write enable set can be useful in an environment that maintains a single line history, but is effective only when the read is done before the write. If there is a FIFO effect (writing is done in length-constrained form before reading), an appropriate table I / O addressing mode is used rather than an image iterator.

The loop process in which only the write enable is set creates a write window for the previous N pixels. This is used with an asynchronous process that reads data from the window. The Artcard read algorithm uses this mode.

Sequential Read and Write Iterator FIG. 17 is an explanatory diagram of a pixel data format. The simplest image iterator is a sequential read iterator and a corresponding sequential write iterator. The sequential read iterator shows the pixels from the channel, one row at a time, from top to bottom, and within the same row, the pixels are shown from left to right. The padding byte is not shown to the client. This is most useful for algorithms that need to perform some sort of process on each pixel from the image, but don't care about the order of the processed pixels, or that require the data to be in this particular order. Is a point. Complementing the sequential read iterator is the sequential write iterator. The client writes the pixel to the output FIFO. The sequential write iterator writes out a valid image using appropriate cache processing and appropriate padding bytes. Each sequential iterator requires access to two cache lines. When reading, 32 pixels are shown from one cache line, but other cache lines can be loaded from memory. At the time of writing, 32 pixels are written to one cache line, while the other pixels are written to the memory. The process of performing the operation independently on each pixel of the image typically uses a sequential read iterator to obtain the pixel and a sequential write iterator to write the new pixel value to the corresponding location in the destination image. Will. Such a process is shown in FIG.

In most cases, the original and target images are different and are represented by two I / O address generators 189 and 190. However, it is also effective to match the original image with the target image. This is because a certain type of input image is read only once. In that case, the same iterator can be used for both input and output, and both the read enable register and the write enable register are set appropriately. For maximum efficiency, it is necessary to use two different cache groups for reading and writing. If the data is created by a VLIW process and is to be written by a sequential write iterator, the PassX and PassY flags are used to generate coordinates, which are then communicated to the input FIFO. The VLIW process can use these coordinates to properly generate output data.

Box Read Iterator The box read iterator is used to present the pixels in the most convenient order for performing operations such as general purpose filters and convolutions. This iterator presents the pixel values in a square box around the sequentially read out pixels. This box is limited so that the width in the X and Y directions is 1, 3, 5 or 7 pixels (the X box size and the Y box size are the same value, or the value of one dimension is 1. Yes, the other dimension value is 3, 5 or 7.) This process is shown in FIG.

Box offset: This dedicated register is used to determine subsampling with respect to which input pixel is used as the center of the box. The normal value is 1, which means that each pixel is used as the center of the box. A value of “2” is useful when reducing an image to 4: 1, such as when building an image pyramid. Using the pixel address from the above figure, the box is centered in the order of pixels 0, 2, 8, and 10. The box read iterator needs to access up to 14 (2 × 7) cache lines. While the pixels are presented from one of the seven lines, other cache lines can be loaded from memory.

Box write iterator There is no corresponding box write iterator. This is because pixel replication is required only at the time of input. The process of using a box read iterator for input will almost certainly be synchronized and will use a sequential write iterator for output. A good example is the convolver. In the convolver, N input pixels are read out to calculate one output pixel. The process flow is as shown in FIG. When using a box read iterator, the original and target images must not use the same memory. This is because subsequent image lines require the original values (not newly calculated).

Vertical Strip Read and Write Iterators Some examples require images to be written in the order of output pixels, but do not know the direction of input pixel coherence with respect to the output pixels. One example is rotation. If the image is rotated 90 degrees and the output image is processed horizontally, there is a complete loss of cache coherence. In contrast, processing the output image simultaneously processes the output image based on one cache line wide pixel (rather than proceeding to the pixel corresponding to the next cache line on the same line) When going to the line, cache coherence for the pixels of the input image is obtained. The same is true if there is a known “block” coherence (such as color coherence) in the input pixel, where the readout dominates the processing order and the writing to be synchronized must follow the same pixel order. Don't be. The order of pixels presented as input (vertical strip readout) or the expected order for output (vertical strip writing) is the same. The order is the same for pixels 0 to 31 on line 0, then pixels 9 to 31 on line 1, and so on for all lines of the image, then pixels 32 to 63 on line 0 followed by. In the last vertical strip, there may not be exactly 32 pixels. In this case, only the actual pixels of the image are presented or scheduled as input. This process is shown in FIG.

Processes that require only a vertical strip writing iterator typically comprise a method of mapping input pixel coordinates given output pixel coordinates. The process accesses the pixels of the input image according to this mapping, and the coherence is determined by providing sufficient cache lines on the “random access” reader for the input image. As shown in the process overview of FIG. 22, the coordinates are typically generated by setting the PassX and PassY flags of the vertical strip write iterator.

Although it is not important to pair the write iterator with the sequential read iterator or the box read iterator, the vertical strip write iterator makes a significant improvement in performance when there is an important mapping between the input and output coordinates.

It is important to pair the vertical strip read iterator with the vertical strip write iterator. In this case, if the input image and the output image are the same, the vertical strip read iterator and the vertical strip write iterator can be assigned to one ALU 188. If coordinates are required, additional iterators must be used with the PassX and PassY flags set. A vertical strip read / write iterator presents pixels to the input FIFO and receives output pixels from the output FIFO. Appropriate padding bytes are inserted when writing. Each input and output requires at least two cache lines for superior performance.

Table I / O addressing mode It is often necessary to look up values in a table (such as an image). The table I / O addressing mode provides this functionality and requires the client to put an index into the output FIFO. The I / O address generator holds the index, examines the data appropriately, and returns the examined value to the input FIFO for later processing by the VLIW client.

One-dimensional, two-dimensional and three-dimensional tables are supported, and certain modes are targeted for interpolation. In order to reduce the complexity of the VLIW on the client side, the index value is handled as a fixed-point number, and the access-only register defines the fixed point and determines the bit to be handled as the integer part of the index. The data format is a limited form of general-purpose image feature in that the pixel offset register is ignored, and the data is assumed to be contiguous in the row, 8 bits or 16 bits (1 byte) per data element. Or 2 bytes) is allowed. A 4-bit address mode register is used to determine the I / O type.

アクセス専用レジスタは次の通りである。

The access-only registers are as follows.

FractX、FractY及びFractZは、インデックスに基づいてアドレスを生成し、有意ビット及び整数／小数コンポーネントに関するインデックスのフォーマットを解釈するため使用される。種々のパラメータは、索引付けされたテーブルの次元数によって必要に応じて定められる。１次元テーブルは、FractXだけが必要であり、２次元テーブルは、FractX及びFractYを必要とする。各Fract値は、対応したインデックスの小数ビットの個数により構成される。例えば、Ｘインデックスは、５：３のフォーマットである。これは、整数が５ビットで、小数が３ビットであることを示す。したがって、FractXは３にセットされる。簡単な１次元ルックアップは、８：０のフォーマットであり、即ち、小数コンポーネントが全く無い。FractXは０になる。零オフセットは、３次元ルックアップの場合に限り必要になり、２種類の解釈を採用する。これは３次元テーブルルックアップのセクションに詳述されている。フラグレジスタ（アクセス専用１）は、データの読み取り（あるケースでは、書き込み）に影響を与える要因を決定するため使用される多数のフラグを含む。フラグレジスタは以下のように構成される。

FractX, FractY and FractZ are used to generate addresses based on the index and interpret the format of the index with respect to significant bits and integer / decimal components. Various parameters are defined as needed by the number of dimensions of the indexed table. A one-dimensional table requires only FractX, and a two-dimensional table requires FractX and FractY. Each Fract value is composed of the number of decimal bits of the corresponding index. For example, the X index has a 5: 3 format. This indicates that the integer is 5 bits and the decimal is 3 bits. Therefore, FractX is set to 3. A simple one-dimensional lookup is an 8: 0 format, i.e. there are no decimal components. FractX becomes 0. Zero offset is only required for 3D lookups and employs two types of interpretation. This is detailed in the 3D table lookup section. The flag register (access only 1) contains a number of flags that are used to determine factors that affect data reading (in some cases, writing). The flag register is configured as follows.

１次元直接ルックアップとＤＲＡＭのＦＩＦＯを除くと、全てのテーブルＩ／Ｏモードは、読み出しだけをサポートし、書き込みをサポートしていない。したがって、上記の二つのモードを除く全てのＩ／Ｏモードでは、読み出しイネーブルビットはセットされ、書き込みイネーブルビットはクリアされる。１次元直接ルックアップは、
読み出しイネーブルビットがセットされ、書き込みイネーブルビットがクリアされている読み出し専用と、
読み出しイネーブルビットがクリアされ、書き込みイネーブルビットがクリアされている書き込み専用と、
読み出しイネーブルビットと書き込みイネーブルビットの両方がセットされているリード・モディファイ・ライトと、
の３モードをサポートする。

Except for one-dimensional direct lookup and DRAM FIFO, all table I / O modes support only read and not write. Therefore, in all I / O modes except the above two modes, the read enable bit is set and the write enable bit is cleared. One-dimensional direct lookup is
Read only with read enable bit set and write enable bit cleared,
Read only with the read enable bit cleared and the write enable bit cleared
A read-modify-write with both the read enable bit and the write enable bit set;
3 modes are supported.

The different modes are described in the one-dimensional lookup section below. DRAM
FIFO mode is
Only one mode is supported, the write / read mode, in which both the read enable bit and the write enable bit are set.

This mode is described in the DRAM FIFO section below. The DataSize flag determines whether the size of each data element in the table is 8 bits or 16 bits. Only two data sizes are supported. 32-bit elements can be created in one of two ways, depending on process requirements:
Read from two 16-bit tables simultaneously and synthesize the results. This is convenient when timing is an issue, but has the disadvantage of wasting two I / O address generators 189 and 190, each 32-bit element being read by the CPU as a 32-bit entity. Can't;
Read from the 16-bit table twice and combine the results. This is convenient because only one lookup is used, but a different index must be generated and communicated to the lookup.

One-dimensional structure direct lookup Direct lookup is a simple indexing into a one-dimensional lookup table. The client sets the appropriate bit in the flag register, thereby allowing three access modes:
It is possible to select between read-only write-only read-modify-write.

The read-only client sends the fixed point index X to the output FIFO and the 8 or 16 bit value of the table [Int (X)] is returned to the input FIFO. The fractional component of index X is completely ignored. If the index is out of range, the edge repeat flag determines whether an edge pixel is returned or a constant pixel is returned. Address generation is as simple as:
If the data size indicates 8 bits, X is the FractX bit barrel shifted right and the result is added to the base address ImageStart of the table;
When the data size indicates 16 bits, X is a bit of FractX barrel-shifted to the right, and the result of 1-bit shift to the left (bit 0 becomes 0) is added to the base address ImageStart of the table .

The resulting 8 or 16 bit data value at the address is placed in the input FIFO. One cycle is required for address generation, and one cycle is also required for transferring the requested data from the cache to the output FIFO (assuming a cache hit). For example, assuming that each entry is 16 bits and the index of a 256-entry table having an 8-: 4 fixed-point format with an index of 12 bits is subtracted, FractX is 4 and the data size is 1. When the index is passed to the lookup, it shifts right by 4 bits, shifts the result left by 1 bit, and adds it to ImageStart.

The write-only client passes the fixed point index X to the output FIFO and then passes the 8 or 16 bit value to be written to the specified location in the table. A complete transfer requires at least two cycles, that is, one cycle for address generation and one cycle for data transfer from the FIFO to the DRAM. Any number of cycles can be provided between the time when the VLIW process accommodates the index in the FIFO and the time when the value to be written is accommodated in the FIFO. Address generation is performed in the same manner as in the read-only mode, but data is not read from that address, but data from the output FIFO is written to that address. If the address is outside the table range, the data is deleted from the FIFO and not written to the DRAM.

The read-modify-write client sends the fixed point index X to the output FIFO and the 8- or 16-bit value of the table [Int (X)] is returned to the input FIFO. The next value contained in the output FIFO is written to the table [Int (X)] and replaces the previously returned value. The general-purpose processing loop is a process of reading a value from a certain place, changing the value, and writing back the value. The overall time is
A cycle to generate an address from the index;
A cycle to return values from the table,
A cycle to change the value in some way,
A cycle to write changed values back to the table;
4 cycles.

There is no specific read / write mode in which the client presents a flag representing “read from X” or “write to X”. The client can simulate “Read from X” by writing the original value and “Write to X” by simply ignoring the returned value. However, the use of such a mode is not facilitated. This is because each operation uses at least three cycles (modification is not required) and uses two data accesses instead of one access as performed by a particular read and write mode. .

Interpolation table This is the same as a direct lookup in read mode, except that for a given fixed point index X, two values are returned instead of one. The returned values are table [Int (X)] and table [Int (X) +1]. If any index is out of range, the edge repeat flag determines whether an edge pixel is returned or a constant pixel is returned. Address generation depends on whether it is 8-bit data or 16-bit data, except that the second address is simply (first address + 1) or (first address + 2). Same as up. Although it takes two cycles to transfer the requested data to the output FIFO (assuming a cache hit), a single 16-bit fetch actually returns two 8-bit values from the cache to the address generator.

DRAM FIFO
A special case of a read / write one-dimensional table is a DRAM FIFO. It is often necessary to simulate a fixed length FIFO using DRAM and associated cache. The DRAM FIFO allows the client to write to the output FIFO as if it were one end of the FIFO without explicitly indexing the table, and whether it is the other end of the same logical FIFO Read from the input FIFO as follows. Two counters track the input and output positions of the simulated FIFO and cache it in DRAM as needed. The client needs to see both the read enable bit and the write enable bit in the flag register.

An example of the usage of a DRAM FIFO is to keep a single line history of a certain value. The initial history is written before processing begins. When the general process completes a line, the value of the previous line is obtained from the FIFO, and the value of this line is entered into the FIFO (this line becomes the previous line when processing the next line). As long as the input and output coincide on average with each other, the output FIFO is always full. As a result, this type of FIFO is efficient with no access delay (however, when the total length of the FIFO is very short, for example, 3 or 4 bytes is excluded, in which case the purpose of the FIFO is Will lose.)

Two-dimensional table direct lookup Two-dimensional direct lookup is not supported. Since all cases of two-dimensional lookups are scheduled to be accessed for bilinear interpolation, special bilinear lookups have been realized.

Bilinear Lookup This type of lookup is necessary for bilinear interpolation of data from a two-dimensional table. Given fixed-point X and Y coordinates (when placed in the output FIFO in the order of Y, X), four values are returned after lookup. Their values are (in order):
Table [Int (X), Int (Y)]
Table [Int (X) +1, Int (Y)]
Table [Int (X), Int (Y) +1]
Table [Int (X) +1, Int (Y) +1]
It is.

The order of the values returned gives the best cache coherence. If the data is 8 bits, two values are returned every cycle between two cycles and the lower byte is the first data element. If the data is 16 bits, one entry per cycle and four values are returned in four cycles. Address generation takes two cycles. In the first cycle, the index (Y) barrel shifted right by FractY bits is multiplied by RowOffset and the result is added to ImageStart. The second cycle shifts the X index to the right by FractX bits and either the result (for 8-bit data) or the result (for 16-bit data) shifted left by 1 bit is the first It is added to the result from the cycle. As a result, the address Adr becomes Table [Int (X),
Int (Y)] is the address:
Adr = ImageStart
+ ShiftRight (Y, FractY) * RowOffset
+ ShiftRight (X, FractX)
A copy of Adr is kept in AdrOld to fetch subsequent entries:
If the data is 8 bits, the timing is 2 cycles of address generation, followed by 2 cycles where data is returned (2 table entries per cycle);
If the data is 16 bits, the timing is 2 cycles of address generation, followed by 4 cycles where data is returned (1 entry per cycle).

The following two tables show address calculation methods for 8-bit data size and 16-bit data size.

両方のケースにおいて、アドレス生成の第１サイクルは、ＸインデックスのＦＩＦＯへの挿入と重なってもよく、実効タイミングは、アドレス生成のための１サイクル、及び返却データの４サイクルと同程度に低い。インデックスの生成がその結果よりも２ステップ進んでいる場合、実効アドレス生成時間は無く、データは単に適切なレート（１セット毎に２サイクル又は４サイクル）で生成される。

In both cases, the first cycle of address generation may overlap with the insertion of the X index into the FIFO, and the effective timing is as low as one cycle for address generation and four cycles of return data. If the index generation is two steps ahead of the result, there is no effective address generation time and data is simply generated at the appropriate rate (2 or 4 cycles per set).

3D Lookup Direct Lookup Since all cases of 2D lookup are expected to be accessed for trilinear interpolation, two special trilinear lookups are implemented. The first is a simple lookup table and the second is for trilinear interpolation from the image pyramid.

Trilinear Lookup This type of lookup is useful for 3D data tables such as color conversion tables. Standard image parameters define a single XY plane of data, ie, each plane is composed of ImageHight rows, each row containing RowOffset bytes. In most environments, assuming a continuous plane, one XY plane is a sequence of ImageHight × RowOffset bytes. Rather than assume or calculate this offset, the software by the CPU must provide the offset in the form of a 12-bit ZOffset register. For this type of lookup, given three fixed-point indices in the order of Z, Y, X, eight values from the lookup table:
Table [Int (X), Int (Y), Int (Z)]
Table [Int (X) +1, Int (Y), Int (Z)]
Table [Int (X), Int (Y) +1, Int (Z)]
Table [Int (X) +1, Int (Y) +1, Int (Z)]
Table [Int (X), Int (Y), Int (Z) +1]
Table [Int (X) +1, Int (Y), Int (Z) +1]
Table [Int (X), Int (Y) +1, Int (Z) +1]
Table [Int (X) +1, Int (Y) +1, Int (Z) +1]
Are returned in order.

The order of the values returned gives the best cache coherence. If the data is 8 bits, two values are returned every cycle for 4 cycles, and the lower byte is the first data element. If the data is 16 bits, 4 values are returned in 8 cycles with one entry per cycle. Address generation takes 3 cycles. In the first cycle, the index (Z) barrel shifted right by FractZ bits is multiplied by 12 bits ZOffset and added to ImageStart. In the second cycle, the index (Y) barrel shifted right by FractY bits is multiplied by RowOffset and the result is added to the result of the previous cycle. The second cycle shifts the X index to the right by FractX bits and either the result (for 8-bit data) or the result (for 16-bit data) shifted left by 1 bit is the second. It is added to the result from the cycle. As a result, the address Adr becomes Table [Int (X),
Int (Y), Int (Z)] is the address:
Adr = ImageStart
+ (ShiftRight (Z, FractZ) * ZOffset)
+ (ShiftRight (Y, FractY) * RowOffset)
+ ShiftRight (X, FractX)
A copy of Adr is kept in AdrOld to fetch subsequent entries:
If the data is 8 bits, the timing is 2 cycles of address generation, followed by 2 cycles where data is returned (2 table entries per cycle);
If the data is 16 bits, the timing is 2 cycles of address generation, followed by 4 cycles where data is returned (1 entry per cycle).

The following two tables show address calculation methods for 8-bit data size and 16-bit data size.

両方のケースにおいて、アドレス生成のサイクルは、ＸインデックスのＦＩＦＯへの挿入と重なってもよく、単一の１回限りのルックアップの実効タイミングは、アドレス生成のための１サイクル、及び返却データの４サイクルと同程度に低い。インデックスの生成がその結果よりも２ステップ進んでいる場合、実効アドレス生成時間は無く、データは単に適切なレート（１セット毎に４サイクル又は８サイクル）で生成される。

In both cases, the address generation cycle may overlap with the insertion of the X index into the FIFO, and the effective timing of a single one-time lookup is one cycle for address generation and the return data As low as 4 cycles. If index generation is two steps ahead of the result, there is no effective address generation time and data is simply generated at the appropriate rate (4 or 8 cycles per set).

During image pyramid lookup brushing, tiling and warping, it is necessary to calculate the average color of a specific area of the image. Rather than calculating values for a given area, these functions make use of an image pyramid. The description and structure of the image pyramid is detailed in the section on internal image formats in the DRAM interface 81 chapter of this document. This section relates to how to address a given pixel in a pyramid with respect to three ordered fixed-point indices: levels (Z), Y, and X. Since the image pyramid lookup assumes an 8-bit data entry, the DataSize flag is completely ignored. After specifying Z, Y and X, the following 8 pixels:
[Int (X), Int (Y)] level Int (Z) pixels
[Int (X) +1, Int (Y)] level Int (Z) pixels
[Int (X), Int (Y) +1] level Int (Z) pixels
[Int (X) +1, Int (Y) +1] level Int (Z) pixels
[Int (X), Int (Y)] level Int (Z) +1 pixel
[Int (X) +1, Int (Y)] level Int (Z) +1 pixels
[Int (X), Int (Y) +1] level Int (Z) +1 pixels
Pixels of level Int (Z) +1 at [Int (X) +1, Int (Y) +1] are returned via the input FIFO.

Eight pixels are returned as 4 × 16 bit entries, and X and X + 1 entries are combined high / low. For example, if the scaled (X, Y) coordinates are (10.4, 12.7), the first four pixels returned are (10, 12), (11, 12), ( 10, 13) and (11, 13). When the coordinates are outside the valid range, the client can choose to repeat the edge pixel or return a constant color value by the edge pixel repeat and constant pixel register (only the lower 8 bits are used). When the image pyramid is built, there is a simple mapping from level 0 coordinates to level Z coordinates. This method simply shifts the X or Y coordinate to the right by Z bits. This needs to be done in addition to the number of bits already shifted to obtain the integer part of the coordinates (ie, FractX bit right shift for the X coordinate and FractY right shift for the Y coordinate). To find the ImageStart and RowOffset for a given level of the image pyramid, the 24-bit ZOffset register is used as a pointer to the level information table. This table is an array of records, where each record represents a given pyramid level ordered by number of levels. Each record has a 16-bit offset ZOffset from ImageStart to its pyramid level (no 64-byte aligned address like the lower 6 bits of the offset) and a 12-bit ZRowOffset for that level Composed. The element 0 of the table has ZOffset of 0, and ZRowOffset matches the general-purpose register RowOffset when simply indicating a full-size image. The Zoffset value at element N of the table will be added to ImageStart to obtain an effective ImageStart of level N of the image pyramid. The RowOffset value of the element N of the table stores the RowOffset value for the level N. Software running on the CPU must properly set up the table before using this addressing mode. The outline of the actual address generation will be described for each cycle below.

上述のアドレス生成は、単一のバレルシフタ、２個のアダー、及び２４ビットを生じる単一の１６×１６乗算／加算ユニットを使用して実現できる。一部のサイクルには２回のシフトが含まれるが、それらは、シフト値が同じであるか（即ち、バレルシフタの出力が２回使用されるか）、又はシフトが１ビットであるかのどちらかであり、ハードワイヤード可能である。次の内部レジスタ：ZAdr, Adr, ZInt, YInt, XInt, ZRowOffset, ZImageStartが必要である。_Int形のレジスタは最大で８ビットだけが必要であるが、他のレジスタは最大２４ビットになり得る。このアクセス方法は、画像ピラミッドから読み出すだけであり、画像ピラミッドへ書き込まないので、キャッシュグループ２(CacheGroup2)が（Zadrを用いて）画像ピラミッドアドレステーブルをルックアップするため使用される。CacheGroup1は、（Adrを用いて）画像ピラミッド自体をルックアップするため使用される。アドレステーブルは、（原画像サイズに応じて）約２２エントリーがあり、各エントリーは４バイトである。したがって、３又は４本のキャッシュラインをキャッシュグループ２に割り付けるべきであり、できるだけ多数のキャッシュラインをCasheGroup1に割り付けるべきである。データのセットを返すためのタイミングは８サイクルであり、サイクル８とサイクル０は動作が重なり合うと考えられ、即ち、次の要求のサイクル０はサイクル８の間に生じる。これが許される理由は、サイクル０がメモリにアクセスせず、サイクル８が特定の演算を含まないからである。

The address generation described above can be accomplished using a single barrel shifter, two adders, and a single 16 × 16 multiply / add unit that yields 24 bits. Some cycles include two shifts, which are either the same shift value (ie, the barrel shifter output is used twice) or the shift is one bit. It can be hard-wired. The following internal registers are required: ZAdr, Adr, ZInt, YInt, XInt, ZRowOffset, and ZImageStart. The _Int type register only requires a maximum of 8 bits, but other registers can have a maximum of 24 bits. Since this access method only reads from the image pyramid and does not write to the image pyramid, Cache Group 2 (CacheGroup2) is used to look up the image pyramid address table (using Zadr). CacheGroup1 is used to look up the image pyramid itself (using Adr). The address table has about 22 entries (depending on the original image size), and each entry is 4 bytes. Therefore, 3 or 4 cache lines should be assigned to cache group 2 and as many cache lines as possible should be assigned to CashGroup1. The timing for returning a set of data is 8 cycles, and cycle 8 and cycle 0 are considered to overlap in operation, ie cycle 0 of the next request occurs during cycle 8. This is allowed because cycle 0 does not access the memory and cycle 8 does not contain a specific operation.

Coordinate Generation Using VLIW Vector Processor 74 Some functions linked to the write iterator require the X and / or Y coordinates of the current pixel being processed in a portion of the processing pipeline. Certain processing needs to be done at the end of each row or column being processed. In most cases, the PassX and PassY flags are sufficient to completely generate all coordinates. However, if there are special requirements, the following function can be used. This calculation can be expanded to multiple ALUs for single cycle generation, or may be performed within a single ALU 188 for multiple cycle generation.

When the sequential [X, Y] generation process sequentially processes a pixel according to a sequential read iterator (or when generating a pixel and writing the pixel to a sequential write iterator), the following process is as shown in FIG. In addition, it can be used to generate X and Y coordinates instead of the PassX flag / PassY flag.

The coordinate generator counts up to the image width ImageWidth with respect to the X coordinate, and increments the Y coordinate once for every ImageWidth pixels. The actual process is shown in FIG. 24, and the following constants are set by software.

以下のレジスタは一時変数を保持するため使用される。

The following registers are used to hold temporary variables.

必要条件は以下のように要約される。

The requirements are summarized as follows:

垂直ストリップ［Ｘ，Ｙ］生成
プロセスが、画素を垂直ストリップ書き込みイタレータへ書くために画素を処理し、ある種の理由からPassXフラグ／PassYフラグを使用できないとき、図２５に示されるようなプロセスがＸ、Ｙ座標を生成するため使用できる。この座標生成器は、Ｘ座標に関して画像幅ImageWidthまでカウントアップし、ImageWidth個の画素毎に１回ずつＹ座標をインクリメントするだけである。実際のプロセスは図２６に示され、以下の定数がソフトウェアによってセットされる。

When the vertical strip [X, Y] generation process processes a pixel to write the pixel to a vertical strip write iterator and the PassX flag / PassY flag cannot be used for some reason, the process as shown in FIG. Can be used to generate X, Y coordinates. This coordinate generator only counts up to the image width ImageWidth with respect to the X coordinate and only increments the Y coordinate once for every ImageWidth pixels. The actual process is shown in FIG. 26, and the following constants are set by software.

以下のレジスタは一時変数を保持するため使用される。

The following registers are used to hold temporary variables.

必要条件は以下のように要約される。

The requirements are summarized as follows:

垂直ストリップ毎に１回ずつ出現する計算（２回の加算、そのうち一つはＭＩＮ演算と関連付けられている）は、一般的なタイミング統計量に含まれない。なぜならば、それらは、画素タイミング毎の実際の一部ではないからである。しかし、それらは、特定の関数用のマイクロコードのプログラミングに考慮しなければならない。

Calculations that occur once per vertical strip (two additions, one of which is associated with the MIN operation) are not included in the general timing statistics. This is because they are not an actual part of every pixel timing. However, they must be considered in programming microcode for specific functions.

Image sensor interface (ISI83)
The image sensor interface (ISI 83) acquires data from the CMOS image sensor and makes it available for storing data in the DRAM. The image sensor has an aspect ratio of 3: 2 and a typical resolution is 750 × 500 samples, yielding 375K (8 bits per pixel). Each 2 × 2 pixel block has a structure as shown in FIG. The ISI 83 is a state machine that transmits control information including a frame synchronization pulse and a PixelClock pulse to the image sensor in order to read an image. Pixels are read from the image sensor and placed in the VLIW input FIFO 78. The VLIW can then process and / or store the pixels. This is illustrated in FIG. The ISI 83 is used in combination with a VLIW program that accumulates detected photographic images in DRAM. Processing takes place in two steps:
A small VLIW program reads pixels from the FIFO and writes the pixels to the DRAM via a sequential write iterator;
The photographic image in the DRAM is rotated 90, 180 or 270 degrees depending on the orientation of the camera when the photograph is taken.

If the rotation is 0 degrees, Step 1 only writes the photographic image to the final photographic image storage location, and Step 2 is not performed. If the rotation is other than 0 degrees, the image is written to a temporary area (eg, a print image storage area) and then rotated during step 2 and put into the final photographic image storage location. Step 1 is a very simple microcode that takes data from the VLIW input FIFO 78 and writes the data sequentially into a write iterator. The rotation in step 2 is performed using an accelerated Vark affine transformation function. Processing is done in two steps to reduce design complexity and reuse the Vark affine transformation rotation logic already needed for the image. This is acceptable because both steps are completed in about 0.03 seconds, which is very little time for the Artcam operator. Even so, the readout process is limited by sensor speed and takes 0.02 seconds to read the entire frame and about 0.01 seconds to rotate the image.

The orientation is important for converting between the detected photographic image and the internal format image. This is because the relative positions of the R pixel, the G pixel, and the B pixel change with the direction. The processed image needs to be rotated during the printing process to be oriented correctly for printing. Artcam's three-dimensional model comprises two image sensors, and their inputs are multiplexed into a single ISI 83 (different microcode, but ACP31 is the same). Since each sensor is a frame storage type, both images can be acquired simultaneously and transferred to the memory at the same time.

Display controller 88
When the “shoot” button on Artcam is pressed halfway, the TFT displays the current image from the image sensor (converted by a simple VLIW process). When the shooting button is fully pressed, a shot image is displayed. When the user presses the print button and image processing starts, the TFT is turned off. When the image is printed, the TFT is turned on again. The display controller 88 is used in the Artcam model that incorporates a flat panel display. An exemplary display is a TFT LCD with a resolution of 240 × 160 pixels. The configuration of the display controller 88 is shown in FIG. The state machine of the display controller 88 includes a register, which controls the timing of synchronization generation, from which the display image is obtained (to the DRAM via the data cache 76 by a particular cache group), and the register stores the TFT in its register Controls whether it should be enabled at the moment (using TFT enable). The CPU can write to these registers via the low speed bus. In order to display an image of 240 × 160 pixels on the RGB TFT, three components are required per pixel. An image acquired from the DRAM is displayed via three DACs, and one DAC is provided for each of the R, G, and B output signals. With an image refresh rate of 30 frames per second (60 fields per second), the display controller 88
It requires a data transfer rate of 240 × 160 × 3 × 30 = 3.5 MB / sec.

This data rate is slower than the rest of the system. However, it is high enough to slow down the VLIW program during intensive image processing. The general principle of TFT operation reflects this.

Image Data Format As described above, the DRAM interface 81 is connected to the other client part of the ACP chip and the RAMBUS.
Connects to DRAM. In effect, each module in the DRAM interface is an address generator:
There are three logical types of images manipulated by ACP. They are,
A CCD image which is an input image captured from the CCD, an internal image format used internally by the Artcam device, and a print image which is an output image format printed by the Artcam.

These images typically differ in color space, resolution, and output and input color spaces that can vary between cameras. For example, the CCD image of a low-price camera differs from the CCD used in a high-price camera in resolution and color characteristics. However, all internal image formats are the same format with respect to color space in every camera.

In addition, the three image types can differ as to which direction is “upward”. The physical orientation of the camera gives rise to the idea of whether it is a person image or a landscape image, and this idea should be maintained throughout the process. For this reason, the internal image is always oriented correctly and rotation is performed on the image acquired from the CCD during the printing operation.

CCD image organization A wide variety of CCD image sensors can be used, but the CCD itself is a 750 × 500 type image sensor, and it is assumed that 375,000 bytes (8 bits per pixel) are generated. Each 2 × 2 pixel block has a structure as shown in FIG.

The CCD image when stored in the DRAM has pixels continuous with a predetermined line adjacent in the memory. Each line is accumulated one after another. The image sensor interface 83 is responsible for acquiring data from the CCD and storing the data in the DRAM in the correct orientation. In this way, the CCD image at 0 degree rotation is obtained by G, R, G, R, G, R.P. . . , B, G, B, G, B, G. . . Including the second line. If the CCD image is a human image and rotated 90 degrees, the first line is R, G, R, G, R, G and the second line is G, B, G, B, G, B ,. . . (And so on).

Pixels are stored in an interleaved format. This is because all color components are required to convert the internal image format.

Note that ACP 31 makes no assumptions about the CCD pixel format. This is because the actual CCD for imaging can change over time between Artcams. All processing performed by the hardware is controlled by the main microcode to extend the usefulness of ACP31.

Internal Image Organization An internal image is typically composed of a number of channels. In the Vark image,
Lab
Labα
LabΔ
αΔ
L
However, it is not limited to these.

L, a and b correspond to components in the Lab color space, α is a matte channel (used for compositing), and Δ is a bump (used during brushing, tiling and illuminating) Map channel.

The VLIW processor 74 requires that the image be organized in a planar structure. Therefore, a Lab image has three separate memory blocks:
1 block for L channel 1 block for a channel 1 block for 1 channel b channel

Within each channel block, the pixels are stored adjacent to a given row (along with some optional padding bytes), and the rows are stored one after the other.

FIG. 31 shows an example of the storage format of the logical image 100. The logical image 100 is stored in a planar format having color components L101, a102 and b103 stored sequentially. Alternatively, the logical image 100 may be stored in a compressed format having an uncompressed L component 101 and a compressed A component 105 and a compressed B component 106.

Referring to FIG. 32, the pixel of line n (110) is stored together before the pixel of line n + 1 (111). Images are stored in adjacent memory within a single channel.

In the case of the 8MB memory model, the final printed image after completion of all processing needs to be compressed in the chrominance channel. The compression of the chrominance channel can be 4: 1 and the overall compression can be 12: 6 or 2: 1.

Except for the final printed image, the Artcam image is typically not compressed. Due to memory constraints, the software may choose to compress the final printed image of the chrominance channels by scaling each of these channels by 2: 1. After this is done, the Print Vark function used to print the image should be instructed to handle the chrominance channel designated as compressed. The print function is the only function that knows how to handle compressed chrominance, but if so, it only processes at a fixed 2: 1 compression ratio.

It is possible to compress the image and then operate on the compressed image to create the final printed image, but this is not recommended because it results in a loss of resolution. Moreover, the image should be compressed only once as a final step before printout. A single compression is virtually unnoticeable, while multiple compressions cause significant image degradation.

Clip images stored in the clip image organization Artcard are not explicitly supported by the ACP 31. The software is responsible for taking images from the current Artcard and organizing the data into a format that can be recognized by the ACP. If the images are stored in compressed form on Artcard, the software is responsible for decompressing them. This is because there is no specific hardware that supports the decompression of Artcard images.

During the brushing, tiling and warping processes utilized to manipulate image pyramid organized images, it is often necessary to calculate the average color of a particular area of the image. Rather than calculating the value for each given area, these functions make use of an image pyramid. As shown in FIG. 33, the image pyramid is a virtual multi-resolution pixel map. The original image 115 is a 1: 1 representation. By low-pass filtering in each dimension and 2: 1 subsampling, a quarter of the original size image 116 is obtained. This process continues until the entire image is represented by a single pixel. The image pyramid is constructed from a primitive internal format image and uses 1/3 the size occupied by the original image (1/4 + 1/16 + 1/64 + ...). For a 1500 × 1000 original image, the corresponding image pyramid is about ½ MB. The image pyramid is constructed by a specific Vark function and used as a parameter for other Vark functions.

The entire print image organization process image is needed at the same time to print it. However, the print image output is the only temporary image format that includes CMY dither images and is used within the print image function. However, it should be noted that color conversion from the internal color space to the print color space is necessary. Moreover, the color conversion can be adjusted separately for each type of camera print roll with different ink characteristics. For example, sepia output is performed using a particular sepia tone Artcard, or using a sepia tone print roll (thus all Artcards function in sepia tone).

Color Space As described above, in Artcam, three color spaces are used corresponding to the types of image types.

ACP has no direct knowledge of a particular color space. Instead, the ACP is a CCD color space, an internal color space, and a printer color space, i.e.
CCD color space: RGB,
Internal color space: Lab
Printer color space: CMY
Depends on the client color space conversion table for conversion between
By removing the color space conversion from ACP31,
-Use different CCDs in different cameras-Use different inks (with different print rolls in the long term)-Separate CCD selection from ACP design pass-Internal for accurate color processing It is possible to define a color space skillfully.

Artcard interface 87
The Artcard interface (AI) obtains data from the linear image sensor when the Artcard passes under the linear image sensor and uses which data is stored in the DRAM. The image sensor generates 11000 8-bit samples per scan line and samples Artcard at 4800 dpi. AI is a state machine that sends control information including a LineSync pulse and a PixelClock pulse to a linear sensor in order to read an image. Pixels are read from the linear sensor and accommodated in the VLIW input FIFO 78. The VLIW can then process the pixels and / or accumulate the pixels. The AI includes several registers:

尚、ＣＰＵは、走査を始動する前にＶＬＩＷ入力ＦＩＦＯ７８をクリアしなければならない。状態レジスタは以下のように解釈されるビットを有する。

Note that the CPU must clear the VLIW input FIFO 78 before starting the scan. The status register has bits that are interpreted as follows.

Ａｒｔｃａｒｄインタフェース（ＡＩ）８７
Ａｒｔｃａｒｄインタフェース（ＡＩ）８７は、Ａｒｄｃａｒｄリーダー３４からＡｒｔｃａｒｄ画像を取得し、それをオリジナルデータ（通常は、Ｖａｒｋスクリプト）に復号化する役割を担う。特に、ＡＩ８７は、ＡｒｔｃａｒｄスキャナリニアＣＣＤ３４から信号を受け取り、カードに印刷されたビットパターンを検出し、ビットパターンをオリジナルデータに変換し、読み出しエラーを訂正する。

Artcard interface (AI) 87
The Artcard interface (AI) 87 is responsible for obtaining an Artcard image from the Ardcard reader 34 and decoding it into original data (usually a Vark script). In particular, the AI 87 receives a signal from the Artcard scanner linear CCD 34, detects a bit pattern printed on the card, converts the bit pattern into original data, and corrects a read error.

If Artcard 9 is not inserted, the image printed from Artcam is only a detected photographic image organized by any standard image processing routine. Artcard 9 is a means for correcting a photo before the user prints it out. The simple task of inserting a specific Artcard 9 into Artcam allows the user to define complex image processing to be performed on the photographic image.

If Artcard is not inserted, the photographic image is processed in the usual way to create a printed image. When a single Artcard 9 is inserted into an Artcam, the Artcard effect is applied to the photographic image to generate a printed image. When Artcard 9 is removed (removed), the printed image reverts to a photographic image processed in a standard manner. When the user presses a button to retrieve Artcard, the event is placed in an event queue maintained by the operating system running on the Artcam central processor 31. When an event is processed (eg, after the current print is performed), the following situation occurs:

If the current Artcard is valid, the printed image is marked invalid and a “process standard” event is placed in the event queue. When the event is finally processed, standard image processing operations are added to the photographic image to produce a printed image.

The motor is started to retrieve the Artcard, and a time-defined “motor stop” event is added to the event queue.

When the art card insertion user inserts Artcard 9, Artcard sensor 49 detects this and reports it to ACP72. As a result, the software inserts the “Artcard inserted” event into the event queue. When this event is processed, several situations occur:
The current Artcard is marked invalid (not "none");
Print images are marked as invalid;
Artcard motor 37 is started to capture Artcard;
Artcard interface 87 is instructed to read Artcard;
The Artcard interface 87 receives a signal from the Artcard scanner linear CCD 34, detects a bit pattern printed on the card, corrects an error in the detected bit pattern, and generates a valid Artcard data block in the DRAM.

Data Reading from Artcard CCD—General Considerations As shown in FIG. 34, the data card reading process has four phases while pixel data is read from the card. The phases are as follows:
Phase 1 Data area detection on Artcard Phase 2 Bit pattern detection from Artcard based on CCD pixels, writing as bytes Phase 3 Descramble byte pattern and XOR operation Phase 4 Data decoding (Reed-Solomon decoding).

As shown in FIG. 35, Artcard 9 must be sampled at least twice the printed resolution to satisfy the Nyquist theorem. In practice, it is better to sample at a higher rate. Preferably, the pixels are sampled at three times the resolution of the dots printed in each dimension (237), and nine pixels are required to define one dot. Therefore, when the resolution of Artcard 9 is 1600 dpi and the resolution of sensor 34 is 4800 dpi, 9450 pixels can be obtained for each column by using a 50 mm CCD image sensor. That is, if 2 MB dot data (9 pixels per dot) is required, 2 MB * 8 * 9/9450 = 15978 columns = about 16000 columns are required. Of course, if the dots are not strictly aligned with the sampling CCD, in the worst case possible, the dots will be detected in the 16 (4 × 4) pixel area (231).

The Artcard 9 is somewhat distorted due to thermal damage, rotates slightly (up to, for example, 1 degree) due to variations in insertion into the Artcard reader, and slightly changes to the true data rate due to the speed variation of the reader motor 37 Deviations may appear. Due to these changes, the column of data from the card cannot be read as the corresponding column of pixel data. As shown in FIG. 36, one rotation of Artcard 9 causes pixels from one column on the card to be read out as pixels over 166 columns.

Finally, Artcard 9 must be read in a time that is reasonable for a human operator. Since the data on Artcard covers most of the Artcard surface, timing issues can be limited to the Artcard data itself. A readout time of 1.5 seconds is appropriate for Artcard readout.

Artcard must be captured within 1.5 seconds. Therefore, all pixel data of 16000 columns need to be read from the CCD 34 within 1.5 seconds, that is, 10667 columns per second. Thus, the time available to read a row is 1/10667 seconds, or 93747 nanoseconds. The pixel data can be written to the DRAM one column at a time, completely independently of the process of reading the pixel data.

The time to write one column of data (4 bits per pixel, and therefore 9450/2 bytes because 2 × 4 bit pixels can be read per byte) to the DRAM is 8 cache lines. It is shortened by using. If 4 lines are written simultaneously, 4 banks can be written independently, and the overlap latency is shortened. In this case, 4725 bytes can be written in 11840 ns (4725/128 * 320 ns). In this way, the time required to write predetermined column data to the DRAM uses less than 13% of the available bandwidth.

Artcard Decoding Simply looking at the data size, if the entire Artcard pixel data (140 MB if each bit is read as a 3 × 3 array) as read by the linear CCD 34 is maintained, then the process is stored in an 8 MB memory 33. It seems impossible to adapt to. For this reason, the linear CCD reading, bitmap decoding, and unbitmap processes should be performed in real time (while Artcard 9 passes by the linear CCD 34), and these processes are performed for all images. Even if data storage is not made available, it must work efficiently.

When Artcard 9 is inserted, the old stored print image and enlarged photographic image are invalidated. The new Artcard 9 may include instructions for creating a new image based on the current captured photographic image. The old print image is invalid and the area holding the enlarged photographic image and image pyramid is invalid, leaving more than 5 MB available for scratch memory during the read process. Strictly speaking, the 1 MB area in which the new Artcard raw data is to be written is available as scratch data during the Artcard read process until the final Reed-Solomon decoding is performed, It is opened again. The read process described here does not use the extra 1 MB area (except for use as the final destination of the data).

It should also be noted that the descrambling process requires two sets of 2 MB memory areas because descrambling cannot be performed in place. Fortunately, the 5 MB scratch area contains enough space for this purpose.

Referring to FIG. 37, there is shown a flowchart 220 of the steps necessary to decode Artcard data. These steps include reading the Artcard 221 and decoding the read data to generate the corresponding encoded, XORed, and scrambled bitmap data 223. A checkerboard XOR operation is then applied to the data to generate encoded and scrambled data 224. This data is then descrambled 227 to produce data 225, which is then subjected to Reed-Solomon decoding to produce the original raw data 226. Alternatively, the descrambling and XOR operation processes may be performed together and do not require separate data paths. Each of the above steps is described in detail below. As described with reference to FIG. 37, the Artcard interface has four phases, the first two of which are time critical and must be performed while pixel data is being read from the CCD. :
Phase 1 Data area detection on Artcard Phase 2 Bit pattern detection from Artcard based on CCD pixels, writing as bytes Phase 3 Descramble byte pattern and XOR operation Phase 4 Data decoding (Reed-Solomon decoding).

These four phases will now be described in detail.

Phase 1: When Artcard 9 passes the CCD 34, the AI must detect the start of the data area by reliably detecting the special target on Arrcard on the left side of the data area. If no special target can be detected, the card is marked invalid. This detection needs to be performed in real time while Artcard 9 passes through the CCD 34.

If necessary, rotation invariance is provided. In this case, the target is repeated on the right side of Artcard, but is associated with the lower right corner, not the upper corner. In this way, if the card is inserted in a “wrong” manner, the target will eventually be oriented correctly. Phase 3 to be described later may be changed to detect the direction of data and consider the possibility of rotation.

Phase 2: Once the data is determined, the main readout process begins, the pixel data from the CCD is put into an “Artcard data window”, the bits from this window are detected, the detected bits are assembled into bytes, and the DRAM Build a byte image on This all needs to be done while Artcard passes the CCD.

Phase 3: When all pixels are read from the Artcard data area, the Artcard motor 37 is stopped and the byte image is descrambled and XORed. Although real-time performance is not required, this process must be fast enough not to bother human operators. This process takes a 2MB scrambled bit image and writes the scrambled / XORed bit image to a separate 2MB image.

Phase 4: The final phase of the Artcard read process is a Reed-Solomon decoding process, where a 2 MB bit image is decoded into a 1 MB valid Artcard data area. Again, real-time performance is not required, but it is necessary to decode quickly from the point of view of a human operator. If the decryption process is valid, the card is marked as valid. If the decoding fails, an attempt is made to decode a copy of the data in the bit image and the process is repeated until there is no duplicate image of data in the bit image.

The above four-phase process requires 4.5 MB of DRAM. 2 MB is reserved for phase 2 output and 0.5 MB is reserved for scratch data between phases 1 and 2. The remaining 2 MB space can hold more than 440 columns with 4725 bytes per column. In practice, the pixel data being read out is ahead of the Phase 1 algorithm by several columns, and in the worst case it is delayed by about 180 columns from Phase 2, but is well within the limit of 440 columns. Fits in.

Next, the actual operation of each phase will be described in detail.

Phase 1-Data Area Detection on Artcard This phase is related to reliably detecting the left side of the data area on Artcard9. Accurate detection of the data area is realized by accurate detection of a special target printed on the left side of the card. These targets are especially designed to be easily detected even when there is a rotation of up to 1 degree.

Referring to FIG. 38, an enlarged view of the left side of Artcard 9 is shown. The left side of the card is divided into 16 bands 239, and a target, for example, 241 is provided at the center of each band. Bands are logical in that no line is drawn to separate the bands. FIG. 39 shows one target. The target 241 is a printed black square containing a single white dot. The idea is to first detect as many targets 241 as possible, and then join at least eight detected white dot locations into a single logical line. If this concept can be implemented, the start of the data area 243 will be a fixed distance from this logical line. If not, the card is rejected as invalid.

As shown in FIG. 38, the height of the card 9 is 3150 dots. Since the target (Target0) 241 is placed at a certain distance of 24 dots from the upper left corner 244 of the data area, the target is the first of 16 areas of equal size of 192 dots (576 pixels). It fits well in the area 239 and there is no target in the last pixel area of the card. The target 241 must be large enough to be easily detected, but not large enough to go outside the area when the card is rotated once. A suitable target size is a black square 241 of 31 × 31 dots (93 × 93 detection pixels) including white dots 242.

If there is a worst one degree rotation, one column shift occurs every 57 pixels. Thus, for a band of 590 pixels in size, if the card is in the worst rotation state, it is impossible to place any part of this symbol at the top or bottom 12 pixels, or at that location in the band Alternatively, the symbols will be detected in the wrong band when reading the CCD.

Therefore, when the rectangular black portion is 57 pixels high (19 dots), at least 9.5 black pixels can be reliably read out in the same column by the CCD (half of the pixels fit in one column). , The worst case where the other half fits into the next column). In order to ensure that at least 10 black dots are read out in the same row, the height needs to be 20 dots. In preparation for erroneous detection at the start edge of black dots, the number of dots is increased to 31, and 15 dots are placed on both sides of the white dot placed at the local coordinates (15, 15) of the target. The 31 dots are 91 pixels, and the shift within the column is 3 pixels at the maximum, and easily fits within the band of 576 pixels.

Thus, each target is a block of 31 × 31 dots (93 × 93 pixels), and each target is
15 columns containing 31 black dots in each column (a column of 45 pixels out of 93 pixels),
A row including 15 black dots (45 pixels), 1 white dot (3 pixels), and 15 additional black dots (45 pixels);
15 columns each containing 31 black dots (a column of 45 pixels out of 93 pixels),
Consists of.

The target detection target is detected not by detecting dots but by reading out the pixel rows one by one at the same time. Within a given band, it is necessary to search a number of columns composed of a number of adjacent black pixels that make up the left side of the target. Next, it is expected to find a white area at the center of the further black column, and finally a black column is found on the left side of the center of the target.

In order to obtain excellent cache performance when reading out pixels, eight cache lines are required. For each logical read, four cache lines are filled with four sub-reads, while the other four cache lines are used. This effectively uses up to 13% of the available DRAM bandwidth.

As shown in FIG. 40, the detection mechanism FIFO for detecting the target requires a filter 245, a run length encoder 246, and a FIFO 247 that requires special wiring of the top three elements (S1, S2, and S3) for random access. including.

The input pixel columns are processed one column at a time until all targets have been detected or a specified number of columns have been processed. To process the columns, the pixels are read from the DRAM, passed through a filter 245 to detect 0 or 1, and then run length encoded 246. The bit value and the number of adjacent bits of the same value are entered in the FIFO 247. Each entry in FIFO 249 fits in 8 bits, 7 bits 250 hold the run length, and 1 bit 249 holds the value of the detected bit.

The run length encoder 246 encodes only adjacent pixels in a 576 pixel (192 dots) region.

The top three elements of FIFO 247 may be accessed 252 in random order. The run length (of pixels) of these entries is filtered into three values: short, medium, and long according to the following table.

ＦＩＦＯ２４７の最上部の３個のエントリーに着目すると、３種類の興味深い特殊ケースが存在する。

Focusing on the top three entries of FIFO 247, there are three interesting special cases.

好ましくは、領域バンド毎に以下の情報が保持される。

Preferably, the following information is held for each region band.

全部で７バイトが与えられる。合計が８バイトであるならば、アドレス生成が簡単化される。そこで、１６個のエントリーは、１６＊８＝１２８バイトが必要であり、これは、４個のキャッシュラインとぴったり合う。アドレスレンジは、０．５ＭＢのＤＲＡＭエリアのスクラッチの内側に入るべきである。なぜならば、他のフェーズが残りの４ＭＢデータエリアを利用するからである。

A total of 7 bytes are given. If the total is 8 bytes, address generation is simplified. So 16 entries require 16 * 8 = 128 bytes, which fits exactly 4 cache lines. The address range should be inside the 0.5 MB DRAM area scratch. This is because other phases use the remaining 4 MB data area.

When starting to process a given pixel column, the register value S2StartPixel 254 is reset to zero. As entries in the FIFO advance from S2 to S1, they are added to the existing S2StartPixel value (255), giving the exact location of the run currently defined in S2. Examining each of the three interesting cases in the FIFO, S2StartPixel can be used to determine the start of the black area of the target (cases 1 and 2) and to determine the start of the white dot in the center of the target. Can be used (Case 3). The algorithm for processing the sequence is expressed as follows:

列の処理に関与するステップ（プロセスColumn）は以下の通りである。

The steps involved in column processing (Process Column) are as follows.

３種類のケース（処理ケース）の各々に対する処理は以下の通りである。

Processing for each of the three types of cases (processing cases) is as follows.

Case 1:

ケース２：
ケース３を識別するためPrevCaseWasCase2フラグ（前のケースはケース２であった）をセットする以外に特別な処理は記録されない（上記の列を処理するステップ３を参照）。

Case 2:
No special processing is recorded other than setting the PrevCaseWasCase2 flag (previous case was case 2) to identify case 3 (see step 3 above processing column).

Case 3:

所与の列を処理する最後に、現在の列がターゲット検出のための列の最大番号と比較される。許容された列の番号を超えたとき、検出されたターゲットの個数をチェックする必要がある。検出されたものが８個未満であるならば、カードは無効であると見なされる。

At the end of processing a given column, the current column is compared with the maximum number of columns for target detection. When the number of allowed columns is exceeded, it is necessary to check the number of targets detected. If less than 8 are detected, the card is considered invalid.

After the process target target is discovered, the target needs to be processed. Either all targets are available, or only some targets are available. Some targets may have been detected in error.

This phase of the process determines a mathematical line through the center of as many targets as possible. As the number of targets through which the straight line passes increases, the target position is detected with higher reliability. The limit is set to 8 targets. A straight line is considered correct if it passes through at least 8 targets.

図３４に示されるように、上記アルゴリズムにおて、TargetA２６１及びTargetBからCurrentLine２６０を決定するため、ターゲット２６１、２６２と、ターゲットＡの位置との間でΔrow（２６４）及びΔcolumn（２６３）を計算することが必要である。Δrow及びΔcolumnを加算することにより、Target0からTarget1、以下同様、へ移動することが可能になる。検出された（実際に検出されたならば）TargetNの位置は、直線上のTargetNの計算された予測位置と比較することができ、許容範囲内に収まるならば、TargetNは直線上にあることが判定される。 Although it is brute force, a simple approach can be adopted. This is because there is time to do so (see below) and the test is simplified by reducing complexity. It is necessary to determine a straight line between Target0 and Target1 (when both targets are considered valid) and to determine the number of targets on this line. Next, a straight line between Target 0 and Target 2 is determined and this process is repeated. Finally, the same process is performed on the straight line between Target 1 and 2, the straight line between 1 and 3, etc., and so on. Finally, the same process is performed on the straight line between Target 14 and Target 15. Assuming that all targets are detected, 15 + 14 + 13 +. . . = Perform 90 calculations (each calculation set requires 16 tests, so in practice 1440 calculations) and select the line with the maximum number of targets detected along that line There is a need to. An algorithm for finding a target location is, for example:

As shown in FIG. 34, Δrow (264) and Δcolumn (263) are calculated between targets 261 and 262 and the position of target A in order to determine CurrentLine 260 from Target A 261 and Target B in the above algorithm. It is necessary. By adding Δrow and Δcolumn, it is possible to move from Target0 to Target1, and so on. The detected TargetN position (if actually detected) can be compared with the calculated predicted position of TargetN on a straight line, and if it falls within an acceptable range, TargetN can be on a straight line Determined.

Δrow and Δcolumn are
Δrow = (row _{TargetA
-row TargetB} ) / (B-A)
Δcolumn =
(column _TargetA -column _TargetB ) / (BA)
Is calculated by Next, the position of Target0
row = rowTargetA-(A * Δrow)
column = columnTargetA-(A * Δcolumn)
Calculate according to

Also, (row,
column) is compared with the actual row _Target0 and column _Target0 . In order to move from a certain prediction target to the next target (for example, Target 0 to Target 1), Δrow and Δcolumn may be added to row and column, respectively. To check whether each target is on a straight line, it is necessary to calculate the predicted position of Target0 and then perform one addition and one comparison on the coordinates of each target.

After completing the comparison of all 16 targets with a maximum of 90 straight lines, the result obtained is the best straight line through the effective target. If the straight line passes through at least 8 targets (ie, MaxFound> = 8), the card can be processed because it is found that sufficient targets have been detected to form the straight line. A card is considered invalid if the best straight line passes through fewer than eight targets.

The resulting algorithm requires 180 divisions to calculate Δrow and Δcolumn, 180 multiplications / additions to calculate Target0 position, and 2880 additions / comparisons. The time for executing this processing is the time required for reading out the 36 columns of pixel data, that is, 337482 ns. If the addition does not take into account the fact that the required time is shorter than the division, 3240 mathematical operations must be performed within 337482 ns. This corresponds to about 1040 ns per operation, or 104 cycles. Therefore, the CPU can safely execute all the processes of the target, and the design complexity is reduced.

Center of gravity update based on data edge boundary and clock mark Step 0: Data area position detection
The distances related to the rows and columns from Target0 (241 in FIG. 38) to the upper left boundary 244 of the data area are predetermined fixed distances. Therefore, TargetA, Δrow and Δcolumn found in the previous stage (Δrow and Δcolumn represent the distance between the targets) are used to calculate the center of gravity or predicted position of Target0 as described above.

Since the fixed pixel offset from Target0 to the data area is related to the distance between targets (192 dots between targets and 24 dots between Target0 and data area 243), simply set Δrow / 8 to the column coordinate of the center of gravity of Target0. What is necessary is just to add (the aspect ratio of a dot is 1: 1). The top coordinate is
(column _DotColumnTop = column _Target0
+ (Δrow / 8)
(row _DorColumnTop = row _Target0
+ (Δcolumn / 8)
Is defined as

The next Δrow and Δcolumn are updated to give the number of pixels between dots in a single row (not between targets) by dividing them by the number of dots between targets:
Δrow = Δrow / 192
Δcolumn = Δcolumn / 192
In addition, by setting -1 in the currentColumn register (see Phase 2), after Step 2, the currentColumn register increments from -1 to 0 when Phase 2 starts.

Step 1: Write initial centroid delta (Δ) and bit history This only writes the setup information needed for Phase 2.

This can be achieved by writing all the Δrow and Δcolumn entries in each row and 0 in the bit history. The bit history is actually a predicted bit history. This is because it is known that there is a boundary row 277 on the left side of the clock mark row 276 and a white area is present in front of it. Therefore, the bit history is 011, 010, 011, 010, or the like.

Step 2: The update bit history of the centroid based on the read actual pixels is set up in Step 1 according to the predicted clock mark and the data boundary. The actual centroid of each dot row is set more accurately (initially 0) by comparing the prediction data with the actual pixel values. The center-of-gravity update mechanism is realized simply by executing Step 3 of Phase 2.

Phase 2-Bit pattern detection from Artcard based on read out pixels and writing as bytes from Artcard 9 requires that at least 9 pixels detected over 3 columns be presented. There are few points at which dot detection calculation is performed for each detected pixel row. The time required for processing should be averaged over the appearance of average dots to maximize the available processing time. As a result, it is possible to keep the dot row processing from Artcard 9 within the time required to read out three rows of data from Artcard. In the most likely case, it takes four columns to present a dot, but the fourth column is the last row of one dot and the first row of the next dot. Therefore, processing should be limited to only 3 columns.

Pixels from the CCD are written to the DRAM within 13% of the available time, so 83% of the time, ie 83747 * 3 83% = 281241 ns 83% = 2233430 ns, process one row of dots. Is available for

It is necessary to detect 3150 dots within the available time and write their bit values to the raw data area of the memory. This process therefore requires the following steps:
For each row of dots on Artcard,
Step 0: Go to the next dot row;
Step 1: Detect the upper and lower ends of the Artcard dot row (check the clock mark);
Step 2: Process the dot sequence, detect the bits, and store the bits appropriately;
Step 3: Update the center of gravity.

In the worst case, the first column is processed until at least 165 columns are read into the DRAM, since Artcard's logical dot sequence is processed and the logical dot sequence may shift over 165 pixels. Can not. Thus, phase 2 ends at the same time after the read process ends. The worst case time is 165 * 93747 ns = 15468255 ns, ie 0.015 seconds.

Step 0: Advance to the next dot In order to advance to the next dot row, Δrow and Δcolumn are added to dotColumnTop to obtain the center of gravity of the dot at the top of the row. When this is done for the first time, since it is at the clock mark string 276 on the left side of the bit image image data area, it proceeds to the first data string. Since Δrow and Δcolumn represent the distance between dots in one row, it is necessary to add Δrow to column _dotColumnTop and Δcolumn to row _dotColumnTop in order to move between the dot rows.

To keep track of the column number being processed, the column number is recorded in a register named CurrentCoulmn. Each time the sensor advances to the next dot row, the CurrentColumn register needs to be incremented. When the CurrentColumn register is first incremented, it is incremented from -1 to 0 (see step 0 of phase 1). The CurrentColumn register determines when to terminate the read process (when maxColumns is reached) and after all 8 bits have been written to the byte (once every 8 dot columns), DataOutPointer sets the next byte information Used to advance to row. The lower 3 bits determine how far into the current byte.

Step 1: Detection of the upper and lower ends of the Artcard dot row In order to process the dot row from Artcard, it is necessary to detect the upper and lower ends of the row. The column should form a straight line between the top and bottom of the column (except for local distortions etc.). Initially, dotCoulumnTop specifies a clock mark string 276. All you need to do is toggle the prediction value, write the prediction value to the bit history, and proceed to step 2. Its first task is to add the values of Δrow and Δcolumn to dotColumnTop to reach the first data dot in the column It is.

Step 2: For Artcard dot rows, if the center of gravity of the top and bottom edges of the processing row is given to the pixel coordinates, the row forms a straight line with as little variation as possible due to distortion or the like between them.

Assuming the process starts at the top of the column (top centroid coordinates) and falls to the bottom of the column, the subsequent predicted dot centroid is
row _next = row + Δrow
column _next = column + Δcolumn
As given.

This gives the address of the predicted centroid of the next dot in the column. However, in order to take into account local distortion and error, another Δrow and Δcolumn based on the dots detected in the predetermined row immediately before are added. In this way, a small drift that accumulates in a percentage of the maximum drift from the straight line joining the top and bottom edges of the column can be taken into account.

So we keep two values per row, but since the row history is used in step 3 of this phase, we store those two values in separate tables:
* Δrow and Δcolumn (2 @ 4 bits = 1 byte each);
* Line history (3 bits per line, 2 lines per byte are stored).

For each row, it is necessary to read Δrow and Δcolumn to determine the change to the center of gravity. The read process requires 5% of bandwidth and two cache lines:
76 * (3150/32) + 2 * 3150 = 13824 ns = 5% of bandwidth.

Once the centroid is determined, the surrounding pixels of the centroid are examined to detect the dot state, ie, the value of the bit. In other words, the dots cover an area of 4 × 4 pixels. However, since sampling is performed at a resolution three times the resolution of the dots, the number of pixels required to detect the dot state, that is, the bit value is much smaller than this. There are only three pixel columns to be accessed at one time.

In the worst case with pixel drift due to 1% rotation, the center of gravity shifts by one column every 57 pixel rows, but since the dot diameter is 3 dots, a column is 171 pixel rows (3 * 57) It will be effective against this. Since a byte contains 2 pixels, the number of bytes valid for each buffer read (4 cache lines) would be 86 (out of 128 reads) in the worst case.

When a bit is detected, the bit needs to be written to DRAM. To minimize DRAM delay, bits from 8 columns are stored as a set of adjacent bytes. Every bit from a given dot sequence corresponds to the next bit position in the data byte, so the old value of that byte can be read, shifted and ORed in the new bit, and the byte written back Is possible.

The read / shift and OR operation / write processes require two cache lines.

When updating the bit history for a given row, it is necessary to read and write that bit history. The bit history required per line is only 3 bits, and two lines of history can be stored in a single byte. The read / shift and OR operation / write processes require two cache lines.

The total bandwidth required for bit detection and storage is listed in the table below.

ドットの検出
重心を与えるドットの値（即ち、ビットの値）を検出するプロセスは、３個の画素値を検査し、ルックアップテーブルから結果を取得することによって実現される。この処理は、かなり簡単であり、図４２に示されている。ドット２９０の半径は約１．５画素である。したがって、重心を支える画素２９１は、その画素内の重心の実際の位置とは無関係に、完全にそのドット値であろう。重心が画素２９１の中心と正確に一致するならば、重心画素の上画素２９２及び下画素２９３、並びに、重心画素の左画素２９４及び右画素２９５は、大部分そのドット値であろう。重心が画素２９５の正確な中心から離れているならば、中心画素以外の画素がそのドットによって１００％覆われる可能性が高い。

The process of detecting the dot value (ie, bit value) that gives the dot detection centroid is accomplished by examining the three pixel values and obtaining the result from a lookup table. This process is fairly simple and is illustrated in FIG. The radius of the dot 290 is about 1.5 pixels. Thus, the pixel 291 that supports the centroid will be completely its dot value, regardless of the actual position of the centroid within that pixel. If the center of gravity exactly matches the center of the pixel 291, the upper and lower pixels 292 and 293 of the center of gravity pixel and the left and right pixels 294 and 295 of the center of gravity pixel will most likely have their dot values. If the center of gravity is away from the exact center of the pixel 295, the pixels other than the center pixel are likely to be 100% covered by the dot.

FIG. 42 shows only the center of gravity that is biased to the left and down of the center, but it is clear that the same relationship holds for the center of gravity of the upper and right sides of the center. In Case 1, the center of gravity exactly matches the center of the central pixel 295. The center pixel 295 is completely covered by the dot, and the top, bottom, left, and right pixels are also well covered by the dot. In Case 2, the center of gravity is on the left side of the center of the central pixel 291. The center pixel is still completely covered by dots, and the center left pixel 294 is completely covered by dots. The upper pixel 292 and the lower pixel 293 are still sufficiently covered. In case 3, the center of gravity is below the center of the central pixel 291. The center pixel 291 is still completely covered by the dots 291 and the pixel at the center left is completely covered by the dots. The central left pixel 294 and right pixel 295 are still fully covered. In Case 4, the center of gravity is at the lower left of the center of the center pixel. The center pixel 291 is still completely covered by the dot 291 and the center left pixel 294 and the center lower pixel 294 are completely covered by the dot.

The algorithm for updating the centroid selects the following three representative pixels and uses the distance from the center of the central pixel 291 to the centroid to determine the dot value:
First pixel: a pixel including the centroid Second pixel: If the X coordinate (column value) of the centroid is <1/2, it is the pixel on the left side of the first pixel, otherwise the pixel on the right side of the first pixel Third pixel: If the Y coordinate (row value) of the center of gravity is <1/2, it is the pixel above the first pixel, otherwise it is the pixel below the first pixel.

As shown in FIG. 43, the value of each pixel is output to a lookup table 301 calculated in advance. The three pixels are fed into a 12-bit lookup table, which outputs a single bit indicating the dot value, ie on or off. The look-up table 301 is constructed at the time of chip definition and is grouped into about 500 gates. The lookup table may be a simple threshold table, except that the center pixel (first pixel) is heavily weighted.

Step 3: The idea of updating the centroid Δs for each row in the column is to use the previous bit history to generate a complete dot at the predicted centroid position for each row in the current column. The actual pixel (from the CCD) is compared with the predicted complete pixel. If both match, the actual centroid position is accurately included in the predicted position, so the centroid Δs is valid and does not need to be updated. Otherwise, it is necessary to perform a process of updating the centroid Δs in order to best fit the predicted centroid position to the actual data. The new centroid Δs is used to process the next row of dots.

The update of the centroid Δs is performed as a subsequent process from step 2 for the following reason:
Reduce design complexity, so that it can be performed as Phase 2 Step 2 with enough bandwidth remaining, allowing DRAM buffers to be reused,
This is because it is guaranteed that all data necessary for the update of the center of gravity can be used at the start of the process without performing special pipelining.

The centroid Δ is treated as Δcolumn and Δrow to reduce complexity.

A given dot is 3 dots in diameter, but is likely to appear in a 4 × 4 pixel area. However, the edge of a certain dot is included in the same pixel as the edge of the next dot as a result. For this reason, in order to update the center of gravity, information other than information regarding a given single dot is required.

In FIG. 44, a single dot 310 from the previous row is shown with a given center of gravity 311. In this example, the dot 310 has a Δ spread in the range of the four pixel columns 312 to 315, and in fact, the dot (coordinate = (Prevcolumn,
Part of CurrentRow)) is in the current row of dots in the current row. If the current row and column dots are white, there is only dot information from the previous column dot (the current column dot is white), so the pixel column 314 on the rightmost side from the previous dot column is Expected to be small. From this, it can be seen that the higher the pixel value in the pixel row 315, the more the center of gravity is on the right side. Of course, if the dot on the right side is also black, information cannot be obtained from the sub-pixel, so that the center of gravity can be adjusted. The same is true for the left, top and bottom dots in dot coordinates (Prevcolumn, CurrentRow).

This shows that a maximum of five pixel columns and pixel rows are necessary. It is possible to simplify the situation by considering the centroids Δs of the rows and columns separately and treating them as the same problem rotated by 90 degrees.

The horizontal case will be described first. If the predicted pixel does not match the detected pixel, it is necessary to change the column centroid Δs. From the bit history, the value of the bit found for the current row of the current dot row, the immediately preceding dot row, and the previous two dot rows are known. The predicted center of gravity position is also known. By using these two sets of information, it is possible to generate a 20-bit predicted bit pattern if reading is complete. The 20-bit bit pattern represents a predicted Δ value for every five pixels in the horizontal dimension. The first nibble represents the rightmost pixel of the leftmost dot. The next three nibbles represent the three pixels that pass through the center of dot 310 from the previous column, and the last nibble represents the leftmost pixel 317 of the rightmost dot (from the current column).

If the predicted centroid is the center of the pixel, the 20-bit pattern is predicted to be based on the following table.

中心ドットの左側及び右側の画素は、ビットが０であるか、又は１であるかに応じて、それぞれ、０又はＤである。中心の３画素は、ビットが０であるか、又は１であるかに応じて、それぞれ、０００又はＤＦＤである。これらの値は、所与の画素に対してドットによって占められた物理的なエリアに基づいている。画素の正確な中心から重心までの距離に依存して、僅かにシフトしたデータを予測し、これは、実際に中心画素の両側の画素だけに影響を与える。１６通りの可能性があるので、中心からの距離を１６で除算し、予測画素をシフトさせるためその量を使用することができる。

The left and right pixels of the center dot are 0 or D, respectively, depending on whether the bit is 0 or 1. The three central pixels are 000 or DFD, respectively, depending on whether the bit is 0 or 1. These values are based on the physical area occupied by the dots for a given pixel. Depending on the distance from the exact center of the pixel to the center of gravity, slightly shifted data is predicted, which actually affects only the pixels on either side of the center pixel. Since there are 16 possibilities, the distance from the center can be divided by 16 and the amount used to shift the predicted pixel.

Once the 20-bit predicted pixel value of 5 bits is determined, it is compared with the actual pixel read out. This is performed by subtracting the prediction pixel from the read actual pixel in units of pixels and finally adding the differences together to obtain the distance from the prediction Δ value.

FIG. 45 is an explanatory diagram of an embodiment for realizing the above algorithm, including a look-up table 320 that receives a bit history 322 and a centroid decimal component 323 and outputs a corresponding 20-bit number 324, and includes a 20-bit number. 324 is subtracted 321 from the center pixel input 326 to generate a pixel difference 327.

This process is performed once for each left and right shift of the center of gravity by an amount of Δcolumn 1 with respect to the predicted center of gravity. The center of gravity with the smallest difference from the actual pixel is considered to be the “winner” and Δcolumn is updated accordingly (“no change” is desirable). As a result, Δcolumn is not changed more than 1 for each dot column.

This process is repeated for vertical pixels, and as a result, Δrow is updated.

Here, the prospect of parallelization is quite high. Depending on the clock rate selected for the ACP unit 31, these units can be placed in series (so that the three Δ tests are performed in consecutive clock cycles) or in parallel. Thus, three types of tests can be performed simultaneously. If the clock rate is fast enough, there is no need for parallelization.

It is necessary to read the old Δs of bandwidth initialization Δs and write them again. This requires 10% of the bandwidth:
2 * (76 (3150/32) + 2 * 3150) = 27648 ns = 10% of bandwidth.

When updating Δs, it is necessary to read the bit history of a given column. Since each byte contains two rows of bit history, it takes 2.5% of the bandwidth:
76 ((3150/32) / 2) 32) + 2 * (3150/2) = 4085 ns = 2.5% of the bandwidth.

In the worst case where the pixel drifts with 1% rotation, the centroid shifts by one column every 57 pixel rows, but the dot diameter is 3 pixels, so a given pixel column is 181 pixel rows (3 * 57). Since a byte contains 2 pixels, the number of bytes valid for a cache read is 86 (out of 128 reads) in the worst case. The worst case timing for 5 columns is therefore 31% bandwidth:
5 * (((9450 / (128 * 2)) * 320) * 128/86) = 88112 ns = 31% of the bandwidth.

The total bandwidth required to update the center of gravity Δ is listed in the table below.

フェーズ２のメモリ使用量
２ＭＢのビットイメージＤＲＡＭエリアは、フェーズ２処理中に読み書きされる。２ＭＢ画素データＤＲＡＭエリアは読まれる。

The bit image DRAM area with 2 MB of memory usage in phase 2 is read and written during phase 2 processing. The 2MB pixel data DRAM area is read.

The 0.5 MB scratch DRAM area is used to store row data. That is,

フェーズ３−生データのスクランブル解除及びＸＯＲ演算
図３７を参照すると、復号化の次のステップは、生データのスクランブル解除及びＸＯＲ演算である。Ａｒｔｃａｒｄから取得されるような２ＭＢバイト画像は、スクランブル処理され、ＸＯＲ演算された形式である。これは、フェーズ４におけるリード・ソロモンデコーダに必要なビットイメージを得るためにスクランブル解除及び再ＸＯＲ演算を必要とする。

Phase 3-Raw Data Descramble and XOR Operation Referring to FIG. 37, the next step of decoding is the raw data descrambling and XOR operation. A 2 MB byte image as obtained from Artcard is in a format that has been scrambled and XORed. This requires a descrambling and re-XOR operation to obtain the bit image required for the Reed-Solomon decoder in phase 4.

Referring to FIG. 46, the descrambling process 330 obtains a 2 MB scrambled byte image 331 and writes the descrambled 2 MB image 332. Since this process cannot be reasonably performed at the same location, two sets of 2MB areas are utilized. The scrambled data 331 is in symbol block order arranged in a 16 × 16 array, and symbol block 0 (334) includes all symbols 0 from any codeword in random order. Symbol block 1 includes all symbols 1 from all codewords in random order, and so on. Since there are only 255 symbols, the 256th symbol block is not used at this time.

The linear feedback shift register is used to determine the relationship between a position in a symbol block, eg, symbol block 334, and the codeword from which the symbol block is derived, eg, codeword 355. This works as long as the same seed is used as when the original Artcard image was generated. Since the time bottleneck is to wait for read / write readiness to DRAM non-sequential addresses, the XOR operation of bytes from other source lines and 0xAA and 0x55 is (temporally) There are virtually no restrictions.

The timing of the descrambling and XOR operation process is essentially a 2MB random byte read and a 2MB random byte write, ie
2 * (2 MB * 76 ns + 2 MB * 2 ns) = 327155571 ns, that is, about 0.33 seconds. This timing does not assume cache processing.

Phase 4-Reed-Solomon Decoding This phase is a loop and is repeated for the copy of data in the bit image, and the copy of the data is reed-Solomon until decoding is successful or there are no more copies to decode. Send to decryption module.

The Reed-Solomon decoder used may be a properly programmed VLIW processor, or it may be a separate hardwired such as L64712 of LSI logic. L64712 has a throughput of 50 Mbits per second (about 6.25 MB per second), so the time is not the memory access time for 2 MB read and 1 MB write (500 MB / second for sequential access) Rather, it is constrained by the speed of the Reed-Solomon decoder. The time required in the worst case is 2 / 6.25 seconds = about 0.32 seconds.

Phase 5 execution of Vark script The total time required to read Artcard 9 and decrypt it is about 2.15 seconds. The apparent delay for the user is actually only 0.65 seconds (sum of phases 3 and 4). This is because Artcard stops moving after 1.5 seconds.

When Artcard is loaded, the Artvark script is interpreted. Instead of executing the script immediately, the script is executed only when the “print” button 13 (FIG. 1) is pressed. The action taken to execute the script is highly dependent on the complexity of the script and must take into account the perceived delay between pressing the print button and the actual print button and the actual printing being performed. Don't be.

Alternative Artcard formats Of course, other artcard formats are also contemplated. The following describes such a new alternative artcard format with a number of preferred features. The following describes an alternative Artcard data format, a mechanism for mapping user data to dots on the alternative Artcard, and a fast alternative Artcard read algorithm for use in embedded systems that lack resources.

Alternative Artcard Overview Alternative Artcard is used in both embedded and PC-type applications and provides an easy-to-use interface to large amounts of data or configuration information.

The backside of the alternative Artcard has the same appearance regardless of the application (since it stores data), but the surface of the alternative Artcard is application dependent. The surface must be meaningful to the user in relation to the application.

Alternative Artcard technology can be independent of print resolution. The idea of storing data as dots on the card means that if more dots can be placed in the same space (by increasing the resolution), those dots can represent more data. Just do it. The preferred embodiment assumes that 1600 dpi printing is utilized on an exemplary Artcard 86 mm × 55 mm card, but other equivalent layouts, data sizes for other card sizes, and / or It is easy to determine other printing resolutions. Regardless of the print resolution, the reading technique is maintained as it is. After considering all decoding and other overheads, the alternative Artcard can store up to 1 megabyte of data at a print resolution up to 1600 dpi. The alternative Artcard is capable of storing several megabytes of data at a print resolution in excess of 1600 dpi. The following two tables summarize the effective data storage capacity of alternative Artcards for a given print resolution.

Alternative Artcard Format The data structure on the alternative Artcard is specially designed to assist in data recovery. This section describes the alternate Artcard data (back) format.

The dots on the data surface of the dot replacement Artcard may be monochrome. For example, black dots are printed on a white background with a predetermined desired print resolution. As a result, “black dots” are physically distinguished from “white dots”. FIG. 47 shows various examples of enlarged views of black dots and white dots. A monochrome scheme of black dots on a white background is preferably selected to maximize dynamic range in a blurry readout environment. Even if black dots are printed at a specific pitch (eg, 1600 dpi), the dots themselves are made slightly larger so that when the dots are printed adjacent to each other, an adjacent straight line is created. The dots may actually be merged together as a result of bleeding, but are not merged in the exemplary image of FIG. The black bruise will be removed more. The alternative Artcard described in the preferred embodiment allows the dot size to change flexibly, but for best results the exact dot size and ink / printing behavior for a particular printing technology Should be examined in more detail.

In the description of this Artcard embodiment, the term dot refers to a dot (ink, heat, electrophotography, silver halide, etc.) that is physically printed on an alternative Artcard. When the alternate Artcard reader scans the alternate Artcard, the dots need to be sampled at least twice the printed resolution to satisfy the Nyquist theorem. The term pixel means a sample value from an alternative Artcard reader device. For example, when a 1600 dpi dot is scanned at 4800 dpi, there are 3 pixels in each dimension of the dot, ie, 9 pixels per dot. The sampling process will be described later.

Referring to FIG. 48, a typical alternative Artcard data plane 1101 is shown. Each alternative Artcard is constituted by an “active” area 1102 surrounded by a white border area 1103. White boundary 1103 does not store data information, but is used to calibrate the white level by alternative Artcard readers. The active area is an array of data blocks, eg 1104, each data block separated from the next data block by a gap of 8 white dots, eg 1106. Depending on the print resolution, the number of data blocks on the alternative Artcard changes. On a 1600 dpi alternative Artcard, the array may be 8x8. Each data block 1104 has a dimension of 627 × 394 dots. When the inter-block gap 1106 is 8 white dots, the active area of the alternative Artcard is 5072 × 3208 dots (8.1 mm × 5.1 mm at 1600 dpi).

Data Block Referring to FIG. 49, a single data block 1107 is shown. The active area of the alternative Artcard is configured by an array of data blocks 1107 having the same structure. The structure of each data block is a data area 1108 surrounded by a clock mark 1109, a boundary 1110, and a Target 1111. The data area contains only encoded data, and clock marks, boundaries, and targets are provided specifically to assist in the location of the data area and to ensure accurate restoration of the data from within the area.

Each data block 1107 has a dimension of 627 × 934 dots. Among these, the central area of 595 × 384 dots is the data area 1108. Surrounding dots are used to hold clock marks, boundaries, and targets.

Boundary and clock mark FIG. 50 shows a data block, and FIGS. 51 and 52 show an enlarged edge portion of the data block. As shown in FIGS. 51 and 52, two 5-dot high boundaries and clock mark areas 1170 and 1177 exist in each data block, one above the data area and the other below the data area. is there. For example, the top 5 dot high region has an outer black dot border 1112 (extending the long side of the data block), a white dot separation line 1113 (guaranteing that the border is independent), and a 3 dot height. And a set 1114 of clock marks. The clock marks alternate between white and black rows, starting with black clock marks in the eighth column from both ends of the black block. There is no separation between the clock mark dots and the dots in the data area.

The clock marks are symmetric in that the same relative boundary / clock mark region appears when the alternate Artcard is inserted rotated 180 degrees. The boundaries 1112 and 1113 are intended to be used by alternative Artcards to follow the vertical direction when data is read from the data area. The clock mark 1114 is intended to track horizontally as data is read from the data area. Separation of the boundary from the white dot line and the clock mark is desirable as a result of the blur effect occurring during reading. The boundary thus becomes a white black line on both sides, which helps to improve the frequency response during readout. A clock mark that alternates between white and black produces a similar result, except that it is a horizontal dimension, not a vertical dimension. Any alternative Artcard must find the location of clock marks and boundaries if it is intended to be used for tracking. The next section deals with targets designed to show clock marks, boundaries and directions to data.

Target in Target Area As shown in FIG. 54, two 15-dot wide target areas 1116 and 1117 exist in each data block. One is on the left side of the data area and the other is on the right side of the data area. The target area is separated from the data area by a row of dots used for orientation. The purpose of the target areas 1116 and 1117 is to specify the clock marks, boundaries, and how to go to the data area. Each target region includes 6 targets, eg, 1118, which are designed to be easily detectable by an alternative Artcard reader. Next, referring to FIG. 53, the structure of a single Target 1120 is shown. Each Target 1120 is a black square of 15 × 15 dots, and includes a central structure 1121 and a run-length encoded target number 1122. The central structure 1121 is a simple white cross, the target number component 1122 is a simple two rows of white dots, and each row is two dots long for each portion of the target number. Accordingly, target id 1122 for target number 1 is 2 dots long, target id 1122 for target number 2 is 4 dots wide, and so on.

As shown in FIG. 54, the target is positioned to have rotational invariance with respect to card insertion. That is, the left target and the right target are the same except that they are rotated 180 degrees. In the left target area 1116, the targets are arranged such that Target1 to 6 are positioned from the upper end to the lower end, respectively. In the right target area, the targets are arranged such that target numbers 1 to 6 are positioned from the lower end to the upper end. The target number id is always half way up to the data area. The enlarged partial view of FIG. 54 clearly shows that the right target is identical to the left target except that it is rotated 180 degrees.

As shown in FIG. 55, Targets 1124 and 1125 are particularly arranged in the target area with the center being 55 dots apart. In addition, the distance from the center of Target1 (1124) to the first clock mark dot 1126 in the upper clock mark area is 55 dots, and the first clock mark dot (not shown) in the lower clock mark area from the center of the target. ) Is 55 dots. The first black clock mark in both areas starts exactly coincident with the target center (the eighth dot position is the center of the 15 dot wide target).

The schematic block diagram of FIG. 55 shows the distance between the target centers and the distance from the upper boundary / clock mark area ON Target1 (1124) to the first dot of the first black clock mark (1126). ing. Since the distance from both the top and bottom targets to the clock mark is 55 dots and both sides of the alternate Artcard are symmetrical (rotated 180 degrees), the card is from left to right or from right Read to the left. Regardless of the readout direction, the orientation need not be determined to extract data from the data area.

As shown in FIG. 56, each data block has two 1-dot width orientation rows 1127 and 1128. One is immediately to the left of the data area and the other is to the right of the data area. An orientation column is provided to provide direction information to the alternative Artcard reader. A single white dot row 1127 is adjacent to the left of the data area (on the right of the left target). A single black dot row 1128 is adjacent to the right side of the data area (to the left of the right target). Since the target is rotationally invariant, these two dot rows allow the alternate Artcard reader to determine the alternate Artcard direction, ie, whether the card was inserted correctly or reversely. From the standpoint of alternative Artcard, assuming there is no dot degradation, there are two possibilities: That is:
* If the dot column on the left side of the data area is white and the column on the right side of the data area is black, the reader recognizes that the card was inserted as it was written;
* If the dot column on the left side of the data area is black and the column on the right side of the data area is white, the reader recognizes that the card is inserted backwards and the data area is rotated accordingly. . The leader must select the appropriate action to correctly restore information from the alternate Artcard.

Data Area As shown in FIG. 57, the data area of the data block is composed of 595 columns each including 384 dots, and there are 228480 dots in total. These dots are interpreted and decoded to obtain the original data. Since each dot represents a single bit, 228480 dots represent 228480 bits, or 28560 bytes. The interpretation of each dot is as follows.

しかし、ドットから得られたビットの実際の解釈は、原データから代替Ａｒｔｃａｒｄのデータ領域内のドットへのマッピングの理解が必要である。

However, the actual interpretation of the bits obtained from the dots requires an understanding of the mapping from the original data to the dots in the data area of the alternative Artcard.

Mapping of original data to data area dots Next, a process of acquiring an original data file having a maximum size of 910082 bytes and mapping it to dots in the data area of 64 data blocks on the 1600 dpi alternative Artcard will be described. To do. The alternate Artcard reader will reverse this process to extract the raw data from the dots on the alternate Artcard. At first glance, it seems simple to map data to dots, ie binary data consists of 1s and 0s, so black and white dots can be easily written to the card. Let's go. However, this scheme does not allow ink to fade and a portion of the card to be damaged by dirt, smudges, or even scratches. There is no way to detect whether the data acquired from the card is accurate unless error detection coding is performed. Further, there is no way to correct detected errors unless redundant coding is performed. Thus, the purpose of the mapping process is to ensure data recovery very reliably and to give the alternative Artcard reader the ability to recognize that the data has been read correctly.

There are three basic steps involved in mapping an original data file to data region dots:
* Redundant encoding of the original data;
* Sequentially shuffle the encoded data to reduce the effects of locally-generated alternative Artcard damage;
* Write the shuffled encoded data as dots in the alternate Artcard data block.

Each of these steps is discussed in detail in the following sections.

The mapping of redundant code data using Reed-Solomon coding to alternative Artcards is highly dependent on the method of using redundant coding. It is preferable to select a Reed-Solomon encoding that handles burst errors and has the ability to efficiently detect and correct errors with minimal redundancy. Reed-Solomon coding is based on Wicker,
S. And Bhargava,
V. By '&apos; Reed-Solomon Codes and
therir application '', IEEE Press, 1994, Rorabaugh,
'' Error Coding Cookbook 'by C, McGraw-Hill,
1994, and Lyppens, H. et al. By '' Reed-Solomon Error
Correction '', De. Dobb's Journal,
Volume 22,
Issue 1,
It is properly described in standard texts such as January 1997.

Various parameters for Reed-Solomon encoding can be used, including different symbol sizes and different redundancy levels. Preferably, the following encoding parameters:
* M = 8
* T = 64
Is used.

Setting m = 8 means that the symbol size is 8 bits (1 byte). The Reed-Solomon encoded block size n means 255 bytes (2 ⁸ -1 symbols). In order to correct up to t symbols, 2t symbols in the final block size must be employed using redundant symbols. T = 64 means that 64 bytes (symbol) can be corrected for each block when there is an error. Each block of 255 bytes has 128 (2 × 64) redundant bytes, and the remaining 127 bytes (k = 127) are used to hold the original data. In this way
* N = 255
* K = 127
It is.

In actual results, the 127-byte original data is encoded, and a 255-byte block of Reed-Solomon encoded data is obtained. The encoded 255-byte block is stored on the alternate Artcard and then returned to the original 127 bytes by decoding the alternate Artcard reader. 384 dots in a single row of the data area of the data block can hold 48 bytes (384/8). The 595 column can hold 28560 bytes. This amounts to 112 Reed-Solomon blocks (each block contains 255 bytes). The 64 data blocks of a complete alternative Artcard can hold a total of 7168 Reed-Solomon blocks (1827840 bytes when 255 bytes per Reed-Solomon block). Two of the 7168 Reed-Solomon blocks are reserved for control information, while the remaining 7166 are used to store data. Each Reed-Solomon block holds 127 bytes of actual data, and the total amount of data that can be stored on the alternate Artcard is 910082 bytes (7166 × 27). If the original data is less than this total amount, the data can be encoded to fit the exact number of Reed-Solomon blocks, and then all 7166 encoded blocks are used. Can be replicated until FIG. 58 is an explanatory diagram of the overall form of encoding used.

Each of the two control blocks 1132 and 1133 has the same encoding information necessary to decode the remaining 7166 Reed-Solomon blocks, i.e.
It contains the number of Reed-Solomon blocks in the full message (16-bit storage low / high) and the number of data bytes in the last Reed-Solomon block of the message (8 bits).

These two numbers are repeated 32 times (consuming 96 bytes), the remaining 31 bytes are reserved and set to 0. Each control block is then Reed-Solomon encoded and the 127-byte control information is converted to 255-byte Reed-Solomon encoded data.

Control blocks are stored twice to increase the likelihood of survival. Moreover, the repetition of data within the control block is particularly important when using Reed-Solomon codes. In an error-free Reed-Solomon coding block, the first 127 bytes of data are correct original data, and if the control block fails to decode (when there are more than 64 symbols in error), Can be referenced to restore the message. In this way, if the control block fails to decode, a set of 3 bytes can be examined to determine the most probable values of the two decoding parameters. It is not guaranteed that it can be restored, but redundancy increases the likelihood. For example, the last 159 bytes of the control block are destroyed and the first 96 bytes are perfectly fine. Referring to the first 96 bytes, a repeating set of numbers is found. These numbers can be exploited to decode the rest of the message in the remaining 7166 Reed-Solomon blocks.

As an example, assume a data file containing exactly 9967 bytes of data. The number of Reed-Solomon blocks required is 79. The first 78 Reed-Solomon blocks are fully utilized and 9906 bytes (78 × 127) are used. The 79th block contains only 61 bytes of data (the remaining 66 bytes are all 0).

The alternative Artcard is composed of 7168 Reed-Solomon blocks. The first two blocks are control blocks, the next 79 blocks are encoded data, the next 79 blocks are a copy of the encoded data, the next 79 blocks are another copy of the encoded data, and so on. The same goes on. After storing the 79 Reed-Solomon blocks 90 times, the remaining 56 Reed-Solomon blocks are another copy of the first 56 blocks of the 79 blocks of encoded data (the end of the encoded data). 23 blocks are not stored any more because there is not enough space on the alternate Artcard). A hexadecimal representation of 127 bytes in each control block data before Reed-Solomon encoding is shown in FIG.

Scramble processing of encoded data If all the encoded blocks are stored adjacent to each other in the memory, a maximum of 1827840 bytes of data (two control blocks and 7166 information blocks in total) 7168 Reed-Solomon coding blocks) can be stored on the alternate Artcard. Preferably, the data is not stored directly on the alternate Artcard at this stage, i.e. not all the 255 bytes of one Reed-Solomon block are physically stored on the card as one unit. . Any debris, dirt, or distortion that causes physical damage to the card can destroy more than 64 bytes in a Reed-Solomon block, making the block unrecoverable. If there is no copy of the Reed-Solomon block, the entire alternative Artcard cannot be decoded.

The solution to this is to take advantage of the fact that there are a large number of bytes on the alternate Artcard, and the alternate Artcard has a reasonable physical size. Thus, the data is scrambled to ensure that symbols from one Reed-Solomon block are not close to each other. Of course, in the abnormal case where the card deteriorates, it is impossible to restore the Reed-Solomon block, but on average, the card can be made very robust by data scrambling. The selected scrambling scheme is simple and is schematically illustrated in FIG. All bytes 0 from the Reed-Solomon block are placed together (1136), and so are all bytes 1 and so on. Therefore, there are 7168 bytes 0, 7168 bytes 1 and the like. Each data block on the alternate Artcard can store 28560 bytes. As a result of this, each of the 64 data blocks on the alternate Artcard contains approximately 4 bytes from each Reed-Solomon block.

Under this scrambling scheme, complete damage to the entire 16 data blocks on the alternate Artcard results in 64 symbol errors per Reed-Solomon block. That is, if there is no other damage to the alternative Artcard, all data can be completely restored even if there is no data replication.

When the scrambled encoded data is written to the alternative Artcard, and the original data is Reed-Solomon encoded, copied, and scrambled, it is necessary to store 1827840 bytes of data in the alternative Artcard. Each of the 64 data blocks on the alternate Artcard stores 28560 bytes.

Since the data is simply written to the alternate Artcard data block, the first data block stores the first 28560 bytes of the scrambled data, the second data block stores the next 28560 bytes, and so on. The same goes on.

As shown in FIG. 61, data is written from left to right with respect to columns within a data block. Therefore, the leftmost column in the data block stores the first 48 bytes of the 28560-byte scrambled data, and the last column stores the last 48 bytes of the 28560-byte scrambled data. To do. Within the column, bytes start at bit 7 and are written one bit at a time from top to bottom, ending with bit 0. If the bit is set (1), a black dot is placed in the alternate Artcard, and if the bit is cleared (0), the dot is not placed and remains the white background color of the card.

For example, a 1827840 byte data set can be created by scrambling 7168 Reed-Solomon encoded blocks to be stored in the alternate Artcard. The first 28560 bytes of data are written to the first data block. The first 48 bytes of the first 28560 bytes are written to the first column of the data block, the next 48 bytes are written to the next column, and so on. The first 2 bytes of 28560 bytes are hexadecimal D3
Assume a case of 5F. Bit 7 of byte 0 is stored first, then bit 6 is stored, and so on. Next, bit 7 of byte 1 to bit 0 of byte 1 are stored. Each “1” is stored as a black dot and each “0” is stored as a white dot, so these two bytes are represented on the alternate Artcard as the following set of dots:
* D3 (1101 0011) is black, black, white, black, white, white, black, black,
* 5F (0101 1111) is white, black, white, black, black, black, black, black.

Alternative Artcard Decoding This section deals with the process of extracting the original data in an accurate and reliable manner from the alternative Artcard. In particular, on the premise of the alternative Artcard format described in the previous chapter, a method for extracting original pre-encoded data from the alternative Artcard will be described.

There are a number of general considerations as part of the premise of decoding an alternative Artcard.

The purpose of the user alternative Artcard is to store data for use in various applications. The user inserts an alternate Artcard into the alternate Artcard reader and expects the data to be loaded within a “reasonable time”. From the user's perspective, motor transport moves the alternate Artcard to the alternate Artcard reader. This is not recognized as a problematic delay. This is because the alternative Artcard is moving. The time after the alternate Artcard stops is recognized as a delay and should be reduced to a minimum in the alternate Artcard readout scheme. In an ideal state, the entire alternative Artcard is read out during movement, so that no delay is recognized after card movement stops.

For the purposes of the preferred embodiment, a reasonable time to physically load the alternate Artcard is determined to be 1.5 seconds. The time for additional decoding after the movement of the alternative Artcard has stopped should be reduced to a minimum. Since the active area of the alternative Artcard covers most of the surface of the alternative Artcard, temporal interest can be limited to that area.

The dots on the dot sampling alternative Artcard are Nyquist's theorem
In order to satisfy (Theorem), it must be sampled by a CCD reader or the like at least twice the printed resolution. In practice, it is better to sample at a higher rate. In an alternative Artcard reader environment, the dots are preferably sampled at a resolution at which the dots were printed for each dimension, i.e. three times the resolution that required 9 pixels to define a single dot. If the alternate Artcard dot resolution is 1600 dpi, the alternate Artcard reader image sensor must scan the pixel at 4800 dpi. Of course, if the dots are not accurately aligned with the image sensor, in the worst and most likely case as shown in FIG. 62, the dots will be detected over a 4 × 4 pixel area.

Each pixel to be sampled is 1 byte (8 bits). The lower 2 bits of each pixel may contain significant noise. Accordingly, the decoding algorithm is required to have noise resistance.

It is very unlikely that the alignment / rotation user will insert the alternate Artcard into the alternate Artcard reader in full alignment without rotation. Certain physical constraints at the reader entrance and motor transport grip ensure that, after being inserted, the alternate Artcard is kept at the initial angle of insertion relative to the CCD. Preferably, this rotation angle is a maximum of 1 degree as shown in FIG. Although the angle may be slightly off due to jitter and motor play during the read process, it is considered to be basically within the limits of 1 degree.

The physical dimensions of the alternative Artcard are 86 mm x 55 mm. One rotation affects the required CCD length because when 86mm passes under the CCD, the effective height of the card is increased by 1.5mm (86sin1 °).

The effect of one degree of rotation on the alternate Artcard readout is that a single scan line from the CCD contains a number of different dot rows from the alternate Artcard. This is shown in an enlarged form in FIG. 63, which shows the drift of dots with respect to a pixel row. Although exaggerated in the figure, the actual drift is one pixel column at the maximum every 57 pixels.

When the alternate Artcard is not rotating, a single dot row can be read by three pixel scan lines. As the rotation of the alternative Artcard increases, the local impact increases. As the number of dots to be read increases, the time during which the influence of rotation is applied becomes longer. Increasing any of these factors increases the number of pixel scan lines that need to be read to obtain a given set of dots from a single row on the alternate Artcard. The following table shows the number of pixel scan lines required for a single dot row in a particular alternative Artcard structure.

代替Ａｒｔｃａｒｄの全体を読むため、８７ｍｍ（８６ｍｍと１°の回転による１ｍｍの和）を読む必要がある。４８００ｄｐｉの場合、これは、１６２５２列の画素列に相当する。

To read the entire alternative Artcard, it is necessary to read 87 mm (86 mm plus 1 mm with 1 ° rotation). In the case of 4800 dpi, this corresponds to 16252 pixel columns.

CCD (or other linear image sensor) length The length of the CCD itself is
-Physical height of alternative Artcard (55mm)
-Vertical movement (1mm) when inserting a physical alternative Artcard
-Rotation during insertion up to 1 ° (86 sin1 ° = 1.5mm)
Must conform to.

Combining these factors, the total length is 57.5 mm.

When the alternative Artcard image sensor CCD of the alternative Artcard reader scans at 4800 dpi, one scan line is 10866 pixels. For simplicity, this number is rounded up to 11000 pixels. The height of the active area of the alternative Artcard is 3208 dots, that is, 9624 pixels. The height of the data area is 384 dots, that is, 1152 pixels.

The amount of memory required for reading and decoding the DRAM size alternative Artcard is ideally reduced to a minimum. A typical alternative Artcard reader arrangement is a self-contained system, where memory resources are valuable. This becomes a more serious problem due to the effects of rotation. As described above, when the rotation of the alternative Artcard increases, the number of scanning lines required to efficiently restore the original dots increases.

There is a trade-off between algorithm complexity, user perceived delay, robustness, and memory usage. One of the simplest reader algorithms would simply scan the entire alternative Artcard and then process the entire data without real-time constraints. This will not only require a large amount of memory capacity, but will also take longer than a reader algorithm that runs in parallel with the alternate Artcard read process.

The amount of memory actually required for reading and decoding the alternate Artcard is twice the amount of space required to hold the encoded data with a small amount of scratch space (1 to 2 KB). For the 1600 dpi alternative Artcard, this means 4 MB of memory requirement. The actual memory usage is detailed in the algorithm description below.

Transfer rate DRAM bandwidth preconditions must be made to account for timing, and in particular, alternative Artcard readers are typically part of an embedded system and thus have some impact on algorithm design .

Rambus Inc's' Direct Rambus
Standard Rambus as described in Technology Disclosure '', Oct 1997
A Direct (Rambus Direct) RDRAM architecture is assumed, and the peak data transfer rate is 1.6 GB / sec. Assuming 75% efficiency (which can be easily realized), an average data transfer rate of 1.2 GB / sec is obtained. Therefore, the average time required to access a 16-byte block is 12 ns.

Contamination data It is possible that an alternative Artcard that has been physically damaged may be inserted into the reader. Alternative Artcards may be scratched or distorted by dirt or debris. It cannot be assumed that the alternative Artcard reader will read everything completely. The impact of contamination data is further exacerbated by the blur effect. This is because contamination data affects the surrounding clean dots.

The blur effect environment blur effect is incorporated into the alternative Artcard reading environment in two ways: * Natural blur effect due to the distance characteristics between the alternative Artcard and the CCD;
* Alternate Artcard distortion.

The natural blur effect of the alternative Artcard image occurs when there is an overlap of data detected from the CCD. The blur effect can be helpful. This is because the blur effect ensures that there is no high frequency in the detected data and that there is no data lost by the CCD. However, if the area covered by the CCD pixels is very large, a very large blur effect appears, and the sampling conditions required to restore the data are not satisfied. FIG. 64 is a schematic explanatory diagram of overlapping detection data.

Another form of blur effect occurs when the alternative Artcard is slightly distorted due to thermal damage. When the vertical dimension is distorted, the distance between the alternative Artcard and the CCD is not constant and the level of the blur effect varies within the area.

Black dots and white dots were chosen so that the alternative Artcard gives the best dynamic range in a blurry readout environment. The blur effect creates a problem when determining whether a given dot is black or white.

As the blur effect increases, a given dot is more affected by surrounding dots. As a result, the dynamic range of a particular dot is reduced. Consider the case where each white and black dot is surrounded by every possible set of dots. The blur effect appears on the nine dots and the center dot is sampled. FIG. 65 is an explanatory diagram of the distribution of central dot values obtained for black dots and white dots.

This figure is intended for a typical blur effect. Curve 1140 from 0 to about 180 shows the range of black dots. Curve 1141 from 75 to 250 shows the range of white dots. However, as the blur effect increases, the two curves shift towards the center of the range, increasing the intersecting area, and determining whether a given dot is black or white is more It becomes difficult. The pixel value at the center point of the intersection is ambiguous, and the possibility that the dot is black is the same as the possibility that it is white.

As the blur effect increases, the probability of read bit errors increases. Conveniently, the Reed-Solomon decoding algorithm can handle these successfully up to t symbol errors. FIG. 65 is a graph of the predicted number of alternative Artcard Reed-Solomon blocks that cannot be recovered for a particular symbol error rate. It should be noted that the Reed-Solomon decoding scheme shows excellent performance and then decreases substantially. If there is no copy of the Reed-Solomon block, it is impossible to restore data only by making an error in one block. Of course, when block duplication is performed, opportunities for alternative Artcard decoding increase.

FIG. 66 shows only symbol (byte) errors corresponding to the number of erroneous Reed-Solomon blocks. There is a trade-off between the amount of blur effect that can be dealt with and the amount of damage done to the card. Since all error detection and correction are performed by a Reed-Solomon decoder, the number of errors that can be handled for each Reed-Solomon data block is finite. As errors introduced by the blur effect increase, the number of errors that can be dealt with among the errors due to the damage of the alternative Artcard decreases.

Overview of Alternative Artcard Decoding As described above, when a user inserts an alternative Artcard into an alternative Artcard reading unit, motor transport ideally carries the alternative Artcard and passes by a monochrome linear CCD image sensor. The card is sampled at a resolution of three times the printed resolution for each dimension. Alternative Artcard reading hardware and software correct for blur effects caused by rotations up to 1 degree, jitter and vibration caused by motor transport, and variations in distance from the alternative Artcard to the CCD. A digital bit image of the data is extracted from the sampled image by a complex method described below. Reed-Solomon decoding corrects arbitrarily distributed data corruption of up to 25% of the raw data on the alternate Artcard. About 1 MB of correction data is extracted from the 1600 dpi card.

The steps related to decoding are as shown in FIG.

The steps required for the decryption process are:
Scanning the alternate Artcard at a resolution of three times the printed resolution (eg, scanning an alternate Artcard of 1600 dpi at 4800 dpi);
A step 1145 for extracting a data bitmap from the dots scanned on the card;
A step 1146 of inverting the bitmap if the alternate Artcard is inserted backwards;
A step 1147 of descrambling the encoded data;
* Step 1148 of Reed-Solomon decoding the data from the bitmap;
It is.

Algorithm Overview Phase 1-Real-time ExtractingBitImage
A simple comparison of the available memory (4MB) with the required memory (172.5MB) to hold all the scanned pixels of the 1600 dpi alternative Artcard, unless the card is read multiple times (realistic Extraction of the bitmap from the pixel data, which is not an option, needs to be performed in real time on the fly while the alternative Artcard is passed by the CCD. In this phase, there are two tasks:
* Task to scan alternate Artcard at 4800 dpi * Task to extract data bitmap from dots scanned on card must be performed.

Bit image rotation and descrambling cannot be performed until the entire bit image has been extracted. Therefore, it is necessary to allocate a memory area to hold the extracted bit image. Bit images are easily accommodated within the range of 2 MB, and 2 MB is reserved for use in the extraction process.

Rather than extracting a bit image while examining only the current scan line from the CCD, it is possible to allocate a buffer functioning as a window to the alternate Artcard and store the latest N scan line readings. The memory requirement does not allow the entire alternative Artcard to be stored in this way (172.5 MB would be required), but allocating 2 MB to store 190 pixel columns (each scan Lines do not require more than 11000 bytes), simplifying the ExtractingBitImage process.

Therefore, the usage of 4MB memory is as follows:
* 2MB for the extracted bit image;
* ~ 2MB for scanned pixels;
* 1.5KB for phase 1 scratch data (on request by algorithm).

The time required for Phase 1 is 1.5 seconds. This is because this is the time it takes for the alternate Artcard to pass by the CCD and be physically loaded.

Phase 2-Data Extraction from Bit Image When a bit image is extracted, the bit image is descrambled and may need to be rotated 180 °. The bit image is then decoded. Phase 2 has no real-time requirement in that the artcard motion is stopped, and the only problem is that the user recognizes the passage of time. Phase 2 thus relates to the remaining task of decoding the alternative Artcard. That is,
* Reorganize the bit image and invert if an alternate Artcard is inserted backwards;
* Descramble the encoded data;
* Reed-Solomon decoding of data from bit images.

The input to Phase 2 is a 2MB bit image buffer. Since descrambling and rotation cannot be performed on the spot, a second 2MB buffer is required. Since the 2 MB buffer used to hold the pixels scanned in phase 1 is no longer needed, it can be used to store the rotated unscrambled data.

The Reed-Solomon decoding task takes an unscrambled bit image and decodes it to 910082 bytes. Decryption can be performed at the original location, but may be performed at another designated location. The decoding process does not require an auxiliary memory buffer.

Therefore, the usage of 4MB memory is as follows:
* 2MB for the extracted bit image (from Phase 1);
* ~ 2MB for a possibly rotated bit image without scramble;
* <1 KB for phase 2 scratch data (on request by algorithm).

The time required for phase 2 depends on the hardware and is limited by the time required for Reed-Solomon decoding. By using a dedicated core such as LSI Logic L64712, or an equivalent CPU / DSP combination, it is estimated that Phase 2 takes 0.32 seconds.

Phase 1-ExtractingBitImage
This is the real-time phase of the algorithm and relates to extracting a bit image from an alternative Artcard as scanned by a CCD.

As shown in FIG. 68, Phase 1 can be broken down into two asynchronous process streams. The first of these simply reads the alternate Artcard pixels from the CCD in real time and writes the pixels to the DRAM. The second stream references the pixels and extracts the bits. The second process stream itself is broken down into two processes. The first process is a global process and relates to detecting the position of the alternative Artcard start. The second phase is specific to ExtractingBitImage.

FIG. 69 is an explanatory diagram of data flow from the viewpoint of data / process.

It is necessary to read a maximum of 16252 pixel columns for the entire timing 1600 dpi alternative Artcard. If the total time for the entire alternate Artcard is 1.5 seconds, this means that the maximum time per pixel column is 92296 ns during the course of various processes.

Process 1—Pixel Reading from CCD The CCD scans the alternative Artcard at 4800 dpi and generates 11000 1-byte pixels per column. This process only acquires data from the CCD and writes it to the DRAM, and is completely independent of other processes reading pixel data from the DRAM. FIG. 70 shows the related steps.

Pixels are written sequentially into a 2 MB buffer that can hold all 190 columns. The buffer always holds the 190 columns that have been read recently. As a result, the process (e.g., process 2 and process 3) attempting to read out pixel data first needs to know where the given column searches and then the requested data is actually It must be fast enough to be sure it is in the buffer.

Since Process 1 makes the current CurrentScanLine available to other processes, it can be assured that other processes do not attempt to access pixels from scan lines that have not yet been read.

The time required to write a single data string (11000 bytes) to the DRAM is 11000/16 * 12 = 8256 ns.

Process 1 therefore uses less than 9% (8256/92296) of the available DRAM bandwidth.

Process 2—Start detection of alternate Artcard This process is associated with the location of the active area on the scanned alternate Artcard. The input to this stage is pixel data from the DRAM (contained therein by process 1). The output is the set of boundaries for the first 8 data blocks on the alternate Artcard and is required as input to Process 3. A high level overview of process 3 is shown in FIG.

The alternative Artcard has a 1 mm vertical movement when inserted. One degree of rotation also causes a vertical movement of 1.5 mm (86 sin 1 °). As a result, the total vertical movement is 2.5 mm. At 1600 dpi, this corresponds to a movement of about 160 dots. Since the height of a single data block is only 394 dots, this movement is slightly less than half of the data block. In order to better estimate where the data block exists, the alternative Artcard itself must be detected.

Therefore, process 2 is
* The part that detects the start of the alternative Artcard, and if detected,
A part that calculates the boundaries of the first 8 data blocks based on the start of the alternative Artcard;
including.

ProcessColumn関数は単純である。走査された列の二つのエリアからの画素は、それらが黒色又は白色のどちらであるかを判定するため閾値フィルタに通される。一定数の白色画素を待ち、所定の個数が検出されたとき、代替Ａｒｔｃａｒｄのスタートを報せることが可能である。画素列を処理するロジックは以下の疑似コードに示されている。列中に代替Ａｒｔｃａｒｄが検出されなかった場合、０が返される。さもなければ、検出された位置の画素数が返される：

データブロック境界の計算
このステージでは、代替Ａｒｔｃａｒｄが検出されている。代替Ａｒｔｃａｒｄの回転に応じて、代替Ａｒｔｃａｒｄの上部又は代替Ａｒｔｃａｒｄの下部の何れかが検出されている。プロセス２の第２ステップはどちらが検出されたかを判定し、フェーズ３のためのデータブロック境界を適切にセットする。 Alternate Artcard Start Position Detection Pixels scanned outside the alternate Artcard area are black (the surface may be black plastic or other non-reflective surface). The border of the alternative Artcard area is white. If the pixel column is processed one by one and the pixel is filtered for either black or white, the transition point from black to white characterizes the start on the alternative Artcard. The highest level process is as follows:

The ProcessColumn function is simple. Pixels from the two areas of the scanned column are passed through a threshold filter to determine whether they are black or white. It is possible to wait for a certain number of white pixels and report the start of the alternative Artcard when a predetermined number is detected. The logic for processing the pixel columns is shown in the following pseudo code. If no alternative Artcard is found in the column, 0 is returned. Otherwise, the number of pixels at the detected position is returned:

Data Block Boundary Calculation At this stage, an alternative Artcard has been detected. Depending on the rotation of the alternative Artcard, either the upper part of the alternative Artcard or the lower part of the alternative Artcard is detected. The second step of process 2 determines which has been detected and sets the data block boundaries for phase 3 appropriately.

Looking at Phase 3, it can be seen that Phase 3 operates on data block segment boundaries. Each data block includes a StartPixel and an EndPixel to determine where to search for a target to locate the data area of the data block.

MaxPixel値はプロセス３で定義され、SetBounds関数は、0及びMaxPixelに対するStartPixel及びEndPixelクリッピングをセットするだけである。 If the pixel value is in the upper half of the card, it can be used as it is as the first StartPixel boundary. If the pixel value is in the lower half of the card, it can be reversed so that the pixel value is at the EndPixel boundary of the last segment. Move forward or backward by the width of the alternate Artcard data size and set an appropriate boundary for each segment. This is ready to start extracting data from the alternative Artcard:

The MaxPixel value is defined in process 3, and the SetBounds function only sets StartPixel and EndPixel clipping for 0 and MaxPixel.

Process 3-Extract Bit Data from Pixels This is the heart of an alternative Artcard reader algorithm. This process is related to bit data extraction processing from CCD pixel data. This process essentially creates a bit image from pixel data based on the scratch information created by process 2 and maintained by process 3. A high level overview of this process is shown in FIG.

Rather than simply reading the alternate Artcard pixel column and determining which pixel belongs to which data block, Process 3 performs the reverse. Process 3 knows where to search for pixels of a given data block. Process 3 does this by splitting the logical alternative Artcard into 8 segments. Each segment includes eight data blocks as shown in FIG.

The segment shown is consistent with the logical alternative Artcard. Physically, the alternative Artcard is likely to have been rotated a certain amount. Segments are independent of rotation because they remain fixed to the logical alternative Artcard structure. A segment can have one of two states:
* LookingForTaregets: In this state, the exact data block location for this segment has not yet been determined. The target is located by scanning pixel column data within the boundary specified by the segment boundary. When the data block is located using the target and the boundaries set for black and white, the state changes to ExtractingBitImage;
* ExtractingBitImage: In this state, the position of the data block is accurately detected, and data bits are extracted one by one at the same time and written to the alternative Artcard bit image. Tracking the data block clock mark gives accurate dot restoration independent of rotation, so segment boundaries are ignored. Once the entire data block has been extracted, a new segment boundary is calculated for the next data block based on the current position. The state changes to LookingForTargets.

The process ends when all eight 64 data blocks have been extracted from each region.

Each data block is composed of 595 data strings, and each data string is 48 bytes. Preferably, two orientation rows for data blocks are extracted every 48 bytes, and 28656 bytes are extracted per data block in total. For simplicity, a 2 MB memory can be divided into 64 × 32K chunks. The nth data block of a given segment is
SrartBuffer + (256k * n)
It is stored in the location represented by

Segment Data Structure Each of the eight segments is associated with a data structure. The data structure that defines each segment is stored in the scratch data area. Its structure can be listed in the table below.

プロセス３の上位レベル
プロセス３は、各セグメントに対して繰り返して、セグメントの現在状態に依存して単一ラインの処理を実行するだけである。擬似コードは以下のように簡潔である：

プロセス３は、指定された時間が経過しても終了しない場合には、外部制御プロセスによって停止される必要がある。これは、データが抽出できなかった場合に限られるであろう。簡単な仕組みは、プロセス１が代替Ａｒｔｃａｒｄの読み出しを終了した後にカウントダウンを開始する仕組みである。プロセス３がその時間までに終了しなかった場合、代替Ａｒｔｃａｒｄからのデータは復元不可能である。

The upper level process 3 of process 3 only repeats for each segment and performs a single line of processing depending on the current state of the segment. The pseudocode is concise as follows:

If the process 3 does not end even after the specified time has elapsed, it needs to be stopped by an external control process. This would only be the case if data could not be extracted. A simple mechanism is a mechanism in which the countdown starts after the process 1 finishes reading the alternative Artcard. If process 3 does not finish by that time, data from the alternate Artcard cannot be recovered.

ProcessPixelColumn関数
各画素列は、ターゲットを識別するある種の画素のパターンを探索するため、指定された境界内（StartPixelとEndPixelの間）で処理される。単一ターゲット（ターゲット番号２）の構造は、既に図５４に示されている通りである。 CurrentState = LookingForTargets
A target is detected by reading out a column of pixels one pixel column at a time, and is a certain pattern of pixels by detecting dots within a band of predetermined pixels (between StartPixel and EndPixel) Is not detected. The pixel columns are processed one column at a time until all targets are detected or a specified number of columns have been processed. At that point, the target can be processed and the data is located using clock marks. The state is changed to ExtractingBitImage to indicate that it is time to extract data. If a sufficient number of valid targets could not be located, the data block is ignored, ensuring that the columns in the failed data block are skipped and the process of searching for targets in the next data block is resumed. This can be represented by the following pseudo code:

ProcessPixelColumn Function Each pixel column is processed within a specified boundary (between StartPixel and EndPixel) to search for a pattern of certain pixels that identify the target. The structure of the single target (target number 2) is as shown in FIG.

From the pixel point of view, the target can be identified by:
* Left black region: This is a number of pixel columns made up of a number of adjacent black pixels, constructing the first part of the target;
* Target center: This is the white area at the center of the further black row;
* Target number: This is a white area surrounded by black, and the white area defines the target number by its length;
* Third black area: This is the two black dot areas after the target number.

An overview of the required process is shown in FIG.

Since the identification is based solely on black or white pixels, the pixels 1150 from each column are passed through a filter 1151 to detect black or white and then run-length encoded 1152. The run length is then sent to state machine 1153, which obtains the last three run lengths and the penultimate color. Based on these values, the target candidate passes through each of the identification stages.

The maximum and minimum collection process GatherMin & Max 1155 only holds the minimum and maximum values found during processing of the segment. These are used to set the BlackMax, WhiteMin, and MidRange values when the target is located.

Each segment holds a set of target structures in the target search. The target structures themselves do not move around in memory, but some segment variables point to a list of pointers to these target structures. These three pointer lists are shown again.

これらのリストポインタの各々と関連したカウンタ、即ち、TargetsFound、PossibleTargetCount、及びAvailableTargetCountが存在する。

There are counters associated with each of these list pointers, TargetsFound, PossibleTargetCount, and AvailableTargetCount.

Before the alternative Artcard is loaded, TargetsFound and PossibleTargetCount are set to 0, and AvailableTargetCount is 28 (the target boundary minimum size is 40 pixels and the data area is about 1152 pixels, so the target structure candidate under investigation The maximum number). An example of the target pointer layout is shown in FIG.

When new target candidates are detected, they are obtained from the AvailableTargets list 1157, the target data structure is updated, and a pointer to this structure is added to the PossibleTargets list 1158. When the target is fully verified, it is added to the Located Targets list 1159. If the target candidate is finally found not to be a target, the target candidate is returned to the AvailableTargets list 1157. As a result of this, there are always 28 target pointers that are always circulating and moving between lists.

The target data structure 1160 can take the following form:

ターゲット検出モジュール１１６２（図７４）内のProcessPixelColumn関数は、全てのランレングスを一つずつ調べ、（StartPixelによって）ランを既存のターゲット候補と比較するか、又は今までに知られていないターゲット候補が検出された場合に、新しいターゲット候補を作成する。全てのケースにおいて、比較はS0.colorが白色であり、S1.colorが黒色である場合に限り行われる。

The ProcessPixelColumn function in the target detection module 1162 (FIG. 74) examines all the run lengths one by one and compares the run (with StartPixel) to an existing target candidate, or a previously unknown target candidate If found, create a new target candidate. In all cases, the comparison is made only if S0.color is white and S1.color is black.

AddToTargetはターゲット検出モジュール内の関数であり、特定のランを所与のターゲットに加算することができるかどうかを判定する：
＊ランがターゲットの開始位置の許容範囲内に入るならば、そのランは現在のターゲットと直接的に関連付けられるので、それに適用することが可能である；
＊ランがターゲットの前で始まる場合、既存のターゲットが依然としてOKであり、そのランとは関係がないと想定する。したがって、ターゲットはそのままの状態で残され、返値FALSEは、ランが適用されなかったことを呼び出し元へ通知する。呼び出し元は、引き続いてランをチェックして、ランがその固有の全く新しいターゲットを開始するかどうかを調べる；
＊ランがターゲットの後で始まる場合、そのターゲットが候補ターゲットではなくなったと考える。状態は、NotATargetに変更され、返値TRUEが返される；
ランをターゲットに適用すべき場合、特別の動作が現在の状態と、S1、S2及びS3のランの組に基づいて実行される。AddToTargetの擬似コードは以下の通りである：

画素ランのタイプはDeterminRunTypeにおいて以下のように識別される。 Next, pseudo code of ProcessPixelColumn is shown. If the first target is positively identified, the last column to check for the target is determined to be within the maximum distance from it. If there is a 1 ° rotation, the maximum distance is 18 pixel columns:

AddToTarget is a function in the target detection module that determines whether a particular run can be added to a given target:
* If the run falls within the tolerance of the target start position, it can be applied to it because it is directly associated with the current target;
* If a run begins before a target, assume that the existing target is still OK and has nothing to do with that run. Therefore, the target is left as is, and a return value of FALSE notifies the caller that the run was not applied. The caller continues to check the run to see if the run starts its own entirely new target;
* If a run begins after a target, consider that the target is no longer a candidate target. The state is changed to NotATarget and the return value TRUE is returned;
When a run is to be applied to a target, a special action is performed based on the current state and the set of S1, S2, and S3 runs. The pseudo code for AddToTarget is as follows:

The type of pixel run is identified as follows in DeterminRunType.

状態推定手続EvaluateStateは現在の状態及びランタイプに依存して動作する。

The state estimation procedure EvaluateState operates depending on the current state and run type.

The operation is shown in tabular form below.

ターゲット処理
位置検出されたターゲット（LocatedTargetsリスト内）は、位置検出された順序で記憶される。代替Ａｒｔｃａｒｄの回転に依存して、これらのターゲットは、画素昇順又は画素降順である。その上、ターゲットから復元されたターゲット番号は、誤りがあるかもしれない。また、間違ったターゲットを復元したかもしれない。クロックマーク推定量が取得できる前に、ターゲットは、無効ターゲットが確実に廃棄されるように処理する必要があり、有効ターゲットは、間違っているならばターゲット番号が修復される（例えば、ごみのために破損されたターゲット番号）。二つの主要なステップが関連している：
＊ターゲットを画素昇順にソートするステップ；
＊誤りのあるターゲット番号を見つけ、修復するステップ。

Targets for which target processing positions have been detected (in the LocateTargets list) are stored in the order in which they have been detected. Depending on the rotation of the alternative Artcard, these targets are in pixel ascending order or pixel descending order. Moreover, the target number recovered from the target may be in error. You may have restored the wrong target. Before the clock mark estimator can be obtained, the target must be processed to ensure that invalid targets are discarded, and valid targets are repaired if the target number is incorrect (for example, because of garbage) Damaged target number). Two main steps are related:
* Sorting targets in ascending pixel order;
* Find and repair the wrong target number.

誤ったターゲット番号を位置検出し修復することは、多少複雑になる。順番に、検出されたＮ個のターゲットの各々が正しいと仮定される。他のターゲットは、この「正しい」ターゲットと比較され、TargetNが正しい場合に変更を必要とするターゲットの個数がカウントされる。変更の回数が０であるならば、全てのターゲットは既に正しい筈である。さもなければ、他のターゲットに対して要求する変更の回数が最も少ないターゲットが変更の基準として使用される。所与のターゲットのターゲット番号及び画素位置が、「正しい」ターゲットの画素位置及びターゲット番号と比較されたときに、相互に関連しないならば、変更が登録される。この変更は、ターゲットのターゲット番号の更新を意味し、或いは、ターゲットの削除を意味する。上昇するターゲットは、（既にソートされているので）昇順の画素を有すると仮定することができる：

多くの場合、この関数は
bestChanges = 0
で終了しており、これは変更が要求されないことを意味している。そうでない場合、変更を適用する必要がある。変更を適用する機能は、targetNumber との比較までは、（上述の擬似コードにおいて）変更をカウントすることと同一である。変更アプリケーションは以下の通りである：

変更ループの最後に、LocatedTargetsリストをコンパクトにして、全てのヌルターゲットを削除する必要がある。 The first step is simple. The nature of the target search means that data should always be stored in either pixel ascending order or pixel descending order. A simple swap sort allows up to 14 comparisons to be performed without swapping if the six targets are already correctly sorted. If the data is not sorted, 14 comparisons are made with 3 3 swaps. The following pseudocode shows the sorting process:

Locating and repairing the wrong target number is somewhat complicated. In turn, each of the detected N targets is assumed to be correct. Other targets are compared to this “correct” target and the number of targets that need to be changed if TargetN is correct is counted. If the number of changes is 0, all targets are already correct. Otherwise, the target that requires the least number of changes to other targets is used as the basis for change. A change is registered if the target number and pixel position of a given target are not interrelated when compared to the “correct” target pixel position and target number. This change means update of the target number of the target or deletion of the target. Ascending targets can be assumed to have ascending pixels (since they are already sorted):

In many cases this function is
bestChanges = 0
This means that no changes are required. If not, changes need to be applied. The function of applying changes is the same as counting changes (in the pseudocode above) until comparison with targetNumber. The modified applications are as follows:

At the end of the change loop, we need to compact the LocateTargets list and remove all null targets.

There may be several targets at the end of this procedure. All remaining targets (at least two targets are required) are used to detect the position of the clock mark and data area.

As shown in FIG. 55, the first clock mark dot 1126 in the upper region is 55 dots away from the center of the first Target 1124 (this is the same as the distance between the target centers). The center of the clock mark dot is further one dot away, and the black boundary line 1123 is further four dots away from the first clock mark dot. The first clock mark dot in the lower area is exactly 7 targets (7 × 55 dots) away from the first clock mark dot 1126 in the upper area.

Since it cannot be assumed that Targets 1 and 6 have been found, the uppermost target and the lowermost target need to be used, and the target number is used to determine which target is being used. At this point, at least two targets are required. Moreover, the target center is only an estimate of the actual target center. It is to detect the target center more accurately. The center of the target is white surrounded by black. Therefore, it is desirable to find local maxima in both pixel dimensions and column dimensions. This requires reconstructing successive images. This is because there is almost no possibility that the maximum value is strictly matched to the integer boundary (the estimated value in this example).

Before building a continuous image around the center of the target, it is necessary to make a better estimate of the two target centers. The existing target center is, in practice, the upper left coordinate of the target center bounding box. Examining each pixel one by one to find the area that defines the center of the target and finding the pixel with the highest value is a simple process. There may be two or more pixels having the same maximum pixel value, but it is sufficient to have one pixel for estimating the center value.

このプロセスの最後に、ターゲット中心座標は、実際の中心の１画素の範囲に収まるべきターゲットの最も白い画素を指示する。ターゲット中心のより正確な位置を構築するプロセスには、推定されたターゲット中心の両側に３個ずつで、ターゲットの７個の走査線スライスの連続信号を再構築することが含まれる。検出された７個の最大値（上記の画素スライスの各々につき１個ずつ）は、列次元において連続信号を再構築し、次に、その次元における最大値の位置を検出するために使用される：

FindMaxは、元の１次元信号に基づくサンプル点を再構築し、最大値の位置及び検出された最大値を返す関数である。使用される信号再構築／再サンプリング方法は、図７６に示されるように、Lanczos3窓sinc関数である。 The pseudocode is simple and is executed for each of the two targets:

At the end of this process, the target center coordinates indicate the whitest pixel of the target that should fall within the range of one pixel in the actual center. The process of building a more accurate location of the target center includes reconstructing a continuous signal of seven scan line slices of the target, three on each side of the estimated target center. The seven maximum values detected (one for each of the above pixel slices) are used to reconstruct a continuous signal in the column dimension and then detect the position of the maximum value in that dimension. :

FindMax is a function that reconstructs a sample point based on the original one-dimensional signal and returns the position of the maximum value and the detected maximum value. The signal reconstruction / resampling method used is a Lanczos 3 window sinc function, as shown in FIG.

Lancoz 3 window sinc functions were collected around the estimated position X from the reconstructed dimension, ie, 7 (pixels) at X-3, X-2, X-1, X, X + 1, X + 2, X + 3. Get a sample. Points are reconstructed at intervals of 0.1 between X-1 and X + 1 to determine the maximum point. The position with the maximum value becomes the new center. Due to the nature of the kernel, only 6 entries are required in the convolution kernel for points between X and X + 1. 6 points are used for X-1 to X, 6 points are used for X to X + 1, and a total of 7 points are required to obtain a pixel value between X-1 and X + 1 It is. This is because some of the requested pixels match.

これにより第１のクロックマークドットが得られる。列位置は、クロックマークの中心へ到達するため、データエリアから更に１ドット遠くへ動かすことが必要である。また、画素位置は、境界線の中心へ到達するため、更に４ドット遠ざける必要がある。deltaColumn及びdeltaPixelの疑似コード値は、５５ドットの距離（ターゲット間の距離）に基づいているので、これらのデルタは、クロックマーク座標に適用される前に、それぞれ、１／５５及び４／５５によって縮小しなければならない。これは以下の通りに表現される：
kDeltaDotFactor =
1/DOTS_BETWEEN_TARGET_CENTRES
deltaColumn *= kDeltaDotFactor
deltaPixel *= 4 * kDelteDotFactor
UpperClock.pixel -= deltaPixel
UpperClock.column -= deltaColumn
LowerClock.pixel += deltaPixel
LowerClock.column += deltaColumn
UpperClock及びLowerClockは、ターゲットの中心とちょうど合致した第１のクロックマークに対する有効クロックマーク推定値である。 If accurate estimates are given from the uppermost target to the lowermost target, the position of the first clock mark dot can be calculated for the upper and lower regions as follows: :

As a result, a first clock mark dot is obtained. In order to reach the center of the clock mark, the column position needs to be moved further one dot away from the data area. Further, the pixel position needs to be further separated by 4 dots in order to reach the center of the boundary line. Since the pseudocode values for deltaColumn and deltaPixel are based on a 55 dot distance (distance between targets), these deltas are 1/55 and 4/55, respectively, before being applied to clock mark coordinates. Must be reduced. This is expressed as follows:
kDeltaDotFactor =
1 / DOTS_BETWEEN_TARGET_CENTRES
deltaColumn * = kDeltaDotFactor
deltaPixel * = 4 * kDelteDotFactor
UpperClock.pixel-= deltaPixel
UpperClock.column-= deltaColumn
LowerClock.pixel + = deltaPixel
LowerClock.column + = deltaColumn
UpperClock and LowerClock are the effective clock mark estimates for the first clock mark that exactly matches the center of the target.

Before extracting setting data of black and white pixels / dot range from the data area, it is necessary to check the pixel ranges of black and white dots. The minimum and maximum values that appear in LookingForTaregets are stored in WhiteMin and BlackMax, respectively, but they do not represent valid values for these variables in terms of data extraction. They are only used for convenience of storage. The following pseudo code shows how to obtain good values for WhiteMin and BlackMax based on the smallest and largest pixels that appeared:
MinPixel = WhiteMin
MaxPixel = BlackMax
MidRange = (MinPixel + MaxPixel) / 2
WhiteMin = MaxPixel-105
BlackMax = MinPixel + 84
CurrentState = ExtractingBitImage
The ExtractingBitImage state is a state where the position of the data block has already been accurately detected by the target, and bit data is being extracted for each dot row at the same time, and is written in the alternative Artcard bit image. Data block clock mark / boundary tracking provides accurate dot restoration independent of rotation and ignores segment boundaries. When the entire data block (597 columns of 595 data columns and 2 bearing columns, 48 bytes each) is extracted, a new segment boundary is calculated for the next data block based on the current position. The This state is changed to LookingForTargets state.

The processing of a given dot sequence requires two tasks:
* The first task is to detect the position of the dot row of specific data using clock marks;
* The second task is to track the dot row and collect bit values one bit per dot.

ドット列の位置検出
所定のドット列は、ドットを読み出し、データを抽出する前に、位置検出されなければならない。これは、データブロックの上側及び下側境界に沿ってクロックマーク／境界線を追跡することによって実現される。位相ロックループと等価的なソフトウェアが使用され、たとえクロックマークが破損されていても、クロックマーク位置の良い推定が確実に行われる。図７７は、例示的なデータブロックの左上の説明図であり、そのデータブロックのコーナーには、ターゲットエリアの外側へ広がる３ドット高のクロックマーク１１６６と、白色行と、黒色境界線と、が現されている。 These two tasks can only be performed if the data in the column is read from the alternative Artcard and transferred to the DRAM. This is determined by checking which scan line process 1 has reached and comparing it to the clock mark train. When dot data is present in the DRAM, it is possible to extract data from the row before updating the clock mark and advancing the clock mark to the estimated value for the next dot row. An overview of this process is shown in the pseudo code below. Specific functions are described below:

Detecting the position of a dot row The position of a predetermined dot row must be detected before reading the dots and extracting the data. This is achieved by tracking clock marks / boundaries along the upper and lower boundaries of the data block. Software equivalent to a phase locked loop is used to ensure a good estimate of the clock mark position even if the clock mark is corrupted. FIG. 77 is an explanatory diagram at the upper left of an exemplary data block. At the corner of the data block, there are a 3-dot high clock mark 1166 extending outside the target area, a white line, and a black border. It is shown.

Initially, an estimate of the position of the first black clock mark is provided (based on the target position). A black border is used to obtain an accurate vertical position (pixel), and a clock mark 1166, for example, is used to obtain an accurate horizontal position (column). These are reflected in the positions of UpperClock and LowerClock.

所定の許容度（MAX_CLOCKMARK_DEVIATION）を上回る偏差が存在する場合、検出された信号は無視され、推定値からの最大許容度に収まる偏差だけが許容される。この点に関して、この機能はフェーズロックループの機能と類似している。したがって、DetermineAccurateUpperDotCenter関数は以下の疑似コードによって実現される：

また、DetermineAccurateLowerDotCenter関数は、境界からクロックマークまでの向きが負方向である（＋３ドットではなく、−３ドットである）点を除くと同様である。 A clock mark estimate is obtained and by examining the neighboring pixels, a continuous signal is reconstructed and the exact center is determined. This is a simple one-dimensional process and needs to be performed twice, since the two dimensions are broken down into clock marks and boundaries. However, this is done only once for a 2-dot row when there is a black clock mark on the other side of the alignment. In the case of a white clock mark, simply use the estimated value and leave the estimated value as it is. Alternatively, the pixel coordinates may be updated based on the boundary for each dot row (because it always exists). In practice, it is sufficient to update both coordinates every other column (including the black clock mark). This is because the resolution during this operation is very fine. So this process looks like this:

If there is a deviation that exceeds a predetermined tolerance (MAX_CLOCKMARK_DEVIATION), the detected signal is ignored and only a deviation that falls within the maximum tolerance from the estimate is allowed. In this respect, this function is similar to that of a phase-locked loop. Therefore, the DetermineAccurateUpperDotCenter function is realized by the following pseudo code:

In addition, the DetermineAccurateLowerDotCenter function is the same except that the direction from the boundary to the clock mark is negative (-3 dots instead of +3 dots).

GetAccuratePixel and GetAccurateColumn are functions that determine an accurate dot center from a one-dimensional viewpoint when given coordinates. Determining the exact dot center is the process of reconstructing the signal and finding the position where the minimum signal value was detected (this is because the target center is white, not black, so the maximum position of the signal is This is different from the process of detecting the position of the target center by detecting it). The method selected for signal reconstruction / resampling for this application is the Lanczos 3 window sinc function as described with reference to FIG.

The clock mark or boundary may have been damaged in some way, possibly by scratching. If the difference between the new center value obtained by resampling and the estimated value exceeds the tolerance, the center value is moved by the maximum tolerance. If it is an invalid position, it should be close enough to be used for data recovery, and additional clock marks are realigned to that position.

Determining Delta to First Data Dot Center and Subsequent Dots Once the exact UpperClock and LowerClock positions are determined, to advance to the first data dot center (CurrentDot) and subsequent dots in the row A delta amount (DataDelta) to be added to the center position can be calculated.

The first thing to do is calculate the dot row delta. This is achieved simply by subtracting UpperClock from LowerClock and dividing by the number of dots between the two points. In practice, it is possible to multiply the inverse of the number of dots. This is because the number of dots is a constant with respect to the alternative Artcard, and multiplication is performed at a higher speed. Different constants can be used to obtain the pixel dimension delta and the column dimension delta. The pixel delta is the distance between the two boundaries, and the column delta is between the centers of the two clock marks. In this way, the function DeternineDataInfo consists of two parts. The first part is represented by the following pseudo code:
kDeltaColumnFactor =
1 / (DOTS_PER_DATA_COLUMN + 2 + 2-1)
kDeltaPixelFactor =
1 / (DOTS_PER_DATA_COLUMN + 5 + 5-1)
delta = LowerClock.column-
UpperClock.column
DataDelta.column = delta *
kDeltaColumnFactor
delta = LowerClock.pixel-UpperClock.pixel
DataDelta.pixel = delta * kDeltaPixelFactor
Next, it is possible to determine the center of the first data in the column. The distance from the center of the clock mark to the center of the first data dot is 2 dots, and the distance from the center of the boundary to the center of the first data dot is 5 dots. The second part of this function is represented by the following pseudo code:
CurrentDot.column = UpperClock.column + (2 * DataDelta.column)
CurrentDot.pixel = UpperClock.pixel +
(5 * DataDelta.pixel)
Tracking the dot row Since the dot row was found from the phase locked loop following the clock mark, all that remains is to sample the dot row down the row at the center of each dot. The variable CurrentDot point is determined at the center of the first dot in the current row. To advance to the next dot in the row, you only need to add DataDelta (two additions: once for row coordinates and once for pixel coordinates). Dot sampling (bilinear interpolation) at predetermined coordinates is performed, and a pixel value representing the center of the dot is determined. The pixel value is then used to determine the bit value of that dot. However, in order to perform better bit determination, it is possible to use the pixel value in association with the center value for two neighboring dots on the same dot line.

It may be assumed that all the pixels for the dots in the extracted dot row are loaded into the DRAM at this point. This is because if both ends of the line (clock marks) are present in the DRAM, the dots between these two clock marks are definitely present in the DRAM. Furthermore, the data block height is low enough (only 384 dots high) to ensure that only a delta is sufficient to exceed the total length of the line. One reason the card is divided into 8 data block heights is because it is impossible to make the same strict guarantees that can be made over a single data block across the entire height of the card. It is.

GetPixel関数はドット座標（固定小数点）を取得し、バイリニア補間によって中心画素値に達するため、４個のＣＣＤ画素をサンプリングする。 The upper level process for extracting a single line of data (48 bytes) is represented by the following pseudo code: The data buffer pointer is incremented as each byte is saved, ensuring that consecutive bytes and columns of data are continuously saved:

The GetPixel function obtains dot coordinates (fixed point) and samples four CCD pixels to reach the center pixel value by bilinear interpolation.

The DetermineCenterDot function obtains a pixel value representing a dot center on both sides of a dot whose bit value has been determined, and tries to intelligently estimate the value of the bit value of the center dot. From the generalized blur effect curve of FIG. 64, the following three general cases can be considered: * The center pixel value of the dot is smaller than WhiteMin, so it is definitely a black dot. Therefore, the bit value is always 1;
* Since the dot center value is larger than BlackMax, it is definitely a white dot. Therefore, the bit value is always 0;
* The dot center value is somewhere between BlackMax and WhiteMin. The dots may be black or white. Therefore, the value of this bit becomes a problem. A number of schemes have been considered to reasonably estimate the bit value. These schemes balance complexity against accuracy, and in certain cases it must be taken into account that there is no guaranteed solution. In the case of erroneous bit determination, an error occurs in the Reed-Solomon symbol of the bit and must be corrected in the Phase 2 Reed-Solomon decoding stage.

The scheme used to determine the dot value when the pixel value falls between BlackMax and WhiteMin is not very complex but gives excellent results. This scheme uses the pixel values of the dot centers at the left and right of the gated dot to determine a more plausible value for the center dot. That is,
* If the two dots on both sides are present on the white side of the MidRange (average dot value), it can be assumed that if the center dot is white, it is “certainly” white. The presence of the central dot in the uncertain area indicates that the dot is black and has been affected by surrounding white dots to make the value more uncertain. Therefore, the bit value is 1 because the dot value is considered to be black;
* If two dots on both sides are present on the black side of the MidRange, it can be assumed that if the center dot is black, it is “certainly” black. The presence of the central dot in an uncertain area indicates that the dot is white and has been affected by surrounding black dots to make the value more uncertain. Therefore, the bit value is 0 because the dot value is considered white.
* If one dot is present on the black side of MidRange and the other dot is present on the white side of MidRange, the center dot value is used as it is. If the central dot is on the black side of MidRange, select black (bit value 1). Otherwise, white (bit value 0) is selected.

ここから、周辺画素値の使用は、中心ドットの状態の優れた指標を与え得ることが認められる。ここで説明した方式は、同じ行からのドットだけを使用しているが、単一のドットライン履歴（前のドットライン）の使用は、明らかに代替的な構成になるであろう。 This logic is expressed as follows:

From this it can be seen that the use of peripheral pixel values can provide an excellent indicator of the state of the central dot. Although the scheme described here uses only dots from the same row, the use of a single dot line history (previous dot line) would obviously be an alternative configuration.

Clock mark update for the next column Once the center of the first data dot in a column is determined, the clock mark value is no longer needed. They are more conveniently updated in preparation for the next column after the data for that column has been acquired. Because the clock mark orientation is perpendicular to the tracking direction of the dots descending along the dot row, use the pixel delta to update the row and use the column delta to update the pixels of both clocks Is possible:
UpperClock.column + = DataDelta.pixel
LowerClock.column + = DataDelta.pixel
UpperClock.pixel-= DataDelta.column
LowerClock.pixel-= DataDelta.column
At this point, an estimated value for the next dot row is obtained.

The timing timing requirement is met as long as the DRAM utilization does not exceed 100%, and the timing is increased by the addition of a parallel algorithm, but the DRAM utilization does not exceed 100%. DRAM utilization is specified with respect to process 1, which writes each pixel once and continuously, consuming 9% of the DRAM bandwidth.

Timing as described in this section indicates that the DRAM can easily accommodate the demands of an alternative Artcard reader algorithm. Thus, the timing bottleneck will be the implementation of algorithms related to logic speed rather than DRAM access. However, the algorithm is designed with architectural simplicity in mind and requires a minimal number of logical operations per memory cycle. In this regard, the target speed will be achieved as long as the implemented state machine or equivalent CPU / DSP architecture is feasible as described in the following subsections.

Target position detection The target is detected by reading out pixels within the boundary of the pixel column. At most one pixel is read out. Considering the case where the run-length encoder operates at a sufficiently high speed, the limit of target position detection is memory access. Thus, access will not be worse than the timing of process 1, ie, 9% utilization of DRAM bandwidth.

Thus, the total utilization of DRAM (including process 1) during target position detection is 18%, and the target locator always follows the alternative Artcard image sensor pixel reader.

Target processing The timing for sorting and checking target numbers is negligible. To find a better estimate of each of the two target centers, 12 sets of 12 pixel readouts are required, ie, a total of 144 readouts. However, accurate target-centric repair is not trivial and requires two sets of estimations. Each target center adjustment requires 8 sets of 20 6-entry convolution kernels. In this way, the sum reaches 8 × 20 × 6 times multiply-accumulate = 960. In addition, 49 memory accesses are required to obtain 7 sets of 7 pixels. Thus, the total number per target is 144 + 960 + 49 = 1153, which is approximately the same as the number of pixels in the column of pixels (1152). Thus, each target estimate consumes the time required to process a row of pixels. When there are two targets, the time required for substantially two pixel columns is consumed.

The target is reliably identified after the target number on the first pixel column. Since there are two dot rows before the azimuth row, there are six pixel rows. The target position detection process uses up the first pixel column, but the remaining five pixel columns are not processed at all. Thus, the data area can be located within 2/5 of the available time without affecting other process times.

The remaining 3/5 of the available time is sufficient for the trivial task of assigning a range of white and black pixels, and one task uses at most 2 to 3 machine cycles.

There are two parts to consider regarding data extraction timing:
* Obtain accurate clock marks and boundary values;
* Extract dot values.

Clock marks and boundary values are collected only every other dot row. However, every time the clock mark estimate is updated for more accuracy, 20 different 6-entry convolution kernels must be evaluated. On average, there are two of them per dot row (4 per 2 dot rows). To update the pixel coordinates based on the boundary, only 7 pixels from the same pixel scan line are required. However, to update the column coordinates, 7 columns from different columns, i.e. different scan lines, are required. Assuming the worst case situation where one cache miss occurs for each scan line entry and two cache misses occur for pixels in the same scan line, a total of eight cache misses occur.

In order to extract dot information, only 4 pixels are read out for each dot (instead of an average of 9 defining dots). Assuming a data area of 1152 pixels (384 dots), at best, this only saves 72 cache reads by reading 4 pixel dots instead of 9 pixel dots. The worst case is a 1 ° rotation that moves one column every 57 pixels, but the degree of saving is only slightly worse.

In the worst case, it may be said that the number of cache lines to be read is smaller than the cache lines used by the pixels in the data area. Thus, access will not worsen than Process 1, ie, 9% utilization of DRAM bandwidth.

Accordingly, the total DRAM utilization during data extraction (including process 1) is 18%, and the data extraction unit always follows the alternative Artcard image sensor pixel reader. In the target processing process thus implemented, target processing is performed in a fairly inefficient manner, but still follows quickly during the data extraction process as needed.

Phase 2-bit image decoding phase 2 is a non-real-time phase of the alternative Artcard data recovery algorithm. At the beginning of phase 2, the bit image has been extracted from the alternative Artcard. The bit image represents bits read from the data area of the alternative Artcard. Some bits are in error, and the entire data is rotated 180 °, probably because the alternate Artcard is rotated on insertion. Phase 2 is concerned with reliably extracting the original data from this encoded bit image. Basically, three steps are performed as shown in FIG. 79:
* Reverse and reorganize the bit image if the alternate Artcard is inserted backwards;
* Descramble the encoded data;
* Reed-Solomon decoding of data from bit images.

Each of the three steps is defined as a separate process, and the output of one process is required as the input of the next process, so it is executed sequentially. Although it is simple to merge the first two steps into a single process, for convenience of explanation, it is treated here as a separate process.

From the data / process point of view, Phase 2 has a structure as shown in FIG.

The timing of processes 1 and 2 can be ignored and consumes less than 1/1000 second during that time. Process 3 (Reed-Solomon decoding) consumes approximately 0.32 seconds, which is the total time required for Phase 2.

Bit image reorganization, and optionally a bitmap in the inversion DRAM, represents the data obtained from the alternative Artcard at this point. However, bit images are not continuous. The bit image is divided into 64 32k chunks, and there is one chunk for each data block. Each 32k chunk is only 28656 valid bytes, ie
48 bytes from the leftmost orientation column,
28560 bytes from the true data area,
48 bytes from the rightmost orientation row,
4112 bytes unused,
Is stored.

A 2MB buffer used for pixel data (stored by Phase 1 Process 1) can be used to hold the reorganized bit image. This is because pixel data is not required during phase 2. At the end of the reorganization, the correctly oriented consecutive bit images are contained in a 2 MB pixel buffer and are ready for Reed-Solomon decoding.

When the card orientation is correct, the leftmost orientation row is white and the rightmost orientation row is black. When the card is rotated 180 °, the leftmost orientation row is black and the rightmost orientation row is white.

データは、この時点で、カードが正しい向きであるかどうかに基づいて認識できるはずである。最も簡単なケースは、カードが正しい向きにされている場合である。この場合、データは、方位列を除去し、データ全体を連続させるために、少しだけ動かす必要がある。これは、以下の擬似コードに示されるように元の位置で非常に簡単に実現される：

それ以外の場合は、データを実際に反転させる必要がある。データを反転させるアルゴリズムは非常に簡単であるが、簡単にするために、２５６バイトのテーブルReverseを用いる。ここで、Reverse[N]の値はNのビット反転である：

両方のプロセスのタイミングは無視することが可能であり、１／１０００秒未満しか消費しない：
＊２ＭＢ連続読み出し（２０４８／１６×１２ｎｓ＝１５３６ｎｓ）；
＊２ＭＢ実効的連続バイト書き込み（２０４８／１６×１２ｎｓ＝１５３６ｎｓ）。 A simple way to determine if the card is oriented correctly is to use each data block one by one until the block with the overwhelming ratio of black to white bits is found. Is to check 48 bytes. The following pseudocode illustrates this method, returning TRUE when the card is in the correct orientation, FALSE otherwise:

The data should now be recognizable based on whether the card is in the correct orientation. The simplest case is when the card is oriented correctly. In this case, the data needs to be moved slightly in order to remove the orientation row and make the entire data continuous. This is achieved very easily in the original position as shown in the following pseudocode:

In other cases, it is necessary to actually invert the data. The algorithm for inverting the data is very simple, but for simplicity, a 256-byte table Reverse is used. Here, the value of Reverse [N] is a bit inversion of N:

The timing of both processes can be ignored and consumes less than 1/1000 second:
* 2 MB continuous reading (2048/16 × 12 ns = 1536 ns);
* 2 MB effective continuous byte write (2048/16 × 12 ns = 1536 ns).

The descrambling bit image of the encoded image is 1827840 consecutive, correctly oriented, scrambled bytes. The bytes must be descrambled to generate 7168 Reed-Solomon blocks, one block of which is 255 bytes long. Although the descrambling process is very simple, the descrambling process cannot be performed in place and requires a separate output buffer. FIG. 80 is an explanatory diagram of a memory in which the descrambling process is performed.

このプロセスのためのタイミングは無視することが可能であり、１／１０００秒未満しか消費しない：
＊２ＭＢ連続読み出し（２０４８／１６×１２ｎｓ＝１５３６ｎｓ）；
＊２ＭＢ非連続バイト書き込み（２０４８×１２ｎｓ＝２４５７６ｎｓ）。 The following pseudo code specifies how to perform the descrambling process:

The timing for this process can be ignored and consumes less than 1/1000 second:
* 2 MB continuous reading (2048/16 × 12 ns = 1536 ns);
* 2 MB non-consecutive byte write (2048 × 12 ns = 24576 ns).

At the end of this process, unscrambled data is ready for Reed-Solomon decoding.

The last part of the Reed-Solomon decoding alternative Artcard read is the Reed-Solomon decoding process, where about 2 MB of unscrambled data is decoded into about 1 MB of valid alternative Artcard data.

This algorithm can simultaneously decode one Reed-Solomon block and (if necessary) execute it in its original location. This is because the encoded block is larger than the decoded block, and redundant bytes are stored after the data bytes.

The first two Reed-Solomon blocks are control blocks and contain information about the size of the data to be extracted from the bit image. This meta information needs to be decrypted first, and the resulting information is used to decrypt the true data. Decoding true data is simply a problem of decoding data blocks one at a time at the same time. If decoding of a particular block fails, a reserved data block can be used.

DecodeBlockは、標準的なリード・ソロモンブロックデコーダであり、ｍ＝８及びｔ＝６４を使用する。 Present the top-level Reed-Solomon decoding in pseudocode:

DecodeBlock is a standard Reed-Solomon block decoder and uses m = 8 and t = 64.

The GetControlData function is simple as long as no decoding error occurs. This function only calls DecodeBlock to decode one control block at a time until successful. The control parameters can be extracted from the first 3 bytes of the decoded data (destBlocks are stored in bytes 0 and 1 and lastBlock
Is stored in byte 2). If there is a decoding error, the function must traverse 32 sets of 3 bytes to determine the most likely value set. One simple method is to find two consecutive equal copies out of 3 bytes and determine that these values are correct. Another method counts the number of types of 3 byte pairs that appear and determines that the most frequently occurring pair is the correct one.

The time required for Reed-Solomon decoding depends on the implementation. A dedicated core (eg L64712 in LSI logic) can be used to perform the Reed-Solomon decoding process, but depending on the application (usually to do something with the decoded data It is preferable to select a CPU / DSP combination that can be used more universally in the entire built-in system. Of course, the decoding time must be sufficiently fast for the CPU / DSP combination.

Since L64712 has a throughput of 50 Mbits per second (approximately 6.25 MB per second), the time is limited by the speed of the Reed-Solomon decoder, not the memory access time of up to 2 MB reads and 1 MB writes. The time required for the worst case (the case where all 2 MBs require decoding) is therefore 2 / 6.25 seconds = about 0.32 seconds. Of course, many improvements can be envisaged, including the following improvements.

The greater the blur effect of the readout environment, the greater the influence of a given dot from surrounding dots. The current readout algorithm according to the preferred embodiment has the ability to use surrounding dots in the same row to make better decisions regarding dot values. Since the dots in the previous row have already been decoded, the dot history in the previous row is useful in determining the values of the dots that fall within the range where the pixel values are uncertain.

Another possibility for the initial stage is to remove it completely, making the initial boundary of the data block wider than needed, making the target processing function more intelligent. This reduces the overall complexity. Care must be taken to maintain data block independence.

The control block mechanism can be made more robust:
* The control blocks may be the first and last blocks rather than being adjacent (as in the case of this example). This will increase protection against certain unusual damage situations;
* A second improvement is to incorporate an auxiliary level of redundancy / error detection into the control block structure that is used if the Reed-Solomon decoding step fails. Simple mechanisms such as parity improve the plausibility of control information if the Reed-Solomon stage fails.

The total time required to read the phase 5 Vark script execution Artcard9 and decrypt it is therefore approximately 2.15 seconds. The delay that is obvious to the user is actually only 0.65 seconds (sum of phases 3 and 4). This is because Artcard stops moving after 1.5 seconds.

Once Artcard is loaded, the Artvark script must be interpreted. Rather than immediately executing the script, the script is executed only when the “print” button 13 (FIG. 1) is pressed. The time it takes to execute the script will vary depending on the complexity of the script and should take into account the perceived delay between pressing the print button, the actual print button, and the actual print operation. .

As described above, the VLIW processor 74 is a digital processing system that accelerates the Vark function, which is expensive to use on a computer. The balance between functions executed in software by the CPU core 72 and functions executed in hardware by the VLIW processor 74 depends on the implementation. The purpose of the VLIW processor 74 is to help all Artcard styles execute in times that are not perceived as slow by the user. As the CPU becomes faster and more powerful, the number of functions that require hardware acceleration continues to decrease. The VLIW processor includes a microcoded ALU subsystem that enables general hardware acceleration of the following time-critical functions:
1) Image access mechanism for general software processing 2) Image convolver 3) Data driven image warper 4) Image scaling 5) Image mosaic 6) Affine transformation 7) Image compositor 8) Color space transformation 9) Histogram collector 10) Illumination of image 11) Brush stamper 12) Histogram collector 13) Conversion from CCD image to internal image 14) Construction of image pyramid (used for brushing by warper).

The following table summarizes the time required for each Vark operation when implemented in the ALU model. A method of implementing a function that uses the ALU model will be described later.

例えば、ＣＣＤ画像を変換するため、ヒストグラム収集、及び（画像強調用）ルックアップ色置換のために１画素当たりに９＋２＋０．５サイクル、即ち、１１．５サイクルを要する。１５００×１０００画像の場合、これは、１７２５０００００、即ち、１コンポーネント当たりに約０．２秒、即ち、全３コンポーネント当たりに０．６秒である。簡単なワープを追加すると、全体で、０．６＋０．３６、即ち、ほぼ１秒になる。

For example, converting a CCD image requires 9 + 2 + 0.5 cycles, or 11.5 cycles per pixel, for histogram collection and lookup color replacement (for image enhancement). For a 1500 × 1000 image, this is 172,500,000, ie about 0.2 seconds per component, ie 0.6 seconds for all three components. Adding a simple warp gives a total of 0.6 + 0.36, or approximately 1 second.

The image convolver convolve is a weighted averaging around the center pixel. This average may be a simple addition, a sum of absolute values, an absolute value of the sum, or a sum rounded by zero.

The image convolver is a general-purpose convolver, and various functions can be realized by changing values in the variable size coefficient kernel. The only kernel sizes supported are 3x3, 5x5, and 7x7.

Referring to FIG. 82, an example convolution process 340 is shown. The pixel component value supplied to the convolver process 341 is provided from the box readout iterator 342. The iterator 342 supplies image data in units of rows and supplies image data for each pixel in each row. The output from the convolver 341 is sent to the sequential writing iterator 344, which stores the composite image in an effective image format.

The coefficient kernel 346 is a lookup table in the DRAM. In the kernel, the coefficients are arranged in the same order as the box read iterator 342. Each coefficient entry is 8 bits. A simple sequential read iterator is used to specify the kernel 346 and provide the coefficients. This simulates an image with an image width ImageWidth that matches the kernel size and the loop option is set so the kernel is fed continuously.

One embodiment of the convolve process on the ALU unit is shown in FIG. The following constants are set by the software:

制御ロジックは、１画素当たりの乗算／加算の回数をカウントダウンするため使用される。カウント（ラッチ２に累算）が０に達するとき、生成される制御信号は（ラッチ１からの）現在のコンボルブ値を書き出し、カウントをリセットするため使用される。このようにして、一つの制御ロジックブロックは、多数の並列コンボルブストリームのために使用される。

The control logic is used to count down the number of multiplications / additions per pixel. When the count (accumulate in latch 2) reaches 0, the generated control signal writes the current convolve value (from latch 1) and is used to reset the count. In this way, one control logic block is used for multiple parallel convolution streams.

In each cycle, the multiplication ALU can perform a single multiplication / addition to incorporate the appropriate portion of the pixel. The number of cycles required to add up all values is therefore the number of entries in the kernel. Since this is a computational constraint, it is appropriate to divide the image into multiple sections and process them in parallel with different ALU units.

On a 7 × 7 kernel, the time required for each pixel is 49 cycles, ie 490 ns. Since each cache line holds 32 pixels, the time available for memory access is ((32-7 + 1) × 490 ns), ie 12740 ns. The time taken to read seven cache lines and write one is worse, 1120 ns (8 * 140 ns assuming all accesses are to the same DRAM bank). As a result, up to 10 pixels can be processed in parallel if the resources are unlimited. If the number of ALUs is limited, up to 4 pixels can be processed in parallel. Therefore, the time required to perform the convolution using the 7 × 7 kernel is 0.18375 seconds (1500 * 1000 * 490 ns / 4 = 1837500000 ns).

On a 5 × 5 kernel, the time required for each pixel is 25 cycles, ie 250 ns. Since each cache line holds 32 pixels, the time available for memory access is ((32-5 + 1) × 250 ns), ie 7000 ns. The time required to read 5 cache lines and write 1 is a worse case, 840 ns (6 * 140 ns assuming all accesses are to the same DRAM bank). As a result, if the resources are unlimited, it is possible to process up to 7 pixels in parallel. If the number of ALUs is limited, up to 4 pixels can be processed in parallel. Therefore, the time required to perform the convolution using the 5 × 5 kernel is 0.09375 seconds (1500 * 1000 * 250 ns / 4 = 93750,000 ns).

On a 3 × 3 kernel, the time required for each pixel is 9 cycles, ie 90 ns. Since each cache line holds 32 pixels, the time available for memory access is ((32-3 + 1) × 90 ns), ie 2700 ns. The time required to read three cache lines and write one is a worse case, 560 ns (4 * 140 ns assuming all accesses are to the same DRAM bank). As a result, up to 10 pixels can be processed in parallel if the resources are unlimited. If the number of ALUs and read / write iterators is limited, up to 4 pixels can be processed in parallel. Therefore, the time required to perform the convolution using the 3 × 3 kernel is 0.03375 seconds (1500 * 1000 * 90 ns / 4 = 33750,000 ns). As a result of this, each output pixel requires a kernel size / 3 cycles to calculate. The actual timing is summarized in the following table.

画像コンポジッター
コンポジット処理は、最終画像における前景及び背景の適切な比率を制御するため、マット又はチャンネルを使用して前景画像を後景画像に加算することである。好ましくは、通常コンポジット処理及び付随コンポジット処理の二つのスタイルのコンポジット処理がサポートされる。二つのスタイルの規則は、
通常コンポジット処理：新しい値＝前景＋（背景−前景）ａ
付随コンポジット処理：新しい値＝前景＋（１−ａ）背景
である。

The image compositor composite process is to add the foreground image to the background image using a mat or channel to control the proper ratio of foreground and background in the final image. Preferably, two styles of composite processing are supported: normal composite processing and accompanying composite processing. The two style rules are:
Normal composite processing: new value = foreground + (background-foreground) a
Accompanying composite processing: new value = foreground + (1-a) background.

The difference is that in the case of accompanying composite processing, the foreground is pre-multiplied by the mat, whereas in normal composite processing it is not. An example of the composite process is shown in FIG.

The alpha channel takes a value from 0 to 255 corresponding to the range 0 to 1.

Normal composite Normal composite is
Background + (Background-Foreground) * α / 255
Implemented as

The division by X / 255 is approximated by 257X / 65536. An example implementation of the composite process is shown in detail in FIG. 84, with the following constants set by software.

４個のイタレータが必要であるため、コンポジットプロセスは１画素当たりに１サイクルを要し、ＡＬＵの半分しか利用しない。コンポジットプロセスは一つのチャンネルだけで実行される。３チャンネル画像を互いにコンポジットするためには、コンポジッターは、各チャンネルに１回ずつの合計３回実行されなければならない。

Since four iterators are required, the composite process takes one cycle per pixel and uses only half of the ALU. The composite process is performed on only one channel. In order to composite 3 channel images with each other, the compositor must be run a total of 3 times, once for each channel.

The time required for compositing a full-size single channel is 0.015 seconds (1500 * 1000 * 1 * 10 ns), and 0.045 seconds are required to composite all three channels.

In order to approximate the division by 255, it is multiplied by 257 and then divided by 65536. This can also be achieved by one addition (256 * x + x) and ignoring the last 16 bits of the result (except for rounding purposes).

As shown in FIG. 42, the composite process requires three sequential read iterators 351-353 and one sequential write iterator 355 and is implemented as microcode that uses an adder with an adder ALU along with a multiplier ALU. Is done. The composite time is one cycle (10 ns) per pixel. The accompanying composite processing and normal composite processing require separate microcodes, but the average time per pixel is the same.

Composite processing is performed on a single channel only. In order to composite one 3-channel image with each other, the compositor is executed three times, once for each channel. Since the a channel is the same for each composite, it needs to be read each time. However, the transfer (reading or writing) of a 4 × 32 byte cache line in the best case takes 320 ns. Since the pipeline gives an average of one cycle per pixel composite and it takes 32 cycles, or 320 ns (100 MHz) to composite 32 pixels, the channel is read out indefinitely. Therefore, all channels
1500/32 * 1000 * 320 ns = 15040000 ns = 0.015 seconds are composited.

The time required for compositing a full-size three-channel image is 0.045 seconds.

Some functions, such as image pyramid construction warping, tiling, and brushing, require an average value for an area of a given pixel. Rather than calculating the value for each given area, these functions preferably utilize an image pyramid. As already shown in FIG. 33, the image pyramid 360 is substantially a multi-resolution pixel map. The original image is a 1: 1 representation. Sub-sampling with 2: 1 in each dimension produces an image that is 1/4 the original size. This process continues until the entire image is represented by a single pixel.

The image pyramid is constructed from the original image and uses 1/3 of the size occupied by the original image (1/4 + 1/16 + 1/64 + ...). If the original image is 1500 × 1000, the corresponding image pyramid is about ½ MB.

An image pyramid can be constructed by a 3 × 3 convolution that is performed on a one-in-four input image that advances the convolution kernel by two pixels in each dimension. The 3 × 3 convolve has the added advantage that it produces higher accuracy than a simple 4-pixel averaging, and the difference in coordinates of different pyramid levels is obtained by shifting one bit per level.

Building the entire pyramid relies on a software loop that calls the pyramid level building function once for each level of the pyramid.

The timing for generating one level of the pyramid is (9/4) * (1/4) of the resolution of the input image. This is because an image having a size ¼ of the original image is generated. In this way, for a 1500 × 1000 image,
Timing to generate level 1 of the pyramid = 9/4 * 750 * 500 = 84750 cycles,
Timing to generate level 2 of the pyramid = 9/4 * 375 * 250 = 210938 cycles,
Timing to generate level 3 of the pyramid = 9/4 * 188 * 125 = 52735 cycles,
The same goes for the following.

The total time is 3/4 cycles per pixel of the original image (the image pyramid is 1/3 of the original image size and each pixel requires 9/4 cycles to calculate, ie (1/3) * ( 9/4) = 3/4). In the case of a 1500 × 1000 image, there are 1125000 cycles at (100 MHz), that is, 0.011 seconds. This timing is for a single color channel, and the three color channel requires a processing time of 0.034 seconds.

The general-purpose data-driven image warper ACP31 can execute an image warping operation for an input image. The principle of image warping is theoretically well known. One complete textbook on the warping process is Gerage
'Digital Image Warping' by Wolberg, IEEE Computer Society Press,
Los Alamitos,
California,
1990. The warping process uses a warp map that forms part of the data supplied by Artcard 9. The warp map can be arbitrarily sized according to the requirements and provides information on the mapping of input pixels to output pixels. Unfortunately, the use of an arbitrarily sized warp map results in a number of problems to be solved by the image warper.

Referring to FIG. 85, the A × B-dimensional warp map 365 is composed of array values of a certain size (for example, 8-bit values from 0 to 255), and these array values represent the coordinates of the theoretical input image. The coordinates of the theoretical input image are then mapped to the corresponding “theoretical” output image having the same array coordinate index. Unfortunately, the output image, eg, output image 366, has a unique dimension C × D, which is quite different from an input image that has a unique dimension E × F. Therefore, it is necessary to make it possible to easily map the warp map 365 so that the output image 366 can determine a corresponding area or region of the input image for each output pixel. The output pixel color data is constructed from the area or region of the corresponding input image. For each output pixel in the output image 366, it is first necessary to determine the corresponding warp map value from the warp map 365. For this reason, it is necessary to bilinearly interpolate the peripheral warp map values when the output image pixels are mapped to decimal positions in the warp map table 365. The result of this process gives the position of the input image pixel in the “theoretical” image that is dimensioned by the size of each data value in warp map 365. These values must be rescaled to map the theoretical image to the corresponding actual input image 367.

In order to determine the actual value, the output image pixel should be chosen to avoid aliasing effects, and the adjacent output image to determine the area of the input image pixel 367 that contributes to the final output image pixel value. It is necessary to examine the pixel. In this regard, an image pyramid is utilized, as will become more apparent below.

The image warper performs several tasks to warp an image:
Scaling the warp map to match the output image size;
-Determining the span of the region of input image pixels represented in each output pixel;
Calculate the final output pixel value by trilinear interpolation from the input image pyramid.

Warp Map Scaling As described above, in the data driven warp, the warp map needs to describe the center of the corresponding input image map for each output pixel. Instead of using a single warp map as described above, storing the interleaved x and y value information allows the X and Y coordinates to be treated as separate channels.

As a result, preferably two warp maps are provided, the X warp map representing X coordinate warping and the Y warp map representing Y coordinate warping. As mentioned above, warp map 365 may have a different spatial resolution than the image being scaled (eg, 32 × 32 warp map 365 suitably describes the warp of 1500 × 1000 image 366). Moreover, the warp map can be represented by 8 or 16 bit values corresponding to the size of the image being warped.

Several steps are included to generate points from a given warp map in the input image space:
1. Determine the corresponding position in the warp map for the output image;
2. Fetch values from the warp map for the next step (this requires scaling in the resolution domain if the warp map is only 8 bit values);
3. Bilinear interpolation of the warp map to determine the actual value;
4). Scale the value to correspond to the input image domain.

The first step is performed by multiplying the current X / Y coordinate in the output image by a scale factor (which may be different for X and Y). For example, if the output image is 1500 × 1000, the warp map is 150 × 100 and both X and Y are scaled by 1/10.

In order to fetch values from the warp map, it is necessary to access two lookup tables. One lookup table reads an 8- or 16-bit entry from the lookup table and always returns a 16-bit value (if the original value has only 8 bits, the upper 8 bits are cleared).

The next step in the pipeline is to bilinearly interpolate the referenced warp map values.

Finally, the result from bilinear interpolation is scaled and accommodated in the same domain as the image to be warped. Thus, if the warp map range is from 0 to 255, X is scaled by 1500/255 and Y is scaled by 1000/255.

The interpolation process is as shown in FIG. 86, and the following constants are set by software.

以下のルックアップテーブルが使用される。

The following lookup table is used.

スパン計算
ワープマップ３６５からの点は入力画像３６７内の画素領域の中心の位置を示す。隣接した出力画像画素の入力画像画素の間の距離は領域のサイズを示し、この距離は、スパン計算によって近似することができる。

The point from the span calculation warp map 365 indicates the position of the center of the pixel area in the input image 367. The distance between adjacent output image pixels to the input image pixels indicates the size of the region, and this distance can be approximated by span calculation.

Referring to FIG. 87, with respect to a predetermined current point of warp map P1, the previous point on the same line is P0, and the point at the same position on the previous line is P2. The absolute distance between X and Y is determined between P1 and P0 and between P1 and P2. The maximum distance of X or Y is a span that is a square that approximates the actual shape.

Preferably, the points are processed in the output order of the vertical strip, P0 is the point preceding the same line in the strip, and when P1 is the first point in the strip, P0 is the corresponding point in the previous strip Represents the last point of the line. Since P2 is the point of the previous line in the same strip, it can be held in a 32-entry history buffer. The principle of the span calculation process is shown in FIG. 88, and details of the process are shown in FIG.

The following DRAM FIFOs are used:

３２ビット精度のスパン履歴が保持されるので、１５００画素幅の画像がワープされる場合、１２０００バイトの一時記憶が必要になる。

Since a span history with 32-bit accuracy is maintained, when an image having a width of 1500 pixels is warped, 12000 bytes of temporary storage is required.

The span 364 calculation uses two adder ALUs (one for span calculation and the other for looping and counting the history of P0 and P2) and requires 7 cycles as follows: .

履歴バッファ３６５、３６６は、キャッシュ型のＤＲＡＭである。「前のライン」（Ｐ２履歴の場合）バッファ３６６は、３２エントリーのスパン精度である。「前の点」（Ｐ０履歴の場合）バッファ３６５は、（ストリップの線の点１から３１の計算のため）殆どの場合に使用される１個のレジスタを必要とし、ＤＲＡＭにバッファされた履歴値の組はストリップの線の点０の計算に使用される。

The history buffers 365 and 366 are cache type DRAMs. The “Previous Line” (for P2 history) buffer 366 has a span accuracy of 32 entries. The “previous point” (for P0 history) buffer 365 requires one register that is used in most cases (for the calculation of strip line points 1 to 31) and history buffered in DRAM. The set of values is used to calculate point 0 of the strip line.

The 32-bit accuracy of span history requires 4 cache lines to hold the P2 history and 2 cache lines for the P0 history. The history of P0 is written to and read from the temporary storage space of (ImageHeight * 4) bytes once for every 8 lines of 32 pixels. Thus, when an image having a height of 1500 pixels is warped, 6000 bytes of temporary storage is required, and a total of 6 cache lines are required.

After determining the center and span of the area from the input image to be trilinear interpolation averaged, the final part of the warping process is to determine the value of the output pixel. Since a single output pixel can theoretically be represented by the entire input image, it can be very time consuming to actually read and average a particular area of the input image that contributes to the output pixel. . Instead, pixel values can be approximated by using an image pyramid of the input image.

If the span is less than 1, it is only necessary to read out the pixels of the original image around the given coordinates and perform bilinear interpolation. If the span is greater than 1, it is necessary to read two appropriate levels of image pyramids and perform trilinear interpolation. Performing linear interpolation between the two levels of the image pyramid is not exactly accurate, but gives acceptable results (it misjudges on the side where the resulting image has a blur effect).

Referring to FIG. 90, it is generally necessary to read the image pyramid levels given by ln ₂ s (370) and ln ₂ s + 1 (371) given a certain span “s”. is there. Ln ₂ s only decodes the upper bit set of s. Bilinear interpolation must be performed to determine the value of the pixel value at each of the two levels 370, 371 of the pyramid, and interpolation must be performed between the levels.

As shown in FIG. 91, to obtain the final output value 373, it is necessary to first interpolate X and Y for each pixel pyramid level before interpolating between the pyramid levels.

The image pyramid address mode generates addresses for pixel coordinates at (x, y) on pyramid levels s and s + 1. Each level of the image pyramid stores consecutive pixels in the x direction. Therefore, reading about x is likely to cause a cache hit.

Moderate cache hit coherence is obtained when the local region of the output image is typically locally coherent with the input image (perhaps at different scales but at the same scale). Although it is not possible to know the relationship between the input and output images, the output pixels are reliably written to the vertical strip (by the vertical strip iterator) to make the best use of cache coherence.

Trilinear interpolation uses four multiplication ALUs and all four adder ALUs as a pipeline, and can be completed in about two cycles on average assuming no memory access. However, since all the interpolated values are derived from the image pyramid, the interpolation speed depends entirely on cache coherence (it goes without saying that the other units are busy performing warp map scaling and span calculations). As many cache lines as possible should be available for image pyramid readout. The best speed is 8 cycles using 2 multiply ALUs.

The output pixel is written to the DRAM by a vertical strip write iterator that uses two cache lines. This speed is therefore limited by a minimum of 8 cycles per output pixel. If the number of cycles required for scaling the warp map is 8 cycles or less, the overall speed does not change. Otherwise, throughput is the time required to scale the warp map. In most cases, the warp map is enlarged until it matches the size of the photo.

When scaling a warp map with 8 cycles or less per pixel, the time required to convert one color component of the image is 0.12 seconds (1500 * 1000 * 8 cycles * per cycle) 10 ns).

Histogram collector A histogram collector is a microcode program that acquires an image channel as input and generates a histogram as output. Each pixel of the channel takes a value that falls within the range 0-255. As a result, there are a total of 256 pixels in the histogram table, each entry is 32 bits, and can fully accommodate the count of all 1500 × 1000 images.

As shown in FIG. 92, the sequential readout iterator 378 is sufficient as an input because the histogram represents a summary of all images. The histogram itself is fully cached and requires 32 cache lines (1K).

The microcode has two passes, with the initialization pass all setting the count to zero, then the “count” stage increments the appropriate counter for each pixel read from the image. The first stage requires an address unit and a single adder ALU and initializes the address of the histogram table 377.

第２のステージは、画像からの実際の画素を処理し、４個のアダーＡＬＵを使用する。

The second stage processes the actual pixels from the image and uses 4 adder ALUs.

アダー２のサイクル２からの零フラグは、入力画素が同じである間、マイクロコードアドレス２に留まるため使用される。入力画素が変化したとき、新しいカウントはマイクロコードアドレス３に書き出され、処理はマイクロコードアドレス２で再開する。マイクロコードアドレス４は最後に使用され、それ以上読み出される画素は存在しない。

The zero flag from cycle 2 of adder 2 is used to stay at microcode address 2 while the input pixels are the same. When the input pixel changes, a new count is written to microcode address 3 and processing resumes at microcode address 2. Microcode address 4 is used last and there are no more pixels to be read.

Stage 1 requires 256 cycles, ie 2560 ns. Stage 2 changes according to the value of the pixel. The worst case time for lookup table replacement is 2 cycles per image pixel if every pixel is not the same as its neighbor. The time required for a single color lookup is 0.03 seconds (1500 × 1000 × 2 cycles per pixel × 10 ns per cycle = 30000000 ns). The time required for the three color components is three times this amount, ie 0.09 seconds.

Color conversion
It is implemented in two main ways: lookup table replacement and color space conversion.

Lookup Table Replacement As shown in FIG. 86, one of the simplest ways to convert the color of a pixel is to encode an arbitrarily complex conversion function into the lookup table 380. The component color value of the pixel is used to look up 381 the new component value of the pixel. For each pixel read from the sequential read iterator, the new value is read from the new color table 380 and written to the write iterator 383 sequentially. The input images are processed in half at the same time in order to efficiently use the memory bandwidth. The following lookup table is used.

総プロセスは、二つの順次読み出しイタレータ及び二つの順次書き込みイタレータを必要とする。二つの新カラーテーブルは、２５６バイト（１バイトからなる２５６個のエントリー）を保持するため、各々に８個のキャッシュラインが必要である。

The total process requires two sequential read iterators and two sequential write iterators. Since the two new color tables hold 256 bytes (256 entries consisting of 1 byte), each requires 8 cache lines.

Therefore, the average time required for look-up table replacement is ½ cycle per image pixel. The time required for a single color lookup is 0.0075 seconds (1500 × 1000 × 1/2 cycle per pixel × 10 ns per cycle = 7500000 ns). The time required for the three color components is three times this amount, ie 0.0225 seconds. Each color component is processed one after another under software control.

Color space conversion Color space conversion is necessary only when moving between color spaces. CCD images are taken in the RGB color space and printed in the CMY color space, while ACP31 clients are likely to process images in the Lab color space. All of the input color space channels are typically required to determine the component value for each output channel. Such a logical process is illustrated at 385 in FIG.

In short, the conversion between Lab, RGB and CMY is fairly simple. However, the individual color profile of a particular device can vary significantly. As a result, the ACP 31 performs color space conversion by trilinear interpolation from the color space conversion lookup table to enable future CCDs, inks, and printers.

Color coherence tends to be area-based rather than line-based. It is best to process the image with vertical strips to aid cache coherence during the trilinear interpolation lookup. Thus, the read iterator 386-388 and the write iterator 389 are vertical strip iterators.

Trilinear color space conversion For each output color component, a single three-dimensional table that maps the input color space to the output color space is required. For example, to convert a CCD image from RGB to Lab, three tables calibrated to the physical characteristics of the CCD, namely:
RGB-> L
RGB-> a
RGB-> b
Is required.

Three tables calibrated to the physical characteristics of the ink / printer for the conversion from Lab to CMY, ie
Lab-> C
Lab-> M
Lab-> Y
Is required.

The 8-bit input color component is treated as a fixed point number (3: 5) for indexing into the conversion table. The integer 3 bits give an index and the decimal 5 bits are used as interpolation. Since 3 bits give 8 values, 512 entries (8 × 8 × 8) are obtained in 3 dimensions. The size of each entry is 1 bit, and 512 bytes are required per table.

Therefore, the color space conversion process is implemented as shown in FIG. 95 and the following lookup table is used.

トライリニア補間は、８個の値の間の補間を返す。ルックアップから各８ビット値を返すために１サイクルを要し、全部で８サイクルを要する。トライリニア補間は、１サイクル当たり２個の乗算ＡＬＵが使用されたとき、８サイクルを使用する。一般的なトライリニア補間情報はこの文書のＡＬＵセクションに記載されている。ルックアップテーブル用の５１２バイトは１６個のキャッシュラインに詰め込まれる。

Trilinear interpolation returns an interpolation between 8 values. One cycle is required to return each 8-bit value from the lookup, and a total of 8 cycles are required. Trilinear interpolation uses 8 cycles when 2 multiply ALUs per cycle are used. General trilinear interpolation information is described in the ALU section of this document. The 512 bytes for the lookup table are packed into 16 cache lines.

The time to convert a single color component of the image is 0.105 seconds (1500 * 1000 * 7 cycles * 10 ns per cycle). It takes 0.415 seconds to convert 3 components. Conveniently, color space conversion for printout is performed on-the-fly during printout.

If color components are converted separately, they must overwrite the input color space component. This is because all color components from the input color space are needed to convert each component.

Since only one multiplication unit is used to perform the interpolation, it is possible to perform the entire Lab-> CMY transformation as a single pass. This requires three vertical strip read iterators and three vertical strip write iterators, which simultaneously access three translation tables. In that case, it is possible to write back to the input image and not use extra memory. However, access to the three conversion tables corresponds to 1/3 of the cache processing for each, and causes high latency in the entire process.

Before compositing an image using an affine transformation photograph, it is necessary to rotate, scale, and translate the image. If the image is only translated, it is faster to use a direct subpixel translation function. However, rotation, magnification, and translation can be incorporated into a single affine transformation.

The general affine transformation can be incorporated as an acceleration function. The affine transformation is limited to two dimensions, and when it is reduced, the input image should be prescaled by a scale function. By providing the general-purpose affine transformation, it is possible to construct an output image one block at a time, and it is possible to reduce the time required to perform many transformations on the image. This is because all of the many transformations can be applied simultaneously.

The transformation matrix must be provided by the client and the matrix must be the inverse of the desired transformation. That is, the input coordinates are obtained by applying the matrix to the output pixel coordinates.

A two-dimensional matrix is usually the following 3x3 array:

によって表現される。

Is represented by

Since the third column is always [0, 0, 1], the client need not specify it. Instead, the client specifies a, b, c, d, e, and f.

When the coordinates (x, y) in the input image in which the coordinates of the upper left image pixel are represented as (0, 0) are given, the input coordinates are specified by (ax + cy + e, bx + dy + f). Once the input coordinates are determined, the input image is sampled to reach that pixel value. Bilinear interpolation of the input image pixels is used to determine the value of the pixel at the calculated coordinates. Since the affine transform preserves parallel lines, the image is processed with a 32 pixel wide output vertical strip to obtain the highest average input image cache coherence.

Three multiplication ALUs are required to perform bilinear interpolation within two cycles. Multiplications ALU1 and 2 perform linear interpolation on X for each of line Y and line Y + 1, and multiplication ALU3 performs linear interpolation on Y between the values output by multiplications ALU1 and 2.

When moving to the right across the output line with respect to X, the two adder ALUs add the “a” to the current X value and the “b” to the current Y value to obtain the actual input image. Calculate the coordinates. When proceeding to the next line (the next line in the vertical strip after processing at most 32 pixels, or the first line of a new vertical strip), let X and Y be pre-computed start coordinates for a given block Update to a value constant.

The process of calculating input coordinates is shown in FIG. 96, where the following constants are set by software.

Once the pixel calculation input image coordinates are obtained, it is necessary to sample the input image. Lookup tables are used to return values at specified coordinates in preparation for bilinear interpolation. The basic process is shown in FIG. 97, and the following lookup table is used.

アフィン変換は、全部で４個の乗算ＡＬＵ及び全部で４個のアダーＡＬＵを必要とし、良好なキャッシュコヒーレンスがある場合、１出力画素当たり平均２サイクルでアフィン変換を実行できる。このタイミングは優れたキャッシュコヒーレンスを前提とし、この前提は画像に歪みが無い場合には成立する。最悪ケースでは、画像がかなり歪み、重要なＶａｒｋスクリプトが含まれている可能性は低い。

The affine transformation requires a total of 4 multiplication ALUs and a total of 4 adder ALUs, and if there is good cache coherence, the affine transformation can be performed with an average of 2 cycles per output pixel. This timing is premised on excellent cache coherence, and this premise holds when there is no distortion in the image. In the worst case, it is unlikely that the image is significantly distorted and contains important Vark scripts.

The time required to convert a 128 × 128 image is 0.00033 seconds (32768 cycles). If this is a clip image with 4 channels (including channel a), the total time required is 0.00131 seconds (131072 cycles).

A vertical strip write iterator is needed to output the pixels. A read iterator is not required. However, since the affine transformation accelerator is constrained by the time it takes to access the input image pixels, as many cache lines as possible should be allocated for reading the pixels from the input image. At least 32 are available, preferably 64 or more are available.

Scaling Scaling is basically image resampling. Image enlargement can be performed using affine transformation functions. Generalized image scaling, including reduction, is performed by a hardware accelerated scale function. Since scaling is done independently with respect to X and Y, different scale factors can be used in each dimension. The generalized scale unit must be compatible with the affine transformation scale function for alignment.

A generalized scaling process is shown in FIG. The scale for X is achieved by Fant's resampling algorithm as shown in FIG.

The following constants are set by the software:

以下のレジスタは一時変数を保持するため使用される。

The following registers are used to hold temporary variables.

Ｙに関するスケールプロセスは図１００に示され、Ｘ画素の順番の処理を考慮するためＦａｎｔの再サンプリングアルゴリズムの僅かに変更されたバージョンによっても実現される。

The scale process for Y is shown in FIG. 100 and is also implemented by a slightly modified version of Fant's resampling algorithm to allow for processing of the order of X pixels.

The following constants are set by the software:

以下のレジスタが一時変数を保持するため使用される。

The following registers are used to hold temporary variables:

以下のＤＲＡＭ ＦＩＦＯが使用される。

The following DRAM FIFOs are used:

画像モザイク処理
画像のモザイク処理はタイリングの一形態である。モザイク処理は、特別に設計された「タイル」を、水平方向及び垂直方向に複数回、第２の（通常はより大きい）画像空間へコピーする処理を含む。モザイク処理されたとき、小さいタイルは継ぎ目のないピクチャーを形成する。この一例は、レンガ壁の区画の小さいタイルである。それは、モザイク処理されたときに、完全なレンガ壁を形成するように設計される。尚、モザイク処理には、スケーリング、又はサブピクセル平行移動が含まれないことに注意すべきである。

Image mosaic processing Mosaic processing of images is a form of tiling. Mosaic processing involves copying specially designed “tiles” multiple times in the horizontal and vertical directions to a second (usually larger) image space. When mosaicked, small tiles form a seamless picture. An example of this is a small tile in a brick wall section. It is designed to form a complete brick wall when mosaicked. Note that mosaic processing does not include scaling or sub-pixel translation.

The most cache-coherent method for performing mosaic processing is to sequentially output an image line by line and repeat the same input image line during the section of the line. When a line ends, the input image should be advanced to the next line (it should be repeated multiple times across the output line).

An overview of the mosaic function is shown at 390 in FIG. The sequential read iterator 392 is set up to read continuously a single line of the input tile (start line is 0, end line is 1). Each input pixel is written to all three write iterators 393-395. The adder ALU counter 397 counts down the number of pixels in the output line and ends the sequence at the end of the line.

At the end of processing the line, a small software routine starts the start and end lines of the sequential read iterator before restarting the microcode and sequential read iterator (which clears the FIFO and repeats line 2 of the tile). Update registers. The write iterator 393-395 is not updated, it just keeps writing to the corresponding part of the output image. The net effect is that the tile is repeated one line across the output line, and the tile is repeated vertically.

This process does not use memory bandwidth completely. This is because the cache coherence in the input image is excellent, but mosaic processing cannot work with tiles of other sizes. This process uses one adder ALU. If each of the three write iterators 393-395 writes to 1/3 of the image (divides the image into tile size boundaries), the entire mosaic process will average a rate of 1/3 cycle per output image pixel. Done in For a 1500 × 1000 image, this corresponds to 0.005 seconds (5000000 ns).

Prior to compositing the subpixel translator image with the background, it is necessary to translate the image by subpixel amounts for both X and Y. Sub-pixel translation can increase the size of the image by one pixel in each dimension. The value of the area outside the image is a constant value (eg, black) or a value determined by the client such as edge pixel repetition. Typically it is better to use black.

The subpixel translation process is illustrated in FIG. The sub-pixel translation of a given dimension is
Pixel _out = Pixel _in *
(1-Translation) + Pixel _in-1 *
Translation
Defined by

This is the form of interpolation, ie
Pixel _out = Pixel _in-1 +
(Pixel _in -Pixel _in-1 )
* Translation
Can be expressed as

Implementation of a single (or average) cycle interpolation engine that uses a combination of a single multiply ALU and a single adder ALU is straightforward. Sub-pixel translation for both X and Y requires two interpolation engines.

Two sequential read iterators 400 and 401 are required to perform Y sub-pixel translation (one reads lines that are ahead of other images from the same image), and one sequential write iterator 403 is required. become.

The first interpolation engine (interpolation with respect to Y) receives data pairs from the two streams and linearly interpolates between them. A second interpolation engine (interpolation with respect to X) receives the data as a single one-dimensional stream and linearly interpolates between the values. Both engines interpolate in one cycle on average.

Each interpolation engine 405, 406 has the ability to perform sub-pixel translation in one cycle per output pixel on average. Thus, the overall time is one cycle per output pixel, requiring two multiplication ALUs and two adder ALUs.

The average time required to output 32 pixels from the sub-pixel translation function is 320 ns (32 cycles). This is sufficient time for a complete four cache line access to the DRAM, and the use of three sequential iterators is well within the timing cost.

The total time required to translate the image by subpixel is therefore one cycle per pixel of the output image. A typical image to be sub-pixel translated is a tile of size 128 * 128. The output image size is 129 * 129. This process is 129 * 129 * 10 ns = 166410 ns.

The image tiling function also uses a sub-pixel translation algorithm, but it is not necessary to write out the sub-pixel translated data and further process the data.

The upper level algorithm for tiling the image tile tiler image is implemented in software. Once the tile placement is determined, appropriate colored tiles must be synthesized. The actual compositing to each tile image is done in hardware by a microcoded ALU. The tile composite processing includes both texture pasting and color pasting to the background image. In certain cases, it may be desirable to compare the actual amount of texture added to the background with the planned amount of texture and use it to scale the color that is pasted. In these cases, it is necessary to paste the texture first.

Since the color pasting function and the texture pasting function are somewhat independent, they are separated into sub-functions.

The number of cycles per 4-channel tile composite for various texture and coloring styles is summarized in the following table.

タイルカラーリング及びコンポジット処理
タイルは、（タイル全体に）一定色を具備するか、又は入力画像から各画素値を取るようにセットされる。これらの両方のケースは、不透明度を拡大縮小するため、テクスチャ処理ステージからのフィードバックがある（シニングペイントと同様）。

Tile coloring and composite processing tiles have a constant color (over the entire tile) or are set to take each pixel value from the input image. Both of these cases have feedback from the texture processing stage (similar to thinning paint) to scale the opacity.

The steps for the four cases can be summarized as follows:
-Translating tile opacity values by sub-pixels;
-Selectively scaling tile opacity (if feedback from texture pasting is enabled);
-Determine the color of the pixel (constantly or from an image map);
-Composite the pixels on the background image.

Each of the four cases is handled separately to minimize the time required to execute the function. The time per color composite processing style for a single color channel is summarized in the table below.

一定色
この場合、タイルは、ソフトウェアによって決定された一定色を具備する。ＡＣＰ３１が一つのタイルを設置する間に、ソフトウェアは次のタイルの配置及びカラーリングを決定することができる。

Constant color In this case, the tile has a constant color determined by the software. While ACP 31 installs one tile, the software can determine the placement and coloring of the next tile.

The color of the tile can be determined by bilinear interpolation to a scaled image of the tiled image. A scaled image can be created and stored at a predetermined location in the image pyramid and need only be executed once for the entire tile operation. If the size is 128 × 128, the image is reduced by 128: 1 for each dimension.

Without feedback from tile texture processing, tiles are only placed at the specified coordinates. The tile color is used for each pixel color, and the opacity for the composite comes from the tile sub-pixel translated opacity channel. In this case, the color and texture channels can be processed completely independently during the tiling pass.

An overview of the process is shown in FIG. Tile sub-pixel translation 410 can be performed using two multiply ALUs and two adder ALUs for an average time of one cycle per output pixel. The output from the sub-pixel translation is a mask to be used when compositing 411 certain tile colors 412 with the background image from the background sequential read iterator.

Composite processing can be performed using one multiplication ALU and one adder ALU for an average time of one cycle per composite. Thus, the prerequisites are 3 multiplication ALUs and 3 adder ALUs. Four sequential iterators 413-416 are required and 320 ns is required to read and write their contents. For the average number of cycles per pixel required for subpixel translation and compositing, there is sufficient time to read and write the buffer.

When there is feedback from the texture processing of the tile with feedback, the tile is placed at the specified coordinates. The tile color is used for each pixel color, and the opacity for the composite comes from the sub-pixel translated opacity channel scaled by the feedback parameter. In this way, the texture value must be calculated before the color value is pasted.

An overview of the process is shown in FIG. The sub-pixel translation of the tile can be performed using two multiply ALUs and two adder ALUs for an average time of one cycle per output pixel. The output from the subpixel translation is a mask to be scaled according to the feedback read from the feedback sequential read iterator 420. This feedback is sent to the scaler (one multiplication ALU) 421.

The composite processing 426 can be executed using one multiplication ALU and one adder ALU in the case of an average time of one cycle per composite. Therefore, the necessary conditions are 4 multiplication ALUs and 4 adder ALUs. The entire process can be completed on average in one cycle, but the bottleneck is memory access. This is because five sequential iterators are required. If sufficient buffering is performed, the average time is 1.25 cycles per pixel.

One way to color the pixels of the color tile from the input image is to extract the color from the pixels of the input image. Again, there are two possibilities for composite processing, with and without feedback from texture processing.

No feedback In this case, the tile color is simply derived from the corresponding pixel in the input image. The opacity for compositing comes from the opacity channel of the subpixel shifted tile.

An overview of the process is shown in FIG. The sub-pixel translation 425 of the tile is 2 multiplication ALUs and 2 adders AL for an average time of 1 cycle per output pixel.
Can be performed using U. The output from the subpixel translation is a mask to be used when compositing 426 the pixel color of the tile (read from the input image 428) with the background image 429. This feedback is sent to the scaler (one multiplication ALU) 431.

The composite processing 434 can be executed using one multiplication ALU and one adder ALU in the case of an average time of one cycle per composite. Thus, the prerequisites are 3 multiplication ALUs and 3 adder ALUs. The entire process can be completed on average within one cycle, but the bottleneck is memory access. This is because five sequential iterators are required. If sufficient buffering is performed, the average time is 1.25 cycles per pixel.

In this case with feedback, the tile color is still simply derived from the corresponding pixel in the input image, but the opacity for composite processing is the relative amount of texture height actually applied during the texture processing pass. Affected by. This process is illustrated in FIG.

The tile sub-pixel translation 431 can be performed using two multiplication ALUs and two adder ALUs for an average time of one cycle per output pixel. The output from the sub-pixel translation is a mask to be scaled 431 according to the feedback read from the feedback sequential read iterator 432.

Composite processing can be performed using one multiplication ALU and one adder ALU for an average time of one cycle per composite.

Thus, the prerequisites are 4 multiplication ALUs and 3 adder ALUs. The entire process can be completed on average within one cycle, but the bottleneck is memory access. This is because six sequential iterators are required. If sufficient buffering is performed, the average time is 1.5 cycles per pixel.

Tile Texture Processing Each tile has a surface texture defined by its texture channel. The texture is moved to subpixels and then pasted into the output image. For texture compositing,
Texture placement,
25% background + tile texture,
An average height algorithm;
There are three types of styles.

In addition, the average height algorithm saves feedback parameters for color composite processing.

The time required for each texture composite processing style is summarized in the table below.

テクスチャ置換
このケースでは、タイルからのテクスチャは、図１０７に示されるように、画像のテクスチャチャンネルを置き換える。タイルのテクスチャのサブピクセル平行移動４３６は、１出力画素当たり１サイクルの平均時間で、２個の乗算ＡＬＵ及び２個のアダーＡＬＵを使用して実行できる。サブピクセル平行移動からの出力は、順次書き込みイタレータ４３７へそのまま供給される。

Texture Replacement In this case, the texture from the tile replaces the texture channel of the image, as shown in FIG. The tile texture sub-pixel translation 436 can be performed using two multiplying ALUs and two adder ALUs with an average time of one cycle per output pixel. The output from the sub-pixel translation is sequentially supplied to the writing iterator 437 as it is.

The time required for texture replacement composite processing is one cycle per pixel. Since 100% of the texture value is always pasted to the background, there is no feedback. Thus, there is no need to process the channels in a specific order.

25% Background + Tile Texture In this example, the texture from the tile is added to 25% of the existing texture value. The new value must be greater than or equal to the original value. Moreover, since the texture channel is only 8 bits, the new texture value is clipped at 255. The process utilized is shown in FIG.

The tile texture sub-pixel translation 440 can be performed using two multiplying ALUs and two adder ALUs with an average time of one cycle per output pixel. The output from the subpixel translation 440 is fed to the adder 441 and is ¼ of the background texture value.
442 is added. The minimum and maximum value functions 444 are provided by two adders that are not used for sub-pixel translation, and the output is written to the write iterator 445 sequentially.

The time required for this style of texture composite processing is one cycle per pixel. There is no feedback because 100% of the texture value is considered pasted on the background (even if it was clipped at 255). Thus, there is no need to process the channels in a specific order.

Average Height Algorithm In this texture pasting algorithm, the average height is calculated based on the tiles, and the height of each image is compared to the average height. If the pixel height is less than average, the stroke height is added to the background height. If the pixel height is above average, the stroke height is added to the average height. In this way, the background peak narrows the stroke. This height is constrained to increase by a minimum amount (but the minimum amount can be zero) to prevent the background from thinning the stroke pasting to zero. The height is clipped at 255 because the texture channel is 8-bit resolution.

There may be feedback of the difference between the pasted texture and the predicted amount pasted. The feedback amount can be used as a magnification when the tile color is pasted.

In both cases, the average texture is calculated by performing bi-level interpolation on the scaled texture map provided by the software. The software determines the average texture height of the next tile while the current tile is being pasted. The software provides a minimum thickness to add, and this minimum thickness is typically constant throughout the tiling process.

Without feedback, the texture is only pasted to the background texture, as shown in FIG.

Four sequential iterators are required, which means that if the process can be pipelined in one cycle, it is sufficient that the high speed of the memory is not delayed.

Tile texture sub-pixel translation 450 can be performed using two multiplying ALUs and two adder ALUs with an average time of one cycle per output pixel. The minimum and maximum value functions 451 and 452 require a separate adder ALU to complete all operations within one cycle. Since 2 is already used by the sub-pixel translation of the texture, there is not enough remaining time for one cycle average time.

Therefore, the average time for processing one pixel texture is two cycles. Note that there is no feedback and the order of the color channels in the composite process is irrelevant.

With feedback, this is conceptually the same as without feedback, except that in addition to the processing of the standard texture pasting algorithm, the percentage of texture actually pasted must be recorded. This ratio can continue to be used as a magnification for compositing the tile color into the background image. The flowchart is shown in FIG. 110, and the following lookup table is used.

ソフトウェアで設けられた１／Ｎテーブル４６０の２５６個のエントリーの各々は１６ビットであり、連続して保持するため１６個のキャッシュラインが必要である。

Each of the 256 entries of the 1 / N table 460 provided by the software is 16 bits, and 16 cache lines are required to hold them continuously.

The tile texture sub-pixel translation 461 can be performed using two multiplying ALUs and two adder ALUs with an average time of one cycle per output pixel. Each of the minimum value function 462 and maximum value function 463 requires a separate adder ALU to complete the entire operation within one cycle. Since 2 is already used by the sub-pixel translation of the texture, there is not enough remaining time for one cycle average time.

Therefore, the average time for processing one pixel texture is two cycles. Sufficient space should be allocated for the feedback data area (tile-sized image channel). The texture should be applied before the tile color is applied. This is because feedback is used to scale the opacity of the tile.

The image acquired from the CCD by the CCD image interpolator ISI 83 (FIG. 3) is 750 × 500 pixels. The orientation of the camera when the image was taken by ISI is used to rotate the pixel by 0, 90, 180 or 270 degrees so that the top of the image corresponds to “up”. Since all pixels have only R, G or B color components (rather than a maximum of 3), the fact that they are rotating must be considered when interpreting the pixel values. Depending on the orientation of the camera, each 2 × 2 pixel block has one of the configurations shown in FIG.

In order to convert a CCD captured image into a form useful for processing, it is necessary to perform some processing on the CCD captured image:
-Up-interpolation of the low sample rate color component of the CCD image (interpreting the correct orientation of the pixels);
Color conversion from RGB to internal color space;
Scaling the internal space image from 750 × 500 to 1500 × 1000;
-Write images in planar format.

The entire image channel must be available at the same time to allow warping. In the small memory model (8 MB), there is only space for holding a single channel at full resolution as a temporary object. For this reason, color conversion is conversion to a single color channel. The limiting factor for this process is color conversion. This is because color conversion includes trilinear interpolation from RGB to the internal color space, and this trilinear interpolation is 0.026 ns per channel (750 × 500 × 7 cycles per pixel × 10 ns per cycle = 26250,000 ns). Because it is a process that requires

It is important to perform color conversion before scaling the internal color space image. This is because the number of scaled pixels (and hence the overall process time) is reduced by a factor of four.

Not all conversion requirements apply to the ALU scheme. The transformation is therefore broken down into two phases:
Phase 1: Up-interpolation of low sample rate color components of CCD image (interpret correct pixel orientation);
Color conversion from RGB to internal color space;
Writing images in planar format;
Phase 2: Scale internal space image from 750 × 500 to 1500 × 1000.

The separation of the scale function means that the small color converted image should exist in memory at the same time as the large image. The output from Phase 1 (0.5 MB) is usually safely written to the memory area reserved for the image pyramid (1 MB). The output from Phase 2 is a typical enlarged CCD image. Scaling separation also allows scaling to be achieved by affine transformation, which allows for various CCD resolutions that are not simple 1: 2 magnification.

Phase 1: Up-interpolation of low sample rate color components Each of the three color components (R, G, and B) needs to be up-interpolated to perform color conversion for a given pixel. Since color conversion uses 7 cycles, there are 7 cycles per pixel to perform the interpolation.

The interpolation of G is simple and is shown in FIG. Depending on the orientation, the actual pixel value G is
It alternates between odd pixels on odd lines and even pixels on even lines and odd pixels on even lines and even pixels on odd lines. In both cases, linear interpolation is all that is required. The interpolation of R and B components is more complex as shown in FIGS. This is because, in the horizontal direction and the vertical direction, as can be seen from the figure, it is required to access three rows of pixels at the same time, so three sequential read iterators are required, and each sequential read iterator is a single row. Offset by In addition, the latch for each row is used to access the previous pixel in the same row.

Thus, each pixel stores one component from the CCD and the other two up-interpolated components. When one component is bilinearly interpolated, the other is linearly interpolated. Since the interpolation factor is a constant 0.5, the interpolation can be calculated (within 1 cycle) by addition and 1-bit right shift, and bilinear interpolation with a factor of 0.5 is 3 additions and 2 It can be calculated by bit shift right (3 cycles). The total number of cycles required is 4 when using a single multiplying ALU.

FIG. 115 shows the case of even-numbered even pixels (EL, EP) and odd-numbered odd pixels (OL, EP) with rotation 0, and FIG. 116 shows even-numbered odd-numbered pixels (EL, OP) and odd-numbered lines with rotation 0. The case of even pixels (OL, OP) is shown. Other rotations are a slightly different form of these two representations.

Color conversion RGB to Lab color space conversion is a process that takes 8 cycles per pixel, implemented using the same method as described for the general color space conversion function. The processing of phase 1 will be described with reference to FIG.

RGB up-interpolation requires 4 cycles (1 multiplication ALU), but color space conversion requires 8 cycles per pixel (2 multiplications ALU) due to lookup transfer time.

Phase 2
Image Scaling This phase involves the up-interpolation of the image from the CCD resolution (750 × 500) to the working photographic resolution (1500 × 1000). Scaling is performed by performing an affine transformation with a 1: 2 scale. The timing of general-purpose affine transformation is 2 cycles per output pixel, and in this case, it means that a scaling time of 0.03 seconds has elapsed.

After the image illumination image is processed, the image can be illuminated by one or more light sources. The light source can be any of the following types:
1. Directional type-because there is an infinite distance, it throws parallel light in a single direction;
2. Omni type-throws unfocused light in all directions;
3. Spot type—projects a focused beam to a specific target point. There are cones and penumbras associated with spot light.

The scene has an associated bump map to change the reflection angle. Ambient light is also present in selectively illuminated scenes.

In the accelerated illumination process, one is interested in illuminating one image channel with a single light source. Multiple light sources can be applied to a single image channel as multiple passes, one pass per light source. Multiple channels can be processed one at a time, with or without a bump map.

The normal surface vector (N) at a pixel is calculated from the bump map, if present. If no bump map exists, the default normal vector is perpendicular to the image plane, ie N = [0, 0, 1].

The viewing vector V is always perpendicular to the image plane, ie V = [0, 0, 1].

In the case of a directional light source, the light source vector (L) from pixel to light source is constant throughout the image and is calculated once for the entire image. In the case of an omni light source (at a finite distance), the light source vector is calculated independently for each pixel.

The ambient light pixel reflection is calculated according to I _a k _a O _d .

Light source pixel scattering and specular reflection, Phong model:
f _att I _p [k _d O _d (N ・ L) + k _s O _s (R ・ V) ⁿ ]
Calculated according to

When the light source is infinite, the light source intensity is constant over the entire image.

Each light source has three contributions per pixel, ie
Ambient light contribution degree Diffuse contribution degree Specular contribution degree

The light source is defined using the following variables:

同じ反射係数（ｋ_ａ，ｋ_ｓ，ｋ_ｄ）がカラーコンポーネント毎に使用される。

The same reflection coefficient _{(k a, k s, k} d) are used per color component.

The value of a given pixel corresponds to the sum of the ambient light contribution and the sum of the diffuse and specular contributions of each light.

Various other calculations are needed to calculate the light calculation subprocess diffuse contribution. Among those calculations are:
1 / ΔX
N
L
N ・ L
R ・ V
f _att
f _cp
Calculation of;
The subprocess is
It is also defined to calculate the contribution of the ambient light diffuse mirror surface.

The sub-process can be used to calculate the overall illumination of the light source. Since there are only four multiplying ALUs, the microcode for a particular type of light source may contain sub-processes that are properly mixed for performance.

The 1 / √X calculation Vark illumination model uses vectors. In most cases, it is important to calculate the reciprocal of the length of the vector for normalization purposes. In order to calculate the reciprocal of the length, it is necessary to calculate 1 / (square root of [x]).

Logically, this process can be expressed as a process with inputs and outputs as shown in FIG. Referring to FIG. 119, this calculation looks up the estimate and then the following function:
V _{n + 1} = 1/2 V _n (3-XV _n ² )
Can be performed by repeating the above once.

The number of repetitions depends on the required accuracy. In this case, 16 bits of accuracy are required. Therefore, the table is 8 bits accurate and only needs to be repeated once. The following constants are set by software.

以下のルックアップテーブルが使用される。

The following lookup table is used.

Ｎの計算
Ｎは表面法線ベクトルである。バンプマップが存在しない場合、Ｎは一定である。バンプマップが存在する場合、Ｎは画素毎に計算しなければならない。

Calculation of N N is the surface normal vector. If no bump map exists, N is constant. If a bump map exists, N must be calculated for each pixel.

When there is no bump map, the fixed normal N has the following properties:
N = [X _N , Y _N , Z _N ] = [0, 0, 1]
|| N || = 1
1 / || N || = 1
Normalized N = N
Have

These properties can be used specifically instead of calculating the normal vector and 1 / || N ||, and other calculations can be optimized.

With bump map as shown in FIG. 120, if a bump map exists, N is calculated by comparing the X and Y dimensional bump map values. FIG. 120 shows the calculation of N for pixel P1 for pixels in the same row and column, but the value at P1 itself is also included. The calculation of N is made independent of resolution by multiplying by the scale factor (same factor for X and Y). This process can be expressed as a process with inputs and outputs (ZN is always 1) as shown in FIG.

Z _N is always 1. X _N and Y _N are not yet normalized (since Z _N = 1). The normalization of N. Because L is delayed until L is calculated, the multiplication by 1 / || N || is once, not three.

The actual process of calculating N is shown in FIG.

The following constants are set by the software:

Ｌの計算
指向性光
光源が無限遠であるとき、光源は実効的な一定光ベクトルＬを有する。Ｌは、
Ｌ＝［Ｘ_Ｌ， Ｙ_Ｌ， Ｚ_Ｌ］
｜｜Ｌ｜｜＝１
１／｜｜Ｌ｜｜＝１
のようにソフトウェアによって正規化され、計算される。

When the L computationally directional light source is at infinity, the light source has an effective constant light vector L. L is
_{L = [X L, Y L} , Z L]
|| L || = 1
1 / || L || = 1
Is normalized and calculated by the software.

These properties can be used specifically instead of calculating L and 1 / || L ||, and other calculations can be optimized. This process is illustrated in FIG.

When the omni light and the spot light source are not at infinity, L is a vector from the current point P to the light source PL. P = [X _P ,
Y _P , 0], so that L is
_{L = [X L, Y L} , Z L]
_{X L} = X P _{-X PL}
Y _L = Y _P −Y _PL
Z _L = −Z _PL
Given by. Each of X _L , Y _L , and Z _L is normalized by multiplying by 1 / || L ||. The calculation of 1 / || L || (used later for normalization) is
V = X _L ² + Y _L ² + Z _L ²
And then
V- ^{1 / 2}
It is executed by calculating

In this case, the calculation of L can be expressed as a process with inputs and outputs as shown in FIG.

X _P and YP are the coordinates of image intensity is computed, Z _P is always zero.

The actual process of calculating L is shown in FIG.

The following constants are set by the software:

Ｎ．Ｌの計算
ベクトルＮとベクトルＬの内積の計算は、
Ｘ_ＮＸ_Ｌ＋Ｙ_ＮＹ_Ｌ＋Ｚ_ＮＺ_Ｌ
によって定義される。

N. The calculation of the inner product of the calculation vector N of L and the vector L is
X _N X _L + Y _N Y _L + Z _N Z _L
Defined by

When there is no bump map without bump map, N is constant [0, 0, 1]. Therefore, N.I. L is simplified into _{Z L.}

When there is a bump map with a bump map, the inner product must be calculated directly. Rather than fetching the normalized N component, normalization is performed after obtaining the inner product of the unnormalized N and the normalized L. L is normalized by software (when constant) or is normalized by the L calculation process. This process is illustrated in FIG.

Since Z _N is defined to be 1, it is not necessary as an input. Instead, however, 1 / || N || is required to normalize the result. N. One actual process for calculating L is shown in FIG.

R · V calculation R · V is required as an input to the specular contribution calculation. Since V = [0, 0, 1], only the Z component is required. Therefore, R · V is
R · V = 2Z _N (N.L) −Z _L
To be simplified.

In addition, since denormalized Z _N = 1, normalized Z _N = 1 / || N ||.

The simplest implementation without a bump map is when N is constant (ie, no bump map). Since N and V are constants, N.V. L and R · V are
V = [0, 0,
1]
N = [0, 0,
1]
_{L = [X L, Y L} , Z L]
N. L = Z _L
R · V = 2Z _N (N.L) −Z _L
= 2Z _L -Z _L
= Z _L
As simplified.

When L is constant (directional light source), the normalized Z _L can be given in the form of a constant when R · V is required. When L changes (for omni and spot lights), the normalized Z _L must be calculated on-the-fly. It is obtained as the output of the L calculation process.

When N with bump map is not constant, the process of calculating R · V is a generalized equation:
R · V = 2Z _N (N.L) −Z _L
Just implement. Inputs and outputs are shown in FIG. 128 and the actual implementation is shown in FIG.

Calculation of attenuation constant When the directional light source is at infinity, the light intensity does not change across the image. Therefore, the attenuation constant f _att is 1. This constant can be used to optimize the illumination calculation of the infinity light source.

When the omni light and the spot light source are not at infinity, the light intensity is
f _att = f ₀ + f ₁ / d + f ₂ / d ²
Changes according to

By appropriately setting the coefficients f ₀ , f ₁ and f ₂ , the light intensity can be attenuated constant, linearly with respect to distance, or by square of distance.

Since d = || L ||, the calculation of f _att can be expressed as a process having inputs and outputs as shown in FIG.

The actual process of calculating f _att is defined in FIG.

The following constants are set by the software:

円錐及び半影ファクタの計算
指向性光及びオムニ光
これらの二つの光源は集光されないので、円錐又は半影を持たない。円錐−半影スケーリングファクタｆ_ｃｐは１である。この定数は、指向性光源及びオムニ光源の照明計算を最適化するために使用できる。

Calculation of cone and penumbra factor Directional light and omni light These two light sources are not condensed and therefore do not have a cone or penumbra. The cone-penumbra scaling factor f _cp is 1. This constant can be used to optimize illumination calculations for directional and omni sources.

Spot light Spot light is focused on a specific target point (PT). The intensity of the spotlight varies depending on whether a particular point in the image is in the cone, penumbra or outside the cone / penumbra area.

Referring to FIG. 132, a graph of f _cp for the penumbra is shown. At the inside 470 of the cone, f _cp is 1, and at the outside 471 of the penumbra, f _cp is 0. From the edge of the cone to the end of the penumbra, the light intensity varies according to a cubic function 472.

Various vectors for the calculation of penumbra 475 and cone 476 are shown in FIGS.

By examining the surface of the image in one dimension as shown in FIG. 134, three corners A, B and C are defined. A is the angle between target point 479, light source 478 and cone end 480. C is an angle between the target point 479, the light source 478, and the penumbra end 481. Both of these are constant for a given light source. B is an angle between the target point 479, the light source 478, and the calculation object 482, and changes for each point calculated on the image.

Here, ranges A to C are normalized from 0 to 1, and B is given by
(BA) / (CA)
Find the distance that fits between that angular range.

The range is rounded to a range of 0 to 1, and this value is used as a lookup for a third order approximation of f _cp .

Thus, the calculation of f _att can be expressed as a process with inputs and outputs as shown in FIG. 135, the actual process of calculating f _cp is shown in FIG. 136, and the following constants are set by software: The

以下のルックアップテーブルが使用される。

The following lookup table is used.

周辺光寄与の計算
像に当てられる光の数とは無関係に、周辺光寄与が各画素に対して１回ずつ実行され、バンプマップには依存しない。

Regardless of the number of lights applied to the ambient light contribution calculation image, the ambient light contribution is performed once for each pixel and is independent of the bump map.

The ambient light calculation process can be expressed as a process having inputs and outputs as shown in FIG. To implement this process, it is necessary to multiply each pixel (O _d ) from the input image by a constant value (I _a k _a ) as shown in FIG. Set.

散光寄与の計算
表面に当てられた各光は拡散照明を生じる。拡散照明は、次式、
散光＝ｋ_ｄＯ_ｄ（Ｎ．Ｌ）
によって表される。２種類の実装形態を考えることができる。

Each light applied to the diffuse contribution calculation surface produces diffuse illumination. Diffuse lighting is:
Diffuse = k _d O _d (N.L)
Represented by Two types of implementations can be considered.

Implementation example 1-constants N and L
When both N and L are constants (when there is no bump map with directional light),
N. L = Z _L
It is. Therefore,
Diffuse = k _d O _d Z _L
It is.

Since only O _d is a variable, the actual process of calculating the diffuse contribution is as shown in FIG. 139 and the following constants are set by the software.

実装例２−非定数Ｎ及びＬ
Ｎ又はＬの何れかが非定数であるとき（ビットマップ、又はオムニ光若しくはスポット光からの照明であるとき）、散光計算は、次式、
散光＝ｋ_ｄＯ_ｄ（Ｎ．Ｌ）
に従って直接実行される。

Implementation Example 2-Non-constant N and L
When either N or L is non-constant (when it is a bitmap or illumination from omni or spot light), the diffuse calculation is:
Diffuse = k _d O _d (N.L)
Will be executed directly according to.

The diffuse calculation process can be expressed as a process with inputs as shown in FIG. N. L is N. Calculated using the L calculation process or given as a constant. The actual process of calculating the diffuse contribution is as shown in FIG. 141, and the following constants are set by software.

鏡面寄与の計算
表面に当てられた各光は鏡面照明を生ずる。鏡面照明は、次式、
鏡面光＝ ｋ_ｓＯ_ｓ（Ｒ・Ｖ）^ｎ
によって表される。ここで、
Ｏ_ｓ＝ｋ_ｓｃＯ_ｄ＋（１−ｋ_ｓｃ）Ｉ_ｐ
である。

Each light applied to the specular contribution calculation surface results in specular illumination. The specular lighting is:
Specular light = k _s O _s (R · V) ⁿ
Represented by here,
_{O s = k sc O d +} (1-k sc) I p
It is.

There are two implementations of the specular light calculation process.

Implementation example 1-constants N and L
The first mounting example is when both N and L are constants (directional light and no bump map). N, L and V are constant, so L and R · V are also constant:
V = [0, 0,
1]
N = [0, 0,
1]
_{L = [X L, Y L} , Z L]
N. L = Z _L
R · V = 2Z _N (N.L) −Z _L
= 2Z _L -Z _L
= Z _L
In this way, the specular light calculation is
Specular light = k _s O _s Z _L ⁿ
= K _s Z _L ⁿ (k _sc O _d + (1−k _sc ) I _p )
= K _s k _sc Z _L ⁿ O _d + (1−k _sc ) I _p k _s Z _L ⁿ
As simplified.

Since only O _d is a variable in the specular light calculation, the calculation of the specular light contribution can be expressed as a process with inputs and outputs as shown in FIG. 142, and the actual process of calculating the specular light contribution is As shown in FIG. 143, the following constants are set by the software.

実装例２−非定数Ｎ及びＬ
この実装例は、Ｎ又はＬの何れかが一定ではないとき（ビットマップ、又はオムニ光若しくはスポット光からの照明であるとき）である。これは、Ｒ・Ｖを与える必要があることを意味し、Ｒ・Ｖ^ｎを計算しなければならない。

Implementation Example 2-Non-constant N and L
This implementation example is when either N or L is not constant (when it is a bitmap, or illumination from omni or spot light). This means that R · V needs to be given, and R · V ⁿ must be calculated.

The specular light calculation process can be expressed as a process having inputs and outputs as shown in FIG. FIG. 145 represents the actual process of calculating the specular light contribution and the following constants are set by the software.

以下のルックアップテーブルが使用される。

The following lookup table is used.

周辺光が唯一の照明であるとき
周辺光寄与が唯一の光源である場合、このプロセスは非常に簡単である。なぜならば、図１４６に示されているようなプロセス全体で周辺光を加える必要がないからである。像は、垂直方向に二つの区画に分割し、周辺光ロジックを複製することにより、半分の各々を同時に処理することが可能である（このようにして、全部で２個の乗算ＡＬＵ及び４個の順次イタレータを使用する）。したがって、タイミングは、周辺光を適用する１画素当たり１／２サイクルである。

This process is very simple if the ambient light contribution is the only light source when the ambient light is the only illumination. This is because it is not necessary to add ambient light throughout the process as shown in FIG. The image can be processed simultaneously in half by dividing the image vertically into two sections and replicating the ambient light logic (in this way a total of two multiplying ALUs and four Use sequential iterators). Therefore, the timing is ½ cycle per pixel to which ambient light is applied.

A typical lighting case is a scene illuminated by one or more lights. In these cases, the ambient light calculation is so easy that the ambient light calculation is accommodated with each light source process. The first light to be processed should have a correct I _a k _a set value, the subsequent light, to avoid I _a k value of _a should be 0 (multiple ambient light contribution ).

If the ambient light is processed as a separate path (rather than the first pass), it is necessary to add the ambient light to the current calculated value (reading and writing to the same address is required). An overview of the process is shown in FIG.

This process uses 3 image iterators and 1 multiplication ALU, and on average takes 1 cycle per pixel.

Infinite light source In the case of an infinite light source, a constant light source intensity is obtained for the entire image. Both L and f _att are constants.

When there is no bump map without a bump map, there is a constant normal vector N [0, 0, 1]. Illumination complexity is significantly reduced by _keeping N, L and f _att constant. The process of shining a single directional light without a bump map is shown in FIG. 147 and the following constants are set by software.

単一の無限光源の場合、図１４８に示されるような論理演算を実行することが求められ、ここで、Ｋ_１からＫ_４は以下の値をもつ定数である。

In the case of a single infinite light source, it is required to perform a logical operation as shown in FIG. 148, where K ₁ to K ₄ are constants having the following values.

このプロセスは、Ｋ_２、Ｋ_３及びＫ_４が定数であるため簡単化することができる。基本的に（乗算ＡＬＵのうちの３個を使用する）鏡面光寄与と散光寄与の計算が複雑であるため、周辺光計算を４番目の乗算ＡＬＵとして安全に加えることが可能である。処理される第１の無限光源は、真の周辺光パラメータＩ_ａｋ_ａを備え、後続の全ての無限光は、Ｉ_ａｋ_ａに０をセットすることができる。周辺光計算は実質的に無くなる。

This process can be simplified because K ₂ , K ₃ and K ₄ are constants. Basically, the calculation of specular and diffuse light contributions (using three of the multiplication ALUs) is complex, so the ambient light calculation can be safely added as the fourth multiplication ALU. The first infinite light source to be processed comprises the true ambient light parameter I _a k _a and all subsequent infinite lights can set I _a k _a to zero. Ambient light calculations are virtually eliminated.

In the case of the first light emitted from the infinite light source, it is not necessary to take in existing contributions from other light sources, the situation is shown in FIG. 149, and the constant takes the following values.

無限光源が照射される第１の光ではない場合、先行して処理された光によって生成された既存の寄与度を取り込む必要があり（同じ定数が当てはまる）、状況は図１４８に示されている。

If the infinite light source is not the first light to be illuminated, it is necessary to capture the existing contribution generated by the previously processed light (the same constant applies) and the situation is shown in FIG. .

In the first case, two sequential iterators 490, 491 are necessary, and in the second case, three sequential iterators 490, 491, 492 are necessary (the extra iterator is the contribution of the preceding light). Is needed to read In both cases, the application of an infinite light source without a bump map requires one cycle per pixel and optionally includes the application of ambient light.

When there is a bump map with a bump map, the normal vector N needs to be calculated for each pixel and is applied to the constant light source vector L. 1 / || N || is used to calculate R · V, which is required as an input to the specular light calculation 2 process. The following constants are set by the software:

バンプマップ順次読み出しイタレータ４９０は、バンプマップの現在ラインを読み出す役割を担う。現在ラインは、Ｘに関するスロープを決定するため入力として与えられる。バンプマップ順次読み出しイタレータ４９１、４９２は、現在ラインの上下のラインを読み出す役割を担う。それらはＹに関するスロープを決定するため入力を提供する。

The bump map sequential read iterator 490 plays a role of reading the current line of the bump map. The current line is given as input to determine the slope for X. The bump map sequential read iterators 491 and 492 serve to read lines above and below the current line. They provide an input to determine the slope for Y.

In the case of an omni light omni light source, the light vector L and the attenuation constant f _att vary from pixel to pixel in the image. Therefore, both L and f _att must be calculated for each pixel.

When there is no bump map without a bump map, there is a constant normal vector N [0, 0, 1]. L must be calculated for each pixel. Both L and R · V is simplified to _{Z L.} When there is no bumpup, the application of omni light is calculated as shown in FIG. 149 and the following constants are set by the software.

このアルゴリズムは、オプションとして、先行の光源からの寄与を取り込み、周辺光計算を含む。周辺光は１回だけ取り込めばよい。他の全ての光パスに対して、周辺光計算プロセスにおける適切な定数は０にセットされるべきである。

This algorithm optionally incorporates contributions from previous light sources and includes ambient light calculations. Ambient light need only be captured once. For all other light paths, the appropriate constant in the ambient light calculation process should be set to zero.

The algorithm as shown requires a total of 19 multiplications / accumulations. The time required for the lookup is one cycle during the calculation of L and four cycles during the specular light contribution calculation. Therefore, the processing time of 5 cycles is the best that can be realized. If the ALU cannot be optimally microcoded for that function, the required time increases to 6 cycles. The speed for applying omni light to an image without an associated bitmap is 6 cycles per pixel.

When omni light with bump map is illuminated onto an image with an associated bump map, N, L, N. All calculations of L and R · V are necessary. The process of applying omni light to an image with an associated bump map is shown in FIG. 150, with the following constants set by software.

このアルゴリズムは、オプションとして、先行の光源からの寄与を取り込み、周辺光計算を含む。周辺光は１回だけ取り込むことが必要である。他の全ての光パスに対し、周辺光計算プロセス内の適切な定数は０にセットされるべきである。

This algorithm optionally incorporates contributions from previous light sources and includes ambient light calculations. Ambient light needs to be captured only once. For all other light paths, the appropriate constant in the ambient light calculation process should be set to zero.

An algorithm as shown requires a total of 32 multiplications / accumulations. The time required for the lookup is one cycle during both L and N calculations and four cycles for specular light contribution calculations. However, the lookups required for N and L are the same (thus two LUs implement three LUs). A processing time of 8 cycles is appropriate. If the ALU for the function cannot be optimally microcoded, the required time is extended to 9 cycles. The rate at which omni light is applied to the associated bitmapped image is 9 cycles per pixel.

Spot light spot light is similar to omni light, _except that the attenuation constant f _att is modified by a cone / penumbra factor f _cp that efficiently collects light around the target.

When there is no bump map without a bump map, there is a constant normal vector N [0, 0, 1]. L must be calculated for each pixel. Both L and R · V is simplified to _{Z L.} FIG. 151 is an explanatory diagram of spot light irradiation on an image, and the following constants are set by software.

このアルゴリズムは、オプションとして、先行の光源からの寄与を取り込み、周辺光計算を含む。周辺光は１回だけ取り込めばよい。他の全ての光パスに対して、周辺光計算プロセスにおける適切な定数は０にセットされるべきである。

This algorithm optionally incorporates contributions from previous light sources and includes ambient light calculations. Ambient light need only be captured once. For all other light paths, the appropriate constant in the ambient light calculation process should be set to zero.

The algorithm as shown requires a total of 30 multiplications / accumulations. The time required for the lookup is one cycle during the calculation of L, four cycles during the specular light contribution calculation, and two sets of four cycle lookups in the cone / penumbra calculation.

When spotlight with bump map is irradiated onto an image with an associated bump map, N, L, N. All calculations of L and R · V are necessary. The process of applying a single spot light to an image with an associated bump map is shown in FIG. 152, with the following constants set by software.

This algorithm optionally incorporates contributions from previous light sources and includes ambient light calculations. Ambient light needs to be captured only once. For all other light paths, the appropriate constant in the ambient light calculation process should be set to zero. The algorithm as shown requires a total of 41 multiplications / accumulations.

Print head 44
FIG. 153 is an explanatory diagram of the logical layout of a single printhead that is logically composed of 8 segments, each printing two layers of cyan, magenta, and yellow on a portion of the page.

Printing Segment Loading Before printing anything, each of the eight segments of the printhead must be loaded with six rows of data corresponding to the following relative rows in the final output image:
Row 0 = Line N, yellow, even dots 0, 2, 4, 6, 8,. . .
Row 1 = Line N + 8, yellow, odd dots 1, 3, 5, 7,. . .
Row 2 = Line N + 10, magenta, even dots 0, 2, 4, 6, 8,. . .
Line 3 = Line N + 18, magenta, odd dots 1, 3, 5, 7,. . .
Row 4 = Line N + 20, cyan, even dots 0, 2, 4, 6, 8,. . .
Line 5 = Line N + 28, cyan, odd dots 1, 3, 5, 7,. . .
Each segment prints a dot on a different part of the page. Each segment prints 750 dots of one color, that is, 375 even dots on one line and 375 odd dots on another line. The eight segments include dots corresponding to the positions.

各ドットは、プリントヘッドセグメントに１ビットで表現される。データは、データをセグメントのＢｉｔＶａｌｕｅ（ビット値）ピンに置くことによって、同時に１ビットだけロードされ、ＢｉｔＣｌｏｃｋ（ビットクロック）に従ってそのセグメントのシフトレジスタへクロック供給されなければならない。データは単一のレジスタにロードされるので、ビットをローディングする順序は正しい筈である。データは１０ＭＨｚの最大レートでプリントヘッドにクロック供給することができる。

Each dot is represented by 1 bit in the printhead segment. Data must be loaded one bit at a time by placing the data on the segment's BitValue pin and clocked to the segment's shift register according to the BitClock. Since the data is loaded into a single register, the order of loading bits should be correct. Data can be clocked into the printhead at a maximum rate of 10 MHz.

When all bits are loaded, the bits are transferred in parallel to the printhead output buffer and ready for printing. This transfer is performed by one pulse on the segment's ParallelXferClock pin.

In order to save print control power, it is not necessary to print all of the print heads simultaneously. A set of control lines allows the printing of specific dots. An external controller such as ACP can change the number of dots printed at the same time, as well as the interval between print pulses, depending on speed and / or power requirements.

Each segment includes five NozzleSelect (nozzle selection) lines, which are decoded to select 32 sets of nozzles per row. Since there are 375 nozzles in each row, there are 12 nozzles in each set. In addition, two Bank Enable (bank enable) lines are provided, one for each even and odd color row. Finally, each segment includes three Color Enable (color enable) lines, one for each of the C, M and Y colors. A pulse on a ColorEnable line causes a designated nozzle in a designated row of color to print. The pulse is typically 2
The interval of s.

If all segments are controlled by the same set of NozzelSelect lines, BankEnable lines, and ColorEnable lines (wired externally to the printhead), the following holds:

If both odd and even banks print simultaneously (if both BankEnable bits are set), 24 nozzles per segment, ie a total of 192 nozzles fire simultaneously, 5.7 Consume watts.

If the odd and even banks print independently, only 12 nozzles per segment, ie a total of 96 fired simultaneously, consuming 2.85 watts.

Print head interface 62
A printhead interface 62 connects the ACP to the printhead and provides both data and appropriate signals to the external printhead. The printhead interface 62 operates in conjunction with both the VLIW processor 74 and software algorithms running on the CPU to print a photo in about 2 seconds.

An overview of input and output to the printhead interface is shown in FIG. An address and data bus is used by the CPU to specify the addresses of the various registers of the printhead interface. A single BitClock output line leads to all 8 segments on the printhead. Eight DataBits (data bit) lines are connected to each segment, one at a time, and clocked in simultaneously to the eight segments of the printhead (based on the BitClock pulse). For example, simultaneously, dot 0 is transferred to segment 0, dot 750 is transferred to segment 1, dot 1500 is transferred to segment 2, and so on.

The VLIW output FIFO stores dithered 2-level C, M, and Y 6000 × 9000 resolution print images in the correct order for output to 8 DataBits lines. Since the ParallelXferClock is connected to each of the eight segments of the printhead, a single pulse causes all segments to transfer their respective bits simultaneously. Finally, the NozzleSelect line, BankEnable line and ColorEnable line are connected to each of the 8 segments so that the printhead interface can control the spacing of the C, M and Y drop pulses and the number of drops printed with each pulse. become. The printhead interface registers can specify pulse intervals from 0 to 6 μs, typically 2 μs.

Before an image print image is placed under the control of an Artcam user, the following two phases:
1. 1. Preparation of an image to be printed It is necessary to print the prepared image.

The image preparation needs to be executed only once. Image printing can be performed as many times as necessary.

Image preparation To prepare an image for printing,
1. 1. Conversion of photographic image to printed image 2. Rotate printed image (internal color space) to align with printer orientation output 3. Up-interpolation of compressed channels (if necessary) It includes color conversion from the internal color space to a CMY color space suitable for specific printers and inks.

At the end of the image preparation, preparation for printing a 4.5 MB correctly oriented 1000 × 1500 CMY color space image is completed.

Conversion of a photographic image into a print image In order to convert a photographic image into a print image, it is necessary to execute a Vark script in order to execute image processing. This script is a default image enhancement script or a Vark script extracted from the currently inserted Artcard. The Vark script is executed by the CPU and accelerated with a function executed by the VLIW vector processor.

The image in the print image rotation memory is initially oriented so that the top edge is upward. Thereby, the Vark process can be simplified. Before the image is printed, the image must be aligned with the orientation of the print roll. Realignment need only be performed once. Subsequent prints of the printed image will have been properly rotated beforehand.

The transformation to be applied is simply the inverse of the transformation applied during capture from the CCD when the user presses the “Camera” button on Artcam. If the original rotation is 0, no conversion is necessary. If the original rotation is +90 degrees, it must be rotated -90 degrees (same as 270 degrees) before printing. The method used to apply this rotation is the Vark accelerated affine transformation function. The affine transformation engine is called to rotate each color channel independently. Note that the color channel cannot be rotated in its original position. Instead, the color channel can utilize the space previously used for the enlarged single channel (1.5 MB).

FIG. 155 is an explanatory diagram of an example of rotation of the Lab image. The a channel and the b channel are compressed to 4: 1. The L channel is rotated to the space that is no longer needed (single channel area), then the a channel is rotated to the empty space left after L, and finally b can be rotated. . The total time required to rotate the three channels is 0.09 seconds. This is an acceptable time lapse before the first image print. Subsequent prints do not incur this overhead.

Up-interpolation and color conversion Lab images must be converted to CMY before printing. Various processes are performed depending on whether the a channel and the b channel of the Lab image are compressed. If the Lab image is compressed, the a and b channels must be decompressed before color conversion is performed. If the Lab image is not compressed, color conversion is the only necessary step. The Lab image is up-interpolated (if the a and b channels are compressed) and converted to a CMY image. A single VLIW process combining scale and color conversion can be used.

The method used to perform the color conversion is a Vark accelerated color conversion function. The affine transformation engine can be called to rotate each color channel independently. The color channel cannot be rotated as it is. Instead, the color channel can take advantage of the space previously used for the enlarged single channel (1.5 MB).

Image Printing Process The image printing process involves capturing a 1000 × 1500 CMY image that is placed in the correct orientation and generating data and signals to be transmitted to an external printhead. This process requires a CPU that cooperates with the VLIW process and the printhead interface.

The resolution of the image in Artcam is 1000 × 1500. The printed image has a resolution of 6000 × 9000 dots and a very simple relationship: 1 pixel = 6 × 6 = 36 dots is obtained. As shown in FIG. 156, since each dot is 16.6 μm, a 6 × 6 square dot is 100 square μm. Since each dot has two levels, it is necessary to dither the output.

The image should be printed in about 2 seconds. In the case of 9000 rows of dots, this means that the time between printing each row is 222 μs. The printhead interface must generate 6000 dots within this time, averaging 37 ns per dot. However, since each dot is composed of three colors, the printhead interface must generate each color component within approximately 12 ns, ie within one ACP clock cycle (10 ns at 100 MHz). One VLIW process is responsible for calculating the next 6000 dot line to be printed. Odd and even C, M, and Y dots are generated by dithering input from six different 1000 × 1500 image lines. The second VLIW process takes the previously calculated 6000 dot line and accurately generates 8 bits of data for 8 segments to be transferred from the printhead interface to the printhead in one transfer. Have a role to play.

The CPU process updates the register in the first VLIW process 3 three times for each print line (one per color component = 27,000 times per second), and prints the register in the second VLIW process to the print line Update once every time (9000 times in 2 seconds). The CPU has advanced one line ahead of the VLIW process to do this.

Finally, the print head interface takes 8-bit data from the VLIW output FIFO, outputs it as it is to the print head, and appropriately generates a BitClock signal. After all the data has been transferred, a ParallelXferClock signal is generated to load the next print line data. In conjunction with the transfer of data to the printhead, separate timers use the NozzleSelect, ColorEnable, and BankEnable lines specified by the printhead interface internal registers to signal for different printhead print cycles. Generate.

In addition, the CPU controls various motors and cutters via a parallel interface during the printing process.

C, M and Y dot generation The input to this process is a 1000x1500 CMY image that is oriented in the correct direction for printing. This image is not compressed. As shown in FIG. 157, the VLIW microcode program takes a CMY image and generates the dithered C, M and Y pixels required by the printhead interface.

This process is performed three times, once for each of the three color components. This process does not include two subprocesses running in parallel, one produces odd dots and the other produces even dots. Each subprocess takes one pixel from the input image and generates three output dots (because 1 pixel = 6 output dots, and each subprocess is associated with either even or odd dots. From). One output dot is generated every cycle, but the input pixel is read only once every three cycles.

The original dither cell is a 64 × 64 type cell, and each entry is 8 bits. This original cell is divided into odd and even cells, so each remains 64 heights, but only 32 entries wide. Even dither cells store the original dither cell pixels 0, 2, 4, etc., while odd dither cells store the original dither cell pixels 1, 3, 5, etc. Since the dither cell is repeated across the line, a single 32-byte line for each of the two dither cells is required during the entire line and can therefore be fully cached. Since odd and even lines of a single process line are alternated by 8 dot lines, it is advantageous to rotate every 8 lines of odd dither cells. Thus, it is possible to use the same offset for both odd and even dither cells. As a result, the even dither cell line corresponds to the even entry of the original dither cell line L, and the odd dither cell line corresponds to the odd entry of the original dither cell line L + 8.

This process is performed three times, once for each of the color components. The CPU software routine must ensure that the sequential read iterator for odd and even lines specifies the correct image line corresponding to the printhead. For example, to generate a set of 18000 dots (3 sets of 6000 dots)
Yellow even dot line = 0, so input yellow image line = 0/6 = 0
Yellow odd dot line = 8, therefore, input yellow image line = 8/6 = 1
Magenta even dot line = 10, so input magenta image line = 10/6 = 1
Magenta odd dot line = 18, so input magenta image line = 18/6 = 3
Cyan even dot line = 20, so input cyan image line = 20/6 = 3
Cyan odd dot line = 28, so input cyan image line = 28/6 = 4
It is. The subsequent set of input image lines is
・ Y = [0,1], M = [1,3], C = [3,4]
・ Y = [0,1], M = [1,3], C = [3,4]
・ Y = [0,1], M = [2,3], C = [3,5]
・ Y = [0,1], M = [2,3], C = [3,5]
・ Y = [0,2], M = [2,3], C = [4,5]
It is.

However, the dither cell data need not be updated for each color component. The three-color dither cells are the same, but are offset by two dot lines for each component.

The dithered output is written sequentially to the write iterator, and the odd and even dithered dots are written to two separate outputs. Since the same two write iterators are used for all three color components, they are continuous within the division of the odd and even dots.

While a set of dots is generated for a print line, the previously generated set of dots is merged by a second VLIW process as described in the next section.

Merge 8-bit dot output generation This process captures a line of dithered dots and generates an 8-bit data stream for output to the printhead interface via the VLIW output FIFO, as shown in FIG. To do. This process requires that the entire line be prepared. This is because this process requires quasi-random access to most of the dithered lines simultaneously. The following constants are set by the software:

順次読み出しイタレータは、前に生成されたドットのラインを指定し、イタレータのレジスタは単一カラーコンポーネントへのアクセスを制限するためセットアップされている。後続の画素の間の距離は３７５であり、あるラインと次のラインとの間の距離は１バイトになるように与えられる。その結果として、８個のエントリーが「ライン」毎に読み出される。単一の「ライン」は、プリントヘッドへロードされる８ビットに対応する。画像の「ライン」総数は３７５にセットされる。少なくとも８個のキャッシュラインを順次読み出しイタレータに割り当てることにより、完全なキャッシュコヒーレンスが維持される。８ビットをカウントする代わりに、８個のマイクロコードステップが暗黙的にカウントを行う。

Sequential readout iterators specify a previously generated line of dots, and the iterator registers are set up to limit access to a single color component. The distance between subsequent pixels is 375, and the distance between one line and the next is given to be 1 byte. As a result, 8 entries are read for each “line”. A single “line” corresponds to 8 bits loaded into the printhead. The total number of “lines” in the image is set to 375. Full cache coherence is maintained by assigning at least eight cache lines to sequential read iterators. Instead of counting 8 bits, 8 microcode steps implicitly count.

The generation process first reads all entries from the even dots, combines 8 entries into 1 byte, and outputs the 1 byte to the VLIW output FIFO. After all 3000 even dots have been read, 3000 odd dots are read and processed. The software routine must update the dot addresses of the odd and even sequential iterators once per color component, ie, three times per line. The two VLIW processes require a total of 8 ALUs and VLIW output FIFOs. As long as the CPU can update the register in two processes as described above, the VLIW processor can generate dithered image dots fast enough to follow the printer.

Data Card Reader FIG. 159 shows one form of card reader 500 into which Artcard 9 can be inserted for reading. FIG. 158 is an exploded perspective view of the reader of FIG. The card reader is interconnected to the computer system and includes a CCD readout mechanism 35. The card reader includes pinch rollers 506 and 507 that sandwich the inserted Artcard 9. One roller, for example roller 506, is driven by an Artcard motor 37 for advancing the card 9 between the two rollers 506 and 507 at a uniform speed. Artcard 9 is passed over a series of LED lights 512, which are housed in a transparent plastic mold 514 having a semi-circular cross section. This cross-section, for example, collects light from the LED 512 on the surface of the card 9 as the card 9 passes by the LED 512. From that surface, the light is reflected to a high resolution linear CCD 34 configured with a resolution of about 480 dpi. Since the surface of Artcard 9 is encoded to a level of about 1600 dpi, linear CCD 34 supersamples the Artcard surface with a multiplier of about three times. The Artcard 9 is further driven at such a speed that the linear CCD 34 can supersample in the direction of movement of the Artcard at a rate of about 4800 readings per inch. The scanned Artcard CCD data is sent from the Artcard reader to the ACP 31 for processing. A sensor 49 that can be configured by an optical sensor functions to detect the presence or absence of the card 13.

The CCD reader has a lower substrate 516 and an upper substrate 514 comprising a transparent molded plastic. A linear CCD array 34 including a thin and long linear CCD array manufactured by a semiconductor manufacturing process is inserted between the two substrates.

Referring to FIG. 160, a partial cross-sectional perspective side view of one configuration example of the CCD reader unit is shown. A series of LEDs, for example LED 512, operates to emit light when the card 9 passes the surface of the CCD reader 34. The emitted light is transmitted through a part of the upper substrate 523. The substrate includes a portion having a curved periphery, such as a portion 529, to collect light emitted from the LED 512 at a point on the surface of the card 9, such as the point 532. The focused light is reflected from the point 532 toward the CCD array 34. A series of microlenses shown exaggerated, for example, microlenses 534 are formed on the surface of the upper substrate 523. The microlens 523 acts to collect light received from the entire surface at a lower focal point 536 corresponding to a point on the surface of the CCD reader 34 that detects light incident on the photosensitive portion of the CCD array 34.

Numerous improvements are conceivable for the above apparatus. For example, the sensing device on the linear CCD 34 may be staggered. Corresponding microlenses 34 can be correspondingly formed to focus the light into a series of staggered spots, corresponding to a staggered CCD sensor.

To aid reading, the data surface area of Artcard 9 may be modulated with a checkerboard-like pattern, as already described with reference to FIG. Other forms of high frequency modulation are possible.

It is clear that the Artcard printer can be provided as a printout of the data on the storage device Artcard. Therefore, the Artcard system can be used as a general form for distributing information to the outside of the Artcam device. Artcard printers can print out Artcards on a high quality print surface, and can print multiple Artcards on the same paper and later separate them. On the second side of Artcard 9, information relating to a file or the like stored in Artcard 9 can be printed for subsequent storage.

Thus, the Artcard system allows for a simplified form of storage device suitable for use in place of other forms of storage devices such as CD-ROMs, magnetic disks and the like. Artcard 9 can be mass produced and can therefore be produced in a substantially low cost form for redistribution.

Print Roll FIG. 162 shows Artcam's print roll 42 and print head section. The paper / film 611 is fed to the printing mechanism 15 in a continuous “web-like” process, which includes additional pinch rollers 616-619 and a print head 44.

The pinch roller 613 is coupled to a drive mechanism (not shown), and when the print roller 613 rotates, “paper” in the form of film 611 is passed through the print mechanism 615 and out of the picture output slot 6. A rotary cutting mechanism (not shown) is used to cut the roll of paper 611 at the requested photo size.

Thus, it is clear that the printer roll 42 serves to supply “paper” 611 to a printing mechanism 615 that prints a picture imaged for photography.

FIG. 163 shows an exploded perspective view of the print roll 42. The print roll 42 includes output printer paper 611 that is output under the operation of pinch rollers 612 and 613.

Next, FIG. 164 shows a more complete exploded perspective view of the print roll 42 of FIG. 163 with the “paper” film roll removed. The print roll 42 includes three main parts, an ink storage section 620, a paper roll section 622, 623, and an outer casing section 626, 627.

Referring initially to the ink storage section 620, an ink reservoir or ink supply section 633 is provided. Printing ink is contained in three bladder-type containers 630 to 632. The printer roll 42 shall provide full color output ink. Accordingly, the first ink reservoir or bladder container 630 contains cyan ink. The second reservoir 631 contains magenta ink, and the third reservoir 632 contains yellow ink. Each of the reservoirs 630-632 is designed to have substantially the same volume size, although the volume dimensions may be different.

The ink storage sections 621 and 633 can be manufactured from a plastic section together with the cover 624, and are designed to be integrated using heat sealing, ultraviolet irradiation, or the like. Each of the equal sized ink reservoirs 630 to 632 is coupled to a corresponding ink channel 639 to 641 to allow ink flow from the reservoir 630 to 632 to the corresponding ink output port 635 to 637. The ink reservoir 632 includes an ink channel 641 and an output port 637, the ink reservoir 631 includes an ink channel 640 and an output port 636, and the ink reservoir 630 includes an ink channel 639 and an output port 7637.

During operation, ink reservoirs 630-632 are filled with corresponding ink and section 633 is joined to section 621. The ink storage sections 630 to 632 may be foldable bladders, allowing ink to traverse the ink channels 639-641 and in fluid communication with the ink output ports 635 to 637. Further, if desired, an air intake can be provided to maintain the pressure associated with the ink channel reservoirs 630-632 as required.

The cap 624 can be joined to the ink storage section 620 to form a pressurized cavity accessible from the pneumatic intake.

The ink storage sections 621, 633 and 624 are designed to be connected together as an integral unit and inserted into the printer roll sections 622, 623. The printer roll sections 622, 623 are designed to be joined together by snap fit using male sections 645-647 that couple with corresponding female sections (not shown). Similarly, female portions 654-656 are designed to mate with corresponding male portions 660-662. The paper roll sections 622, 623 are thus designed to be snapped together. One end of the film in the roll is clamped between the two sections 622, 623 when the two sections 622, 623 are joined together. The print film can be rotated on the print roll sections 622, 625 as required.

As described above, the ink storage sections 620, 621, 633, 624 are designed to be inserted inside the paper roll sections 622, 623. Printer roll sections 622, 623 can rotate around stationary ink storage sections 621, 633, and 624 to provide film on demand.

Outer casing sections 626 and 627 are further designed to be coupled around print roll sections 622, 623. In addition, each end of a pinch roller, e.g., 612, 613, is designed to be fastened to a corresponding cavity, e.g., 670, in the cover 626, 627, and the roller 613 feeds the print film from It is driven from the outside (not shown) for discharging.

Finally, a cavity 677 can be provided in the ink storage section 620, 621 for inserting and guiding a silicon chip integrated circuit type device 53 for storing information related to the print roll 42.

As shown in FIGS. 155 and 164, the print roll 42 is designed to be inserted into the Artcam camera device and coupled to the coupling unit 680, and the coupling unit 680 is a connector for wiring with the silicon chip 53. A pad 681 is included. Further, the connector 680 includes four end connectors for connection with the ink supply ports 635 to 637. The ink supply port is then connected to an ink supply line, eg, 682, which in turn is interconnected to a printhead supply port, eg, 687, to pass ink to the printhead 44 as needed. Shed.

The “media” 611 used to form the roll can be composed of a variety of materials designed to print an appropriate image on top. For example, an opaque, rotatable plastic material can be utilized, transparency can be used by using a transparent plastic sheet, and metallic printing can be performed by utilizing a metallic sheet film. In addition, it is possible to utilize a fabric in the printer roll 42 and print an image on the fabric, but it should be noted that only materials with suitable stiffness or suitable backing can be utilized.

When the print media is plastic, it is possible to cover the print media with a layer that fixes and absorbs ink. In addition, some types of printing media such as opaque white mats, transparent films, matte transparent films, lenticular array films for three-dimensional solid printing, metallized films, gratings or holograms of embossing optically. Variable devices, media pre-printed on the back side, and media including a magnetic recording layer can be used. When utilizing metal stays, the metal stays can have a polymer base and are covered with a thin (a few microns) aluminum or other metal deposition layer, and then the ink is passed through an ink printer mechanism. Covered with a transparent protective layer adapted to accept.

The print roll 42 used is inserted into the camera device and is designed to provide ink and paper for printing an image as required. Ink output ports 635 to 637 are compatible with corresponding ports in the camera device. The pinch rollers 672 and 673 are operated so that paper can be supplied to the camera device under the control of the camera device.

As shown in FIG. 164, the mounted silicon chip 53 is inserted into one end of the print roll 42. In FIG. 165, the authentication chip 53 is shown in more detail, which includes four communication leads 680-683 that communicate details from the chip 53 to the corresponding camera in which the chip is inserted.

Referring to FIG. 165, a chip can be made alone by placing a miniature integrated circuit 687 in epoxy and bonding bonding leads, eg, 688, to external communication leads 680-683. The integrated chip 687 is about 400 square microns and the scribe boundary is 100 microns. Subsequently, the chip is applied to the appropriate surface of the cavity of the print roll 42. FIG. 166, which is an exploded view of the device of FIG. 165, shows an integrated circuit 67 interconnected to bonding pads 681,682.

In FIGS. 164A through 164E, reference numeral 1100 generally indicates the print cartridge 1100. The print cartridge 1100 includes an ink cartridge 1102 according to the present invention.

Print cartridge 1100 includes a housing 1104. As shown in more detail in FIG. 2, the housing 1104 is defined by an upper molded part 1106 and a lower molded part 1108. Molded parts 1106 and 1108 are sandwiched between clips 1110. The housing 1104 is covered by a label 1112 that gives the cartridge 1100 an attractive appearance. The label 1112 carries information for enabling the user to use the cartridge 1100.

The housing 1104 defines a chamber 1114 in which the ink cartridge 1102 is contained. The ink cartridge 1102 is fixedly supported on the chamber 1114 of the housing 1104.

A supply of print media 1116, including a roll 1126 of film / media 1118 wound around the former, is contained in a chamber 1114 of the housing 1104. The former 1120 is slidably accommodated in the ink cartridge 1102, and is rotatable with respect to the ink cartridge.

As shown in FIG. 164B, the upper molded part 1106 and the lower molded part 1108 are clipped together, an exit slot 1122 is defined, and the tongue of the paper 1118 is ejected through the exit slot.

The cartridge 1100 includes a roller assembly 1124 that serves to decurl the paper 1118 as the paper 1118 is fed from the roll 1126 and to push the paper 1118 through the slot 1122. The roller assembly 1124 includes a driving roller 1128 and two driven rollers 1130. The driven roller 1130 is rotatably supported by the rib 1132, and the rib 1132 rises from the floor 1134 of the lower molded part 1108 of the housing 1104. Roller 1130 is integral with drive roller 1128 to provide positive traction to paper 1118 to control the speed and position of paper 1118 as paper 1118 is ejected from housing 1104. Roller 1130 is an injection molded part of a suitable synthetic plastic material such as polystyrene. In this regard, the upper molded part 1106 and the lower molded part 1108 are similarly injection molded parts of a suitable synthetic plastic material such as polystyrene.

The drive roller 1128 includes a drive shaft 1136 that is rotatably anchored between mating recesses 1138 and 1140 defined in the respective side walls of the upper molded part 1106 and the lower molded part 1108 of the housing 1104. The opposite end 1142 of the drive roller 1128 is rotatably held by the upper molding part 1106 and the lower molding part 1108 of the housing 1140 with an appropriate configuration (not shown).

The drive roller 1128 is a two-shot injection molded part that includes a high impact polystyrene shaft 1136 on which a bearing means in the form of an elastomer or rubber roller portion 1144 is molded. These portions 1144 securely lock the paper 1118 and prevent the paper 1118 from slipping when the paper 1118 is fed from the cartridge 1100.

The end of the roller 1128 protruding from the housing 1104 has an engagement body in the shape of a cruciform structure 1146 (FIG. 164A), which is printed on a device such as a camera to which the print cartridge 1100 is mounted. Coupled with a geared drive interface (not shown) of the head assembly. This structure ensures that the ink is accurately aligned on the paper 1118 because it reliably synchronizes the speed at which the paper 1118 is fed to the printhead with printing by the printhead.

The ink cartridge 1102 includes a container 1148 that is in the form of a right cylindrical extruded part. The container 1148 is extruded from a suitable synthetic plastic material such as polystyrene.

In a preferred embodiment of the present invention, the print head with which the print cartridge 1100 is used is a multicolor print head. Accordingly, the container 1148 is divided into a plurality, more specifically four compartments, or reservoirs 1150. Each reservoir 1150 contains a different color or type of ink. In one embodiment, the ink stored in the reservoir 1150 is cyan ink, magenta ink, yellow ink, and black ink. In another embodiment of the present invention, three types of colored inks, cyan ink, magenta ink, and yellow ink are contained in three of the reservoirs 1150, and the fourth reservoir 1150 is visualized only in the infrared spectrum. Contains ink.

As clearly shown in FIGS. 164C and 164D, one end of the container 1148 is closed by an end cap 1152. The end cap 1152 has a plurality of openings 1154 defined therein. Since the opening 1154 is associated with each reservoir 1150, the interior of the reservoir 1150 is maintained at atmospheric pressure at the end of the container 1148 provided with the end cap 1152.

The seal structure 1156 is provided in the container 1148 on the end side where the end cap 1152 is provided. Seal structure 1156 includes a quadrant pellet 1158 made of a gel material slidably received in each reservoir 1150. The gel material of the pellet 1158 is a compound composed of a thermosetting rubber and a hydrocarbon. Hydrocarbons are white mineral oils. Thermoset rubber is a copolymer that gives the mineral oil sufficient rigidity so that the pellet 1158 maintains its shape at normal operating temperatures while at the same time the pellet 1158 can slide within its associated reservoir 1150. is there. A suitable thermosetting rubber is the thermosetting rubber sold under the name “Kalton” ® by the Shell Petroleum Company. The copolymer is present in the compound in an amount sufficient to give each pellet 1158 a gel-like viscosity. Typically, the copolymer will be present in an amount of about 3% to 20% weight percent, depending on the type used.

In use, the compound is heated, resulting in a fluid. As each reservoir 1150 is filled with a particular type of ink, the molten compound is injected into each reservoir 1150 and the compound sets to form pellets 1158. The atmospheric pressure at the back of the pellet 1158, ie, the atmospheric pressure at the end of the pellet facing the end cap 1152, causes the self-lubricated pellet 1148 to slide toward the opposite side of the container 1148 when the ink is withdrawn from the reservoir 1150. Guarantee that. The pellet 1158 stops the ink from emptying out of the inverted container, prevents the ink in the reservoir 1150 from becoming dirty, and prevents the ink in the reservoir 1150 from drying out. Furthermore, the pellet 1158 is hydrophobic to prevent ink leakage from the reservoir 1150.

The opposite end of the container 1148 is closed by an ink fountain molded part 1160. A baffle 1162 supported on the molded part 1160 receives an elastomer seal molded part 1164. A sealing curtain 1166 is defined in an elastomer seal molded part 1164 which is hydrophobic. Each sealing curtain 1166 is provided with a slit 1168 so that alignment pins (not shown) from the printhead assembly can be inserted through the slit 1168 and are in fluid communication with the reservoir 1150 of the container 1148. . The hollow protrusion 1170 protrudes from the opposite side of the ink fountain molded part 1160. Each protrusion 1170 is shaped to fit snugly with the associated reservoir 1150 to position the ink fountain molded part at the end of the container 1148.

Referring to FIG. 164C, the ink fountain molded part 1160 is maintained in the correct position by the carrier or concealed board molded part 1172. The hidden plate molding part 1172 defines a four-leaf clover-shaped window 1174 through which the elastomer seal molding part 1164 can be accessed. The hidden plate molded part 1172 is anchored between the upper molded part 1106 and the lower molded part 1108 of the housing 1104. The hidden plate molded part 1174 and the webs 1176 and 1178 extending from the respective inner surfaces of the upper molded part 1106 and the lower molded part 1108 of the housing 1104 define a compartment 1180. The air filter 1182 is housed in the compartment 1180 and is held in place by the end molded part 1174. Air filter 1182 cooperates with the printhead assembly. Air is blown all over the nozzle guard of the print head assembly to clean the nozzle guard. This air is filtered by drawing it through the air filter 1182 using a pin (not shown) accommodated in the inlet side opening 1184 in the hidden plate 1172.

Air filter 1182 is shown in more detail in FIG. 164E. The air filter 1182 includes a filter medium 1192. Filter media 1192 is based on synthetic fibers and is arranged in a fluted form to increase the available surface area for filtering purposes. Instead of the paper-based filter media 1192, other fibrous core materials may be used.

The filter medium 1192 is accommodated in the sealed container 1194. The sealed container 1194 includes a bottom molded part 1196 and a lid 1198. To accommodate the compartment 1180 of the housing 1104, the sealed container 1194 is partially annular, i.e., a horseshoe. In this way, the sealed container 1194 has a pair of opposed ends 1200. An intake opening 1202 is defined at each end 1200.

An exhaust opening 1204 is defined in the lid 1198. The exhaust opening 1204 is initially closed by a film or membrane 1206. When the filter 1182 is installed in the correct position in the compartment 1180, the exhaust opening 1204 is aligned with the opening 1184 in the hidden plate molded part 1172. Pins from the print head assembly penetrate film 1206 and draw air from the atmosphere through air filter 1182 before air is blown onto the nozzle guard and print head assembly print head.

The bottom molded part 1194 includes positioning structures 1208 and 1210 for placing the filter media 1192 in the correct location on the sealed container 1194. The positioning structure 1208 is in the form of a plurality of pins, while the positioning structure 1210 is in the form of a rib that locks the end 1214 of the filter media 1192.

After the filter media 1192 is placed in the correct position on the bottom molded part 1196, the lid 1198 is secured to the bottom molded part 1196 by ultrasonic welding or similar methods to seal the bottom molded part 1196 with the lid 1198. .

When the print cartridge 1100 is assembled, the membrane or film 1186 is placed over the outer edge of the hidden plate molding part 1172 to close the window 1174. This membrane or film 1186 is pierced or ruptured by a pin for use. Film 1186 prevents debris from entering ink reservoir 1150.

Authentication means in the form of an authentication chip 1188 is received in the opening 1190 of the hidden plate molded part 1172. The authentication chip 1188 is queried by the printhead assembly 1188 to ensure that the print cartridge 1100 is compatible and compliant with the printhead assembly of the device.

Authentication chip authentication chip 53
The authentication chip 53 of the preferred embodiment is responsible for ensuring that only properly manufactured print rolls are utilized in the camera system. The authentication chip 53 utilizes techniques that are generally useful when used with any consumable that is not limited to a print roll system. Manufacturers of other systems that require consumables (such as laser printers that require toner cartridges) struggle with the problem of authenticating consumables, and the level of success varies. Most of them rely on specialized packaging. However, this does not limit the refilling work at home and the production of duplicates. Preventing copying is important to prevent incompletely manufactured substitute consumables from damaging the base system. For example, inadequately filtered ink can clog the print nozzles of an ink jet printer, making the consumer responsible for the system manufacturer and not allowing the use of unauthorized consumables.

In order to solve the authentication problem, the authentication chip 53 includes an authentication code and a circuit specially designed for copy protection. The chip is manufactured using standard flash memory manufacturing processes and is inexpensive enough to be incorporated into consumables such as ink and toner cartridges. Once programmed, the authentication chip complies with NSA export guidelines as described below. Certification is a very large and constantly growing field. The following description relates only to consumable authentication.

Symbol names The following symbol names are used throughout the description of this embodiment.

基本用語
Ｍで示されたメッセージは平文である。Ｍを、Ｍの実体が隠された暗号文Ｃに変換するプロセスは、暗号化と呼ばれる。ＣをＭへ逆変換するプロセスは、復号化と呼ばれる。暗号化関数をＥで表し、復号化関数をＤで表すことにより、以下の恒等式：
Ｅ［Ｍ］＝Ｃ
Ｄ［Ｃ］＝Ｍ
が得られる。したがって、以下の恒等式：
Ｄ［Ｅ［Ｍ］］＝Ｍ
が成り立つ。

The message indicated by the basic term M is plain text. The process of converting M to ciphertext C in which the entity of M is hidden is called encryption. The process of transforming C back to M is called decoding. By representing the encryption function as E and the decryption function as D, the following identity:
E [M] = C
D [C] = M
Is obtained. Thus, the following identity:
D [E [M]] = M
Holds.

Symmetric encryption method Symmetric encryption algorithm:
The encryption function E depends on the key K ₁ ;
The decryption function D depends on the key K ₂ ;
K ₂ can be derived from K ₁ ;
K ₁ can be derived from K ₂ ;
It is an algorithm.

In most symmetric algorithms, K ₁ is usually equal to K ₂ . However, even if K ₁ is not equal to K ₂ , a single key is sufficient for mathematical definition if one key can be derived from the other. . Therefore,
E _K [M] = C
D _K [C] = M
It can be expressed.

There are numerous symmetric algorithms, from textbooks that everyone knows to sophisticated modern algorithms. Most of them are uncertain in that the current cryptanalysis techniques can succeed in the attack to the extent that K can be derived. The security of a particular symmetric algorithm is usually a function of two things: the algorithm strength and the key length. The following algorithm includes features suitable for use with authentication chips:
DES;
Blowfish;
RC5;
IDEA.

DES
DES (Data Encryption Standard) is a US and international standard, and the same key is used for encryption and decryption. The key length is 56 bits. It is implemented in hardware and software, but the original design was hardware only. The original algorithm used in DES is described in US Pat. No. 3,962,539. The DES variant, called the triple DES variant, is more secure, but requires _three keys: K ₁ , K ₂ and K ₃ . These keys are
E _K3 [D _K2 [E _K1 [M]]] = C
D _K3 [E _K2 [D _K1 [C]]] = M
It is used in the form of

The main advantage of triple DES is that it can be made more secure than single key DES using existing DES implementations. In particular, triple DES provides protection equivalent to a 112 bit key length. Triple DES does not provide protection equivalent to a 168-bit key (3 × 56) as simply expected. Devices that perform Triple DES decoding and / or encoding cannot be taken from the United States.

Blowfish
Blowfish is a symmetric block cipher originally proposed by Schneier in 1994. This uses a variable length key from 32 bits to 448 bits. Furthermore, it is much faster than DES. The Blowfish algorithm is composed of two parts, a key expansion unit and a data encryption unit. In key expansion, a key having a maximum of 448 bits is converted into a plurality of subkey arrays having a total of 4168 bytes. Data encryption is performed by a 16-round Feistel network. All operations are XORs and additions over 32-bit words, including 4 index array lookups per round. It should be noted that decryption is the same as encryption except that the subkeys are used in reverse order. Thus, the implementation complexity is reduced over other algorithms that do not have such symmetry.

RC5
RC5 designed by Ron Rivest in 1995 has variable block size, key size, and rounds. However, typically RC5 uses a 64-bit block size and a 128-bit key. The RC5 algorithm is composed of two parts, a key extension part and a data encryption part. Key expansion converts the key into 2r + 2 subkeys (where r = number of rounds), where each subkey is w bits. In the case of a 64-bit block size and 16 rounds (w = 32, r = 16), the subkey arrangement is a total of 136 bytes. Data decoding is used with addition of modulo ^{2 w,} and XOR, a bit rotation, the.

IDEA
The first embodiment of the IDEA cryptosystem developed by Lai and Massey in 1990 is called PES. After the differential cryptanalysis was discovered by Biham and Shamir in 1991, the algorithm was enhanced and the results were published as IDEA in 1992. IDEA uses a 128-bit key to manipulate 64-bit plaintext blocks. The same algorithm is used for encryption and decryption. This algorithm is generally considered to be the safest block algorithm currently available. This is described in US Pat. No. 5,214,703, issued in 1993.

An algorithm that can be used for the asymmetric encryption method is an asymmetric algorithm. What is an asymmetric encryption algorithm:
The encryption function E depends on the key K ₁ ;
The decryption function D depends on the key K ₂ ;
K ₂ cannot be derived from K ₁ within a reasonable time;
K ₁ cannot be derived from K ₂ within a reasonable time;
It is an algorithm.

Therefore,
E _K1 [M] = C
D _K2 [C] = M
It is.

These algorithms are also referred to as public keys because _one key K1 is made public. Thus, anyone can encrypt a message (using K ₁ ), but only those who have the corresponding decryption key (K ₂ ) can decrypt and read the message. In most cases, the following identity:
E _K2 [M] = C
D _K1 [C] = M
Is established.

This identity is very important. This is because anyone who has the public key K ₁ can see M, and know that M is from the owner of K ₂ . Others cannot generate C. This is because it may generate C is because it means that knows K _2. Inability derive K ₁ from K ₂ within a reasonable time, as well, can not derive the K ₂ from K ₁ to the contrary, of course, become obscured by the concept of a reasonable time. As has been proved again, calculations that are expected to take a long time can be realized by introducing faster computers, new algorithms, and the like. The safety of asymmetric algorithms has two problems: factoring large numbers (more specifically, large numbers that are the product of two large prime numbers) and the difficulty of computing discrete logarithmic functions in finite numbers. Based on the difficulty of one of them. It is speculated that large number factoring is a difficult problem in understanding modern mathematics. However, the problem is that prime factorization is faster and easier than expected. Ron
Rivest said in 1977 that a 125-digit prime factorization would take 40,000 trillion years. In 1994, 129-digit prime factorization was performed. According to Schneier, a 1024-bit number is required to currently obtain a level of security derived from a 512-bit number in the 1980s. If the key lasts for several years, 1024 bits is not enough. Rivest revised the key length assessment in 1990 and proposed 1628 bits to maintain high security until 2005, and 1884 bits for high security that could last until 2015. Yes. Schneier, on the other hand, suggests that 2048 bits are needed to get protection from businesses and governments until 2015.

There are a number of public key encryption algorithms. Most of them are practically unimplementable, many of which generate very large C for a given M or require huge keys. In addition, others, even if safe, are so slow that they cannot be implemented in a few years. For this reason, many public key systems are hybrid and the public key mechanism is used to send a symmetric session key, and the session key is used for the actual message. All algorithms are problematic in terms of key selection. Random numbers are inadequately secure. Large prime numbers p and q need to be carefully selected, and certain combinations have the disadvantage that they can be easily factored (some weak keys can be examined). But nevertheless, key selection is not as simple as simply selecting 1024 bits randomly. As a result, the key selection process must also be secure.

Among the practical algorithms in use under public scrutiny, the following algorithms:
RSA;
DSA;
EIGamal;
Is suitable for use.

RSA
The RSA cryptosystem, named after Rivest, Shamir, and Adleman, is the most widely used public key cryptosystem and is the de facto standard in many parts of the world. The security of RSA is thought to depend on the difficulty of a large number of prime factors, which is the product of two prime numbers (p and q). There are a number of constraints on the generation of p and q. They both need to be large, have the same number of bits, and must not be close to each other (otherwise pq approximates √pq). Moreover, it is pointed out that in many cases p and q must be strong prime numbers. The RSA algorithm patent was issued in 1983 (US Pat. No. 4,405,829).

DSA
DSA (Digital Signature Standard) is an algorithm designed as part of the Digital Signature Standard (DSS). As specified, this algorithm cannot be used for general encryption. In addition, DSA is 10 to 40 times slower for signature verification than RSA. DSA explicitly uses the SHA-1 hash algorithm (see the definition of the one-way function below). DSA key generation relies on the detection of two prime numbers p and q such that q divides p-1. According to Schneier, a p-value of 1024 bits is required for long-term DSA security. However, the DSA standard does not allow the value of p to be greater than 1024 bits (p must also be a multiple of 64 bits). The US government owns the DSA algorithm and holds at least one related patent (US Pat. No. 5,231,688 issued in 1993).

EIGal
The EIGal method is used for both encryption and digital signatures. Its safety relies on the difficulty of computing discrete logarithmic functions with bounds. Key selection requires selection of a prime number p and two random numbers g and x such that g and x are smaller than p. Next, y = gx
Calculate mod p. Public keys are y, g, and p. The secret key is x.

The general principle of cryptographic challenge-response and zero knowledge proof challenge-response protocols is to provide identity authentication suitable for camera systems. The simplest form of challenge-response takes the form of a secret password. A asks B for a secret password, and when B responds with the correct password, A asserts that B is authentic. There are three major problems with this type of simple protocol. First, once B announces the password, any observer C knows what the password is. Second, A needs to know the password to verify the password. Third, if C impersonates A, B (assuming C is A) gives C a password, thereby putting B at risk. Using copyrighted text (eg haiku) is a weak substitute. This is because anyone can copy the password (eg, in countries where intellectual property rights are not respected). The idea of a cryptographic challenge-response protocol is that a single entity (requester) does not reveal the secret information itself to the verifier during the protocol by demonstrating knowledge of the secret information known about the entity, Passing your identity to others (verifiers). In the generalized case of cryptographic challenge-response protocols, the verifier knows the secret information in some way, otherwise the secret information is not even known to the verifier. Since the description of this embodiment is particularly concerned with authentication, the actual cryptographic challenge-response protocol that is tried for authentication is detailed in the appropriate section. However, the concept of zero knowledge proof is explained here. First, the zero-knowledge proof protocol revealed by Feige, Fiat, and Shamir is widely used in smart cards for authentication purposes. The effectiveness of this protocol is based on the assumption that computing the square root modulo of large composite integers with unknown prime factorization is not feasible with computer capability. This is probably equivalent to the assumption that large integer prime factorization is difficult. It should be noted that the requester need not have significant computational power. Smart cards achieve this type of authentication using only a few modulo multiplications. A zero knowledge proof protocol is described in US Pat. No. 4,747,668.

The one-way function F acts on the input X and returns F [X] such that X cannot be determined from F [X]. If there is no restriction on the format of X and F [X] stores fewer bits than X, there should be a collision. Collision, the same F [X] value two different X values produce, i.e., _{X 1} is ≠ _{X 2, moreover, F [X 1] = F} [X 2] become such _{X 1} and _{X 2} Is defined as If X stores more bits than F [X], the input must be compressed in some way to produce an output. In most cases, X is divided into blocks of a specific size and compressed over a number of rounds, with the output of one round being the input to the next round. The output of this hash function is the last output after X is exhausted. The pseudo-collision of the compression function CF is defined as _two different initial values V ₁ and V ₂ , and the two inputs X ₁ and X ₂ (which may be identical) are CF (V ₁ , X ₁ ). =
It is given to be CF (V ₂ , X ₂ ). Note that the presence of pseudo-collisions does not simplify the calculation of X ₂ for a given X ₁ .

I am only interested in one-way functions that can be computed at high speed. Furthermore, we are only interested in deterministic one-way functions that are reproducible in various implementations. As an example, consider F where F [X] is the time between calls to F. For a given F [X], X can be determined. This is because X is not evenly used by F. However, the output from F is different in various implementations. Therefore, I am not interested in this kind of F.

Within the scope of the description of the implementation of the authentication chip of this embodiment, a one-way function of the following form:
Encryption using an unknown key;
Random number sequence;
Hash function;
Message authentication code;
Only interested in.

When an encrypted message using an unknown key is encrypted using an unknown key K, the encryption function E is effectively unidirectional. In the absence of this key, obtaining M without K from E _K [M] is not feasible in terms of computer capability. The encryption function is unidirectional as long as the key is hidden. The encryption algorithm does not generate collisions. This is because E generates E _K [M] such that M cannot be reconstructed using function D. As a result, F [X] stores at least as many bits as X if the one-way function F is E (no information is lost). Symmetric encryption algorithms (above) are advantageous over asymmetric algorithms that generate one-way functions based on encryption. The reason is,
The key for a given encryption algorithm is shorter for a symmetric algorithm than for an asymmetric algorithm,
This is because the symmetric algorithm can calculate faster and requires less software / silicon.

The selection of a good key depends on the chosen encryption algorithm. Certain keys are not strong for a particular encryption algorithm, so it is necessary to test the strength of all keys. The more tests that have to be performed for key selection, the less likely that the key will remain hidden.

Random number sequence Random number sequence R ₀ , R ₁ ,. . . , R _i , R _{i + 1} . Define a one-way function such that F [X] returns the Xth random number in the random number sequence. However, it must be ensured that F [X] is reproducible for a given X in various implementations. Thus, the random number sequence is not truly random. Instead, the random number sequence is a pseudo-random number and the generator uses a special seed.

There are a number of problems with defining a good random number generator. Knuth explains the factors that "make" a generator (including statistical tests), and general issues related to generator construction. Most random number generators generate the i th random number from the (i−1) th state, and the only way to determine the i th number is to iterate from the 0 th number to the i th number. is there. If i is large, it is not practical to wait for i iterations. However, certain types of random number generators do not allow random access. Blum, Blum and Shub, for an ideal generator, “To quickly generate a long (bit) sequence from a short seed that appears to have been completely generated by successive inversions of calibrated coins. Pseudorandom sequence generator is preferred ". They are X ²
A mod n type generator, generally called a BBS generator, has been defined. Given the certain assumptions on which current encryption schemes rely, the BBS generator has shown to pass very strict statistical tests.

The BBS generator relies on the choice of n being a Blum integer (n = pq, p and q are large prime numbers, p ≠ q, p
mod 4 = 3 and q
mod 4 = 3). The initial state of the generator is given by x ₀ , where x ₀ = x ²
mod n, where x is a random integer that is relatively better than n. i th pseudorandom bit is the least significant bit of _{x i, x i = x i} -1 2
mod n. As an additional property, the knowledge of p and q is x _i = x ₀ ^y
mod n, where y = 2 ⁱ
mod ((p−1) (q−1)) allows the direct calculation of the i th number in the sequence.

Without knowledge of p and q, the generator must perform an iteration (the security of the computation depends on the difficulty of factoring large numbers). When originally defined, the main problem with the BBS generator was the amount of work required for a single output bit. This algorithm was considered too slow for most applications. However, with the advent of Montgomery-type reduction operations, more practical implementations have occurred. Moreover, Vazirani and Vazirani have shown that depending on the size of n, more bits can be safely extracted from x _i without compromising the security of the generator. If only one bit is needed per x _i , N bits (and thus N iterations of the bit generator function) are needed to generate an N-bit random number. There is no way for an external observer to determine the next bit with a probability other than half when given a particular set of bits. If x, p and q are kept secret, they act as keys, taking the output bitstream and calculating x, p and q is not computationally feasible. Also, determining the value of i used to generate a given set of pseudo-random bits is not feasible in terms of computer capabilities. This last feature makes the generator unidirectional. Different values of i may result in the same bit sequence of a predetermined length (eg, 32 random bits). Even if x, p and q are known, for a given F [i], i can only be derived as a set of probabilities and not as a specific value. (Of course, if i's domain is known, the probability set is further limited). However, there are problems regarding the choice of good p and q and good seed x. In particular, Ritter describes the problem of selecting x. The nature of this problem is that the BBS generator does not generate a single cycle of known length. Instead, the BBS generator generates various length cycles, including degenerate (zero length) cycles. Thus, the BBS generator cannot be initialized with a random state that may be a short cycle.

A special one-way function known as a hash function hash function maps an arbitrarily long message to a fixed length hash value. The hash function is expressed as H [M]. Since the input is of arbitrary length, the hash function has a compression component to produce a fixed length output. The hash function also includes an obfuscation component to make it difficult to detect collisions and to determine information about M from H [M]. Since a collision actually exists, most applications have difficulty finding X ₂ such that H [X ₁ ] = H [X ₂ ] for a given X ₁ , Requires the hash algorithm to be pre-image resistant. Moreover, most applications find it difficult to find _two messages X ₁ and X ₂ such that the hash algorithm is collision resistant (ie H [X ₁ ] = H [X ₂ ]). ) Whether a collision-resistant hash function can exist in an ideal sense is an undetermined question. The main application of the hash function is to transform the input message into a digital “fingerprint” before applying the digital signature algorithm. One problem of conflicts with digital signatures is illustrated in the following example.

A has a long message M ₁ that says "I have a $ 100 loan to B." A signs H [M ₁ ] with his private key. A greedy B is H [M ₂ ] = H [M ₁ ] and a collision message M ₂ where M ₂ is advantageous to B, eg, “I have a $ 1 million borrow in B” Explore. Clearly, it is A's benefit to ensure that finding such M ₂ is difficult.

Examples of collision-resistant one-way hash functions are SHA-1, MD5, and RIPEMD-160, all of which are derived from MD4.

MD4
Ron Rivest announced MD4 in 1990. Since all other one-way hash functions are derived from MD4 by some method, MD4 will be described here. MD4 is now considered completely broken in that it can be computed rather than searched for collisions. In the above example, B can use the same hash value as the original message M _1, and generates an alternative message M ₂ normally.

MD5
Ron Rivest announced MD5 in 1991 as MD4 with increased safety. Similar to MD4, MD5 generates a 128-bit hash value. Dobertin describes the state of MD5 after a recent attack. He explained how to find a pseudo-collision in MD5, pointed out the weakness of the compression function, and more recently a collision was discovered. That is, MD5 should not be used for compression of digital signature schemes where the presence of collisions can have serious consequences. However, MD5 can still be used as a one-way function. Moreover, the structure of HMAC-MD5 is not affected by these recent attacks.

SHA-1
SHA-1 is very similar to MD5, but has a 160-bit hash value (MD5 has only a 128-bit hash value). SHA-1 was designed and published by NIST and NSA for use in the Digital Signature Standard (DSS). The first published explanation was called SHA, but soon afterwards it was probably revised to correct SHA safety flaws and became SHA-1 (however, the NSA was the math that caused the change) For no reason). There is no known cryptographic attack on SHA-1. Also, SHA-1 is more resistant to brute force attacks than MD4 or MD5 simply because the hash result is longer. The US government owns the SHA-1 and DSA algorithms (digital signature authentication algorithms defined as part of DSS) and holds at least one related patent (US Pat. No. 5,231,688 issued in 1993).

RIPMD-160
RIPEMD-160 is a hash function derived from its advance, RIPEMD (developed for the European Commission's RIPE project in 1992). As can be seen from the name, RIPEMD-160 generates a 160-bit hash result. RIPMD-160, combined for a 32-bit architecture software implementation, is scheduled to provide a high level of security for over a decade. Although there are no successful attacks on RIPEMD-160, RIPEMD-160 is relatively new and has not been deciphered on a large scale. The first RIPEMD algorithm was specially designed to combat cryptographic attacks against conventional MD4. Recent attacks on MD5 have revealed similar weaknesses in the RIPEMD 128-bit hash function. Although this attack only showed a theoretical weakness, Dobertin, Preneel and Bosselars further enhanced RIPEMD to a new algorithm, RIPEMD-160.

Message Authentication Code Message authentication issues can be summarized as follows:
How does A believe that a message that is supposed to be from B is really a message from B?
Message authentication is different from entity authentication. In the case of entity authentication, one entity (requester) illuminates its identity to others (verifier). In the case of message authentication, it is related to guaranteeing that a predetermined message is a message from a person who thinks it, that is, ensuring that the message has not been tampered with from the source to the destination. One-way hash functions are not adequately protected against messages. A hash function such as MD5 is based on the fact that a hash value representative of the original input is generated and the original input cannot be derived from the hash value. A simple attack from E between A and B is to steal the message from B and replace it with your own message. Even if A sends the hash of the original message, E can easily replace it with the hash of his new message. When using only a one-way hash, A has no way of knowing whether B's message has changed. One solution to the problem of message authentication is a message authentication code, or MAC. When B sends message M, B sends MAC [M] so that the recipient knows that M is really a message from B. To achieve this, only B can generate M's MAC, and A needs to be able to verify M against MAC [M]. Note that this differs from M-MAC encryption being effective when M does not need to be kept secret. The simplest way to construct a MAC from a hash function is to encrypt the hash value using a symmetric algorithm, ie
Hash input message H [M]
Encrypt hash E _K [H [M]]
It is.

This is more secure than first encrypting the message and then hashing the encrypted message. Any symmetric or asymmetric encryption function can be used. However, instead of techniques that use encryption (as described above), it is advantageous to use a key-dependent one-way hash function in several ways:
Speed: Because one-way hash functions generally run faster than encryption;
Message size: E _K [H [M]] is at least as large as M, but H [M] is a fixed size (usually much smaller than M);
Hardware / software requirements: Keyed one-way hash functions are typically much less complex than their cryptographic-based alternatives;
The implementation of the one-way hash function is not considered to be an encryption device or a decryption device, and is not subject to US export restrictions.

It should be noted that the hash function was not originally designed to store keys or support message authentication. As a result, certain ad hoc methods have been proposed that use hash functions to perform message authentication, including various functions that concatenate messages with secret prefixes, suffixes, or both. Most of these ad hoc methods have been successfully attacked by sophisticated means. Additional MACs have been proposed based on the XOR scheme and the Toeplitz matrix (including the special case of LFSR-based structures).

HMAC
In particular, the HMAC structure is beginning to be recognized as a solution for Internet message authentication security protocols. The HAMC structure acts as a wrapper, using the underlying hash function as a black box. The replacement of the hash function is straightforward if required for security or performance reasons. However, the main advantage of the HMAC structure is that if the underlying hash function has some reasonable cryptographic strength, the HMAC structure can prove secure, ie the strength of the HMAC is the strength of the hash function. It is directly related to. Since the HMAC structure is a wrapper, any iterative hash function can be used in the HMAC. Examples thereof include HMAC-MD5, HMAC-SHA1, HMAC-RIPEMD160, and the like. Definitions such as:
H = Hash function (eg, MD5 or SHA-1)
Number of bits output from n = H (for example, 160 bits for SHA-1 and 128 bits for MD5)
M = data to which the MAC function is to be applied K = 2 The secret key shared by the parties ipad = 0x36 64 iterations Opad = 0x5C iterations 64 Given the HMAC algorithm:
Extend K to 64 bytes by appending 0x00 bytes to the end of K;
XORing the 64-byte character string created in (1) with ipad;
Append data stream M to the 64 bytes created in (2);
Apply H to the stream generated in (3);
XOR the 64-byte character string created in (1) with opad;
Append the result H from (4) to the 64 bytes resulting from (5);
Apply H to the output of (6) and output the result;
become that way. Therefore,
HMAC [M] = H [(KAopad) | H [(KAipad) | M]]
It is.

The recommended key length is at least n bits, but should not be longer than 64 bytes (the length of the hashed block). Keys longer than n bits do not add function security. The HMAC can optionally perform final output rounding, for example rounding from 160 bits to 128 bits. The HMAC Designer's Code (RFC) was published in 1997, one year after the algorithm was first published. Designers have announced that the strongest known attack on HMAC is based on the frequency of collisions on hash function H, and at the very least it is not practical for a suitable hash function. More recently, the HMAC protocol with anti-replay function has been defined to prevent capture and playback of M, HMAC [M] combinations within a predetermined time.

The use of a random number generator as a one-way function of random numbers and time-varying messages has already been described. However, the theory of random number generators is very intertwined with cryptography, security, and authentication. There are a number of problems with defining a good random number generator. Knuth describes the common problems associated with building generators (including statistical tests) and building generators. One use of random numbers is to ensure that messages change over time. Assume a system where A encrypts a command and sends it to B. If the encryption algorithm produces the same output as a given input, the attacker can easily record the message and play it back to deceive B. The attacker does not have to destroy the encryption mechanism, but only needs to know the message to be played to B (while impersonating A). As a result, messages often include random numbers and time stamps to ensure that the message (and hence its encrypted counterpart) always changes. Random number generators are often also used to generate keys. Thus, at present, all generators can be said to be uncertain for this purpose. For example, the Berlekamp-Massey algorithm is a conventional attack on LFSR random number generators. If the length of the LFSR is n, a 2n bit sequence is sufficient to determine the LFSRs that make up the key generator. However, if the only role of the random number generator is to ensure that the message changes over time, the security of the generator and seed is not as important as in the case of session key generation. However, some protocols may be left open to the attacker if the random seed generator is compromised and the attacker can calculate future “random numbers”. New protocols should be considered for this situation. The actual type of random number generator required will depend on the implementation and the intended use of the generator. The generator includes Ron
River, Blum, and Shub stream ciphers such as RC4 by Rivest, hash functions such as SHA-1 and RIPEMD-160, and LFSR (Linear Feedback Shift Register) and their more recent counterpart FCSR (Carry A conventional generator such as a feedback shift register) is included.

Attacks This section describes various types of attacks that can be performed to break an authentication cryptosystem such as an authentication chip. Attacks are broadly divided into physical attacks and logical attacks. A physical attack represents a way to destroy the physical implementation of the cryptographic system (eg, break and open the chip to retrieve the key), and a logical attack represents an attack on the cryptographic system that is independent of the implementation. Including. A logical type of attack works against a protocol or algorithm and tries to do one of three things:
Completely avoid the authentication process;
Obtain a private key by force or reasoning so that any question can be answered;
In order to give each question a correct answer without using a key, the authentication question and the nature of the answer are fully detected.

Next, the style and form of the attack will be described in detail. Regardless of the algorithm and protocol used by the security chip, the authenticator circuitry of the chip can be subject to physical attacks. Physical attacks appear in four main forms, but the form of attack can vary:
Avoid authentication chips completely;
Physical inspection of the chip during operation (destructive and non-destructive);
Physically disassembling the chip;
The chip is physically modified.

Next, the style and form of the attack will be described in detail. This section does not propose a solution for these attacks. This section only describes each attack type. The investigation is limited to the situation of the authentication chip 53 attached to a system (rather than some other system like Internet authentication).

Logical attacks These attacks are independent of the physical implementation of the cryptographic system. These affect the protocol, the security of the algorithm, and the random number generator.

Ciphertext only attack This is the case when an attacker has one or more encrypted messages that are all encrypted using the same algorithm. The attacker's goal is to obtain a plaintext message from the encrypted message. Ideally, the key can be recovered, so that all future messages can be recovered.

Known plaintext attack This is the case when the attacker has both plaintext and plaintext encryption form. In the case of an authentication chip, a known plaintext attack is when the attacker can see the flow between the system and the authentication chip. Inputs and outputs can be observed (without being screened by the attacker) and analyzed to find weaknesses (eg, by a birthday attack or by searching for distinguishable interesting input / output pairs). The known plaintext attack is a weaker type of attack than the selected plaintext attack. This is because the attacker can only observe the data flow. Known plaintext attacks can be performed by connecting a logic analyzer to the wiring between the system and the authentication chip.

Selected plaintext attack A selected plaintext attack is when a decryptor has the ability to send any selected message to the cryptographic system and observe the response. If the cryptanalyst knows the algorithm, there will be a relationship between the input and output available by supplying a specific output to the input of another function. In a system using a built-in authentication chip, it is difficult to prevent a selected plaintext attack. This is because a decryptor can logically pretend to be the system and send any selected bit pattern stream to the authentication chip.

Adaptive Selected Plaintext Attack This type of attack is similar to a selected plaintext attack, except that the attacker can change the subsequent selected plaintext based on the results of the previous test. This is certainly the case when the system / authentication chip scenario described above is used for consumables such as copiers and toner cartridges. This is especially because both systems and consumables are made available to everyone.

A guaranteed way to break a brute force attack key-based cryptosystem algorithm is to simply try all the keys. Eventually the correct key will be found. This is known as a brute force attack. However, as the number of key candidates increases, more keys need to be tried, and the time required (on average) to find the correct key increases. If there are N keys, a maximum of N trials are required. If the key length is N bits, a maximum of 2 ^N trials are required, and the chance of finding the key after only half the trials (2 ^N-1 ) is 50%. The longer N is, the longer it takes to find the key, and the more secure the key. Of course, there is a possibility that the attack will hit the key on the first attempt, but this possibility decreases as the key gets longer. Here, consider a 56-bit key length. In the worst case, it should be a total of effected a ^{2 56} times of the test (7.2 × ^{10 16} times of the test) to find the key. In 1977, Diffie and Hellman announced a DES decryption machine with 1 million processors, each processor capable of running 1 million tests per second. Such a machine takes 20 hours to break any DES code. Consider the case where the key length is 128 bits. In the worst case, a total of 2 ¹²⁸ tests (3.4 × 10 ³⁸ tests) must be performed to find the key. This takes 1 billion years on 1 trillion processors, each running 1 billion tests per second. If the key length is long enough, a Bruce Force attack takes a very long time and is not worth the effort of the attacker.

Guess Attack This type of attack is when an attacker simply tries to "guess" a key. The attack is the same as the Bruce Force attack, and the probability of success depends on the key length.

To decrypt a quantum computer attack n-bit key, a quantum computer (NMR, optical, or caged atom) with n qubits embedded in the appropriate algorithm must be constructed. A quantum computer effectively exists in 2 ⁿ simultaneous coherent states. The point is to extract the correct coherent state without causing decoherence. So far this has been achieved by a 2 qubit system (which exists in 4 coherent states). In a few years, it seems possible to extend this to 6 qubits (with 64 simultaneous coherent states).

Unfortunately, each time a qubit is added, the relative strength of the signal representing the key is halved. This quickly becomes a major obstacle to key retrieval, especially for long keys used in cryptographically secure systems. As a result, attacks on cryptographically secure keys (eg, 160 bits) using quantum computers are unlikely to be carried out, and quantum computers may be used during the commercial lifetime of the authentication chip. The possibility of achieving a qubit number greater than 50 qubits is very low. Even using a 50 qubit quantum computer, 2 ¹¹⁰ tests are required to decrypt a 160 bit key.

Intentional error attacks For some algorithms, an attacker can gather useful information from bad input results. The range of useful information ranges from the error message text to the time taken to generate the error. A simple example is a user ID / password scheme. If the error message is usually "wrong user ID", the attacker knows that the user ID is correct when he gets the message "wrong password". If the message is always "wrong user ID / password", much less information is given to the attacker. A more complex example is the recently announced method of decrypting cryptographic codes from secure websites. This attack involves sending specific messages to the server and observing error message responses. The response gives sufficient information to learn the key, and some information can be obtained even if there is no response. As an example of an algorithm, time can be known in the case of an algorithm that returns an error immediately when an erroneous bit is detected in the input message. Depending on the hardware implementation, it is a simple way for an attacker to time the response and change it bit by bit based on the time required for the error response, thus obtaining the key. . In the case of the chip mounting form, it is certain that the required time can be observed with a considerably higher accuracy than that via the Internet.

Birthday Attack This attack was named after the famous “Birthday Paradox” (not really a paradox at all). The probability that a person has the same birthday as another person is 1/365 (not considering leap years). Therefore, the probability that one of 183 people in the room has the same birthday as you should be higher than 50%. However, if there are 23 people in the room, the probability that any two of them have the same birthday exceeds 50%. The reason is that 253 pairs are generated by 23 people. Birthday attacks are common attacks on hashing algorithms, particularly algorithms that combine hashing and digital signatures. If the message was generated and already signed, the attacker would need to look for a collision message that hashes to the same value (similar to finding someone with the same birthday as you). However, if an attacker can generate a message, a birthday attack will start to work. The attacker looks for two messages with the same hash value (similar to the case where either two have the same birthday), only one message is accepted by the person who signed it, and the other Useful for attackers. If the person has signed the original message, the attacker only has to claim that the person has signed another message and mathematically indicates which message is the original. There is no way. This is because both messages hash to the same value. If a brute force attack is the only way to determine a match, the n-bit key weakening due to the birthday attack is 2 ^{n / 2} . A 128-bit long key that can be subject to a birthday attack is effectively only 64 bits long.

Chain attacks These attacks are attacks that are performed on the chain nature of the hash function. Chain attacks focus on the compression function of the hash function. The idea is based on the fact that a hash function generally takes an input of arbitrary length and produces an output of a certain length by processing the input n bits simultaneously. The output from one block is used as a chain variable set to the next block. Rather than finding a collision on the entire input, the idea is to find a substitute block that, when given an input chain variable set, yields the same output chain variable as the original message. The number of selections for a particular block is based on the length of the block. If the chain variable is c bits, the hashing function behaves like a random mapping, and the block length is b bits, the number of such b bit blocks is about 2b / 2c. The challenge to find a substitute block is that such a block is a sparse subset of all possible blocks. In the case of SHA-1, the number of 512-bit blocks is about 2 ^{512/2 160} , that is, 2 ³⁵² . Expected to find a block by brute force search is one of 2 ¹⁶⁰ minutes.

If the number of messages that may have been sent to the surrogate chip with a full lookup table is small, the replica manufacturer is not required to decrypt the key. Instead, the replica manufacturer can incorporate into the chip a ROM that records all of the response from the genuine chip to the code sent by the system. The key becomes longer, the response becomes longer, and the space required for such a lookup table becomes larger.

If a message sent to a surrogate chip with a sparse lookup table is not effectively random and is predictable to some extent, the replica manufacturer may not provide a complete lookup table. For example:
If the message is just a serial number, the replica manufacturer may provide a look-up table that stores past and future predicted serial numbers. It is unlikely that these serial numbers will exceed 10 ¹⁹ ;
If the test code is just a date, the replica manufacturer can generate a lookup table using the date as the address;
If the test code is a pseudo-random number using a serial number or date as a seed, the replica manufacturer just needs to decipher the pseudo-random number generator in the system. This is probably not difficult. This is because the replica manufacturer can obtain the object code of the system. The replica manufacturer will use these codes to access the stored authorization codes and generate content addressable memory (or other sparse array lookups).

Differential decryption Differential decryption represents an attack in which pairs of input streams are generated using known differences and the differences in the encoded stream are analyzed. Known differential attacks are highly dependent on the structure of the S-box as used in DES and other similar algorithms. Other algorithms like HMAC-SHA1 do not have an S-box, but the attacker
A differential attack can be performed by performing a statistical analysis of the minimum differential input and the corresponding minimum differential output and the corresponding input.

Most algorithms were augmented against differential decryption after the differential decryption process was described. This is symmetric in a section dedicated to each encryption scheme. However, some secretly developed recent algorithms have been broken. This is because developers did not consider certain styles of differential attacks and did not target their algorithms for public scrutiny.

Message replacement attack In certain protocols, an intermediary can replace some or all of a message. This is the case when a true authentication chip is plugged into a reusable replica chip in the consumable. The replica chip can steal all messages between the system and the authentication chip and perform multiple replacement attacks. Consider as an example a message that contains a header followed by the contents. An attacker cannot generate a valid header, but can replace the specific content of the message, especially if the valid response is something like "Yes, your message has been received." Even if the return message is "Yes, the following message has been received ...", the attacker may replace the original message before returning an acknowledgment to the original sender. it can. Message authentication codes have been developed to combat most message replacement attacks.

If a reverse-engineered pseudo-random number generator of the key generator is used to generate the key, the replica manufacturer may obtain the generator program or infer the random seed used. This is how the Netscape security program was first broken.

Avoidance of the entire authentication The authentication protocol can have problems that allow the entire authentication process to be avoided. For this type of attack, the key is completely irrelevant, and the attacker does not need to recover or infer the key. Consider as an example a system that authenticates at power-on but does not authenticate at other times. A reusable consumable including a copy authentication chip may use a genuine authentication chip. The duplicate authentication chip 53 uses the authentic chip for authentication call, and then simulates the status data of the subsequent authentic authentication chip. Another example of avoiding authentication is when the system authenticates only after the consumable is used. The duplicate authentication chip can perform simple authentication avoidance by simulating a connection failure after the consumable is used but before the authentication protocol is completed (or before it is started). A notorious attack called the “Kentucky Fried Chip” hack replaces the microcontroller chip for satellite television systems. When the subscriber stops paying the fee, the system will send an “unavailable” message. However, the new microcontroller only detects this message and does not pass it to the consumer's satellite television system.

Anyone who knows the robbery / bribery attack key may teach it to someone else. This secret exposure can be done for compulsory (bribery, robbery, etc.), retaliation (eg, dissatisfied employees), or simply because of beliefs. These attacks are usually cheaper and easier than other attempts to guess the keys. As an example, many people who have requested to be involved in the development of the Divx standard have recently (May / June 1998) wished to participate in the development of the Divx specification decryption device, that is, depending on their beliefs Claimed in a DVD newsgroup.

Physical attacks The following attacks are based on the assumption that an authentication mechanism is implemented on a silicon chip that can be physically accessed by an attacker. The first attack, ROM read, represents an attack when the key is stored in ROM, and the remaining attacks assume that the private key is stored in flash memory.

If the ROM read key is stored in ROM, the key can be read directly. A ROM can thus be used safely to hold a public key (for use in asymmetric cryptography), but cannot be used to hold a secret key. In symmetric cryptography, ROM is not at all relevant. Using copyrighted text (eg haiku) as a key is insufficient. This is because it is assumed that the key is duplicated in a country where intellectual property rights are not observed.

Chip Reverse Engineering In chip reverse engineering, an attacker disassembles the chip and analyzes the circuit. Once the circuit has been analyzed, the internal operation of the chip algorithm can be restored. Lucent Technology is an abbreviation of TOBIC (two-photon OBIC) for visualizing the circuit.
Beam Induced Current Acronym for light beam induced current. ) Developed an active method called. A process developed primarily for static RAM analysis removes all backing, polishes the back to a mirror finish, and focuses light on the surface. The excitation wavelength is specifically selected so as not to induce current in the IC. In the 19th century, Kerkhofs made the basic hypothesis on cryptanalysis that if the internal operation of the algorithm was just a secret of the scheme, the scheme would be as if it had been decrypted. Kerkhofs clarifies that confidentiality must be completely in the key. As a result, the best protection against reverse engineering of the chip is to make internal operations irrelevant.

Unauthorized use of the certification process Every replica manufacturer must assume that both the system and consumable designs are available. If the same channel is used for communication between the system and the trusted system authentication chip, as well as between the unreliable consumables authentication chip, the untrusted chip is trusted to get a “correct answer” There is a possibility to inquire about a possible chip. If so, the replica manufacturer will not need to determine the key. They only need to cheat the system using the response from the system authentication chip. Another method of improperly using the authentication process is to simulate a connection failure with the system and to simulate a power shutdown or the like when the authentication process is performed according to the same method as the logical attack “avoidance of the authentication process”.

System changes This type of attack is an attack in which the system itself is modified to accept replication consumables. Attacks are system ROM changes, consumable rewiring, or, in extreme cases, a complete replication system. This type of attack requires that each individual system be changed and in most cases will require the consent of the owner. Usually, a clear benefit is needed for consumers to make such changes. This is because such changes typically invalidate the warranty and in most cases are expensive. An example of such a change that would be of obvious benefit to the consumer is a software patch that changes the fixed region DVD player to a region free DVD player.

If the direct view of chip movement by conventional probing can be directly monitored using STM or electron beam, the key can be recorded as it is read from the internal non-volatile memory and loaded into the working register. Three conventional forms of probing require direct access to the top or front surface of the IC while the IC is energized.

If the non-volatile memory direct view chip is sliced to expose the flash memory floating gate without discharging, the key can probably be monitored directly using STM or SKM (scanning Kelvin microscope). . However, slicing the chip at this level without discharging the gate is probably not feasible. Wet etching, plasma etching, ion milling (chemical ion beam etching), or chemical mechanical polishing almost certainly releases the small charges present on the floating gate.

A small amount of infrared energy is emitted when the state of the monitoring gate of a light burst caused by a state change changes. Since silicon is transparent to infrared, these changes can be observed by looking at the circuit from the underside of the chip. The emission process is weak but shines to the extent that it can be detected using an ultra-sensitive instrument developed for use in astronomy. This technology developed by IBM is called PICA (picosecond imaging circuit analyzer). If the state of the register is known at time t, monitoring the register switch time reveals the exact value at time t + n, and if the data is part of the key, that part is decrypted.

When the EMI monitoring electronics are operating, a weak electromagnetic signal is always emitted. A relatively low cost (thousands of dollars) device can monitor these signals. This gives enough information for an attacker to guess the key.

Even if the monitoring key for the I _dd variation cannot be monitored, a current variation always occurs when the register changes state. If the signal-to-noise ratio is high enough, the attacker can monitor the I _dd difference that occurs when programming the high or low bit. A change in I _dd may indicate information about the key. Such attacks are already used to crack smart cards.

Differential Fault Analysis This attack assumes that bit errors are captured by ionization, microwave radiation, or environmental stress. In most cases, such an error is not a useful change that would reveal the key, but is likely to adversely affect the chip (eg, destroy program code). Target failures such as ROM overwriting, gate destruction, etc. are even less likely to produce useful results.

Clock malfunction attack chips are typically designed to operate properly within a certain clock speed range. Some attackers attempt to cause the logic to fail by operating the chip at a very high clock speed, or to cause a clock malfunction at a specific time at specific intervals. The idea is to create a turbulent state where the circuit does not function properly. As an example, an AND gate always passes through input ₁ rather than a logical AND of input ₁ and input ₂ (due to a turbulent state). If the attacker knows the internal structure of the chip, the attacker can take into account a turbulent state at the exact point in time during the algorithm execution, thereby revealing information about the key (or in the worst case) , Reveal the key itself).

Instead of causing a malfunction in the power attack clock signal, the attacker can cause the power supply to malfunction, and the power is increased or decreased outside the operating voltage range. The net effect is the same as in the case of a clock malfunction, causing an error in the execution of a specific instruction. The idea is to stop the CPU from XORing the key or to stop shifting data by one bit position. Specific instructions are targeted to reveal information about the key.

A single bit of the ROM overwriting ROM can be rewritten to 1 or 0 depending on the logic orientation using a laser cutter microscope. Given a set of opcodes / operands, it is easy for an attacker to rewrite a conditional jump to an unconditional jump, or perhaps to change the destination of a register transfer. If the target command is carefully selected, the key will be revealed.

EEPROM / Flash Modifications EEPROM / flash attacks are similar to ROM attacks, except that laser cutter microscopy techniques can be used for both individual bit sets and resets. This greatly expands the scope of algorithm changes.

Gate Destruction Anderson and Kuhn explain gate destruction in a ramp session of a 1997 workshop on high-speed software encryption that Biham and Shamir announced an attack on DES. This attack used a laser cutter to break individual gates in a hardware implementation of a conventional block cipher (DES). The net effect of this attack is to force certain bits in the register to be “valueless”. Biham and Shamir describe the effect of forcing a particular register to have such an effect—the least significant bit of the output from the rounding function is set to zero. By comparing the least significant 6 bits of the left half and the right half, several bits of the key can be recovered. When a large number of chips are damaged in this way, sufficient information about the key can be revealed to facilitate complete key recovery. An encryption chip modified in this way will have the property that encryption and decryption are no longer reversed.

Instead of reading the overwrite attack flash memory, the attacker simply sets a single bit using a laser cutter microscope. Even if the attacker doesn't know the previous value, the new value is known. If the chip continues to operate, the original state of that bit should be the same as the new state. If the chip ceases to operate, the original state of the bit is a logical NOT NOT of the current state. An attacker can perform this attack on each bit of the key and use up to n chips to obtain an n-bit key (if the new bit matches the old bit, the new bit Not needed to determine the bits).

Test Circuit Attack Most circuits have a test circuit that is specifically designed to inspect manufacturing defects. This includes BIST (built in self test) and scan path. Very often, scan paths and test circuitry include access and readout mechanisms for all the built-in latches. In some cases, the test circuit may be used to provide information regarding the contents of a particular register. Test circuits are often disabled by cutting certain wires in the chip in some cases once the chip passes all manufacturing tests. However, a motivated attacker can reconnect the test circuit to make it operational.

The residual memory value remains in the RAM for a long time after power is removed, but does not remain for a long time until it is considered non-volatile. The attacker can remove the power supply and read the value from the RAM after the confidential information is transferred to the RAM (eg, working register). This attack is most effective for a security system having a normal RAM chip. As a typical example, the security system was designed with an automatic power shutdown function that is triggered when the computer case is opened. The attacker simply opened the case, removed the RAM chip, and used the remaining memory to retrieve the key.

If there are multiple stages in the lifetime of a chip theft attack authentication chip, each of these stages needs to be examined in terms of security derivation issues if the chip is stolen. For example, if the information is programmed into the chip at multiple stages, the attacker may obtain key information or reduce the effort for the attack if the chip is stolen between stages. Similarly, if the chip is just manufactured and stolen before programming, will there be a logical or physical benefit to the attacker?
Conventional solutions to the problem of authenticating consumables typically rely on patents for physical packaging. However, this does not stop replicating in countries where household refilling and industrial property rights are weak. As a result, a higher level of protection is required. Thus, the authentication mechanism is built in an authentication chip 53 that allows the system to authenticate consumables safely and easily. Here, if limited to only systems that authenticate consumables (not considering consumables that authenticate the system), two levels of protection can be considered:
Presence-only authentication In this case, only the presence of an authentication chip is tested. The authentication chip can be reused with another consumable without reprogramming;
Consumable lifetime authentication In this case, not only is the presence of the authentication chip tested, but the authentication chip 53 must last only for the lifetime of the consumable. If the chip is reused, the chip must be completely erased and reprogrammed. These two protection levels handle different requirements. Here, we are primarily interested in consumable lifetime authentication to prevent duplicates of mass-produced consumables. In this case, each chip must maintain secure state information regarding the consumable item to be authenticated. Note that the consumable lifetime authentication chip can be used in any situation that requires a presence-only authentication chip. Authentication requirements, data storage integrity requirements, and manufacturing requirements should be considered separately. The following sections briefly explain each requirement.

Authentication requirements for both authentication presence-only authentication and consumable lifetime authentication are limited to cases where the system authenticates consumables. For presence-only authentication, it must be ensured that the authentication chip is physically present. In the case of consumable lifetime authentication, it is necessary to ensure that the status data is actually derived from the authentication chip and that the status data has not been tampered with. These problems cannot be separated, and the falsified data is from a new transmission source. If the transmission source cannot be determined, the falsification problem cannot be solved. It is not enough to provide authentication methods based on home-grown security laws that are confidential and not scrutinized by security experts. Thus, a key requirement is to provide authentication by means that have endured expert scrutiny. The authentication scheme used by the authentication chip 53 must be able to resist breaking by logical means. The logical type of attack is expensive and has three things:
Completely avoid the authentication process;
Obtain a private key by force or reasoning so that any question can be answered;
Fully detect the nature of the authentication question and the answer to give each question the correct answer without using a key;
Try to do one of them.

Although the data storage integrity authentication protocol counters the guarantee of data integrity in communicated messages, data storage integrity is also required. The following two types of data:
Authentication data such as a private key;
Consumables status data such as serial number and media remaining amount;
Must be stored in the authentication chip.

The access requirements for these two data types are quite different. Accordingly, the authentication chip 53 requires a storage / access control mechanism that takes into account each type of integrity requirement.

Authentication data Authentication data must be kept confidential. The authentication data needs to be stored on the chip during the manufacturing / programming stage of the chip's lifetime, but should not be allowed to be retrieved from the chip thereafter. Authentication data must be able to resist being read from non-volatile memory. The authentication scheme serves to ensure that the key cannot be obtained by inference, and the manufacturing process serves to ensure that the key cannot be obtained by physical means. The size of the authentication data storage area must be large enough to hold the necessary keys and confidential information as required by the authentication protocol.

Consumable Status Data Each authentication chip 53 can store 256 bits (32 bytes) of the consumable status data. The consumable status data is classified into the following types. Depending on the application, the number of each of the following types of data items is different: Assume that the maximum number for a single data item is 32 bits:
Read only;
Read / write;
Decrement only.

Read-only data must be able to be stored on the chip during the manufacturing / programming stage of the chip's lifetime, but no further changes should be allowed. Examples of read-only data are consumable batch numbers and serial numbers.

The read / write data is state information that can be changed, and is, for example, the last time when a specific consumable was used. Read / write data items can be read and written an unlimited number of times during the life of the consumables. They can be used to store any state information about the consumables. The only requirement for this data is that it should be kept in non-volatile memory. Since an attacker can gain access to the system (write to read / write data is possible), the attacker may change this type of data field. This data type should not be used for confidential information and should be considered uncertain.

Decrement-only data is used to count down the availability of consumable resources. For example, the toner cartridge of the copier stores the remaining amount of toner as a decrement-only data item. Ink cartridges for color printers store the amount of each ink color as a decrement-only data item and require three (one for each of cyan, magenta, and yellow), or five to six decrement-only data items And The requirement for this type of data item is that once programmed at the manufacturing / programming stage with initial values, it is only possible to reduce the values. It cannot decrease any further once the minimum value is reached. Decrement-only data items are required only for consumable lifetime authentication.

The manufacturing authentication chip 53 should ideally be low manufacturing cost so that it can be incorporated as an authentication mechanism for low cost consumables. The authentication chip 53 should use a standard manufacturing process such as flash. this is:
Allow a very wide selection of manufacturing locations;
Use clear and working technology;
Reducing costs;
is required.

Regardless of the authentication scheme used, the chip's authenticator circuit must be able to resist physical attacks. Physical attacks appear in four main forms, but the form of attack can vary:
Avoid authentication chips completely;
Physical inspection of the chip during operation (destructive and non-destructive);
Physically disassembling the chip;
The chip is physically modified.

Ideally, the authentication chip 53 should not be usable as a secure encryption device because the chip should be taken out of the United States. This is a low priority requirement. This is because there are many companies that can manufacture authentication chips in other countries. In any case, export restrictions from the United States can change.

Conventional solutions to the problem of authenticating certified consumables typically rely on patents for physical packaging. However, this does not stop replicating in countries where household refilling and industrial property rights are weak. Therefore, a higher level of protection is required. It is not enough to provide authentication methods based on home-grown security laws that are confidential and not scrutinized by security experts. Netscape's first independently developed system and the GSM fraud prevention network used privately by mobile phones are security vulnerabilities created by design secrecy. Both security systems were broken by conventional means that would have been able to be detected if those companies were following an open design process. The solution is to provide authentication by means of withstanding expert scrutiny. A number of protocols can be used for consumable authentication. Here we use only the published security method and use the actions known by this new method. In all protocols, the security of the scheme relies on a secret key, not a secret algorithm. All of the protocols depend on time-varying challenges (i.e. different challenges each time) and the response depends on the challenge and the secret information. Because the challenge includes a random number, the observer cannot collect useful information about subsequent identification. Two protocols are proposed for each of presence-only authentication and consumable lifetime authentication. The protocol differs in the number of authentication chips required for the authentication process, but in all cases the system authenticates the consumable. Some protocols operate on one or two chips, while other protocols operate on only two chips. Regardless of whether one or two authentication chips are used, the system is responsible for making an authentication decision.

Single Chip Authentication When a single authentication chip 53 is used for the authentication protocol, the single chip (referred to as chip A) is responsible for telling the system (referred to as system) that chip A is authentic. is there. At the start of the protocol, the system cannot be sure of the authenticity of chip A. The system performs a challenge-response protocol with chip A to determine the authenticity of the chip. In all protocols, the authenticity of the consumable is directly based on the authenticity of the chip. That is, if the chip A is considered authentic, the consumable is also considered authentic. The data flow is shown in FIG. In the case of a single chip authentication protocol, the system can be software, hardware, or a combination of both. Importantly, the system is not secure and an attacker can easily reverse engineer by examining the ROM or by examining the circuit. The system is not specifically designed to be secure by itself.

In the protocol different from the double chip authentication, two authentication chips are required as shown in FIG. A single chip (chip A) is responsible for telling the system (called system) that chip A is authentic. As part of the authentication process, the system utilizes a trusted authentication chip (referred to as chip T). In a double chip authentication protocol, the system can be software, hardware, or a combination of both. However, the chip T must be a physical authentication chip. In some protocols, chip T and chip A have the same internal structure, and in other protocols, chip T and chip A have different internal structures.

Presence-only authentication (uncertain status data)
In this level of consumables authentication, we are only interested in confirming the presence of the authentication chip 53. Although the authentication chip can store state information, the transmission of the state information is not considered secure. Two protocols are proposed. Protocol 1 requires two authentication chips, and protocol 2 can be implemented using one or two authentication chips.

Protocol 1
Protocol 1 is a double chip protocol (two authentication chips are required). Each authentication chip stores the following values:
Key for K F _K [X]. Must be secret;
R Current random number. It does not have to be secret, but it must be seeded with a different initial value for each chip instance. It changes every time the Random function is called.

Each authentication chip stores the following logical functions:
Random [] Returns R and advances R in the sequence;
F [X] Returns F _K [X], which is the result of applying the one-way function F to X based on the secret key K.

The protocol is as follows:
The system requests Random [] from chip T;
Chip T returns R to the system;
The system requires F [R] from both chip T and chip A;
Chip T returns F _KT [R] to the system;
Chip A returns F _KA [R] to the system;
The system compares F _KT [R] with F _KA [R]. If they match, chip A is considered valid. If they do not match, chip A is considered invalid.

The data flow is shown in FIG. The system does not need to understand the F _K [R] message. The system need only check that the response from chip A and the response from chip T are the same. Thus, the system does not require a key. Protocol 1 is secure in two places:
Safety of F [X]. Since only the authentication chip stores the private key,
There must be an authentic chip that can generate F [X] from X that matches F [X] generated by the trusted authentication chip 53 (chip T);
The domain of R generated by all authentication chips should be large and non-deterministic. If the R domain generated by all the authentication chips is small, the replica manufacturer need not decrypt the key. Instead, the replica manufacturer can incorporate in his chip a ROM in which all the responses from the genuine chip to the code sent by the system are recorded. The Random function may not be strictly stored in the authentication chip. This is because the system has the potential to generate the same random number sequence. However, it simplifies the design of the system and the security of the random number generator is the same for all implementations that use the authentication chip, reducing the possibility of errors in the system implementation.

Protocol 1 has several advantages:
K is not revealed during the authentication process;
Given X, the replica chip cannot generate F _K [X] without using K or accessing the actual authentication chip;
The system can be easily designed, especially in low cost systems such as inkjet printers. Because encryption or decryption is not required by the system itself;
There is a wide range of keyed one-way functions, including symmetric encryption, random number sequences, and message authentication codes;
One-way functions require fewer gates and are easier to verify than asymmetric algorithms;
The secure key size for a keyed one-way function does not need to be as large as for an asymmetric (public key) algorithm
If F [X] is a symmetric encryption function, a minimum of 128 bits can provide adequate security.

But there are problems with this protocol:
This is subject to selected text attacks. The attacker can plug the chip into his own system, generate the selected R, and observe the output. To find the key, the attacker can also search for R that generates a particular F [M]. Because many authentication chips can be tested in parallel;
Depending on the selected one-way function, key generation can be complicated. The method of choosing a good key depends on the algorithm used. Certain keys are weak against certain algorithms;
The choice of the keyed one-way function itself is not important. In some cases, a license derived from patent protection is required.

The intermediary can act on the plaintext message M before passing the plaintext message M to chip A, which is preferred if the intermediary does not see M until chip A sees M. It is more preferable when the mediator does not see M at all.

If F is symmetric encryption, the chip cannot be taken out of the United States due to the key size required for proper security. This is because the chip can be used as a strong encryption device.

If the protocol is implemented using F as an asymmetric cryptographic algorithm, there is no advantage over the symmetric case, the key needs to be longer and the encryption algorithm in silicon is more expensive. Protocol 1 must be implemented using two authentication chips to maintain key security. That is, each system requires an authentication chip, and each consumable needs an authentication chip.

Protocol 2
In some cases, the system has a large processing capacity. Or, for example, in the case of a system manufactured in large quantities, it is desirable to integrate the chip T into the system. The use of an asymmetric cryptographic algorithm allows the chip T portion of the system to be uncertain. Therefore, protocol 2 uses an asymmetric cryptosystem. For this protocol, each chip stores the following values:
It is a key for K E _K [X] and D _K [X]. Should be kept secret in chip A. There is no need to keep it secret in chip T;
R Current random number. It does not have to be secret, but it must be seeded with a different initial value for each chip instance. It changes every time the Random function is called.

The following functions are defined:
E [X] Only chip T. Returns E _K [X]. Where E is the asymmetric encryption function E;
D [X] Only chip A. Returns D _K [X]. Where D is an asymmetric encryption function D;
Random [] Only chip T. R | E _K [R] is returned. R is a random number based on the seed S. Advance R in the random sequence.

Public key _{K T} is in the chip T, on the other hand, there is the secret key _{K A} chip A. The K _T is in the tip T has an advantage of realizing a tip T in software or hardware (provided that seed R is different for each chip or system). Therefore, protocol 2 can be realized as a single chip protocol or a double chip protocol. The protocol for authentication is as follows:
The system calls the Random function of chip T;
Chip T returns R | E _KT [R] to the system;
The system calls function A on chip A and passes E _KT [R];
Chip A returns R acquired by D _KA [E _KT [R]];
The system compares R from chip A with the original R generated by chip T. If they match, chip A is considered valid. If they do not match, chip A is considered invalid.

The data flow is shown in FIG. Protocol 2 has the following advantages:
K _A (private key) is not revealed during the authentication process;
When E _{KT [X]} is given, replication chip without using a K _A, or without access to the actual chip A, can not generate X;
Since K _T ≠ K _A , the chip T can be realized entirely in software, with uncertain hardware, or as part of a system. Only chip A (in the consumable) is required to be a secure authentication chip;
If chip T is a physical chip, the system can be easily designed;
From among a large number of well documented and decrypted asymmetric algorithms, one can select algorithms for implementation, including solutions that are patent free and license free.

However, protocol 2 has a number of inherent problems:
For sufficient security, each key needs to be 2048 bits (in contrast, for protocol 1 symmetric cryptography, the minimum is 128 bits). The associated intermediate memory used by the encryption and decryption algorithms will be correspondingly larger;
Key generation is important. Random numbers are not good keys;
When chip T is implemented as a core, it is difficult to couple chip T to a given system ASIC;
When chip T is implemented as software, not only is the system implementation vulnerable to programming errors and less rigorous testing, but the integrity of the compiler and mathematical primitives is strictly checked for each system implementation. Must. This is more complex and costly than simply using a well-tested chip;
Numerous symmetric algorithm Despite the enhanced especially differential cryptanalysis (selected text attack by being based) To order counter, easily subject to the secret key K _A is selected text attack;
If chip A and chip T are instances of the same authentication chip, each chip needs to have both asymmetric encryption and decryption functions. As a result, each chip is larger, more complex and more expensive than the chip required for protocol 1;
If the authentication chip is split into two chips to save cost and reduce design / test complexity, the two chips still need to be manufactured, reducing the scale merit. This is offset by the relative number of consumables in the system, but still needs to be considered;
Protocol 2 authentication chips cannot be taken out of the United States. Because they are considered strong encryption devices.

Even if the process of selecting a key for protocol 2 is simple, protocol 2 is not practical at this time due to the high cost of silicon implementation (both key size and functional implementation). Therefore, protocol 1 is selected as the protocol for presence-only authentication.

Replication consumables protocols 1 and 2 that use a real authentication chip only check whether chip A is a real authentication chip. They do not check to see if the consumables themselves are valid. As a basic premise for authentication, if the chip A is valid, the consumable is valid. Therefore, the replica manufacturer can insert a genuine authentication chip into the replica consumable. Consider the following two cases:
In the case where status data is not written to the authentication chip, the chip is completely reusable. Accordingly, the replica manufacturer can recycle a valid consumable into a replica consumable. This is made difficult by fusing the authentication chip into the physical packaging of the consumable, but will not stop the refiller;
In the case where status data is written to the authentication chip, the chip is new, partially used, or completely used up. However, this does not stop the use of the piggyback attack by the replica manufacturer, and the replica manufacturer constructs a chip with a real authentication chip as the piggyback. The attacker's chip (chip E) is an intermediary. When the power is turned on, the chip E reads all storage state values from the genuine authentication chip 53 to the specific memory. Chip E then examines the request from the system and takes various actions depending on the request. The authentication request can be passed directly to the real authentication chip 53, while the read / write request can be simulated by a memory similar to the operation of the real authentication chip. In this way, the authentication chip 53 always appears as if it is new when the power is turned on. Chip E can do this because data access is not authenticated.

In order to deceive the system and make the data access of the system appear to be successful, chip E still needs a real authentication chip, and in the second case, a duplicate chip is needed in addition to the real chip. As a result, protocols 1 and 2 are effective when the replica manufacturer is not cost-effective to embed a genuine authentication chip 53 in the consumable. If the consumables cannot be recycled or simply refilled, the use of protocol 1 or protocol 2 provides sufficient protection. In order for the replication operation to be successful, each replication consumable must have a valid authentication chip. The chips must be stolen together or removed from old consumables. The amount of these playback chips (along with the effort to regenerate them) is insufficient to make it a business foundation, and there is no practical benefit of adding protection for secure data transfers (see protocols 3 and 4).

Key Life The general problem with the above two protocols is that once an authentication key is selected, it is not easy to change it. In some situations, it does not matter that the key is compromised, but in other cases it is fatal. For example, in a car / car key system / consumable scenario, the consumer has only one set of car / car key. Each car has a different authentication key. As a result, the loss of the car key only jeopardizes the individual car. If the owner considers this a problem, the owner must obtain a new “car key” by replacing the system chip in the car electronics. The owner's key needs to be reprogrammed / exchanged to work with the new “car system authentication chip”. On the other hand, compromising the key to the mass consumables market (e.g., printer ink cartridges) allows duplicate ink cartridge manufacturers to manufacture their own authentication chips. The only solution for existing systems is to update the system authentication chip, which is a costly and logistical challenge. In any case, the consumer system is already functioning normally and there is no motivation to interfere with existing equipment.

Consumable Lifetime Authentication At this level of consumables authentication, we are interested in confirming the presence of the authentication chip and ensuring that the authentication chip lasts for the same period as the consumable. In addition to confirming that the authentication chip exists, writing and reading of the authentication chip's memory space must be authenticated as well. In this section, we assume that the data storage integrity of the authentication chip is secure, some of the memory is read-only, some is read / write, and others are decrement-only (more details See the chapter titled Data Storage Integrity). There are two protocols. Protocol 3 requires two authentication chips and protocol 4 can be implemented using one or two authentication chips.

Protocol 3
This protocol is a double chip protocol (requires two authentication chips). For this protocol, each authentication chip stores the following values:
Key for calculating K ₁ F _K1 [X]. Must be kept secret;
Key for calculating K ₂ F _K2 [X]. Must be kept secret;
R Current random number. It does not have to be secret, but it must be seeded with a different initial value for each chip instance. Changes for each successful authentication as defined by the Test function;
M Memory vector of authentication chip 53. A part of this space should be different for each chip (it may not be a random number).

Each authentication chip stores the following logical functions:
F [X] For internal functions only. Return F _K [X], which is the result of applying the one-way function F to X based on either key K ₁ or key K ₂ ;
Random [] R | F _K1 [R] is returned;
Test [X, Y] F _K2 [R | X] = Y, return 1 and advance R. Otherwise 0 is returned. The time taken to return 0 must be the same for all bad inputs;
Read [X. If Y] F _K1 [X] = Y, return M | F _K2 [X | M]. Otherwise 0 is returned. The time taken to return 0 must be the same for all bad inputs;
Write [X] Writes X to a properly rewritable part of M.

To authenticate chip A and read the memory M of chip A,
The system calls the Random function of chip T,
Chip T generates R | F _K [R] and returns it to the system,
The system calls the Read function of chip A and passes R, F _K [R]
Chip A returns M and F _K [R | M]
The system calls the test function of chip T and passes M and F _K [R | M]
The system checks the response from chip T. If the response is 1, chip A is considered authentic. If it is zero, chip A is considered invalid.

To authenticate the writing of _Mnew to the memory M of chip A,
The system calls the write function of chip A, passes M _new ,
The authentication procedure for reading is executed,
If chip A is authentic and M _new = M, the write is successful. Otherwise, the write failed.

The data flow for read authentication is shown in FIG. The first thing to note about protocol 3 is that F _K [X] cannot be called directly. Instead, F _K [X] is called indirectly by Random, Test and Read:
Random [] calls F _K1 [X]. X is not selected by the caller. X is selected by the Random function. An attacker must perform a brute force search using multiple calls of Random, Read, and Test to obtain the desired X, F _K1 [X] pair;
Test [X, Y] calls F _K2 [X]. Rather than return the result as is, compare the result with Y and return 1 or 0. Attempts to infer K ₂ by trying different values of F _K2 [R | X] for a given X and invoking Test multiple times reduce the brute force search that R cannot choose by an attacker Done;
Read [X, Y] calls F _K1 [X]. Since X and F _K1 [X] must be supplied by the caller, the caller must know the X, F _K1 [X] pair in advance;
This call returns 0 if Y ≠ F _K1 [X], so the caller can use the Read function for a brute force attack on K ₁ ;
Read [X, Y] calls F _K2 [X | M]. Although X is supplied by the caller, X is can take only the values given previously by the Random function (because, X and Y are verified by K _1). For this reason, a selected text attack must first collect pairs from Random (in effect, a brute force attack). Moreover, only a part of M is used in the selected text attack. This is because part of M is constant (read only), and the decrement only part of M can be used once for each consumable. In the next consumable, the read-only part of M is different.

Invoking F _K [X] indirectly prevents selected text attacks on the authentication chip. A brute force attack on K ₁ is to K _{2 because the} attacker can only acquire the R, F _K1 [R] pair selected by calling Random, Read, and Test multiple times until the desired R appears. Restricted selection is necessary to perform a text attack. Attempts at selecting text attack on the K ₂ will be limited. This is because the text cannot be completely selected, part of M is read-only and still different for each authentication chip. The second thing to note is that two keys are used. Given a small size M, the two keys K ₁ and K ₂ are used to ensure that there is no correlation between F [R] and F [R | M]. Therefore, K ₁ is used to protect K ₂ from differential attacks. It is insufficient to use one longer key. This is because M is only 256 bits and only a portion of M changes during the lifetime of the consumable. Otherwise, the attacker determines the effect of a limited M change on a particular R bit combination by some undiscovered technique, and based on F _K1 [X] F _K2 [ X | M] may be calculated. As an additional pre-computation, chip A's Random and Test functions should be invalidated, so that the attacker creates an instance of chip T to generate a pair of R, F _K [R] Each of which is more expensive than chip A (because the system must be acquired for each chip T). Similarly, the delay between Random, Read, and Test calls should be minimized, so that an attacker cannot call these functions at high speed. In this way, each chip reveals only a certain number of X, F _K [X] pairs at a given time. The only unique timing requirement for Protocol 3 is that a return value of 0 (indicating a bad input) should be generated at the same time, regardless of where the error is in the input. . Therefore, the attacker cannot learn what the error regarding the input value is. This is true for both the RD function and the TST function.

Another thing to note about protocol 3 is that reading of data from chip A also requires authentication of chip A. When F _K2 [R | M] is correctly returned, the system can confirm that the contents of the memory (M) are the contents claimed by the chip A itself. The duplicate chip may pretend that M is a certain value (eg, it appears to be full of consumables), but the duplicate chip may be F _K2 [for any R passed by the system. R | M] cannot be returned. Thus, the de facto signature, F _K2 [R | M], assures the system not only that the authentication chip A has sent M, but that M has not been tampered with between the chip A and the system. . Finally, the above Write function does not authenticate the write. To authenticate the write, the system must perform a read after each write. Protocol 3 has several basic advantages:
K ₁ and K ₂ are not revealed during the authentication process;
Given X, the replica chip cannot generate F _K2 [X | M] without using K or without accessing real chip A;
Since the system itself does not require encryption or decryption, the system can be easily designed, especially in low cost systems such as inkjet printers;
There is a wide range of key-based one-way functions including symmetric cryptography, random number sequences, and message authentication codes;
Keyed one-way functions require fewer gates and are easier to verify than asymmetric algorithms. );
The secure key size for a keyed one-way function need not be as large as in the case of an asymmetric (public key) algorithm. If F [X] is a symmetric encryption function, a minimum of 128 bits can provide adequate security.

Thus, for protocol 3, the only way to authenticate chip A is to read the contents of chip A's memory. The security of this protocol depends on the underlying F _K [X] scheme and the R domain across all system sets. F _K [X] may be a keyed one-way function, but there is no advantage in realizing it as an asymmetric cipher. The key needs to be longer and the encryption algorithm becomes more expensive in silicon. This results in a second protocol for use with the asymmetric algorithm, namely protocol 4. Protocol 3 needs to be implemented with two authentication chips to keep the key secret. That is, each system requires an authentication chip, and each consumable requires an authentication chip.

Protocol 4
In some cases, the system has a large processing capacity. Or, for example, in the case of a system manufactured in large quantities, it is desirable to integrate the chip T into the system. The use of an asymmetric cryptographic algorithm allows the chip T portion of the system to be uncertain. Therefore, protocol 4 uses an asymmetric cryptosystem. For this protocol, each chip stores the following values:
It is a key for K E _K [X] and D _K [X]. Should be kept secret in chip A. There is no need to keep it secret in chip T;
R Current random number. It does not have to be secret, but it must be seeded with a different initial value for each chip instance. Changes for each successful authentication as defined by the Test function;
M Memory vector of authentication chip 53. A part of this space should be different for each chip (it may not be a random number).

There is no point in verifying something with the Read function. This is because anyone can encrypt using a public key. Thus, the following function is defined:
E [X] Internal function only. Returns E _K [X]. Where E is the asymmetric encryption function E;
D [X] Internal function only. Returns D _K [X]. Where D is an asymmetric encryption function D;
Random [] Only chip T. Return E _K [R];
Test [X, Y] D _K [R | X] = Y, return 1 and advance R. Otherwise 0 is returned. The time taken to return 0 must be the same for all bad inputs;
Read [X] M | E _K [R | M] is returned. Where R = D _K [X] (do not test input);
Write [X] Writes X to a properly rewritable part of M.

Public key _{K T} is in the chip T, on the other hand, there is the secret key _{K A} chip A. The fact that _KT is on chip T has the advantage that chip T can be implemented in software or hardware (where R is seeded with a random number that varies from system to system). To authenticate chip A and read the memory M of chip A,
The system calls the Random function of chip T,
Chip T generates E _KT [R] and returns it to the system,
The system calls the Read function of chip A, passes E _KT [R],
Chip A returns M | E _KA [R | M] and first obtains R by D _KA [E _KT [R]]
The system calls the test function of chip T and passes M and E _KA [R | M]
Chip T calculates D _KT [E _KA [R | M]], compares it with R | M,
The system checks the response from chip T. If the response is 1, chip A is considered authentic. If it is zero, chip A is considered invalid.

To authenticate the writing of _Mnew to the memory M of chip A,
The system calls the write function of chip A, passes M _new ,
The authentication procedure for reading is executed,
If chip A is authentic and M _new = M, the write is successful. Otherwise, the write failed.

The data flow of read authentication is shown in FIG. Only valid chip A knows the value of R. Because R is not passed to the authentication function (it is passed as an encrypted value). R must be obtained by decoding the E [R], this is done only by using the secret key K _A. Once R is acquired, R is added to M and the result is re-encoded. Chip T can verify the decrypted form E _KA [R | M] = R | M, so that chip A is valid. Since K _T ≠ K _A , E _KT [R] ≠ E _KA [R]. Protocol 4 has the following advantages:
K _A (private key) is not revealed during the authentication process;
When E _{KT [X]} is given, replication chip without using a K _A, or without access to the actual chip A, can not generate X;
Since K _T ≠ K _A , the chip T can be realized entirely in software, with uncertain hardware, or as part of a system. Only chip A is required to be a secure authentication chip;
Since the chip T and the chip A store different keys, it does not reveal anything about K _A by vigorous testing of the chip T;
If chip T is a physical chip, the system can be easily designed;
From among a large number of well documented and decrypted asymmetric algorithms, one can select algorithms for implementation, including solutions that are patent free and license free;
Even if the system can be rewired so that the demand for chip A is directed to chip T, chip T cannot answer chip A because K _T ≠ K _A Attacks should be directed to the system ROM itself to circumvent the authentication protocol.

However, protocol 4 has a number of problems:
All authentication chips need to accommodate both asymmetric encryption and decryption functions. As a result, each chip is larger, more complex and more expensive than the chip required for protocol 3;
For sufficient security, each key needs to be 2048 bits (in contrast, for protocol 1 symmetric cryptography, the minimum is 128 bits). The associated intermediate memory used by the encryption and decryption algorithms will be correspondingly larger;
Key generation is important. Random numbers are not good keys;
When chip T is implemented as a core, it is difficult to couple chip T to a given system ASIC;
When chip T is implemented as software, not only is the system implementation vulnerable to programming errors and less rigorous testing, but the integrity of the compiler and mathematical primitives is strictly checked for each system implementation. Must. This is more complex and costly than simply using a well-tested chip;
A large number of symmetric algorithm despite has been enhanced especially the difference decryption order to compete in (and are based on the selected text attack), likely to be the subject of the secret key K _A is selected text attack.

Protocol 4 authentication chips cannot be taken out of the United States. Because they are considered strong encryption devices.

As with protocol 3, the only specific timing requirement for protocol 4 is that a return value of 0 (indicating a bad input) is generated at the same time, regardless of where the error is in the input. It should be. Therefore, the attacker cannot learn what the error regarding the input value is. This is true for both the RD function and the TST function.

TST call change If two authentication chips are used, in theory, the replica manufacturer can replace the system authentication chip with one that returns 1 (success) for each call to TST. . The system can test this by calling TST N times with the wrong hash value many times and can predict that the result will be zero. When TST is last called, the true return value from chip A is passed and the return value is reliable. The number of times TST is called becomes a problem. The number of calls must be random. As a result, the replica chip manufacturer cannot know the number of times in advance. If the system has a clock, the bits from the clock are used to determine how many incorrect TST calls must be made. Otherwise, the return value from chip A can be used. In the latter case, the attacker can still rewrite the system to allow the duplicate chip T to monitor the return value from chip A, thereby knowing the correct hash value. The worst case is, of course, the case where the system is completely replaced by a replication system that does not require authorized consumables, which is the limit case of rewriting and changing the system. Thus, changing the TST call is optional and depends on the system, consumables, and the likelihood that changes will be made. Adding such logic to the system is not considered worthy (for example, in the case of a small desktop printer) as the system becomes more complex. On the other hand, adding such logic to the camera is considered worth it.

It is important to decrement the remaining amount of consumables before using the duplicate consumable consumable part that uses a genuine authentication chip. This is because, if the consumable is used first, the duplicate consumable will appear as a new consumable, pretending to be incomplete contact during writing to a special known address. It is important that this attack still require a genuine authentication chip for each consumable.

Key Life The general problem with the above two protocols is that once an authentication key is selected, it is not easy to change it. In some situations, it does not matter that the key is compromised, but in other cases it is fatal.

Protocol Selection Even though the key selection for protocols 2 and 4 is simple, both protocols are not practical at this time due to the high cost of silicon implementation (both key size and functional implementation). Therefore, the protocols selected are protocols 1 and 3. However, protocols 1 and 3 include a number of the same components,
Both require write and read access,
Both require the implementation of a keyed one-way function,
Both require a random number generation function.

Protocol 3 requires an additional key (K ₂ ) and some minimal state machine changes:
A state machine change that can call F _K1 [X] during Random;
A Test function that calls F _K2 [X];
A change to the Read function of the state machine is required to call F _K1 [X] and F _K2 [X].

Protocol 3 requires only minimal changes to protocol 1. Protocol 3 is more secure and can be used in all situations where presence-only authentication (protocol 1) is required. It is therefore a particularly good protocol. The selection of the one-way function will be described in more detail when both protocols 1 and 3 use a keyed one-way function. The following table outlines the applicable option attributes. The attributes are shown so that they are an advantage.

この表を検討することによって、事実上、３種類のＨＭＡＣ構造とランダムシーケンスとの間で選択されることがわかる。鍵サイズ及び鍵生成の問題は、ランダムシーケンスを除外する。多数の攻撃が既にＭＤ５に対して実行されていることを考えると、ハッシュ結果は１２８ビットに過ぎないので、ＨＭＡＣ−ＭＤ５も除外される。したがって、ＨＭＡＣ−ＳＨＡ１とＨＭＡＣ−ＲＩＰＥＭＤ１６０の間で選択を行うことになる。ＲＩＰＥＭＤ−１６０は比較的新しく、ＳＨＡ１のようには大規模に暗号解読されていない。しかし、ＳＨＡ−１はＮＳＡによって設計されているので、これは、否定的な属性として見られることもある。

By examining this table, it can be seen that there is effectively a choice between three types of HMAC structures and random sequences. Key size and key generation issues exclude random sequences. Considering that many attacks have already been performed against MD5, HMAC-MD5 is also excluded because the hash result is only 128 bits. Therefore, a selection is made between HMAC-SHA1 and HMAC-RIPEMD160. RIPEMD-160 is relatively new and not as extensively decrypted as SHA1. However, since SHA-1 is designed by the NSA, this may be seen as a negative attribute.

If there is no significant difference between the two, SHA-1 will be used for the HMAC structure.

Selection of random number generator Each protocol (1-4) described above requires a random number generator. The generator must be “good” in the sense that the random numbers generated over the lifetime of all systems are unpredictable. If the random number is the same for each system, the attacker can easily record the correct response from the real authentication chip and place the response in the ROM lookup for the duplicate chip. . In such an attack, K ₁ or K ₂ may not be acquired. Therefore, the random numbers from each system need to be sufficiently different to be unpredictable, i.e., nondeterministic. In this way, the initial value of R (Random Seed) was physically generated collected from physically random phenomena where there is no information about whether a particular bit is 1 or 0 Should be programmed with random numbers. The R seed must never be generated by a random number generator running on a computer. Otherwise, the generator algorithm and seed are deciphered, allowing the attacker to generate and know all R pairs of all systems.

The difference in the R seed for each authentication chip is that the first R is random and unpredictable on every chip. Therefore, how to generate the subsequent R in each chip becomes a problem.

In the basic case, R is not changed at all. As a result, R and F _K1 [R] are the same for each call to Random []. If they are the same, F _K1 [R] can be a constant rather than being calculated. An attacker can use a single valid authentication chip to generate a valid look-up table and use that look-up table in a replica chip specifically programmed for the system . The constant R is not safe.

The simplest conceptual way to change R is to increment R by one. Since R starts with a random number, its value in various systems is likely to be random. However, given the initial R, R values of all subsequent can be determined directly (even without repeated 10000 times, R represents takes a value from _{R 0} to _R 0 +10000). Incrementing R does not have to worry about the above attack on the constant R. Since R is always different, there is no way to build a lookup table for a particular system without wasting as many authentication chips as replicated chips to be substituted.

Besides incrementing using adders, another way to change R is to implement it as an LFSR (Linear Feedback Shift Register). This has the advantage of having less silicon than the adder, and also has the advantage that the attacker cannot directly determine the range of R for a particular system, since the LFSR value domain is determined by sequential access. In order to determine the value that a given initial R generates, the attacker must iterate through the possibilities and list them. The benefits of changing R are also evident in the case of LFSR solutions. Since R is always different, there is no way to build a look-up table for a particular system without using up the same number of authentication chips as the replacement chips (which are dedicated to a particular system) replace. Therefore, there is no advantage of introducing a more complicated function to change R. Regardless of the function, the attacker can always repeat the set of values throughout the lifetime through simulation. The inherent safety is in the initial randomness of R. The use of LFSR for changing the R is only the advantage of not being limited to (apart from the point a small silicon amount than adders) consecutive numeric range (i.e., R, calculate R _N The attacker must repeat the LFSR N times).

Therefore, the random number generator in the authentication chip is a 160-bit LFSR. Bits 159, 4 as shown in FIG. 173 by 160-bit tap selection for the maximum period LFSR (ie, LFSR cycles through 2 ¹⁶⁰ −1 states since 0 is not a valid state). 2 and 1 are obtained. The LFSR is a sparse structure and not many bits are used for feedback (only 4 bits out of 160 bits are used). This is a problem for cryptographic applications, but not for non-sequential number generation applications. The 160-bit seed value of R can take any random number other than zero. This is because an LFSR packed with 0 generates a stream with no limit of 0. Since the above-described LFSR is the maximum period LFSR, all 160 bits can be used as R as they are. There is no need to build a number in order from the output bit b _0. Each time the TST call is successful, the random number (R) is advanced by XORing bits 1, 2, 4 and 159 and shifting the result to the higher bits. The new R and the corresponding F _K1 [R] can be retrieved on the next call to Random.

Logical attack countermeasure protocol 3 is an authentication scheme used by the authentication chip. For this reason, it should be resistant to invalidation by logical means. Having described the effects of various types of attacks on Protocol 3, this section details each type of attack from the Protocol 3 perspective.

Brute force attack A brute force attack is guaranteed to break Protocol 3. However, the key length means that the time required for an attacker to perform a brute force attack is so long that it is not worth the effort. An attacker need only break the K ₂ for building a replication authentication chip. K ₁ is only provided to strengthen K ₂ against other forms of attack. Brute-force attack on the K ₂ is, therefore, must be breaking the 160-bit key. Attacks against K _2, it is necessary to up to ^{2 160 trials,} the probability of detecting the key after ^{two 159} trials is 50%. Assuming an array of 1 trillion processors, each running 1 million tests per second, 2 ¹⁵⁹ tests (7.3 x 10 ⁴⁷ ) require 2.3 x 10 ²³ years, Is longer than the lifetime of the universe. There are only 100 million personal computers in the world. Even if they are connected during an attack (eg, via the Internet), the number is 10,000 times less than the trillion processor attacks described above. Furthermore, even if one trillion processors can be manufactured in the nanocomputer era, the time required to acquire a key is longer than the lifetime of the universe.

It is theoretically possible for a guessing attacker to simply “guess a key”. In practice, if there is enough time and trying any number, the attacker will get the key. This is the same as the brute force attack described above, and 2 ¹⁵⁹ attempts must be made before a 50% success rate is achieved. There are 2 ¹⁶⁰ opportunities for a person to simply guess the key on the first attempt. On the other hand, won the first prize a person in the United States lottery, the probability of death by lightning on the same day, only 1 of 2 ²⁶ minutes. The probability that a person to guess the key on the first attempt is one of 2 ¹⁶⁰ minutes, which, by comparable to that two out of all the atoms of the earth to select exactly the same atoms Yes, i.e. very unlikely.

To break a quantum computer attacks K _2, it is necessary to build a quantum computer comprising 160 qubits, incorporated in a suitable algorithm. Attacks against 160-bit keys are not feasible. The most promising prospect of a quantum computer is to achieve 50 qubits within 50 years. Even using a 50 qubit quantum computer, 2 ¹¹⁰ tests are required to decrypt a 160 bit key. Assuming an array of 100 million 50 qubit quantum computer, if the quantum computer is to be able to try 2 ⁵⁰ times key (beyond the most violent estimated at the present time) to 1 microsecond, It takes 18 billion years on average to find the key.

Attack attacker only ciphertext ciphertext against K ₂ by monitoring to leave only attack ciphertext for K ₁ by calling monitor calls to RND and RD, and calls to the RD and TST It is possible to just attack. However, if all these calls reveal plaintext and plaintext hash form, the attack will turn into a more powerful form of attack, namely the known plaintext attack.

It is easy to connect a known plaintext attack logic analyzer to the wiring between the system and the authentication chip, thereby monitoring the data flow. Flow of this data occurs the known plaintext, and a hashed form of the plaintext can be used to exit the known plaintext attack against both K ₁ and K _2. To get out attacks on K _1, it is necessary to perform a large number of calls to the RND and TST (call to TST is successful, it is necessary to call to RD of effective chip). This is simple and the attacker needs to have both a system authentication chip and a consumables authentication chip. As each pair of K ₁ , X, and H _K1 [X] becomes clear, a pair of K ₂ , Y, and H _K2 [X] also becomes clear. The attacker must collect these pairs for further analysis. The problem arises of how many pairs should be collected in order to use this data for a meaningful attack. An example of an attack that requires collection of data for statistical analysis is differential cryptanalysis. However, since there is no known attack on SHA-1 or HMAC-SHA1, there is currently no use for the collected data.

If chosen plaintext attack cryptanalyst has the capability to change the subsequent selection plaintext based on the results of previous studies, K ₂ adaptively selected in the form of stronger attack than a simple selection plaintext attacks Unprotected in some forms of plaintext attack. Chosen-plaintext attack can not occur for K _1. Because the caller, because there is no way to change the R to be used as input to the RND function (RND is the only function that gives the results of hashing by K _1). Clearing R is not useful because it has the effect of clearing the key, and the SSI command calls CLR before storing the new R value.

Adaptive chosen plaintext attack this type of attack can not occur with respect to K _1. This is because, K ₁ is because not allow the chosen plaintext attack. However, some forms of this attack can occur with respect to K _2, in particular, both of consumables system, typically an attacker because available (system specific In some cases, such as a car, it cannot be accessed by an attacker). The HMAC structure provides security against any form of chosen plaintext attack. This is mainly because the HMAC structure has two secret input variables (the original hash result and the secret key). In this way, detecting the collision of the hash function itself when the input variable is secret is more difficult than finding the collision of the plaintext hash function. The reason is that the former requires direct access to SHA-1 (not allowed in protocol 3) in order to generate an input / output pair from SHA-1. The only values that can be collected by the attacker are HMAC [R] and HMAC [R | M]. These reduce the attack to a differential cryptanalysis attack that examines statistical differences between collected data, rather than attacks on the SHA-1 hash function itself. If there are no known differential cryptanalysis attacks against SHA-1 or HMAC, protocol 3 counters adaptive choice plaintext attacks.

An intentional error attacker can only make an intentional error attack against the TST function and the RD function. This is because these are the only functions that validate the input to the key. For both TST and RD functions, a zero value is generated when an error is found in the input and no further information is given. In addition, the time required to generate a zero result does not depend on the input, and the attacker cannot obtain information about which bits have errors. Therefore, intentional error attacks are ineffective.

Chain Attacks All types of chain attacks assume that the message to be hashed is spread over several blocks or that the input variables can be set in some way. The HMAC-SHA1 algorithm used in protocol 3 can only hash a single 512-bit block at the same time. As a result, a chain attack is not possible against protocol 3.

Birthday attack The strongest attack known for HMAC is a birthday attack based on the frequency of hash function collisions. However, this is not practical at all for a minimal and reasonable hash function such as SHA-1. A birthday attack can be realized only when an attacker manages a signed message. Protocol 3 uses hashing as a digital signature format. The system sends a number to be incorporated into the response from the valid racial chip. The authentication chip must respond using H [R | M], but since the input value R cannot be controlled, the birthday attack is not feasible. This is because the message has already been generated and signed in effect. Instead, the attacker must search for collision messages hashed to the same value (similar to finding someone with the same birthday as you). Therefore, the replica chip must try to find a new value R ₂ such that the hash of R ₂ and the selected M ₂ yields the same hash value as H [R | M]. However, the system authentication chip does not reveal the correct hash value (the TST function only returns 1 or 0 depending on whether the hash value is correct). Therefore, the only way to find the correct hash value (to find a collision) is to query a real authentication chip. However, finding the correct value means updating M, and since M's decrement-only part and M's read-only part cannot be changed, the replica consumables are genuine before attempting to find a collision. Consumables must be updated. An alternative is a brute force attack search into the TST function to find success (each replicated consumable needs to have access to the system consumable). As described above, the brute force search takes longer than the lifetime of the universe for each authentication in this example. Collecting hash values in a timely manner means that a real consumable must be decremented, so there is no point in replicating consumables carrying out this type of attack.

The random number seed of each substitute system according to the complete lookup table is 160 bits. In the worst case situation of the authentication chip, the state data is not changed. As a result, a constant value is returned as M. However, the duplicate chip must continue to return F _K2 [R | M], which is a 160-bit value. Assuming a 160-bit lookup table with a 160-bit result, this requires 7.3 × 10 ⁴⁸ bytes, ie 6.6 × 10 ³⁶ terabytes, and is definitely more than space that can be implemented in the near future. large. This of course does not take into account even the method of collecting values for the ROM. Thus, a complete lookup table is not completely feasible.

The alternative sparse lookup table by the sparse lookup table can be implemented only when the message sent to the authentication chip is not effectively random and can be predicted to some extent. The random number R is seeded with an unknown random number collected from natural random events. The replica manufacturer is unlikely to know the range of possible Rs for all hesitations. This is because the probability that each bit is 1 or 0 is 50%. Since the range of R in all systems is unknown, it is impossible to construct a sparse lookup table that can be used in all systems. Therefore, a general sparse lookup table attack is unthinkable. However, the replica manufacturer can know the range of R for a particular system. This can be achieved by loading the LFSR with the current result from a call to the RND function of a particular system authentication chip and repeating it several times for the future. If this is done, a special ROM that stores only the response to a particular R range, ie a ROM dedicated to the consumables of that particular system, can be constructed. However, the attacker must keep correct information in ROM. Therefore, the attacker needs to detect an effective authentication chip and call it for each value of R.

Assume that the duplicate authentication chip reports a complete consumable and can be used only once before simulating the loss of connection and the insertion of a new complete consumable. Duplicate consumables must store responses to full consumable authentication and responses to partially used consumables. In the worst case, the ROM stores entries for full consumables and partially used consumables for R over the life of the system. However, the valid authentication chip is used to generate information and should be used in part in the process. If a given system produces only n R values, the required sparse lookup ROM is 10n bytes multiplied by the number of different values for M. The time required to build the ROM depends on the time imposed between RD calls.

Nevertheless, replica manufacturers are forced to rely on consumables returned for refilling. This is because the cost of building a ROM in the first place consumes one consumable. The replica manufacturer's business in this situation results in refilling. Time and cost depend on the size of R and the number of types of M values to be incorporated into the lookup. In addition, custom replicated ROMs must be adapted to every single system, and different valid authentication chips must be used for each system (to provide complete and partially used data). . The use of an authentication chip in the system must therefore be examined to determine whether this type of attack is valuable to the replica manufacturer. An example is a camera system that has about 10,000 prints during its lifetime. Assume that there is a one-second delay between a single decrement-only value (remaining print count) and RD invocation. In such a system, the sparse table takes about 3 hours to build and consumes 100K. It should be noted that the cost must be higher than the sum of one consumable and duplicate consumables, since it is necessary to consume a valid authentication chip in order to construct a ROM. In this way, performing this function on a single consumable is not cost effective (unless the duplicate consumable is somehow housed in a number of equivalent certified consumables). If the replica manufacturer does not spare the effort to build a custom ROM for each owner of the system, a simpler approach would be to update the system to completely ignore the authentication chip, so This attack can be implemented as a system-by-system attack, and the possibility of such occurrence for a given system / consumable combination must be determined. The possibilities include consumable and authentication chip costs, consumable life, consumable profit margin, time required to generate the ROM, the size of the ROM that can be obtained, the same replica chip, etc. Depends on whether or not to return to the replica manufacturer for refilling.

Differential cryptanalysis known differential attacks are highly dependent on the structure of the S-box as used in DES and other similar algorithms. Other algorithms like HMAC-SHA1 used in protocol 3 do not have an S box, but the attacker
A differential attack can be performed by performing a statistical analysis of the minimum differential input and the corresponding minimum differential output and the corresponding input.

To attack this type of attack, input / output pair pairs are collected. Collection from protocol 3 is by known plaintext attack or partially adaptive selected plaintext attack. Obviously, the latter is more useful. In general, hashing algorithms are designed to counter differential analysis. In particular, SHA-1 has been enhanced, especially by an 80-word extension, so a minimal difference in input will produce an output that changes a number of bit positions (compared to a 128-bit hash function). Let's go. Moreover, the information collected is not a direct SHA-1 input / output due to the nature of the HMAC algorithm. The HMAC algorithm hashes a known value with an unknown value (key) and the result of this hash is then re-hashed with a separate unknown value. Since the attacker does not know the secret value or the result of the first hash, the input and output from SHA-1 are unknown, making any differential attack extremely difficult. In the following, the input and output of the minimum difference from the authentication chip will be described in more detail.

In this case, the attacker obtains a set of X values, F _K [X] values, where the X value is the minimum difference, and the statistical value between the outputs F _K [X] Check for differences. Attacks rely on X values that differ by a minimum number of bits. The question is how to obtain the minimum difference X value to compare F _K [X] values.

For K ₁ : K ₁ , the attacker must statistically examine the minimum difference X, F _K1 [X] pairs. However, the attacker cannot select the X value and acquire the associated F _K1 [X] value. Since the X, F _K1 [X] pair can only be generated by calling the RND function on the system authentication chip, the attacker calls RND multiple times and records each observed pair in a table. Next, to perform a statistical analysis of F _K1 [X], a sufficiently minimal difference X value must be searched among the observed values.

For K ₂ : K ₂ , the attacker must statistically examine the minimum difference X, F _K2 [X] pairs. _X, the only way to generate _F K2 [X] pair is by RD function to generate _F K2 [X] for a given _{Y, F} K1 _[Y] pairs, where, X = Y | M. This means that the changeable parts of Y and M can be selected within a limited range by the attacker. The amount of selection should therefore be as high as possible.

The first way to limit the attacker's choice is to limit Y. This is because RD requires input of format Y, F _K1 [Y]. A valid pair can be easily obtained from the RND function, but it is a pair by selection of RND. An attacker only if acquired the appropriate pair from the RND, or only if you know the K _1, it is possible to provide a unique Y. In order to obtain an appropriate pair from the RND, a Bruce Force search is required. Knowing K ₁ is logically feasible only by performing a decryption on the pair obtained from the RND function, ie only by a de facto known text attack. RND is that number of times determined to call per second, K ₁ are common with the system of the chip. Thus, known pairs can be generated in parallel. A second way to limit the attacker's selection is to limit M, or at least limit the attacker's ability to select M. The restriction of M is realized by making a part of M read-only, making it different for each authentication chip, and making the other part of M decrement-only. Since the read-only part of M should ideally differ from one authentication chip to another, information such as a serial number, batch number, or random number may be used. The decrement-only part of M can decrement only that part of M as many times as the attacker tries a different M, and that part can change again after the decrement-only part of M drops to zero It means no. Acquiring a new authentication chip 53 gives a new M, but the read-only portion is different from the read-only portion of the previous authentication chip, thus further reducing the attacker's ability to select M. As a result, the attacker can only slightly increase the chance of selecting values for Y and M.

In this case, the attacker obtains a set of X values, F _K [X] values, where the F _K [X] values represent the minimum difference, and the statistical value between the X values Check for differences. The attack relies on F _K [X] values that differ by a minimum number of bits. For both K ₁ and K ₂ , the attacker has no way to generate an X value for a given F _K [X]. To do so, it would violate the fact that F is a one-way function. Consequently, the only way for an attacker to incorporate an attack with this property is to record all observed X, F _K [X] pairs in a table. The search is performed within the observed values to find a sufficiently minimal difference F _K [X] value to perform a statistical analysis of the X values. If this requires more work than a minimal difference input attack (which is very limited due to restrictions on M and restrictions on the choice of R), this attack is useless.

Message substitution attack In order to perform this type of attack, the replica manufacturer needs to store a real authentication chip 53, but it does not diminish and is practically reusable. The duplicate authentication chip steals the message and replaces it with its own message. However, this attack is not successful for the attacker. The duplicate authentication chip may choose not to pass the WR command to the real authentication chip. However, subsequent RD commands must return the correct response (as if the WR was successful). In order to return a correct response, the hash values for a particular R and M must be known. As explained in the birthday attack section, the attacker cannot determine the hash value without actually updating M in the real chip, but the attacker does not want to do this. Even changing the R sent by the system is useless. This is because the system authentication chip must match R during the subsequent TST. Therefore, the message replacement attack is not successful. This is true only when the system updates the remaining amount of consumables before using the consumables.

If a reverse-engineered pseudo-random number generator of the key generator is used to generate the key, the replica manufacturer may obtain the generator program or infer the random seed used. This is how the Netscape security program was first broken.

The authentication avoidance protocol 3 requires the system to update the consumable status data before using the consumable and continue reading after all writes (to authenticate the write). In this way, authentication is requested every time the consumable is used. If the system adheres to these two simple rules, the replica manufacturer must simulate authentication by the method described above (such as a sparse ROM lookup).

Authentication Chip Reuse As described above, protocol 3 requires that the system update the consumable status data before using the consumable and continue reading after all writes (to authenticate writing). In this way, authentication is requested every time the consumable is used. When the consumable is used up, the appropriate status data value of the authentication chip has decreased to zero. Therefore, the chip cannot be used with another consumable. Note that this is only true for authentication chips that hold decrement-only data items. If there is no state data that is decremented every time it is used, there is nothing to prevent the reuse of the chip. This is the fundamental difference between presence-only authentication and consumable lifetime authentication. Protocol 3 makes both possible. In conclusion, if the consumable has a decrement-only data item that is used by the system, the authentication chip is not reusable unless it is completely reprogrammed by a valid programming station that knows the secret key.

Management decisions that omit authentication to save costs While not limited to external attacks, decisions that omit authentication in future systems to save costs will have different impacts on different markets. It should be noted that in the case of mass production consumables, it is extremely difficult to introduce certification after the start of sales. This is because systems that require authenticated consumables do not work with older consumables in circulation. Similarly, it is impractical to abort authentication at some stage. This is because older systems do not work with new uncertified consumables. In the second case, older systems can be individually modified by replacing the system authentication chip with a simple chip that has the same programming interface but whose TST function is always successful. is there. Of course, the system may be programmed to test and stop a TST function that always succeeds. For specialized pairs such as car / car keys or door / door keys, or other similar situations, omitting authentication in future systems is not important and has no effect. This is because the consumer purchases the entire system and consumable authentication chip set at the same time.

Burglary / Bribery Attack This form of attack can only be successful in one of two environments:
K ₁ , K ₂ , and R have already been recorded by the chip programmer, or the attacker can force future K ₁ , K ₂ , and R values to be recorded.

When a person or computer system outside the programming station does not know the key, it cannot be revealed by violence or bribery. The level of security against this type of attack is ultimately a decision made by the system / consumables owner depending on the level of service desired. For example, an automobile company wants to keep a record of all keys that it has manufactured, so it can request the creation of a new key for the car. However, this can compromise the entire key database and allow an attacker to create a key for the manufacturer's existing car. This does not allow an attacker to create a new car key. Of course, the key database itself is encrypted with another key, and in order to access the key database, it is necessary to combine the key portions of a certain number of people into one. If there is no record of which key is used for a particular car, there is no way to create an auxiliary key if the key is lost. In this case, the owner will have to exchange all of his car's authentication chip and his car key. This is not necessarily a problem. In contrast, in the case of consumables such as printer ink cartridges, a single key combination is used for all systems and all consumables. Of course, if no key backup is stored, no one knows the key and no attack is possible. However, the situation without backup is not desirable for consumables such as ink cartridges. This is because if the key is lost, no more consumables can be manufactured. Therefore, the manufacturer should store a backup of the key information in several parts, and some people need to combine the parts together to reveal the information for the entire key. is there. This is required when the chip programming station has to be reloaded. In any case, these attacks are not attacks on the protocol 3 itself. This is because people are not involved in the authentication process. Instead, this is an attack on the programming stage of the chip.

HMAC-SHA1
The mechanism for authentication is the HMAC-SHA1 algorithm,
HMAC-SHA1 (R ₁ , K ₁ ) or HMAC-SHA1 (R | M, K ₂ )
It operates according to one of the following.

Next, the HMAC-SHA1 algorithm will be examined in more detail than described so far, and optimization of the algorithm that requires less memory resource than the original definition will be described.

HMAC
The HMAC algorithm proceeds assuming the following definition:
H = hash function (eg, MD5 or SHA-1);
Number of bits output from n = H (for example, 160 bits for SHA-1 and 128 bits for MD5);
M = data to which the MAC function is applied;
K = private key shared by both parties;
ipad = 0x36 64 repetitions;
64 repetitions of opad = 0x5C.

The HMAC algorithm is as follows:
Extend K to 64 bytes by appending 0x00 bytes to the end of K;
Perform an XOR logical operation on the 64-byte character string created in (1) with ipad;
Add data stream M to the 64-byte character string created in (2);
Apply H to the stream generated in (3);
XOR logical operation of the 64-byte character string created in (1) with opad;
Append the result H from (4) to the 64-byte string obtained from (5);
H is added to the output of (6), and the result is output.

This
HMAC [M] = H [(KAopad) | H [(KAipad) | M]]
Is
The HMAC-SHA1 algorithm is simply an HMAC where H = SHA-1.

SHA-1
The SHA-1 hashing algorithm is defined in the algorithm summarized below.

Nine 32-bit constants are defined. Five constants are used to initialize the chain variable and there are four additional constants.

最適化されていないＳＨＡ−１は、全部で２９１２ビットのデータ記憶を必要とする：
５個の３２ビット連鎖変数：Ｈ_１、Ｈ_２、Ｈ_３、Ｈ_４及びＨ_５が定義される；
５個の３２ビット作業変数：Ａ、Ｂ、Ｃ、Ｄ及びＥが定義される；
１個の３２ビット一時変数：ｔが定義される；
８０個の３２ビット一時レジスタ：Ｘ_０−７９が定義される。

Non-optimized SHA-1 requires a total of 2912 bits of data storage:
Five 32-bit chain variables: H ₁ , H ₂ , H ₃ , H ₄ and H ₅ are defined;
Five 32-bit work variables: A, B, C, D and E are defined;
One 32-bit temporary variable: t is defined;
80 32-bit temporary registers: X _0-79 are defined.

The following functions are defined for SHA-1.

ハッシングアルゴリズムは、最初に、５１２ビットの倍数になるまで入力メッセージをパディングし、連鎖変数Ｈ_１−５をｈ_１−５で初期化する。パッドされたメッセージは、次に、５１２ビットチャンクで処理され、出力ハッシュ値は、連鎖変数：Ｈ_１｜Ｈ_２｜Ｈ_３｜Ｈ_４｜Ｈ_５の連結によって与えられる最後の１６０ビット値である。次に、ＳＨＡ−１アルゴリズムのステップを詳細に説明する。

The hashing algorithm first pads the input message until it is a multiple of 512 bits, and initializes the chain variable H _1-5 with h _1-5 . The padded message is then processed in 512-bit chunks, and the output hash value is the last 160-bit value given by the concatenation of the chain variables: H ₁ | H ₂ | H ₃ | H ₄ | H ₅ . Next, the steps of the SHA-1 algorithm will be described in detail.

Step 1 The first step of the process SHA-1 stuffs the input message until it becomes a multiple of 512 bits as follows, and initializes the chain variable.

ステップ２ 処理
詰め込まれた入力メッセージが処理可能である。５１２ビットのブロック内でメッセージを処理する。各５１２ビットのブロックは、１６×３２ビット語の形であり、ＩｎｐｕｔＷｏｒｄ_０−１５として表される。

Step 2 Processed input message can be processed. Process the message in a 512-bit block. Each 512-bit block is in the form of a 16 × 32-bit word and is represented as InputWord _0-15 .

ステップ３ 終了
詰め込まれた入力メッセージの全ての５１２ビットのブロックが処理された後、出力ハッシュ値は、Ｈ_１｜Ｈ_２｜Ｈ_３｜Ｈ_４｜Ｈ_５によって与えられる最後の１６０ビット値である。

Step 3 After all 512-bit blocks of the stuffed input message have been processed, the output hash value is the last 160-bit value given by H ₁ | H ₂ | H ₃ | H ₄ | H ₅ .

Optimization SHA-1 for hardware implementation Step 2 procedure is not optimized for hardware. In particular, 80 temporary 32-bit registers use up valuable silicon on hardware implementations. This section describes an optimization to the SHA-1 algorithm that uses only 16 temporary registers. The silicon reduction is a reduction from 2560 bits to 512 bits, saving over 2000 bits. This is not important for some applications, but in the case of an authentication chip, the storage space should be reduced as much as possible. The basis of the optimization is that the original 16 word message block is expanded to an 80 word message block, but 80 words are not updated during the algorithm. Moreover, since the word depends only on the previous 16 words, the expanded word can be computed on the fly during processing as long as 16 words are retained for backward reference. A circular counter is required to keep track of which registers are being used, but the effect is to save a large amount of storage. Rather than indexing X with a single value j, a 5-bit counter is used to count through iterations. This is accomplished by initializing a 5-bit register with 16 or 20 and decrementing it until it reaches zero. Since 16 temporary variables are updated as if they were 80, 4 indexes are required, and each index is a 4-bit register. All four indexes are incremented in the middle of the algorithm (with wraparound).

ラウンド０及び１Ａの間のＮ_１、Ｎ_２及びＮ_３のインクリメントは随意的である。ソフトウェア実装は、時間を要するので、それらをインクリメントせずに、１６巡のループの最後に、全４個のカウンタは元の値になる。ハードウェアの設計者は、制御ロジックを節約するため、全４個のカウンタを同時にインクリメントしようとする。呼び出し元が５１２ビットのＸ_０−１５をロードするならば、ラウンド０は完全に省くことができる。

Incrementing N ₁ , N ₂ and N ₃ during rounds 0 and 1A is optional. The software implementation takes time, so at the end of the 16-round loop, all four counters return to their original values without incrementing them. Hardware designers try to increment all four counters simultaneously to save control logic. If the caller loads 512 bits of X _0-15 , round 0 can be omitted entirely.

HMAC-SHA1
In the case of an authentication chip implementation, the HMAC-SHA1 unit only performs hashing on two types of inputs: R using K ₁ and R | M using K ₂ . Since the inputs are two fixed lengths, instead of providing HMAC and SHA-1 as separate entities on the chip, they can be combined to optimize the hardware. Message padding (1 bit, 0 bit string, and message length) in step 1 of SHA-1 is necessary to ensure that different messages do not look the same after padding. Since only two types of messages are handled, the padding may be a constant 0. In addition, an optimized version of the SHA-1 algorithm is used, and only 16 32-bit words are used for temporary storage. These 16 registers are loaded directly by the optimized HMAC-SHA1 hardware. Although nine 32-bit constant _{h -1-5} and _{y 1-4} is still necessary, it is that they are constant, it is advantageous for hardware implementation. The hardware optimized HMAC-SHA-1 requires a total of the following 1024 bits of data storage:
Five 32-bit chain variables: H ₁ , H ₂ , H ₃ , H ₄ and H ₅ are defined;
Five 32-bit work variables: A, B, C, D and E are defined;
Five 32-bit variables for temporary storage and final result: Buff160 _1-5 are defined;
One 32-bit temporary variable: t is defined;
Sixteen 32-bit temporary registers: X _0-15 are defined.

The following two sections describe the steps for two types of calls to HMAC-SHA1.

H [R, K ₁ ]
In the case of generating a keyed hash of R using K ₁ , the original input message R is 160 bits long. This can therefore be exploited advantageously during processing. Rather than loading _{X 0-15} during the first part of the SHA-1 algorithm to load _{the X 0-15} directly, thereby, the round 0 of SHA-1 optimization process block (Step 2) Omit. The pseudo code performs the following steps:

Ｈ［Ｒ｜Ｍ，Ｋ_２］
Ｋ_２を使用してＲ｜Ｍの鍵付きハッシュを生成するケースでは、元の入力メッセージは、４１６（２５６＋１６０）ビットの一定長さである。したがって、処理中にこのことを有利に活用することができる。ＳＨＡ−１アルゴリズムの最初の部分の間にＸ_０−１５をロードするのではなく、Ｘ_０−１５を直接ロードし、これにより、ＳＨＡ−１の最適化プロセスブロック（ステップ２）のラウンド０を省く。擬似コードは次のステップを行う。

H [R | M, K ₂ ]
In the case of using K ₂ to generate a keyed hash of R | M, the original input message is a constant length of 416 (256 + 160) bits. This can therefore be exploited advantageously during processing. Rather than loading _{X 0-15} during the first part of the SHA-1 algorithm to load _{the X 0-15} directly, thereby, the round 0 of SHA-1 optimization process block (Step 2) Omit. The pseudo code performs the following steps:

データ記憶完全性
確認証チップは、認証プロトコル３によって要求される変数を保持するため、不揮発性メモリを含む。以下の不揮発変数が定義される。

The data storage integrity verification chip includes a non-volatile memory to hold variables required by the authentication protocol 3. The following non-volatile variables are defined:

これらの変数がフラッシュメモリにあるならば、古い値を置き換えるため新しい値を書き込むことは簡単ではないことに注意する必要がある。メモリは、最初に消去され、次に、適切なビットをセットされる。これは、フラッシュメモリベースの変数を変更するため使用されるアルゴリズムに影響を与える。例えば、フラッシュメモリはシフトレジスタとして簡単に使用できない。汎用演算でフラッシュメモリ変数を更新するためには、以下のステップが必要になる：
Ｎビット値全体を汎用レジスタに読み込む；
汎用レジスタに演算を実行する；
変数に対応したフラッシュメモリを消去する；
汎用レジスタにセットされたビットに基づいてフラッシュメモリロケーションのビットをセットする。

Note that if these variables are in flash memory, writing a new value to replace the old value is not easy. The memory is first erased and then the appropriate bit is set. This affects the algorithms used to change flash memory based variables. For example, flash memory cannot be easily used as a shift register. To update a flash memory variable with a general purpose operation, the following steps are required:
Read the entire N-bit value into a general-purpose register;
Perform operations on general-purpose registers;
Erase the flash memory corresponding to the variable;
Set the bit in the flash memory location based on the bit set in the general register.

The reset of the authentication chip does not affect these non-volatile variables.

M and access mode variables M [0] to M [15] are used to hold consumable status data such as serial number, batch number, and consumable remaining amount. Each M [n] register is 16 bits, and the entire M vector is 256 bits (32 bytes). The client cannot read or write individual M [n] variables. Instead, the entire vector represented by M is read or written with a single logical access. M is read using an RD (read) command and written using a WR (write) command. These commands succeed only if both K1 and K2 are finalized (SIWritten = 1) and the authentication chip is a consumable untrusted chip (IsTrusted = 0). M stores a number of different data types, but they only differ in write permission. Each data type can always be read. Once in the client memory, the 256 bits can be interpreted in any way selected by the client. The entire 256 bits of M are read at the same time instead of being read in small increments for security reasons as described in the authentication section. Various write permissions are listed in the table below.

書き込みに要求される保護を実現するため、２ビットアクセスモード値がＭ［ｎ］毎に定められる。以下の表は、２ビットアクセスモードビットパターンの解釈を定義する。

In order to realize the protection required for writing, a 2-bit access mode value is determined for each M [n]. The following table defines the interpretation of the 2-bit access mode bit pattern.

１６個のＭ［ｎ］レジスタのための１６組のアクセスモードビットは、単一のアクセスモードＡｃｃｅｓｓＭｏｄｅレジスタにまとめられる。３２ビットのＡｃｃｅｓｓＭｏｄｅレジスタは、以下のように、ｎ付きのＭ［ｎ］に対応する。

The 16 sets of access mode bits for the 16 M [n] registers are grouped into a single access mode AccessMode register. The 32-bit AccessMode register corresponds to M [n] with n as follows.

各２ビット値は、上位／下位のフォーマットで記憶される。したがって、Ｍ［０−５］がアクセスモードＭＳＲであり、Ｍ［６−１５］がアクセスモードＲＯであるならば、３２ビットＡｃｃｅｓｓＭｏｄｅレジスタは、
１１−１１−１１−１１−１１−１１−１１−１１−１１−１１−０１−０１−０１−０１−０１−０１
である。

Each 2-bit value is stored in an upper / lower format. Therefore, if M [0-5] is the access mode MSR and M [6-15] is the access mode RO, the 32-bit AccessMode register is
11-11-11-11-11-11-11-11-11-11-01-01-01-01-01-01-01
It is.

During execution of the WR (write) command, AccessMode [n] is examined for each M [n] to determine whether the new M [n] value replaces the old one. The AccessMode register is set using the SAM (Set Access Mode) command of the authentication chip. Decrement-only comparisons are unsigned, and decrement-only values that require a negative range must be shifted to the positive range. For example, a consumable containing decrement-only data items in the range −50 to 50 is shifted in range from 0 to 100. The system interprets the range 0-100 as -50-50. In most cases, the decrement only range is N to 0 and no range shift is necessary. In the case of a decrement-only data item, data is arranged forward from M [n] in order from the most significant 16-bit amount to the least significant 16-bit amount. The access mode for the most significant 16 bits (stored in M [n]) should be set to MSR. The remaining registers (M [n + 1], M [n + 2], etc.) should have their access mode set to NMSR. If incorrectly set to NMSR and there is no associated MSR region, each NMSR region is considered independent, not a double precision comparison.

K ₁
K ₁ is a 160-bit secret key used to convert R during the authentication protocol. K ₁ is programmed with _{K 2} and R with SSI (the secret information set) command. Since K ₁ must be kept secret, the client can not read the K ₁ directly. Command to use the K ₁ is the RND and RD. RND returns a pair of R, F _K1 [R], where R is a random number, while RD requests an X, F _K2 [R] pair as input. K ₁ is used in a keyed one-way hash function HMAC-SHA1. Thus, K ₁ is physically collected from random phenomenon, it should be programmed in a physically generated random number. K ₁ is in no way shall be generated by the random number generator is run on a computer. The security of the authentication chip depends on K ₁ , K ₂ and R generated in a non-deterministic way. For example, to set the K _1, some people toss 160 times a fair coin, record the table as 1, and records the back as zero. K ₁ is automatically cleared to 0 at the time of execution of the CLR command. It can be set to a non-zero value only by an SSI command.

K ₂
K ₂ is a 160-bit secret key used to convert M | R during the authentication protocol. K ₂ is programmed with _{K 1} and R with SSI (the secret information set) command. Since K ₂ must be kept secret, the client can not read the K ₂ directly. Command to use the K ₂ is the RD and TST. RD returns a pair of M, F _K2 [M | X], where X is passed to the RD function as one of the parameters. TST required as input an M, F _K2 [M | R] pair, where R was obtained from the RND function of the authentication chip. K ₂ is used in a keyed one-way hash function HMAC-SHA1. Thus, K ₂ are physically collected from random phenomenon, it should be programmed in a physically generated random number. K ₂ is in no way shall be generated by the random number generator is run on a computer. The security of the authentication chip depends on K ₁ , K ₂ and R generated in a non-deterministic way. For example, to set the K _2, some people toss 160 times a fair coin, record the table as 1, and records the back as zero. K ₂ is automatically cleared to 0 at the time of execution of the CLR command. It can be set to a non-zero value only by an SSI command.

R and IsTrusted
R is a 160-bit random number seed which is programmed with _{K 1} and R with SSI (the secret information set) command. R need not be kept secret. Because it is freely given to the caller by the RND command. However, only the authentication chip can change R and cannot be set to any selected value by the caller. R is used during the TST command to ensure that R from the previous RND call was used to generate F _K2 [M | R] at the untrusted authentication chip (Chip A). Both RND and TST are used only with a trusted authentication chip (chip T).
IsTrusted is a 1-bit flag register that determines whether the authentication chip is a trusted chip (chip T):
If the IsTrusted bit is set, the client is able to call the RND and TST functions (not RD or WR) because the chip is considered to be a trusted chip;
If the IsTrusted bit is cleared, the chip is not considered a trusted chip. Therefore, the RND function and the TST function cannot be called (instead, RD and WR may be called). The system never needs to call RND or TST on the consumables (because the duplicate chip only returns 1 to functions like TST and only returns a constant value for RND).

The IsTrusted bit has the added benefit of reducing the number of available R, F _K1 [R] that can be acquired by an attacker, while maintaining the integrity of the authentication protocol. To obtain a valid R, F _K1 [R] pair, an attacker needs a system authentication chip, which is more expensive and difficult to obtain than consumables. Both R and IsTrusted bits are cleared to 0 by the CLR command. Both of these are written by issuing an SST command. The IsTrusted bit can only be set by an SSI command by storing a non-zero seed value in R (R is non-zero because it is a valid LFSR state, which is quite reasonable). R is modified by a 160-bit maximum period LFSR with bits 1, 2, 4 and 159 tapped, and only when the call to TST is successful (if 1 is returned) .

An authentication chip that is scheduled to become a trusted chip in the system (chip T) should have its IsTrusted bit set during programming and the authentication chip (chip A) used in the consumable is Its IsTrusted bit should be left clear (by storing 0 in R with an SSI command during programming). There is no command for directly reading and writing the IsTrusted bit. The security of the authentication chip does not depend solely on the randomness of K ₁ and K ₂ and the strength of the HMAC-SHA1 algorithm. To prevent an attacker from building a sparse lookup table, the security of the authentication chip also depends on the range of R over the lifetime of all systems. Thus, R should be programmed with physically generated random numbers collected from physically random phenomena. R should never be generated by a random number generator running on a computer. The generation of R must not be deterministic. For example, to generate R for use with a reliable system chip, one person tosses a fair coin 160 times, records the front as 1, and the back as 0. 0 is the only invalid initial value for reliable R (or the IsTrusted bit is not set).

SIWritten
SIWriten (Secret Information Written) 1-bit register holds the state of the secret information stored in the authentication chip. The secret information is K ₁ , K ₂ and R. The client cannot directly access the SIWriteten bit. Instead, the SIWritten bit is cleared by the CLR command (this command also clears K ₁ , K ₂ and R.) The authentication chip uses the SSI command (regardless of the value written). When programmed with a secret key and random number seed, the SIWritten bit is automatically set, although R is not strictly secret, but the attacker obtains the selected R, F _K1 [R] pair. Can generate their own unique random number seed R must be written with K ₁ and K _{2. The} SIWritten status bit is used by all functions that access K ₁ , K ₂ or R. The SIWritten bit is cleared. RD, WR, RND, and TST calls are interpreted as CLR calls.

MINTICKS
There are two mechanisms that prevent an attacker from generating multiple calls to the TST and RD functions in a short time. The first mechanism is a clock limit hardware component that prevents the internal clock from operating at speeds above a certain maximum value (eg, 10 MHz). The second mechanism is a 32-bit MinTicks register, which is used to specify the minimum number of clock ticks that must elapse between calls to the keyed function. The MinTicks variable is cleared to 0 by the CLR command. The bit is set by an SMT (MinTicks set) command. The input parameters to the SMT include a bit pattern that represents which bits of MinTicks should be set. The actual effect is that the attacker can only increase the value in units of MinTicks (because the SMT function only sets the bit). Moreover, no function is provided that allows the caller to read the current value of this register. The value of MinTicks depends on the operating clock speed and how to determine a reasonable time between keyed function calls (application specific). The interval of one tick depends on the operating clock speed. This is the maximum value of the input clock speed and the clock limit hardware of the authentication chip. For example, the clock limit hardware of the authentication chip is set to 10 MHz (this cannot be changed), but the input clock is 1 MHz. In this case, the value of 1 tick is 1 MHz instead of 10 MHz. If the input clock is 20 MHz instead of 1 MHz, the value of 1 tick is based on 10 MHz (because the clock speed is limited to 10 MHz).

Once the interval of one tick is known, the MinTicks value can be set. The value of MinTicks is the number of MinTicks that are required to elapse between calls to the keyed RD and TST functions. This value is a real-time number divided by the length of the operating ticks. Assume that the input clock speed matches the maximum clock speed of 10 MHz. If the minimum value of 1 second is determined during the keyed function call, the value of MinTicks is set to 10000000. Consider the case where an attacker is trying to collect X, F _K1 [X] pairs by calling RND, RD and TST many times. If the MinTicks value is set so that the time between calls to the TST is 1 second, 1 second is required to generate each pair. 2 The attacker needs more than a year to generate ²⁵ pairs (only 1.25 Gb of storage is needed). An attack that requires 2 ⁶⁴ pairs would require 5.84 x 10 ¹¹ years if 1 chip is used, and 584 if 1 billion chips are used. Such an attack is totally unrealistic in time (not to mention the memory requirements!).

With respect to the K _1, MinTicks variable is to slow down the attacker, only to increase the cost of the attack. This is because MinTicks does not stop an attacker from using many system chips in parallel. However, MinTicks really make the attack on the _{K 2} more difficult. This is because each consumable has a different M (part of M is random read-only data). In order to enter a differential attack, a minimum difference input is required, which can be realized by a single consumable (including effectively a constant part of M). The minimal difference input requires the attacker to use a single chip, and MinTicks slows down the use of a single chip. If it takes a year to get the data to start searching for a value to start a differential attack, this will increase the cost of the attack and reduce the effective sales period of the replication consumables.

The authentication chip command system communicates with the authentication chip by a simple operation command set. This section details the actual commands and parameters required for Protocol 3 implementation. An authentication chip is defined as a minimum implementation that communicates with the system via a serial interface. It is conventional to define an equivalent chip that operates with a wider interface (eg, 8, 16 or 32 bits). The interpretation of the opcode may depend on the current value of the IsTrusted bit and the current value of the IsWriteten bit. The following operations are defined:

Ｏｐ＝オペコード、Ｔ＝ＩｓＴｒｕｓｔｅｄ値、Ｗ＝ＩｓＷｒｔｔｅｎ値、
Ｍｎ＝ニューモニック、［ｎ］＝パラメータに必要なビット数。

Op = opcode, T = IsTrusted value, W = IsWrtten value,
Mn = Pneumonic, [n] = Number of bits required for the parameter.

Commands not defined in this table are interpreted as NOP (no operation). Examples include opcodes 110 and 111 (which are independent of IsTrusted value or IsWriteten value) and opcodes other than SSI when IsWriteten = 0. Note that the RD and RND opcodes are the same, and the WR and TST opcodes are the same. The actual command that is moved after receiving the opcode depends on the current value of the IsTrusted bit (as long as IsWritten is 1). Where the IsTrusted bit is cleared, the RD and WR functions are called. When the IsTrusted bit is set, the RND and TST functions are called. The two sets of commands are mutually exclusive between a trusted authentication chip and an untrusted authentication chip, and the same opcode enforces this relationship. Each command is discussed in detail in subsequent sections. Some algorithms are specifically designed. This is because the flash memory is assumed to be a non-volatile variable implementation form.

ＣＬＲ（クリア）コマンドは、全ての認証チップメモリの内容を完全に消去するため設計される。これは、全ての鍵及び秘密情報と、アクセスモードビットと、状態データと、を含む。ＣＬＲコマンドの実行後、認証チップは、新しく製造されたときと丁度同じように、プログラマブル状態になる。それは、新しい鍵として再プログラムして、再使用することができる。ＣＬＲコマンドは、ＣＬＲコマンドオペコードだけにより構成される。認証チップはシリアルであるため、これは同時に１ビットずつ転送される。ビット順序は、各コマンドコンポーネントのＬＳＢからＭＳＢである。したがって、ＣＬＲコマンドは、ＣＬＲオペコードのビット０から２として送られる。全部で３ビットが転送される。ＣＬＲコマンドはいつでも直ちに呼び出すことができる。消去の順序は重要である。ＳＩＷｒｉｔｔｅｎｂｉｔ値は、（ＲＮＤ、ＴＳＴ、ＲＤ及びＷＲのような）鍵アクセス関数への更なる呼び出しを禁止するため、最初に消去されなければならない。ＡｃｃｅｓｓＭｏｄｅビットがＳＩＷｒｉｔｔｅｎの前にクリアされると、攻撃者は、それらがクリアされた後のある時点で電源を取り外し、Ｍを操作し、これにより、部分選択テキスト攻撃で秘密情報を取り出すよい機会を得る。ＣＬＲコマンドは以下のステップで実現される。

The CLR (clear) command is designed to completely erase the contents of all authentication chip memories. This includes all keys and secret information, access mode bits, and status data. After execution of the CLR command, the authentication chip is in a programmable state just as it was newly manufactured. It can be reprogrammed as a new key and reused. The CLR command is composed only of the CLR command opcode. Since the authentication chip is serial, it is transferred one bit at a time. The bit order is from LSB to MSB of each command component. Therefore, the CLR command is sent as bits 0 to 2 of the CLR opcode. A total of 3 bits are transferred. The CLR command can be called immediately at any time. The order of erasure is important. The SIWrittenbit value must first be erased to prohibit further calls to key access functions (such as RND, TST, RD, and WR). If the AccessMode bit is cleared before SIWritten, the attacker will have the opportunity to remove power at some point after they are cleared and manipulate M, thereby extracting the secret information in a partial-select text attack. obtain. The CLR command is realized by the following steps.

チップがクリアされると、チップは再プログラミングと再使用の準備が完了する。ブランクチップは攻撃者にとって無用である。なぜならば、Ｍに対して任意の値を作成することが可能であるとしても（Ｍは読み出し及び書き込みが可能である）、Ｋ_１及びＫ_２が間違っているので鍵付き関数は全く情報を提供しないからである。ＣＬＲがＣＬＲ以外の任意のオペコードのために要求されるならば、入力パラメータビットを費やす必要がない。攻撃者はチップをリセットするだけでよい。ＣＬＲを呼び出す理由は、全ての秘密情報が破壊され、チップが攻撃者にとって無用になることを保証するためである。

Once the chip is cleared, the chip is ready for reprogramming and reuse. Blank chips are useless for attackers. Because even if it is possible to create any value for M (M can be read and written), K ₁ and K ₂ are wrong, so the keyed function provides no information at all. Because it does not. If CLR is required for any opcode other than CLR, there is no need to spend input parameter bits. The attacker only has to reset the chip. The reason for calling CLR is to ensure that all secret information is destroyed and the chip is useless to the attacker.

SSI-Secret Information Set (SET SECRET INFORMATION)
Input: K ₁ , K ₂ , R = [160 bits, 160 bits, 160 bits]
Output: None Change: K ₁ , K ₂ , R, SIWritten, IsTrusted.

SSI (Secret Information Set (Set Secret)
Information)) command is used to load the K ₁ , K ₂ and R variables and set the SIWriteten and IsTrusted flags to later call the RND, TST, RD and WR commands. The SSI command includes an SSI command opcode followed by secret information to be stored in the K ₁ , K ₂ and R registers. Since the authentication chip is serial, it should be transferred bit by bit at the same time. The bit order is from LSB to MSB of each command component. Accordingly, SSI command includes a bit 0-2 SSI opcode, followed, and bits 0-159 of new values for _{K 1,} and bit 0-159 of new value for _{K 2,} finally, the seed values for R Of bits 0-159. A total of 483 bits are transferred. The K ₁ , K ₂ , R, SIWritten, and IsTrusted registers are all cleared to 0 by the CLR command. They can only be set by using SSI commands.

The SSI command uses the flag SIWritten to store the fact that data has been loaded into K ₁ , K ₂ and R. If the SIWriteten and IsTrusted flags are cleared (this is the case after the CLR instruction), new values are loaded into K ₁ , K ₂ and R. If any flag is set, a CLR command is executed by an SSI call attempt. This is because only the attacker or the wrong client tries to change the key or random seed without first calling the CLR. The SSI command also sets the IsTrusted flag depending on the value for R. If R = 0, the chip is considered untrustworthy and therefore IsTrusted remains zero. If R ≠ 0, the chip is considered trustworthy and IsTrusted is set to 1. Note that the IsTrusted bit is set only between SSI commands. If the authentication chip is reused, the CLR command must be invoked first. The key is then securely programmed with the SSI command and new state information is loaded into M using the SAM and WR commands. The SSI command is executed in the following steps.

ＲＤ−読み出し
入力：Ｘ、Ｆ_Ｋ１［Ｘ］＝［１６０ビット、１６０ビット、１６０ビット］
出力：Ｍ、Ｆ_Ｋ２［Ｘ｜Ｍ］＝［２５６ビット、１６０ビット］
変更内容：Ｒ。

RD-read input: X, F _K1 [X] = [160 bits, 160 bits, 160 bits]
Output: M, F _K2 [X | M] = [256 bits, 160 bits]
Details of change: R.

The RD (Read) command is used to securely read the entire 256 bits of status data (M) from an untrusted authentication chip. Only valid authentication chips respond correctly to RD requests. The output bit from the RD command can be supplied as an input bit to the TST command on the trusted authentication chip for verification, the first 256 bits (M) are later if the TST returns 1 (as expected). Saved for use in. Since the authentication chip is serial, the command and input parameters should be transferred one bit at a time. The bit order is from LSB to MSB of each command component. Thus, the RD command is bits 0-2 of the RD opcode, followed by bits 0-159 of X, and bits 0-159 of F _K1 [x]. A total of 320 bits are transferred. X and F _K1 [X] are obtained by calling the trusted authentication chip's RND command. The 320 bits output from the trusted chip's RND command can thus be supplied directly to the untrusted chip's RD command, and those bits may not be stored by the system. The RD command can only be used when the following conditions are met:
SIWritten = 1 indicates that K ₁ , K ₂ , R have been set up by the SSI command;
IsTrusted = 0 Indicates that the chip is not trusted because it is not allowed to generate a random number sequence.

In addition, the RD call must wait until the MinTicksRemaining register reaches zero. After that, the registers are reloaded with MinTicks, ensuring that a minimum amount of time elapses between calls to the RD. When MinTicks is reloaded into MinTicksRemaining, the RD command verifies that the input parameters are valid. This internally generate _F K1 [X] with respect to the input X, then carried out by comparing the results obtained with the input _F K1 [X]. This generation and comparison must use the same time regardless of whether the input parameters are correct. If the times are not the same, the attacker can obtain information about which bits of F _K1 [X] are wrong. An input parameter is invalid if it is the wrong system (passing the wrong bit), if the wrong system has the wrong consumables, a bad trusted chip (creating a bad pair), or an authentication chip Limited to attacks on A constant value of 0 is returned if the input parameter is incorrect. Since the time taken to return 0 must be the same for every bad input, the attacker cannot learn any information about what is invalid. When the input parameters are verified, the output value is calculated. The 256-bit contents of M are transferred in the order of bits 0-15 of M [0], bits 0-15 of M [1], and bits 0-15 of M [15]. F _K2 [X | M] is calculated and output as bits 0-159. The R register is used to store a value during verification of the X, F _K1 [X] pair. The reason is that RND and RD are mutually exclusive. The RD command is implemented in the following steps.

ＲＮＤ−ランダム
入力：無し
出力：Ｒ、Ｆ_Ｋ１［Ｒ］＝［１６０ビット、１６０ビット］
変更内容：無し。

RND-Random input: None Output: R, F _K1 [R] = [160 bits, 160 bits]
Changes: None.

The RND (Random) command is used by the client to acquire a valid R, F _K1 [R] pair for use in subsequent authentication by the RD and TST commands. Since there are no input parameters, the RND command is simply bits 0-2 of the RND opcode. The RND command can only be used if the following conditions are met:
SIWritten = 1 indicates that K ₁ and R have been set up by SSI command;
IsTrusted = 1 Indicates that the chip is allowed to generate a random number sequence.

The RND returns both R and F _K1 [R] to the caller. The 288 bit output of the RND command can be provided as an input parameter to the RD command of the untrusted chip. The client does not need to store them at all. Because they are not required again. However, the TST command succeeds only if the random number passed to the RD command is the one originally obtained from the RND command. If the caller calls only RND multiple times, the same R, F _K1 [R] pair will be returned each time. R proceeds to the next random number in the sequence after a successful call to TST. See the description of TST for more details. The RND command is implemented in the following steps.

ＴＳＴ−テスト（ＴＥＳＴ）
入力：Ｘ、Ｆ_Ｋ２［Ｒ｜Ｘ］＝［２５６ビット、１６０ビット］
出力：１又は０＝［１ビット］
変更内容：Ｍ、Ｒ及びＭｉｎＴｉｃｋｓＲｅｍａｉｎｉｎｇ（又は攻撃が検出されたならば全てのレジスタ）。

TST-Test (TEST)
Input: X, F _K2 [R | X] = [256 bits, 160 bits]
Output: 1 or 0 = [1 bit]
Changes: M, R, and MinTicksRemaining (or all registers if an attack is detected).

A TST (Test) command is used to authenticate the reading of M from an untrusted authentication chip. The TST (test) command is composed of a TST command opcode followed by input parameters: X and F _K2 [R | X]. Since the authentication chip is serial, it must be transferred bit by bit at the same time. The bit order is from LSB to MSB of each command component. Therefore, the TST command is bits 0-2 of the TST opcode, followed by bits 0-255 of M, and bits 0-159 of F _K2 [R | M]. A total of 419 bits are transferred. The last 416 input bits are obtained as output bits from the RD command to the untrusted authentication chip, so the entire data does not need to be saved by the client. Instead, the bit can be passed directly to the TST command of the trusted authentication chip. Only 256 bits of M should be kept secret to the RD command. The TST command can only be used when the following conditions are met:
SIWritten = 1 indicates that K ₂ and R have been set up by SSI command;
IsTrusted = 1 Indicates that the chip is allowed to generate a random number sequence.

In addition, the call to TST must wait until the MinTicksRemaining register reaches zero. After that, the registers are reloaded with MinTicks, ensuring that a minimum amount of time elapses between calls to the TST. TST causes the internal M value to be replaced by the input M value. Next, F _K2 [M | R] is calculated and compared with the 160-bit input hash value. The single output bit generated is 1 if they are the same and 0 if they are different. Using an internal M value saves space on the chip and is the reason that RD and TST are mutually exclusive commands. If the output bit is 1, R is updated to be the next random number in the sequence. This forces the caller to use a new random number each time the RD and TST tests are called. Since the resulting output bits are not output until the entire input string is compared, the time required to evaluate the comparison in the TST function is always the same. In this way, the attacker cannot compare the execution time or the number of bits processed before the output is given.

The next random number is generated from R using a LFSR with a maximum period of 160 bits (tap selected by bits 159, 4, 2 and 1). The initial 160-bit value of R is set up by an SSI command and can be any random number other than 0 (LFSR filled with 0 produces an unlimited stream of 0). R is converted by performing an XOR logic operation on bits 1, 2, 4 and 159 together and shifting the entire 160 bits to the right by 1 bit using the result of the XOR logic operation as an input to b ₁₅₉ . The new R is returned on the next RND call. Note that the time required to return 0 from the TST is the same for all bad inputs, and the attacker cannot learn anything about invalid inputs.

The TST command is implemented in the following steps.

尚、ステップ７では、単純にそのままＲを進めることができない。なぜならば、Ｒはフラッシュメモリであり、セットされたビットが０になるためには消去しなければならないからである。電源が、古いＲの値を消去した後で新しいＲの値が書き込まれる前に、ステップ７の間に認証チップから取り外された場合、Ｒは消去されるが再プログラミングされない。したがって、ＩｓＴｒｕｓｔｅｄ＝１でありながら、Ｒ＝０であるという状況が生まれ、この状況が起こり得るのは攻撃者が原因となる場合だけである。ステップ４はこの事象を検出し、攻撃が検出された場合に動作する。この問題は、Ｒのための第２の１６０ビットフラッシュレジスタと有効性ビットとを設け、新しい値がロードされた後にトグル切替することで防ぐことができる。これは、スペースの理由で本実施形態には組み込まれていないが、チップ空間がそれを許容するならば、補助的な１６０ビットフラッシュレジスタがこの目的のために有用であろう。

In step 7, R cannot be simply advanced as it is. This is because R is a flash memory and must be erased in order for the set bit to become zero. If the power supply is removed from the authentication chip during step 7 after erasing the old R value and before the new R value is written, R is erased but not reprogrammed. Therefore, the situation that IsTrusted = 1 but R = 0 is born, and this situation can only occur if the attacker is responsible. Step 4 detects this event and operates when an attack is detected. This problem can be prevented by providing a second 160-bit flash register for R and a validity bit and toggling after a new value is loaded. This is not incorporated into this embodiment for space reasons, but if the chip space allows it, an auxiliary 160-bit flash register would be useful for this purpose.

WR-write (WRITE)
Input: M _new = [256 bits]
Output: None Change content: M.

The WR (Write) command is used to update the writable part of M that stores the authentication chip state data. The WR command itself is not secure. The WR command is followed by an authenticated M read (by the RD command) to ensure that the changes have been made as specified. The WR command is invoked by passing the WR command opcode followed by new 256-bit data to be written to M. Since the authentication chip is serial, the new value of M must be transferred one bit at a time. The bit order is from LSB to MSB of each command component. Therefore, the WR command includes bits 0-2 of the WR opcode, followed by bits 0-15 of M [0], bits 0-15 of M [1], and bits 0-15 of M [15]. is there. A total of 259 bits are transferred. The WR command can only be used when SIWritten = 1, ie, indicating that K ₁ , K ₂ , R have been set up by the SSI command (if SIWritten is 0, K ₁ , K ₂ , R Is not set up yet, the CLR command is called instead). The possibility of writing to a particular M [n] is governed by the corresponding access mode bit as stored in the AccessMode register. The AccessMode bit can be set using a SAM command. When writing a new value to M [n], it must be taken into account that M [n] is a flash memory. After erasing all bits of M [n], the appropriate bit must be set. Since the two steps are performed in separate cycles, the possibility of attack remains unprotected. An attacker can remove the power supply after erasure, but before programming a new value. However, attackers do not benefit from doing this:
Changing the read / write M [n] to 0 by this means has no benefit. This is because an attacker can write arbitrary values using the WR command;
By changing the read-only M [n] to 0 by this means, an additional text pair can be known (where M [n] is 0 instead of the original value). When using M [n] values in the future, they are already 0, so no information is given;
When the decrement only M [n] is changed to 0, the time until the consumable is used up is only accelerated. This does not give the attacker any new information that can be gained by using consumables.

The WR command is implemented in the following steps.

ＳＡＭ−アクセスモードセット（ＳＥＴ ＡＣＣＥＳＳ ＭＯＤＥ）
入力：ＡｃｃｅｓｓＭｏｄｅ_ｎｅｗ＝［３２ビット］
出力：ＡｃｃｅｓｓＭｏｄｅ＝［３２ビット］
変更内容：ＡｃｃｅｓｓＭｏｄｅ。

SAM-access mode set (SET ACCESS MODE)
Input: AccessMode _new = [32 bits]
Output: AccessMode = [32 bits]
Details of change: AccessMode.

The SAM (Set Access Mode) command is used to set the 32 bits of the AccessMode register and is only valid for use with the consumables authentication chip (when the IsTrusted flag = 0). The SAM command is invoked by passing a SAM command opcode followed by a 32-bit value that is used to set a bit in the AccessMode register. Since the authentication chip is serial, data must be transferred one bit at a time. The bit order is from LSB to MSB of each command component. Thus, the SAM command is bits 0-2 of the SAM opcode, followed by bits 0-31 of bits set in AccessMode. A total of 35 bits are transferred. The AccessMode register is cleared to 0 only when the CLR command is executed. Since an access mode of 00 indicates an RW (read / write) access mode, not setting the AccessMode bit after CLR means that all M can read and write. Only the SAM command sets the bits in the AccessMode register. Therefore, the client changes the access mode bit for M [n] from RW to RO (read only) by setting the appropriate bit in the 32-bit word and calling SAM with the 32-bit value as the input parameter. Is possible. This allows programming of access mode bits at various points in time, perhaps at various stages of the manufacturing process. For example, read-only random data can be written during the initial key programming stage, while a second programming stage for items such as consumable serial numbers is allowed.

Since the SAM command only sets a bit, the effect is that the access mode bit corresponding to M [n] can be advanced from RW to either MSR, NMSR or RO. Note that the MSR access mode can be changed to RO, but this does not help the attacker. This is because the authentication of M after writing to an illegally processed authentication chip detects that the writing was unsuccessful and stops the operation. The setting of the bit corresponds to the way that the flash memory works best. The only way to clear a bit in the AccessMode register is, for example, to use the CLR command to change the decrement only M [n] to read / write. The CLR command not only erases (clears) the AccessMode register, but also clears the key and all M. In this way, the AccessMode [n] bit corresponding to M [n] can be effectively changed only once between CLR commands. The SAM command returns the new value of the AccessMode register (after the appropriate bits are set by the input parameters). By calling the SAM with the input parameter set to 0, the AccessMode is not changed, so the current value of the AccessMode is returned to the caller.

The SAM command is implemented in the following steps.

ＧＩＴ−ＩｓＴｒｕｓｔｅｄを取得（ＧＥＴ ＩＳＴＲＵＳＴＥＤ）
入力：無し
出力：ＩｓＴｒｕｓｔｅｄ＝［１ビット］
変更内容：無し。

Acquired GIT-IsTrusted (GET ISTRUSTED)
Input: None Output: IsTrusted = [1 bit]
Changes: None.

The GIT (Get IsTrusted) command is used to read the current value of the IsTrusted bit on the authentication chip. If the returned bit is 1, the authentication chip is a trusted system authentication chip. If the bit is 0, the authentication chip is a consumable authentication chip, the GIT command contains only the GIT command opcode, and since the authentication chip is serial, it must be transferred bit by bit at the same time. The order is from LSB to MSB of each command component, so the SAM command is transmitted as bits 0-2 of the SAM opcode, a total of 3 bits are transferred, the GIT command is implemented in the following steps: .

ＳＭＴ−ＭｉｎＴｉｃｋｓセット（ＳＥＴ ＭＩＮＴＩＣＫＳ）
入力：ＭｉｎＴｉｃｋｓ_ｎｅｗ＝［３２ビット］
出力：無し
変更内容：ＭｉｎＴｉｃｋｓ。

SMT-MinTicks set (SET MINTICKS)
Input: MinTicks _new = [32 bits]
Output: None Changed content: MinTicks.

The SMT (Set MinTicks) command is used to set the MinTicks register bit and determine the number of MinTicks that should elapse between calls to TST and RD. The SMT command is invoked by passing an SMT command opcode followed by a 32-bit value used to set a bit in the MinTicks register. Since the authentication chip is serial, data must be transferred one bit at a time. The bit order is from LSB to MSB of each command component. Thus, the SMT command is bits 0-2 of the SMT opcode, followed by bits 0-31 of the bits set in MinTicks. A total of 35 bits are transferred. The MinTicks register is cleared to 0 only when the CLR command is executed. A value of 0 indicates that ticks do not have to pass between keyed function calls. Thus, this function is called at a frequency that the clock speed limit hardware allows the chip to operate.

Since the SMT command only sets the bit, the effect is to allow the client to set the value and only increase the time delay if further calls are made. Setting a bit that has already been set has no effect, and only setting a bit that has been cleared serves to further slow down the chip. The setting of the bit corresponds to the way that the flash memory works best. The only way to clear a bit in the MinTicks register, for example, is to use the CLR command to change the value of 10 ticks to a value of 4 ticks. However, the CLR command not only clears the MinTicks register to 0, but also clears all keys and M. Therefore it is useless for attackers. In this way, the MinTicks register can be effectively changed only once between CLR commands.

The SMT command is implemented in the following steps.

認証チップのプログラミング
認証チップは物理的に安全な環境内で論理的に安全な情報でプログラミングしなければならない。その結果として、プログラミング手続は論理的セキュリティと物理的セキュリティの両方を取り扱う。論理的セキュリティは、Ｋ_１、Ｋ_２、Ｒ及び乱数Ｍ［ｎ］の値が、コンピュータではなく、物理的にランダムなプロセスによって生成されることを保証するプロセスである。また、それは、チップの部品のプログラミングされる順序が最も論理的に安全であることを保証するプロセスでもある。物理的セキュリティは、プログラミングステーションが物理的に安全であり、したがって、鍵生成ステージ期間と鍵の記憶の耐用期間の両方で、Ｋ_１及びＫ_２が秘匿されることを保証するプロセスである。その上、プログラミングステーションは、鍵を獲得又は破壊する物理的攻撃に対抗しなければならない。認証チップは、Ｋ_１及びＫ_２が秘匿されることを保証するための固有のセキュリティメカニズムを備えているが、プログラミングステーションはＫ_１及びＫ_２を安全に保持しなければならない。

Authentication Chip Programming The authentication chip must be programmed with logically secure information in a physically secure environment. As a result, the programming procedure deals with both logical security and physical security. Logical security is a process that ensures that the values of K ₁ , K ₂ , R, and random number M [n] are generated by a physically random process, not a computer. It is also a process that ensures that the order in which the chip components are programmed is the most logically secure. Physical security is a process that ensures that the programming station is physically secure, and therefore K ₁ and K ₂ are kept secret during both the key generation stage period and the lifetime of key storage. In addition, the programming station must counter physical attacks that acquire or destroy keys. Although the authentication chip has its own security mechanism to ensure that K ₁ and K ₂ are concealed, the programming station must keep K ₁ and K ₂ secure.

Overview After manufacture, the authentication chip needs to be programmed before it can be used. For all chips, the values of K ₁ and K ₂ must be determined. If the chip is intended for a system authentication chip, the initial value of R must be determined. If the chip is intended for a consumable authentication chip, R must be set to 0 and the initial values for M and AccessMode must also be set. Therefore, the following stages are confirmed:
Determine the iteration between the system and consumables;
Determine keys for systems and consumables;
Determine MinTicks for the system and consumables;
Program keys, random sheets, MinTicks and unused M;
Program state data and access mode.

When no consumable or system is needed, the attached authentication chip can be reused. This can be easily achieved by starting again at stage 4 and reprogramming the chip. Each stage is described in the following sections.

Stage 0: The manufacture of the authentication chip does not require any special security. At the manufacturing stage, there is no secret information programmed into the chip. The algorithm and chip process are not special. A standard flash process is used. The theft of the authentication chip between the chip manufacturer and the programming station only gives the replica manufacturer a blank chip. This hardly compromises the sale of the authentication chip and nothing is authenticated by the authentication chip. Since the programming station is the only mechanism that contains the consumable and system product key, the replica manufacturer cannot program the chip with the correct key. Replica manufacturers can program blank chips for their own systems and consumables, but it would be difficult to sell without seeing those items. In addition, it would be difficult to base the business on a single theft.

Stage 1: Determining what is the interaction between the system and the consumable, and what is the consumable, should be determined before the authentication chip can be programmed. A determination needs to be made as to which consumables are available on which systems. This is because only the connected system and consumables must share the same key information. They should also share a state data usage mechanism, even if interpretation of some of the state data is undetermined. A simple example is the case of a car and a car key. The car itself is a system, and the car key is a consumable item. Each car has several car keys, and each key stores the same key information as a particular car. However, each car (system) will have a different key (shared with its car key). This is because the car key of one car is not required to function in another car. Another example is the case of a copier that requires a specific toner cartridge. Simply put, the copier is a system and the toner cut ridge is a consumable item. However, it is necessary to determine what compatibility is available between the cartridge and the copier. This determination is conventionally made by physical packaging of the toner cartridge, and certain cartridges may or may not be compatible with the new model copier based on the design decision of the new model copier. When an authentication chip is used, the components to be linked must share the same key information.

Furthermore, consumables require different division methods for M (status data) for each type. The usage of M varies from application to application, and the allocation method for M [n] and AccessMode [n] is similar:
Define consumable status data for specific applications;
Reserve some M [n] registers for future use (if necessary). Set them to 0 and set them to read-only. Values can be tested on the system for compatibility;
Set the remaining M [n] registers (at least one, but not M [15]) read-only, and make the contents of each M [n] completely random. This makes it more difficult for replicators to attack authentication keys;
The following example shows how the state data is organized.

Example 1
Assume a car with a car key attached. A 16-bit key number is sufficient to uniquely identify a car key for a given car. The 256 bits of M are divided as follows.

車製造者が全ての車の全ての論理的鍵を保持するならば、車キーを紛失したときに、新しい物理的な車キーを製造することは些細な問題である。新しい車キーは、Ｍ［０］に新しい鍵番号を格納するが、その車の認証チップと同じＫ_１及びＫ_２を含む。車システムは、特定の鍵番号を無効にすることができる（例えば、鍵を紛失したとき）。このようなシステムは、鍵０（マスター鍵）を最初に挿入し、次に全ての有効な鍵を挿入し、次に、鍵０をもう一度挿入することを要求するかもしれない。これらの有効な鍵だけが車と連動するようになる。最悪のケースでは、例えば、全ての車キーを紛失すると、その車に対する新しい論理的な鍵の組を生成し、必要に応じて、関連した物理的な車キーを生成することができる。カーエンジン番号は、その鍵を特定の車に結びつけるために使用される。将来用データは、レンタル情報、例えば、ドライバ／貸出者の詳細情報のような事項を格納する。

If the car manufacturer holds all the logical keys for all cars, it is a trivial problem to make a new physical car key when the car key is lost. The new car key stores the new key number in M [0] but includes the same K ₁ and K ₂ as the car's authentication chip. The car system can invalidate a specific key number (eg, when the key is lost). Such a system may require that key 0 (master key) be inserted first, then all valid keys inserted, and then key 0 inserted again. Only these valid keys will work with the car. In the worst case, for example, if all car keys are lost, a new logical key pair for the car can be generated, and the associated physical car key can be generated if necessary. The car engine number is used to bind the key to a specific car. The future data stores items such as rental information, for example, detailed information of the driver / lender.

Example 2
Consider an example of a copier image unit that must be replaced after every 100,000 copies. 32 bits are required to store the remaining number of pages. The 256 bits of M are divided as follows.

１００００回のコピー毎に交換しなければならない低品質画像ユニットが製作されたとき、３２ビットのページカウントは、既存コピー機との互換性のためにそのまま使用される。これにより、数タイプの消耗品を同じシステムで使用できるようになる。

When a low quality image unit is produced that must be replaced after every 10,000 copies, the 32-bit page count is used as is for compatibility with existing copiers. This allows several types of consumables to be used in the same system.

Example 3
Consider an example of a polaroid camera that stores 25 photos. In order to store the number of remaining pictures, only a 16-bit countdown is required. The 256 bits of M are divided as follows.

Ｍ［２］の残り写真数は、同じカメラシステムと使用するための多数の消耗品タイプの構築を可能にさせる。例えば、写真数３６の新しい消耗品は簡単にプログラムできる。カメラの発表後、２年が経過した場合を考えると、新しいタイプのカメラが発表されている。それは、古い消耗品を使用することが可能であるが、新しいフィルムタイプも処理できる。Ｍ［３］はフィルムタイプを定義するため使用できる。古いフィルムタイプは０であり、新しいフィルムタイプは他の新しい値である。新しいシステムはこれを利用することができる。元のシステムはＭ［３］に非零値を検出し、新しいフィルムタイプとの非互換性を認識する。新しいシステムは、Ｍ［３］の値を理解し、適切に応じる。古い消耗品との互換性を保つため、新しい消耗品及びシステムは、古いものと同じ鍵情報をもつことが必要である。新しいシステム及びその専用消耗品と完全に断ち切るためには、新しい鍵セットが必要になるであろう。

The remaining number of photos in M [2] allows the construction of multiple consumable types for use with the same camera system. For example, a new consumable with 36 photos can be easily programmed. Considering the case where two years have passed since the announcement of the camera, a new type of camera has been announced. It can use old consumables but can also handle new film types. M [3] can be used to define the film type. The old film type is 0 and the new film type is another new value. New systems can take advantage of this. The original system detects a non-zero value in M [3] and recognizes the incompatibility with the new film type. The new system understands the value of M [3] and responds appropriately. To maintain compatibility with old consumables, new consumables and systems need to have the same key information as the old ones. A new key set will be required to completely disconnect from the new system and its dedicated consumables.

Example 4
Consider an example of a printer consumable that stores three inks, cyan, magenta, and yellow. Each ink amount should be decremented separately. The 256 bits of M are divided as follows.

ステージ２：システム及び消耗品の鍵の決定
どのシステムとどの消耗品が同じ鍵を共有すべきであるかに関する決定がなされると、それらの鍵を定義しなければならない。したがって、Ｋ_１及びＫ_２の値が決定される。殆どのケースでは、Ｋ_１及びＫ_２は全体で１回だけ生成される。連動すべき（現在及び将来の）全てのシステム及び消耗品は、同じ鍵Ｋ_１及びＫ_２の値をもつ。したがって、Ｋ_１及びＫ_２は秘匿されなければならない。なぜならば、システム／消耗品の組み合わせに対するセキュリティメカニズム全体は、鍵が見られてしまうと、役に立たなくなるからである。鍵が暴露されると、その損害は、システム及び消耗品の数、並びに、それらを新たな暴露されていない鍵で再プログラミングすることの容易さに依存する。トナーカートリッジを用いるコピー機の場合、最悪のケースでは、複製品製造者は固有の認証チップを製造し（又は、さらに悪い場合には、それらを購入し）、判明した鍵でチップをプログラムし、固有の消耗品に挿入する。車キー付きの車のケースでは、各車は、異なる鍵の組をもつ。このことにより、２通りの一般的なシナリオが生まれる。第１のシナリオでは、車及び車キーがそれらの鍵を用いてプログラムされた後、Ｋ_１及びＫ_２が削除され、それらの値の記録が保持されず、即ち、Ｋ_１及びＫ_２を暴露する方法がない。しかし、その車のための予備の車キーは、車の認証チップを再プログラミングしない限り製作できない。第２のシナリオでは、車製造者はＫ_１及びＫ_２を保持し、その車のための新しいキーを製作可能である。Ｋ_１及びＫ_２の暴露は、ある者が特定の車専用の車キーを製作できることを意味する。

Stage 2: Determining System and Consumable Keys Once a decision is made regarding which systems and which consumables should share the same key, those keys must be defined. Therefore, the values of _{K 1} and _{K 2} are determined. In most cases, K ₁ and K ₂ are generated only once in total. All systems and consumables (current and future) to be linked have the same key K ₁ and K ₂ values. Thus, _{K 1} and _{K 2} must be concealed. This is because the entire security mechanism for the system / consumable combination is useless once the key is seen. When a key is exposed, the damage depends on the number of systems and consumables and the ease with which they can be reprogrammed with a new unexposed key. In the case of a copier using a toner cartridge, in the worst case, the replica manufacturer manufactures unique authentication chips (or, in the worst case, purchases them), programs the chip with the found key, Insert into unique consumables. In the case of cars with car keys, each car has a different set of keys. This creates two general scenarios. In the first scenario, after cars and car keys are programmed with those keys, K ₁ and K ₂ are deleted and no record of their values is kept, ie exposing K ₁ and K ₂ There is no way to do it. However, a spare car key for that car cannot be made without reprogramming the car's authentication chip. In the second scenario, the car manufacturer holds K ₁ and K ₂ and can make a new key for the car. Exposure of K ₁ and K _2, means that a person can be fabricated the car keys in a particular car only.

Thus, the key and random data used in the authentication chip should be generated by non-deterministic means (fully computer generated pseudo-random numbers cannot be used because it uses the generator seed Because knowing is deterministic in that all future numbers can be obtained). K ₁ and K ₂ should be generated by a physically random process, not a computer. However, random bit generators based on random natural sources are susceptible to external causes and are prone to failure. It is inevitable that such devices should be tested periodically for statistical randomness.

A simple but useful random number generator is the Lavarand® system from SGI. This generator uses a digital camera that captures six rubber lamps at intervals of a few minutes. Lava lamps include disordered turbulent systems. The resulting digital image is supplied to the SHA-1 implementation, which generates a 7-way hash and produces a 160-bit value for every 7 bytes from the digitized image. The total of seven sets of 160 bits is 140 bytes. This 140 byte value is supplied to the BBS generator to determine the start position of the output bitstream. The output 160 bits from the BBS is a key or an authentication chip 53.

An extreme example of a non-deterministic random process is toss a coin 160 times for K ₁ and toss 160 times for K _{2 in} a clean room. Depending on the front or back, 1 or 0 is entered into the panel of the key programmer device. This process is performed with several observers (for verification) silent (someone may have a hidden microphone). Importantly, secure data entry and storage is not so easy. The physical security of the key programmer device and the accompanying programming station requires its own complete documentation. After the keys K ₁ and K ₂ are determined, they must be retained as long as the authentication chip needs to use the key. In the first car / car key scenario, K ₁ and K ₂ are destroyed after a single system chip and several consumable chips are programmed. In the case of a copier / toner cartridge, K ₁ and K ₂ must be held only while the toner cartridge is useful for the copier. They should be kept secret.

Stage 3: Determining MinTicks for system and consumables The value of MinTicks depends on the operating clock speed of the authentication chip (system specific) and the rationale for the reasonable time between RD or TST function calls (application specific) . The interval between single ticks depends on the operating clock speed. This is the maximum value of the input clock speed and the clock limit hardware of the authentication chip. For example, the authentication chip clock limit hardware is set to 10 MHz (which cannot be changed), but the input clock is 1 MHz. In this case, the value of 1 tick is 1 MHz instead of 10 MHz. If the input clock is 20 MHz instead of 1 MHz, the value of 1 tick is relative to 10 MHz (because the clock speed is limited to 10 MHz). Once the interval of one tick is known, the MinTicks value can be set. The value of MinTicks is the number of MinTicks that are required to elapse between calls to the keyed RD or ST function. Assume that the input clock speed matches the maximum clock speed of 10 MHz. If the minimum value of 1 second is determined during the TST call, the value of MinTicks is set to 10000000. A value such as 2 seconds is quite reasonable for a system such as a printer (one page is generated every 2 to 3 seconds with one authentication per page).

Stage 4: The key, random seed, final tick and unused M program authentication chips are in unknown state after manufacture. Alternatively, the authentication chip has already been used with one consumable and must be reprogrammed for use with another consumable. Each verification chip is cleared and programmed with a new key and new status data. Authentication chip clearing and subsequent programming is performed in a secure programming station environment.

If the trusted system authentication chip programming chip is a trusted system chip, an R seed value is generated. It is a random number extracted from a physically random process and is not zero. The following tasks are performed in a secure programming environment in the following order:
Reset Reset chip CLR []
Load R
Load physical random data into R (160-bit register) SSI [K ₁ , K ₂ , R]
SMT [MinTicks _System ].

Here, the authentication chip is ready for insertion into the system. The authentication chip is fully programmed. If the system authentication chip is stolen at this point, the replica manufacturer can use R, F _K1 [to go into a known text attack on K ₁ or a partial text attack on K ₂ . R] can be used to generate pairs. This is no different from purchasing multiple systems, each containing a trusted authentication chip. Security depends on the strength of the authentication protocol and the randomness of K ₁ and K ₂ .

If the programming chip of the untrusted consumable authentication chip is an untrusted consumable authentication chip, the programming is slightly different from the programming of the trusted system authentication chip. Initially, the seed value for R is zero. It does additional programming for the values of M and AccessMode. Unused M [n] is programmed with 0, and random M [n] is programmed with random data. The following tasks are performed in a secure programming environment in the following order:
Reset Reset chip CLR []
Load R
Load 0 into R (160-bit register) SSI [K ₁ , K ₂ , R]
Load X
Load 0 to X (256-bit register) Set Set physical random data to X corresponding to appropriate M [n] WR [X]
Load Y
Load 0 to Y (32-bit register) Set Set read-only access mode to Y corresponding to appropriate M [n] SAM [Y]
SMT [MinTicks _Consumable ].

Here, the untrusted consumable chip is ready to be programmed with general status data. If the authentication chip is stolen at this point, the attacker can perform a restricted choice text attack. In the best situation, part of M is read-only (0 and random data) and the rest of M is fully selected by the attacker (using the WR command). Numerous RD calls by an attacker obtain F _K2 [M | R] for a limited M. In the worst case, M is completely selected by the attacker (because all 256 bits are used for state data). However, in either case, the attacker cannot select the value of R. This is because R is supplied from the system authentication chip by a call to RND. The only way to acquire the selected R is a brute force attack. Note that if stages 4 and 5 are performed in the same programming station (preferred and ideal situation), the authentication chip cannot be removed between stages. In this case, there is no possibility that the authentication chip is stolen at this point. The decision to program the authentication chip once or twice is up to the system / consumable manufacturer's request.

Stage 5: Status Data and Access Mode Program This stage is only required for consumable authentication chips. This is because the M and AccessMode registers cannot be changed on the system authentication chip. Unused and random values of M [n] are already programmed in stage 4. The remaining state data needs to be programmed and the associated AccessMode value is set. Note that the speed of this stage is limited by the value stored in the MinTicks register. This stage is separated from stage 4 in view of the physical location difference or time difference where stage 4 and stage 5 are executed. Ideally, stages 4 and 5 are executed simultaneously in the same programming station. Stage 4 generates valid authentication chips but does not load them with initial state values (other than 0). This can make chip programming consistent with the operation of the consumable production line. Stage 5 can be executed multiple times, each time a different state data value and access mode value can be set, but only once, all remaining state data values are set, all remaining More likely to set the access mode value. For example, the production line is set up, and the batch number and serial number of the authentication chip are generated according to the physical consumable to be produced. This can be quite difficult if the state data is loaded at a physically different factory.

The stage 5 process first performs a test to ensure that the chip is a consumable chip, which includes an RD that collects data from the authentication chip, and then the WR of the initial data value. In addition, a SAM is performed to permanently set new data values. A summary of these steps follows:
IsTrusted = GIT [];
If IsTrusted is set, exit with error (wrong kind of chip!);
Call RND of valid system chip to get valid input pair;
Call the RD of the chip to be programmed, passing a valid input pair;
Load the result from the RD of the authentication chip into X (256 bit register);
Call the TST of the active system chip to ensure that X and the consumable chip are valid;
If TST returns 0, exit with an error (wrong consumable chip for the system).
Set the initial state value in the X bit;
WR [X];
Load 0 into Y (32-bit register);
Set the Y bit corresponding to the access mode for the new status value;
SAM [Y].

Of course, verification (stages 1 to 7) may not be performed if stages 4 and 5 are performed one after another at the same programming station. However, this verification should be done in all other situations where stage 5 is executed as a separate programming process from stage 4. If these authentication chips are stolen here, they are already programmed for use with specific consumables. An attacker can attach a stolen chip to a replica consumable. Such theft limits the number of replicated products to the number of stolen chips. A single theft does not provide enough supplies for a replica manufacturer to do a cost-effective business. Another use of this chip is that an attacker does not have to purchase the same number of consumables with an authentication chip in order to enter a partial selection text attack or brute force attack. Even with such an attack, there are no special security flaws in the key.

The circuit of the manufacturing authentication chip must be able to counter physical attacks. An overview of the manufacturing implementation guidelines is given, followed by a description of the physical defense specifications of the chip (in order of attack).

Manufacturing guidelines The following are general guidelines for mounting certified chips for manufacturing:
Standard process minimum size (as much as possible)
Clock filter Noise generator Tamper prevention and detection circuit Protected memory with tamper detection Boot circuit for loading program code Key data path FETs for data path Specially mounted FET layer for data connection Under-over-power detection unit test circuit None.

Standard process authentication chips can be implemented with standard manufacturing processes (such as flash). This is clear to allow a very wide selection of manufacturing locations and is necessary to meet the cost savings of using technology that works properly. Note that the standard process allows physical protection mechanisms.

The minimum size authentication chip 53 must have a low manufacturing cost so that it can be incorporated as an authentication mechanism for low cost consumables. Therefore, it is desirable to keep the chip size as small as possible. Each authentication chip requires a 802-bit non-volatile memory. Moreover, the storage capacity required for the optimized HMAC-SHA1 is 1024 bits. The rest of the chip (chosen to implement the state machine, processor, CPU, or protocol 3) is minimal so that the number of transistors is minimized and the cost per chip is minimized To the limit. The area of the circuit that holds the secret key information, and the area of the circuit that may leak information about the sub-h, key should be minimized (see the non-flushing CMOS below for special data paths).

The clock filter authentication chip circuit is designed to operate in a specific clock speed range. Since the user provides the clock signal directly, the attacker can cause the circuit to become messed up at a particular point in the process. An example of this is when a high clock speed (higher than the circuit was designed for) prevents the XOR from operating properly and the first of the two inputs is returned as is. Such a transient failure attack is very effective in restoring the secret key information. What should be learned is that the input clock signal is not reliable. The input clock signal is unreliable and should be restricted to operate up to the maximum frequency. This can be achieved in various ways. One way to filter the clock signal is to use an edge detection unit that passes “edge on” to the delay, and then allows the input clock signal to pass through the delay. FIG. 174 is an explanatory diagram of the clock signal flow in the clock filter. The delay is set so that the maximum clock speed is a specific frequency (eg, 4 MHz). Note that this delay is not programmable and is fixed. The filtered clock signal is further divided internally as needed.

Noise Generator Each authentication chip includes a noise generator that generates continuous circuit noise. The noise interferes with other electromagnetic radiation from the normal operation of the chip and adds noise to the I _dd signal. The installation of the noise generator does not cause problems with the authentication chip due to the length of the emission wavelength. The noise generator generates electronic noise, multiple state changes per clock cycle, and is used as a pseudo-random bit source for tamper prevention and detection circuits. A simple implementation of a noise generator is a 64-bit LFSR seeded with a non-zero number. The clock used by the noise generator runs at the maximum clock rate of the chip to generate as much noise as possible.

A set of tamper prevention and detection circuits is required for testing and preventing physical attacks on the authentication chip. However, those who are actually detected as attacks may not be intentional physical attacks. Therefore, the following two types of attacks within the authentication chip:
It is important to distinguish when you can be sure of the appearance of a physical attack.

The two types of detection differ in that they are executed as a result of detection. In the first case, that is, when it can be certain that a physical attack appears, the erasure of flash memory key information is an appropriate operation. If the second case is uncertain that a physical attack will appear, there may be some mistake. Although the operation should be performed, it is not an operation for deleting the secret key information. A suitable action to take in the second case is a chip reset. If what is detected is an attack that permanently damages the chip, then the same condition appears again and the chip resets again. On the other hand, if what is detected is part of the normal operating environment of the chip, this reset does not compromise the key.

A good example of an event where the circuit cannot understand the situation is a power supply malfunction. This malfunction may be a deliberate attack that attempts to reveal information about the key. However, it may be a misconnection or simply the start of a power down sequence. Therefore, it is best to only reset the chip and not erase the key. If the chip is powered down, nothing is lost. If the system has failed, the consumer will repair the system by repeated resets. In both cases, the consumables are intact. A good example of an event that allows the circuit to understand the situation is the disconnection of a data line in the chip. If this attack is detected in any way, this is limited to a chip failure (manufacturing defect) or the result of the attack. In any case, erasure of secret information is a step that should be selected for the purpose.

As a result, each authentication chip includes two tamper detection lines as shown. One is for clear attack and the other is for attack potential. A number of tamper detection test units are connected to these tamper detection lines, and each unit tests various forms of tamper rings. Moreover, it is desirable to ensure that the tamper detection line and the circuit itself cannot be tampered with.

One end of the tamper detection line is a source of pseudo-random bits (clocking faster than a typical operating circuit). The noise generator circuit described above is a suitable source. The generated bits go through two types of paths, one carrying the original data and the other carrying the inversion of the original data. The wiring for carrying these bits is in a layer above a general chip circuit (for example, a memory, a key operation circuit, etc.). The wiring also covers the random bit generator. Bits are recombined at multiple locations by XOR gates. If the bits are different (should be), 1 is output and used by a particular unit (eg, each output bit from the memory read is ANDed with this bit value). The line eventually gathers in the flash memory erase circuit, where a complete erase is triggered by a zero from XOR. A number of triggers are attached to the line, each detecting a physical attack on the chip. Each trigger has a large nMOS transistor connected to GND. The tamper detection line physically passes through this nMOS transistor. When the test fails, the trigger sets the tamper detection line to zero. The XOR test fails either on this clock cycle or the next clock cycle (on average) and resets or erases the chip. FIG. 175 is an explanatory diagram of the basic principle of a tamper detection circuit using a test and an XOR connected to either the erase circuit or the reset circuit.

The tamper detection line passes through the drain of the output transistor of each Test, as shown in the large nMOS transistor layout of FIG. The tamper detection line cannot be cut. This is because it stops the flow of 1s and 0s from random sources. As a result, the XOR test fails. Since the tamper detection line physically passes through each test, it is impossible to remove a specific test without cutting the tamper detection line. Importantly, XOR takes values from various locations along the tamper detection line to reduce the chance of attack. FIG. 177 illustrates a situation where multiple XORs are taken from tamper detection lines used in various parts of the chip. Each of these XORs can be regarded as generating a chip normal bit ChipOK used in each unit or subunit.

As a sample application, an OK bit is provided in each unit, and this is ANDed with the ChipOK bit given in each cycle. The OK bit is loaded with 1 at reset. If the OK bit is 0, the unit has failed (failed) until the next reset. If the tamper detection line functions correctly, the chip resets or erases all key information. If the reset circuit or erasure circuit is destroyed, this unit will not function, thus preventing an attacker. The reset and erase lines and associated circuit destinations are very sensitive to the situation. It should be protected in a manner very similar to the individual tamper test. If the attacker can easily cut the wiring connected to the reset circuit, it does not make sense to generate a reset pulse. The actual implementation largely depends on the items to be cleared by reset and the method of clearing those items.

Finally, FIG. 178 shows how the tamper line protects the chip noise generator circuit. The generator and NOT gate are at the same height, while the tamper detection line passes higher than the generator.

It is not sufficient to store the protected memory secret information or program code with tamper detection only in the flash memory. Flash memory and RAM must be protected from attackers who modify (or set) specific bits of program code or key information. The mechanism used is compliant with the mechanism used in the tamper detection circuit (described above). The first part of the solution is to ensure that the tamper detection line passes just above each flash or RAM bit. This ensures that an attacker cannot probe the contents of flash or RAM. The breakage of the protection part of the wiring is the cutting of the tamper detection line. This damage sets the erase signal so that the contents of the memory are erased. High frequency noise on the tamper detection line also hinders passive observation.

The second part of the solution for flash is to use multilevel data storage, and only a subset of those multilevels are used for valid bit representation. In general, when multilevel flash storage is used, a single floating gate holds more than one bit. For example, a 4-voltage state transistor can represent 2 bits. If the minimum and maximum voltages are represented by 00 and 11, respectively, the two intermediate voltages represent 01 and 10. In the case of an authentication chip, two intermediate voltages are used to represent a single bit, and two at both ends are considered invalid. When an attacker attempts to force the state of a bit in one direction or another by closing or disconnecting the gate circuit, an invalid voltage (and therefore an invalid state) appears.

The second part of the solution for RAM is to use parity bits. The data portion of the register can be examined using parity bits (which do not match after an attack). Bits coming from flash and RAM are verified by a number of test units (one per bit) connected to a common tamper detection line. The tamper detection circuit is the first circuit through which data passes (thus preventing the attacker from cutting the data line).

Boot circuit program code loading program code should be held in multi-level flash, not ROM. This is because ROM is easy to change in a way that cannot be tested. Therefore, the boot mechanism is necessary for taking the program into the flash memory (the flash memory is in an intermediate state after manufacture). The boot circuit is not placed in the ROM and a small state machine is sufficient. Otherwise, the boot code is changed in such a way that it cannot be detected. The boot circuit erases all the flash memory, confirms that the erase has worked correctly, and then loads the program code. The flash memory must be erased before loading the program. Otherwise, the attacker can place the chip in the boot state and load a program that has just extracted the existing key. Before loading the new program code, the state machine performs a check to ensure that all flash memory has been cleared (to ensure that the attacker has not cut the erase line). The loading of the program code is done by a secure programming station before secret information (such as a key) is loaded.

Special FET Implementation for Key Data Path A typical FET implementation situation for a CMOS inverter (including a pMOS transistor combined with an nMOS transistor) is shown in FIG. During the transition, there is a short period in which both the nMOS transistor and the pMOS transistor have intermediate resistance values. The resulting power supply-ground short circuit temporarily increases the current and in effect accounts for the majority of the current consumed by the CMOS device. A small amount of infrared light is emitted during the short circuit and can be observed through the silicon substrate (silicon transmits infrared). A small amount of light is emitted during charging and discharging of the transistor gate capacitance and the transmission line capacitance.

In the case of a circuit that operates secret key information, such information must be kept secret. Alternative non-flashing CMOS implementations are used for all data paths that operate on a key or a value that is partially calculated based on that key. The use of two non-overlapping clocks φ1 and φ2 provides a non-flashing mechanism. φ1 is connected to the second gates of all nMOS transistors, and φ2 is connected to the second gates of all pMOS transistors. Transitions occur when combined with a clock. Since φ1 and φ2 do not overlap, the pMOS and nMOS transistors do not take intermediate resistance values at the same time. The configuration is shown in FIG.

Finally, conventional CMOS inverters can be placed in the vicinity of important non-flashing CMOS components. These inverters take their input signals from the tamper detection line. Since tamper detection lines operate many times faster than normal operating circuits, the net effect is a burst of light that occurs at a high rate near each location of the non-flashing CMOS component. Since bright light interferes with observation of weak light in the vicinity, the observer cannot detect what switching operation is being performed on the chip itself. These conventional CMOS inverters effectively increase the amount of circuit noise, lower the signal-to-noise ratio, and mask useful EMI.

The use of non-flushing CMOS presents a number of side effects:
The effective chip speed is reduced by twice the clock rise time per clock cycle. This is not a problem for authentication chips;
The amount of current drawn by the non-flashing CMOS is reduced (because no short circuit appears). However, this is offset by the use of a normal CMOS inverter;
This is because the clock wiring increases the chip area, and in particular, multiple versions of φ1 and φ2 are required to take into account various levels of propagation. The estimated amount of chip area is twice that of normal packaging;
The design of the non-flash area of the authentication chip is somewhat more complicated than when using a normal CMOS design. In particular, standard cell components cannot be used and the area is fully customized. This is not a problem when the authentication chip is small, particularly when the entire chip does not have to be protected by this method.

Wiring in the polysilicon layer where possible Where possible, the wiring through which the key or secret data flows should be made in the polysilicon layer. If necessary, it is provided on the metal 1, but it should never be provided on the uppermost metal layer (including the tamper detection line).

Under / over power detection unit Each authentication chip needs a under / over power detection unit to prevent power supply attacks. The under / over power detection unit detects power supply malfunctions and tests the power supply level against a voltage reference to ensure that the power supply level is within a certain tolerance. This unit includes a single voltage reference and two comparators. The under / over power detection unit may be connected to a reset tamper detection line, thereby causing a reset when triggered. A side effect of the under / over power detection unit is that when the voltage drops during power down, a reset is triggered and the working register is erased.

Test hardware on an authentication chip without a test circuit can easily introduce vulnerabilities. As a result, the authentication chip should not include a BIST or scan path. Thus, the authentication chip is made testable using the external test vector. This can be realized because the authentication chip is not complicated.

Read ROM This attack relies on the key being stored in an addressable ROM. This attack is irrelevant because each authentication chip stores its authentication key in an internal flash memory rather than an addressable ROM.

Chip Reverse Engineering Chip reverse engineering is useful only if the security of the authentication is derived from an algorithm. However, the authentication chip of the present invention relies on a secret key, not on algorithm confidentiality. On the other hand, the authentication algorithm of the present invention is open to the public, and in any case, it is considered that the mass production consumable attacker may have obtained a detailed design drawing inside the chip. In view of these factors, reverse engineering of the chip itself is not a threat as opposed to stored data.

Unauthorized use of the authentication process There are several possible forms of this attack, and each form varies in the degree of success. In all cases, the replica manufacturer will be able to obtain both system and consumable designs. Some attackers try to build a chip that tricks the system into returning a valid code, rather than generating an authentication code. This attack is not feasible for two reasons. The first reason is that the system authentication chip and the consumables authentication chip are programmed separately even if they are physically the same. In particular, the RD opcode and the RND opcode are the same, and the WR opcode and the TST opcode are the same. The system authentication chip cannot execute the RD command. This is because all calls are interpreted as calls to RND, not RD. The second reason that this attack fails is that separate serial data lines are provided from the system to the system authentication chip and the consumable authentication chip. As a result, neither chip can see the content sent to or received from the other. If the attacker builds a duplicate chip that ignores the WR command (decrements the remaining consumables), protocol 3 ensures that subsequent RDs detect that no WR has appeared. The system therefore stops using consumables and prevents attackers. This is also the case when an attacker simulates a loss of contact before authentication, and authentication is not performed, so consumables are not used. Therefore, the attacker is limited to a range in which each system is changed so that a copy consumable is accepted.

The simplest way to change the system is to replace the system's authentication chip with a chip that only reports success on every call to the TST. This is hampered by a system that calls the TST many times for each authentication, gives a false value the first few times, and requests notification of failure from the TST. The last call to TST is expected to succeed. The number of false calls to TST may be determined from RD or a portion of the result returned from the system clock. Unfortunately, since the attacker can simply rewrite the system, the new system copy authentication chip 53 can monitor the results returned from the consumable chip or clock. The replication system authentication chip only returns success when the monitored value is given to its TST function. The replication consumable can return any value as a hash result of the RD, and the replication system chip reveals that the value is valid. Therefore, it does not make sense for the system to call the system authentication chip many times. This is because the rewrite attack works correctly only on rewired systems, not on all systems. A similar form of attack on the system is a system ROM replacement. The ROM program code can be changed so that authentication is not performed. I can't do anything about this. This is because the system is in the hands of consumers. Of course, this voids any warranty, but consumers would consider it worth changing if the copy consumables are very cheap and easier to obtain than the real ones. .

The system / consumable manufacturer must therefore determine how likely this type of attack can occur. Such surveys naturally include the pricing structure of the system and consumables, the frequency of system services, the consumer's benefit when making physical changes, and the places where consumers go to perform the changes. The limiting case of changing the system is when the replica manufacturer provides a complete replica system that requires replica consumables. This is just competition or patent infringement. In any case, this is outside the scope of the authentication chip and depends on the technology and service being replicated.

In order to observe the direct chip operation of the conventional probing chip operation, the chip must be operating. However, the tamper prevention and detection circuit covers the area of the chip that processes or holds the key. It is impossible to see these areas through the tamper-proof line. The attacker cannot simply slice the chip through the anti-tamper layer. This is because doing so breaks the tamper detection line and erases all keys when the power is turned on. Simply destroying the erase circuit is not sufficient. This is because a large number of ChipOK bits (all 0 at this time) supplied to a large number of units in the authentication chip stop the function of the normal operation circuit of the chip. In order to set up a chip for an attack, the attacker needs to delete the tamper detection line, stop erasing the flash memory, and somehow rewrite the component that trusted the ChipOK line. Even if all this can be done, slicing the chip to this level will almost destroy the charge pattern in the non-volatile memory holding the key, making the process useless.

If the non-volatile memory direct view authentication chip is sliced to expose the flash memory floating gate and does not discharge them, the key will probably be viewable directly using STM or SKM. However, it is probably impossible to slice the chip to this level without discharging the gate. Wet etching, plasma etching, ion milling, or chemical mechanical polishing almost certainly releases the small charge present on the floating gate. This is true for normal flash memory, but is more certain for multilevel flash memory.

All areas of the circuit that manipulates the monitoring secret key information of the optical burst caused by the state change are realized in the non-flashing CMOS. This prevents the emission of most light bursts. Conventional CMOS inverters placed in close proximity to non-flashing CMOS obscure the slight radiation caused by capacitor charging and discharging. Since the inverter is connected to the tamper detection circuit, the inverter changes its state many times (at a high clock rate) for every non-flashing CMOS state change.

EMI Monitoring The noise generator described above induces circuit noise. Noise interferes with other electromagnetic radiation from the normal operation of the chip, obscuring significant readouts of internal data transfers.

Monitoring I _dd variation A solution to this type of attack is to reduce the signal-to-noise ratio in the I _dd signal. This is achieved by increasing the amount of noise in the circuit and decreasing the amount of signal. The noise generator circuit (which also serves as a defense against EMI attacks) causes enough state changes every cycle to obscure important information in I _dd . Moreover, a special non-flashing CMOS implementation of the data path that propagates the key of the chip prevents current from flowing when a state change occurs. This has the advantage of reducing the amount of signal.

Differential Fault Analysis Differential fault bit errors are captured rather than targeted by ionization, microwave radiation, or environmental stress. The most likely effect of this type of attack is a flash memory change (causing an invalid state) or a RAM change (bad parity). Invalid states and bad parity are detected by the tamper detection circuit and cause key erasure. Since the tamper detection line covers the key operation circuit, any error taken into the key operation circuit is reflected by an error in the tamper detection line. If the tamper detection line is affected, the chip continually resets or simply erases the key at power-up, making the attack useless. Rather than relying on untargeted attacks and not wanting “just the right part of the chip to be affected in the right way”, the attacker wants to bring the targeted failure (overwrite attack, gate destruction, etc.) It is better to try it.

The clock malfunction attack (above) eliminates the feasibility of a clock malfunction attack.

The power attack (above) under / over power detection unit excludes the possibility of power attack.

The ROM overwrite authentication chip stores the program code, key, and secret information in the flash memory, not in the ROM. Therefore, this attack is not possible.

The EEPROM / flash change authentication chip stores the program code, key, and secret information in the flash memory. However, the flash memory is covered by two tamper prevention and detection lines. If any one of these lines breaks (in the process of breaking the gate), the attack is detected at power-up and each chip resets (continuously) or erases the key from flash memory . However, even if an attacker can somehow access a bit in flash memory and destroy or omit the gate holding a particular bit, this forces that bit to Set to no charge or full charge. Both of these are invalid states for use of the multi-level flash memory of the authentication chip (only two intermediate states are valid). When the data value is transferred from the flash, the detection circuit triggers the erase tamper detection line, thereby erasing the remaining portion of the flash memory and resetting the chip. Therefore, the EEPROM / flash memory modification attack is useless.

Gate destruction attacks Gate destruction attacks rely on the attacker's ability to change a single gate and expose information to the chip during operation. However, the circuit that manipulates the secret information is covered by one of the two tamper prevention and detection lines. If any one of these lines breaks (in the process of breaking the gate), the attack is detected at power-up and each chip resets (continuously) or erases the key from flash memory . To participate in this type of attack, the attacker must first reverse engineer the chip to determine which gate should be targeted. After the target gate location is determined, the attacker must break the protected tamper detection line, stop erasing the flash memory, and somehow rewrite the component that trusted the ChipOK line. is there. Circuit rewriting cannot be performed without slicing the chip, and even if it can be performed, the operation of slicing the chip to this level almost destroys the charge pattern in the nonvolatile memory holding the key, and Will be useless.

Overwrite attacks Overwrite attacks rely on being able to set individual bits of a key without knowing their previous values. It relies on probing the chip like a conventional probing attack and destroying the gate like a gate breaking attack. Both of these attacks are unsuccessful because they use tamper prevention and detection circuitry and the ChipOK line (as described in the respective sections). However, even if an attacker can somehow access a bit in flash memory and destroy or omit the gate holding a particular bit, this forces that bit to Set to no charge or full charge. Both of these are invalid states for use of the multi-level flash memory of the authentication chip (only two intermediate states are valid). When the data value is transferred from the flash, the detection circuit triggers the erase tamper detection line, thereby erasing the remaining portion of the flash memory and resetting the chip. Similarly, the erase tamper detection line is triggered by a parity check of the tampered value read from the RAM. Thus, overwriting attacks are useless.

The working register or RAM in the residual memory attack authentication chip holds a part of the authentication key when the power is removed. The working register and RAM keep the information for a certain time after the power supply is removed. When the chip is sliced and the register / RAM gates are exposed without discharging them, the data will probably be directly viewable using STM. The first defense can be understood from the description of the defense against the power supply malfunction attack described above. When the power supply is removed, all registers and RAM are cleared, just as the reset condition causes the memory to be cleared. The chances of this attack being successful are less than the chances of reading the flash memory. RAM charge is (essentially) lost more easily than flash memory. Chip slicing to reveal the RAM ensures that the charge (even if not simply lost due to the memory not being refreshed and the time it takes to perform the slicing). To lose. Therefore, this attack is useless.

There are distinct phases in the lifetime of a chip theft attack authentication chip. A chip can be stolen at any of the following stages:
After manufacturing, before programming the key;
After programming the key and before programming the state data;
After programming the status data, before inserting it into a consumable or system;
After being inserted into the system or consumables.

The theft between the chip manufacture and the programming station only gives the replica manufacturer a blank chip. This only jeopardizes the sale of the authentication chip and nothing is authenticated by the authentication chip. The programming station is the only mechanism with consumables and system product keys, and the replica manufacturer cannot program the chip with the correct key. While replica manufacturers can program blank chips for their own systems and consumables, it would be difficult to sell without seeing those items. The second form of theft can only occur in situations where the authentication chip goes through two or more separate programming phases. This can happen but is unlikely. In any case, the worst situation is when no state data is programmed and all M are read / write. In this case, the attacker can go on an adaptive selected text attack against the chip. The HMAC-SHA1 algorithm can counter this attack. A third form of theft takes place between the programming station and the installation shop. The authentication chip is already programmed for a specific system or a specific consumable. As the only use of these chips, thieves house them in a replication system or replication consumables. The replication system is misplaced and the replication system will not even require the authentication chip 53. For replica consumables, such theft limits the number of replicas to the number of stolen chips. A single theft does not provide enough supplies for a replica manufacturer to do a cost-effective business. The final form of theft is theft of the system or the consumable itself. If theft occurs on the manufacturer side, the physical security protocol must be strengthened. If theft takes place elsewhere, it is only a matter of the item owner and the police or insurance company. The security mechanism used by the authentication chip assumes that consumables and systems are under the control of the general public. As a result, even if they are stolen, there is no significant impact on the security of the keys.

Authentication chip design The authentication chip has physical and logical external interfaces. The physical interface defines how the authentication chip can be connected to the physical system, and the logical interface defines how the system can communicate with the authentication chip.

The physical interface authentication chip is a small 4-pin CMOS package (the actual internal size is about 0.30 mm ² using a 0.25 μm flash process). The 4th pin is a ground (GND), a clock (CLK), a power (Power), and data (Data). Power is a nominal voltage. If the voltage fluctuates more than a certain amount from the nominal voltage, the chip resets. The recommended clock speed is 4 to 10 MHz. Internal circuitry filters the clock signal to ensure that the safe maximum clock speed is not exceeded. Data is transmitted and received one bit at a time along the serial data line. The chip performs a reset at power-up and power-down. In addition, the tamper detection and prevention circuit in the chip causes the chip to reset or erase the flash memory (depending on the detected attack) if an attack is detected. Special programming mode is enabled by holding the CLK voltage at a specific level. This is further explained in the next section.

The logical interface authentication chip has two operation modes, a normal mode and a programming mode. The reason that the two modes are required is that the operating program code is stored in flash memory instead of ROM (for safety reasons). The programming mode is used for test purposes after manufacture and is packed with operating program code, while the normal mode is used for subsequent chip use.

Programming Mode Programming mode is enabled by maintaining a specific voltage on the CLK line for a predetermined time. When the chip enters programming mode, all flash memory is erased (including all secret key information and program code). The authentication chip then verifies the erasure. If the erase is successful, the authentication chip receives 384 bytes of data corresponding to the new program code. The bytes are transferred in the order of byte ₀ to byte ₃₈₃ . Bits are transferred in the order of bit ₀ to bit ₇ . When all 384 bytes of program code are loaded, the authentication chip hangs. If the erase is not successful, the authentication chip hangs without loading any data into the flash memory. After the chip is programmed, the chip can be restarted. When the chip is reset with normal voltage on the CLK line, it enters normal mode.

The normal mode authentication chip is in the normal mode when not in the programming mode. When the authentication chip starts up in normal mode (for example, power-up reset), the authentication chip then executes the program stored in the program code area of the flash memory. The program code incorporates a communication mechanism between the system and the authentication chip, receives commands and data from the system, and generates output values. Since the authentication chip performs serial communication, bits are transferred one bit at a time. The system communicates with the authentication chip via a simple set of operation commands. Each command is defined by a 3-bit opcode. The interpretation of the opcode depends on the current values of the IsTrusted and IsWriten bits.

The following operations are defined:

Ｏｐ＝オペコード、Ｔ＝ＩｓＴｒｕｓｔｅｄ値、Ｗ＝ＩｓＷｒｉｔｔｅｎ値、
Ｍｎ＝ニューモニック、［ｎ］＝パラメータに必要なビット数。

Op = opcode, T = IsTrusted value, W = IsWritten value,
Mn = Pneumonic, [n] = Number of bits required for the parameter.

Commands not defined in this table are interpreted as NOP (no operation). Examples include opcodes 110 and 111 (which are independent of IsTrusted value or IsWriteten value) and opcodes other than SSI when IsWriteten = 0. Note that the RD and RND opcodes are the same, and the WR and TST opcodes are the same. The actual command that is moved after receiving the opcode depends on the current value of the IsTrusted bit (as long as IsWritten is 1). Where the IsTrusted bit is cleared, the RD and WR functions are called. When the IsTrusted bit is set, the RND and TST functions are called. The two sets of commands are mutually exclusive between a trusted authentication chip and an untrusted authentication chip. To execute a command on the authentication chip, a client (such as a system) sends a command opcode followed by the input parameters required for that opcode. The opcode is sent from the least significant bit to the most significant bit. For example, to send an SSI command, bits 1, 0 and 0 are sent in this order. Each input parameter is sent in the same way, first from the least significant bit, and finally to the most significant bit. The return value is read in this way, the least significant bit is read first, and the most significant bit is read last. The client must know how many bits to acquire.

In some cases, the output bits from one chip command are supplied as input bits to another chip command. An example of this is an RND command and an RD command. The output bits from the call to RND on the trusted authentication chip may not be retained by the system. Instead, the system can transfer the output bit as is to the input of the RD command of the authentication chip that is not trusted. The description of each command indicates whether that is the case. Each command is discussed in detail in subsequent sections. Some algorithms are specially designed because resident registers are held in flash memory.

The memory in the register authentication chip includes a non-volatile memory for storing variables required by the authentication protocol. The following non-volatile memory (flash) variables are defined:

アーキテクチャ概要
このセクション・章は、認証チップの必要な機能を実装することができる専用ＣＰＵの上位定義を与える。このＣＰＵは汎用ＣＰＵではないことに注意する必要がある。認証ロジックを実装するためオーダーメードされている。ＷＲＩＴＥ、ＴＳＴ、ＲＮＤ等のような認証チップのユーザが目にする認証コマンドは、ＣＰＵ命令セットでかかれた小型プログラムとして全て実装される。ＣＰＵは、３２ビットのアキュムレータ（殆どのオペレーションで使用される）と、多数のレジスタと、を含む。ＣＰＵは、認証ロジックを実装するため専用に仕立てられた８ビット命令を含む。各８ビット命令は、典型的に、４ビットのオペコードと、４ビットのオペランドと、により構成される。

Architecture Overview This section / chapter gives a high-level definition of a dedicated CPU that can implement the necessary functions of the authentication chip. It should be noted that this CPU is not a general purpose CPU. Made to order to implement authentication logic. The authentication commands seen by the user of the authentication chip such as WRITE, TST, RND, etc. are all implemented as a small program written in the CPU instruction set. The CPU includes a 32-bit accumulator (used for most operations) and a number of registers. The CPU includes 8-bit instructions tailored specifically to implement the authentication logic. Each 8-bit instruction is typically composed of a 4-bit opcode and a 4-bit operand.

The operating speed internal clock frequency limiter unit prevents the chip from operating at a speed faster than a predetermined frequency. This frequency is built into the chip during manufacture and cannot be changed. It is recommended that this frequency be about 4 to 10 MHz.

The composition and block diagram authentication chip includes the following components:

図１８１は認証チップの概略ブロック図である。タンパー防止及び検出回路は図示されない。雑音発生器、過小過大電力検出ユニット、及びプログラミングモード検出ユニットは、タンパー防止及び検出回路に接続され、残りのユニットへは接続されない。

FIG. 181 is a schematic block diagram of the authentication chip. The tamper prevention and detection circuit is not shown. The noise generator, under / over power detection unit, and programming mode detection unit are connected to the tamper prevention and detection circuit and not to the rest of the units.

Memory Map FIG. 182 shows an example of a memory map. The authentication chip does not have an external memory, but has an internal memory. The internal memory is addressed by 9 bits and is 32 or 8 bits wide (depending on the address). A 32-bit wide memory is used to hold non-volatile data, variables used for HAMC-SHA1, and constants. 8-bit wide memory is used to hold the program and the various jump tables used by the program. Address partitioning (including spare memory ranges) is designed to optimize address generation and decoding.

Constant FIG. 183 is an explanatory diagram of an example of a constant memory map. The constant area is composed of 32-bit constants. These are simple constants (such as all 32 bits are 0 and all 32 bits are 1), constants used by the HAMC algorithm, and constants y ₀₋ required for use in the SHA-1 algorithm. ₃ and h _0-4 . These values are not affected by the reset. The only opcode that uses constants is LDK. In this case, the operands and memory placement are closely related to minimize address generation and decoding.

RAM
FIG. 184 is an explanatory diagram of an example of a RAM memory map. The RAM area contains 32 32-bit registers with parity check that are necessary for the general purpose function of the authentication chip, but only during chip operation. The RAM is a volatile memory and the value is lost once the power is removed. In practice, the memory maintains its value for some time after power-down (due to memory remaining), but is not considered available after power-up. This is a security issue described in other sections of this document. RAM is a variable used in the HMAC-SHA1 algorithm: AE, temporary variable T, space H for 160-bit working hash value, for temporary storage of hash results (required by HMAC) Space B160 and space X for 512 bits of extended hashing memory are included. All RAM variables are cleared to 0 after reset, but the program code must not rely on this. The operation codes that use the RAM address are LD, ST, ADD, LOG, XOR, and PRL. In all cases, operands and memory placement are closely related to minimize address generation and decoding (multiword variables are stored most significant word first).

Flash Memory-Variable FIG. 185 is an explanatory diagram of an example of a flash memory variable memory map. The flash memory area stores non-volatile information in the authentication chip. The flash memory maintains its value after the power is removed and can be expected to remain unchanged the next time the power is turned on. The non-volatile information held in the multi-state flash memory includes two 160-bit keys (K ₁ and K ₂ ), a current random value (R), state data (M), a MinTicks value (MT), It includes an AccessMode value (AM), an IsWritten (ISW) flag, and an IsTrusted (IST) flag. The flash value does not change upon reset, but is cleared (to 0) when entering programming mode. Operations that use the flash address are LD, ST, ADD, PRL, ROR, CLR, and SET. In all cases, the operands and memory placement are closely related to minimize address generation and decoding. Multiword variables K ₁ , K ₂ and M are stored first from the most significant word due to addressing requirements. The addressing scheme used is a base address offset with an index starting with N and ending with 0. In this way, M _N is the word which is first accessed, M ₀ is the last 32-bit word to be accessed by the loop. The multiword variable R is stored from the least significant word first to simplify LFSR generation using the same indexing scheme.

Flash Memory-Program FIG. 186 is an explanatory diagram of an example of a program memory map of the flash memory. The second multi-state flash memory area is 384 × 8 bits. This area stores an address table for JSR, JSI, and TBR instructions, an offset for DBR commands, a constant, and the program itself. The flash memory is not changed by reset, but is cleared (to 0) when entering the programming mode. Once in programming mode, the 8-bit flash memory can be loaded with a new set of 384 bytes. When this is finished, the chip is reset and normal chip operation is performed.

Register A number of registers are defined in the authentication chip. They are used as temporary storage during function execution. Some are for arithmetic functions, some are used for counting and indexing, and others are used for serial I / O. These registers need not be held in non-volatile (flash) memory. They can be read or written without an erase cycle (unlike flash memory). Temporary storage registers that store secret information must be protected from physical attacks by tamper prevention and detection circuitry and parity checks. All registers are cleared to 0 after reset. However, the program code should not assume a specific state, but should set the register values appropriately. Note that these registers may not include the various OK bits defined for tamper protection and detection circuitry. The OK bits are scattered across the various units and are set to 1 after reset.

The cycle 1 bit cycle value determines whether the CPU is a fetch cycle (0) or an execution cycle (1). The cycle is actually derived from a 1-bit register that holds the previous cycle value. Cycles cannot be accessed directly from the instruction set. It is for internal registers only.

A 9-bit program counter array (PCA) is defined which is 6 levels deep. It is indexed by a 3 bit stack pointer (SP). The current program counter (PC) stores the address of the currently executing instruction and is effectively PCA [SP]. In addition, a 9-bit Adr register is defined to store the current memory-based resolved address (for indexed or indirect memory access). The PCA, SP and Adr registers are not directly accessible from the instruction set. These are dedicated to internal registers.

CMD
The 8-bit CMD register is used to hold the command currently being executed. The CMD register register is not directly accessible from the instruction set, but is dedicated to internal registers.

The accumulator and Z flag accumulator are 32-bit general purpose registers. This is a register used as one of the inputs to arithmetic operations and used to transfer information between memory registers. The Z register is a 1-bit flag and is updated when the accumulator is written. The Z register stores the zero value of the accumulator. If the last value written to the accumulator is zero, Z = 1, and if the last value written is non-zero, Z = 0. Both accumulators and Z registers are directly accessible from the instruction set.

Counter A number of dedicated counter / index registers are defined.

これらの全てのカウンタ・レジスタは、命令セットから直接アクセス可能である。これらに特定の値をロードするための専用命令が存在し、別の命令は、それらをデクリメント又はインクリメントし、或いは、特定のカウンタが零であるかどうかに応じて分岐する。２個の専用フラグ（レジスタではない）がＣ１及びＣ２に関連付けられ、これらのフラグはＣ１又はＣ２の零性を保持する。これらのフラグは、ループ制御のために使用され、以下に列挙される。これれのフラグはレジスタではないが、レジスタと同様にテストできる。

All these counter registers are directly accessible from the instruction set. There are dedicated instructions for loading a particular value, and another instruction decrements or increments them or branches depending on whether a particular counter is zero. Two dedicated flags (not registers) are associated with C1 and C2, and these flags retain the zeroness of C1 or C2. These flags are used for loop control and are listed below. These flags are not registers, but can be tested in the same way as registers.

フラグ
ＣＰＵ動作モードに対応する多数の１ビットフラグが定義される。

A number of 1-bit flags corresponding to the flag CPU operation mode are defined.

これらの全ての１ビットフラグは命令セットから直接アクセス可能である。専用命令がこれらのフラグをセット及びクリアするため設けられる。

All these 1-bit flags are directly accessible from the instruction set. Dedicated instructions are provided to set and clear these flags.

Registers used for write integrity

Ｉ／Ｏのため使用されるレジスタ
４個の１ビットレジスタがクライアント（システム）と認証チップの間の通信のため定義される。これらのレジスタは、InBit（入力ビット）、InBitValid（入力ビット有効）、OutBit（出力ビット）、及びOutBitValid（出力ビット有効）である。InBit及びInBitValidは、クライアントがコマンド及びデータを認証チップへ送るための手段を提供する。OutBit及びOutBitValidはクライアントが認証チップから情報を受けるための手段を提供する。クライアントは、コマンド及びパラメータビットを、同時に１ビットずつ認証チップへ送る。認証チップはスレーブ装置であるため、認証チップの観点では：
InBitからの読み出しは、InBitValidがクリアされている間はハングする。InBitValidは、クライアントが次の入力ビットをInBitに書き込むまでクリアされたままである。InBitの読み出しはInBitValidビットをクリアし、次のInBitをクライアントから読み出せるようになる。クライアントは、InBitValidビットがクリアされるまでビットを認証チップに書き込めない；
OutBitへの書き込みは、OutBitValidがセットされている間はハングする。OutBitValidは、クライアントがOutBitからビットを読み出すまでセットされたままである。OutBitの書き込みはOutBitValidビットをセットし、次のOutBitをクライアントが読み出せるようにする。クライアントは、OutBitValidビットがセットされるまで認証チップからビットを読み出せない。

Registers used for I / O Four 1-bit registers are defined for communication between the client (system) and the authentication chip. These registers are InBit (input bit), InBitValid (input bit valid), OutBit (output bit), and OutBitValid (output bit valid). InBit and InBitValid provide a means for clients to send commands and data to the authentication chip. OutBit and OutBitValid provide a means for the client to receive information from the authentication chip. The client sends the command and parameter bits to the authentication chip one bit at a time. Since the authentication chip is a slave device, from the authentication chip perspective:
Reading from InBit hangs while InBitValid is cleared. InBitValid remains cleared until the client writes the next input bit to InBit. Reading InBit clears the InBitValid bit so that the next InBit can be read from the client. The client cannot write bits to the authentication chip until the InBitValid bit is cleared;
Writing to OutBit hangs while OutBitValid is set. OutBitValid remains set until the client reads a bit from OutBit. Writing OutBit sets the OutBitValid bit so that the client can read the next OutBit. The client cannot read the bit from the authentication chip until the OutBitValid bit is set.

Register used for timing access One 32-bit register is defined for use as a timer. An MTR (MinTicksRemaining) register is decremented each time an instruction is executed. When the MTR register goes to 0, it stays at zero. MTR is associated with a 1-bit flag MTRZ, and MTRZ stores MTR zeroness. If MTRZ is 1, the MTR register is zero. If MTRZ is 0, the MTR register is not yet zero. The MTR always starts with the MinTicks value (after reset or after a specific key access function) and eventually decreases to zero. MTR can be set and MTRZ can be tested with a dedicated instruction, but the value of MTR cannot be read directly by the instruction.

Register Summary The following table summarizes all temporary registers (in order of register name). The table lists the register name, size (in bits), and where the specified register is located.

命令セット
ＣＰＵは、認証ロジックを実現するため特別に用意された８ビット命令で動作する。８ビット命令の大半は、４ビットオペコードと、４ビットオペランドと、により構成される。上位４ビットはオペコードを含み、下位４ビットがオペランドを含む。

The instruction set CPU operates with a specially prepared 8-bit instruction for realizing the authentication logic. Most 8-bit instructions are composed of a 4-bit opcode and a 4-bit operand. The upper 4 bits contain an opcode and the lower 4 bits contain an operand.

Opcodes and operands (summary)
The opcodes are summarized in the following table.

以下の表は、どのオペランドがどのオペコードと共に使用できるかを要約した表である。この表は、オペコードのニューモニックによってアルファベット順に並べられている。各オペランドのバイナリ値は、その次の表に示されている。

The following table summarizes which operands can be used with which opcodes. The table is arranged in alphabetical order by the mnemonic of the opcode. The binary value of each operand is shown in the next table.

以下のオペランド表は、４ビットオペランドの解釈を示し、全ての４ビットが直接解釈のため使用されている。

The following operand table shows the interpretation of 4-bit operands, and all 4 bits are used for direct interpretation.

以下の命令は、オペランドの最上位ビットに基づいて選択を行う。

The following instructions make a selection based on the most significant bit of the operand.

オペランドの下位３ビットは、オフセット（ＤＢＲ，ＴＢＲ）か、専用テーブル（ＳＣ）からの値であるか、ＬＯＧの場合のように、それらは論理演算のための第２の入力を選択する。解釈は、ＡＤＤ、ＬＤ、及びＳＴオペコードの解釈と一致する。

The lower 3 bits of the operand are an offset (DBR, TBR), a value from a dedicated table (SC), or they select a second input for a logical operation, as in LOG. The interpretation is consistent with the interpretation of ADD, LD, and ST opcodes.

ＡＤＤ−アキュムレータへの加算
ニューモニック： ＡＤＤ
オペコード： １０００
用法： ＡＤＤ
値。

ADD-Addition mnemonic to accumulator: ADD
Opcode: 1000
Usage: ADD
value.

The ADD instruction adds the specified operand to the accumulator by modulo 2 ³² addition. Operands are A, B, C, D, E, T, AM, MT, AE [C1], H [C1], B160 [C1], R [C1], K [C1], M [C1], or One of X [N4]. Also, the Z flag is set during this calculation depending on whether the loaded value is zero or non-zero.

CLR-Clear Bit Numonic: CLR
Opcode: 0110
Usage: CLR
Flag / register.

The CLR instruction causes the designated internal flag or flash memory register to be cleared. In the case of flash memory, the CLR instruction takes some time, but the next instruction is deferred until the flash memory is finished erasing. The registers that can be cleared are WE and K2MX. The flash memories that can be cleared are R, M [C1], Group1, and Group2. Group 1 is an IST and ISW flag. When they are cleared, the only valid upper level command is the SSI instruction. Group2 is an MT, AM, K1, and K2 register. R is erased separately. This is because R is updated on every call to TST. Also, M is erased by the index mechanism and individual parts of M can be updated. There is also a corresponding SET instruction.

DBR-decrement and branch mnemonics: DBR
Opcode: 0001
Usage: DBR
Counter, offset.

This instruction provides a mechanism for building a simple loop. The high bit of the operand selects the C1 test or the C2 test (two counters). If the specified counter is non-zero, the counter is decremented and a predetermined offset value (sign extension) is added to the PC. If the specified counter is zero, the counter is decremented and processing continues with PC + 1. An 8-entry offset table is stored at address 011000000 (the 64th entry in the program memory). The 8 bits of the offset are treated as a signed number. In this way, 0xFF is treated as -1 and 0x01 is treated as +1. Typically, the value is negative for use in the loop.

JSI-Subroutine Indirect Jump Pneumonic: JSI
Opcode: 01001
Usage: JSI (Acc).

The JSI instruction allows a jump to a subroutine depending on the current value of the accumulator. This instruction pushes the current PC onto the stack and loads the PC with a new value. The upper 8 bits of the new PC are loaded from jump table 2 (the offset is given by the lower 5 bits of the accumulator) and the least significant bit of the PC is cleared to zero. In this way, all subroutines must start with even addresses. The stack provides 6 execution levels (5 subroutine depths). The programmer is responsible for ensuring that this depth is not exceeded, or that the return value is overwritten (because the stack wraps).

JSR-Subroutine Jump Pneumonic: JSR
Opcode: 001
Usage: JSR
offset.

The JSR instruction provides the most common usage of subroutine structures. This instruction pushes the current PC onto the stack and loads the PC with a new value. The upper 8 bits of the new PC are given from address table 1 and the offset to the table is given by a 5-bit operand (32 addresses can be implemented). The least significant bit of the PC is cleared to 0. In this way, all subroutines must start with even addresses. The stack provides 6 execution levels (5 subroutine depths). The programmer is responsible for ensuring that this depth is not exceeded, or that the return value is overwritten (because the stack wraps).

LD-Accumulator Road Pneumonics: LD
Opcode: 1011
Usage: LD
value.

The LD instruction loads into the accumulator from the specified operand. Operands are A, B, C, D, E, T, AM, MT, AE [C1], H [C1], B160 [C1], R [C1], K [C1], M [C1], or One of X [N4]. Also, the Z flag is set during this calculation depending on whether the loaded value is zero or non-zero.

LDK-constant road mnemonic: LDK
Opcode: 1110
Usage: LDK
constant.

The LDK instruction loads the constant specified in the accumulator. The constants are the 32-bit values needed for HMAC-SHA1, and all 0s and all 1s that are most useful for general purpose processing. Therefore,
0x00000000
0x36363636
0x5C5C5C5C
0xFFFFFFFF
Or, there are options from the h and y constant tables indexed by C1. The constant table of h and y holds a 32-bit tabular constant necessary for HMAC-SHA1. Also, the Z flag is set during this calculation depending on whether the loaded value is zero or non-zero.

LOG-Logic Pneumonic: LOG
Opcode: 1001
Usage: LOG
Calculated value.

The LOG instruction performs a 32-bit bit logical operation on a value designated as an accumulator. The two operations supported by the LOG instruction are AND and OR. Bit NOT and XOR operations are supported by the XOR instruction. The 32-bit value that is ANDed or ORed with the accumulator is one of A, B, C, D, E, T, MT, and AM. Also, the Z flag is set during this operation depending on whether the resulting 32-bit value (loaded into the accumulator) is zero or non-zero.

ROR-Right Rotating Numonic: ROR
Opcode: 1100
Usage: ROR
value.

The ROR instruction provides a way to rotate the accumulator right by a set number of bits. The bit appearing at the top of the accumulator (becomes bit 31) may be the value of bit 0 before the accumulator, or it may be from an external 1-bit flag (such as a flag or serial input connection). Bits overflowing by rotation may be output from the serial connection or may be combined with an external flag. Permitted operands are InBit, OutBit, LFSR, RLFSR, IST, ISW, MTRZ, 1, 2, 27, and 31. Also, the Z flag is set during this operation depending on whether the resulting 32-bit value (loaded into the accumulator) is zero or non-zero. In the simplest form, the operand for the ROR instruction is one of 1, 2, 27, 31 indicating the number of bit positions that rotate the accumulator. For these operands, there is no external input or output, and the accumulator bits are simply rotated to the right. If the operand is IST, ISW and MTRZ, the appropriate flag is transferred to the most significant bit of the accumulator. The remaining part of the accumulator is shifted right by one bit position (bit 31 becomes bit 30, etc.) and the least significant bit of the accumulator is shifted and overflows. If the operand is InBit, the next serial input bit is transferred to the most significant bit of the accumulator. Next, the InBitValid bit is cleared. If no input bits are available from the client, execution is suspended until an input bit appears. The remaining part of the accumulator is shifted right by one bit position (bit 31 becomes bit 30, etc.) and the least significant bit of the accumulator is shifted and overflows.

If the operand is OutBit, the accumulator is shifted right by one bit position. The bit sent out by shifting from bit 0 is stored in the OutBit flag, and the OutBitValid flag is set. It is therefore ready for reading by the client. If the OutBitValid flag is already set, execution of the instruction is postponed until the OutBit is read by the client (and the OutBitValid flag is cleared). The new bit shifted to bit 31 is considered garbage (actually the value of the current InBit register). Finally, the RB and XBR operands allow for the implementation of LFSR and double precision shift registers. In the case of RB, the bit sent out by shift (formally bit 0) is written to the RTMP register. The current register of the RTMP register becomes the new bit 31 of the accumulator. Several RORs with several 32-bit values
Executing the RB command realizes double-precision right rotation / shift. XRB operates in the same way as RB, and the current value of the RTMP register becomes accumulator on new bit 31. However, in the case of an XRB instruction, a bit that is formally known to be bit 0 does not simply replace RTMP (for the RB instruction). Instead, the bit is XORed and RTMPed, and the result is stored in RTMP. Thereby, a long LFSR required by the authentication protocol can be realized.

RPL-bit replacement mnemonic: RPL
Operand: 1101
Usage: ROR
value.

The RPL instruction was designed to incorporate higher level WRITE commands into the authentication chip. This instruction intends to replace the upper 16 bits of the accumulator with the value that was ultimately written to the M array (depending on the access mode value). This instruction takes three operands: Init, MHI and MLO. The Init operand sets all internal flags and prepares the RPL unit in the ALU for subsequent processing. The accumulator is transferred to the internal AccessMode register. When the accumulator executes the WRITE command, the accumulator
It should be loaded from the AM flash memory location prior to the call to Init, and should be loaded with 0 when executing a TST command. The accumulator remains unchanged. The MHI and MLO operands refer to whether the upper 16 bits or lower 16 bits of M [C1] are (always) used for comparison with the upper 16 bits of the accumulator. Each executed MHI and MLO instruction uses the following 2 bits from the initialized AccessMode value. The first MHI or MLO run uses the least significant 2 bits, the next run uses the next 2 bits, and so on.

Return mnemonic from RTS-subroutine: RTS
Opcode: 01000
Usage: RTS.

The RTS instruction resumes execution at the instruction after the JSR or JSI instruction executed immediately before. Therefore, this is called return from subroutine. In practice, this instruction pulls the saved PC off the stack, adds 1, and resumes execution from the resulting address. Six execution levels (five subroutines) are provided, but it is the programmer's responsibility to match each of the JSR and JSI instructions with the RTS. The RTS executed without the preceding JSR starts execution from the address pulled from the stack.

SC-Counter Set Numonic: SC
Opcode: 0101
Usage: SC
Counter value.

The SC instruction is used to load a specific value into the counter. The operand determines which of the counters C1 and C2 is loaded. The value to be loaded is one of 2, 3, 4, 7, 10, 15, 19 and 31. The counter value is used for loop processing and index processing. Both C1 and C2 are used for the loop structure (when combined with the DBR instruction), while only C1 can be used to index the 32-bit portion of a double precision variable.

SET-bit set mnemonic: SET
Opcode: 0111
Usage: SET
Flag / register.

The SET instruction allows a specific flag or flash memory to be set. There is also a corresponding CLR instruction. Each of the WE operand and the K2MX operand sets a designated flag for later processing. Each of the IST and ISW operands sets the appropriate bit in the flash memory, while the MTR operand transfers the current value of the accumulator to the MTR register. SET
The Nx command loads the following constants from N1 to N4.

参照される各初期Ｘ［Ｎ_ｎ］は、最適化されたＳＨＡ−１アルゴリズムのインデックスＮ１からＮ４に対する初期状態と一致することに注意すべきである。各インデックス値Ｎｎがデクリメントするとき、有効なＸ［Ｎ］がインクリメントする。なぜならば、Ｘ個のワードは、最上位ビットの方からメモリに格納されるからである。

Note that each initial X [N _n ] referenced corresponds to the initial state for the optimized SHA-1 algorithm indices N1 through N4. As each index value Nn decrements, valid X [N] increments. This is because X words are stored in the memory from the most significant bit.

ST-Accumulator Preservation Numonic: ST
Opcode: 1111
Usage: ST
Location.

The ST instruction stores the current value of the accumulator at a specified location. Locations are A, B, C, D, E, T, AM, MT, AE [C1], H [C1], B160 [C1], R [C1], K [C1], M [C1], or One of X [N4]. The X [N4] operand has the side effect of advancing the N4 index. After storage is done, N4 points to the next element in the X array. N4 is decremented by 1, but since the X array is ordered from high to low, decrementing the index advances to the next element in the array. When the storage destination is flash memory, the effect of the ST instruction is that the bit of the flash memory corresponding to the bit in the accumulator is set. To ensure that the correct value from the accumulator is stored, a CLR instruction that erases the appropriate memory location first should be used reliably.

TBR-test and branch mnemonics: TBR
Opcode: 0000
Usage: TBR
Value index.

Test and branch instructions test whether the accumulator is zero or non-zero and branch to a given address if the current state of the accumulator matches the state of the test partner. If the Z flag matches the TRB test, PC is replaced with a 9-bit value, where bit0 = 0 and the upper 8 bits are provided by the MU. Otherwise, increment PC by 1. The value operand is 0 or 1. 0 indicates a test for the accumulator being zero. 1 indicates that the test is for a non-zero accumulator. The index operand indicates where to jump execution when the test is successful. The remaining 3 bits of the operand index index the lowest 8 entries of jump table 1. The upper 8 bits are taken from the table and the least significant bit (bit 0) is cleared to 0. CMD is cleared to 0 at reset. 0 is translated as TBR0, which means to branch to the address stored at address offset 0 if accumulator = 0. Since the accumulator and Z flag are also cleared at reset, the test is true and the net effect is a jump to the address stored in the 0th entry of the jump table.

XOR-Exclusive OR mnemonic: XOR
Opcode: 1010
Usage: XOR
value.

The XOR operation performs a 32-bit bit XOR with the accumulator and stores the result in the accumulator. The operand is one of A, B, C, D, E, T, AM, MT, X [N1], X [N2], X [N3] or X [N4]. Also, the Z flag is set during this operation depending on the result (ie, depending on the value loaded into the accumulator). Bit NOT operations are performed by XORing the accumulator with 0xFFFFFFFF (using the LDK instruction). The X [N] operand has the side effect of moving the appropriate index to the next value (after the operation). After XOR is done, the index points to the next element in the X array. N4 is ST
It is advanced by the X [N4] instruction. The index is decremented by one, but since the X array is ordered from high to low, decrementing the index advances to the next element in the array.

Programming mode detection unit The programming mode detection unit monitors the input clock voltage. If the clock voltage is a specific value, the erase tamper detection line is triggered to erase all keys, program codes, secret information, etc. and enter program mode. The programming mode detection unit can be implemented using a normal CMOS. This is because the key does not pass through this unit. This unit may not be realized by non-flashing CMOS. There is no special need to cover the programming mode detection unit with a tamper detection line. This is because the attacker can move the chip into programming mode with a CLR input. Using the erase tamper detection line as a signal to enter the programming mode means that if the attacker wants to use the programming mode as part of an attack, the erase tamper detection line must be activated and function Means. This makes it very difficult to attack the authentication chip.

Noise generator The noise generator can be realized by a normal CMOS. Because the key does not pass through this unit. This unit may not be realized by non-flashing CMOS. However, the noise generator should be protected by both tamper detection and protection lines, and as a result, if an attacker tampers with this unit, the chip will reset or all confidential information To clear. Moreover, the LFSR bits should be verified to ensure that they have not been tampered with (ie, parity check). If the parity check fails, the erase tamper detection line is triggered. Finally, all 64 bits of the noise generator are OR'ed into a single bit. If this bit is 0, the erase tamper detection line is triggered. This is because 0 is an invalid state of LFSR. There is no point in using the OK bit setup. This is because the noise generator is used only by the tamper detection and prevention circuit.

The state machine state machine generates two operating cycles of the CPU, stretches during long command operations, and stores opcodes and operands during the operating cycle. The state machine can be implemented in normal CMOS because the key does not pass through this unit. This unit may not be realized by non-flashing CMOS. However, the opcode / operand latch should be parity checked. The logic and registers stored in the state machine should be covered by both tamper detection lines. This ensures that the executed instructions are not changed by the attacker.

The authentication chip does not require the high speed and throughput of a general purpose CPU. It only needs to operate fast enough to run the authentication protocol and need not be faster. Rather than having dedicated circuitry (and all associated complexity) to optimize branch control or execute the opcode while fetching the next opcode, the state machine adopts a simplified concept To do. This helps minimize design time and reduce the possibility of implementation errors.

The general operation of a state machine is to generate the following set of cycles:
Cycle 0: fetch cycle. This is the cycle in which the opcode is fetched from program memory and the effective address is generated from the fetched opcode;
Cycle 1: execution cycle This is the cycle in which the operand is looked up (may be) by the effective address generated (in cycle 0) and the operation itself is executed.

Under normal conditions, the state machine will cycle: 0, 1, 0, 1, 0, 1. . . Is generated. However, in some cases, the state machine performs a stretch and generates cycle 0 at each clock tick until the stretch condition ends. The extension condition includes an erase cycle waiting for a flash cycle, a client serial data read / write wait, or an invalid opcode (by tampering). If the flash memory is being erased then the next instruction cannot be executed until the flash memory is completely erased. This is determined by a Wait signal sent from the memory unit. If Wait = 1, the state machine generates only cycle 0. There are two cases of stretching due to serial I / O operations:
Opcode is ROR
Case of OutBit and already OutBitValid = 1. This is the case when the current operation requires the bit to be output to the client, but the client has not yet read the last bit;
Operation is ROR
Case of InBit and InBitValid = 0. This is the case when the current operation requests reading a bit from the client, but the client has not yet supplied that bit.

In both cases, the state machine must stretch until the stretching condition ends. Thus, the next “cycle” depends on the old or previous cycle and the current values of CMD, Wait, OutBitValid, and InBitValid. Wait is derived from the MU, and OutBitValid and InBitValid are derived from the I / O unit. When the cycle is 0, the 8-bit opcode is fetched from the memory unit and stored in the 8-bit CMD register. The write enable for the CMD register is -cycle. The two outputs from this unit are cycle and CMD. Both values are passed to all other processing units in the authentication chip. A 1-bit cycle value informs each unit whether a fetch or execute cycle is taking place, while an 8-bit CMD allows each unit to perform the appropriate operation for a command associated with a particular unit. To.

FIG. 187 is a diagram showing the data flow and relationship between the components of the state machine,

である。

It is.

Both Old and CMD are cleared to 0 by reset. With this result, the first cycle is 1, which causes 0CMD to be executed. 0 is TBR
If it is translated as 0, that is, Accumulator = 0, it branches to the address stored at address offset 0. Since the accumulator is also cleared to zero on reset, the test is true and the net effect is a jump to the address stored in the zeroth entry of the jump table. Two VAL units are designed to verify the values that pass through them. Each VAL includes an OK bit connected to both tamper prevention and detection lines. The OK bit is set to 1 at reset and ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. For VAL ₁ , the effective cycle is always 0 if the chip has been tampered with. In this way, no program code is executed since cycle 1 is never reached. There is no need to check whether the Old has been tampered with. This is because if the attacker stops the Old state, the chip will not execute any more instructions. For VAL ₂ , the effective 8-bit CMD value is always 0 if the chip has been tampered with, which is the TBR
0 instruction. This stops the execution of any program code. VAL ₂ performs a parity check on the bits from the CMD to ensure that the CMD has not been tampered with. If the parity check fails, the erase tamper detection line is triggered.

I / O unit The I / O unit communicates serially with the outside world. The authentication chip operates as a slave serial device, receives serial data from the client, processes a command, and serially transmits the obtained data to the client. The I / O unit can be realized by a normal CMOS. This is because the key does not pass through this unit. This unit may not be realized by non-flashing CMOS. Further, the latch need not be parity checked. This is because there is no benefit for the attacker to destroy or change them. The I / O unit outputs 0 and inputs 0 when any of the tamper detection lines is broken. This is only effective when the attacker has disabled the reset and / or erase circuit. This is because breaking any of the tamper detection lines will cause a reset or erase of the entire flash memory.

The 1-bit registers of InBit, InBitValid, OutBit, and OutBitValid are used for communication between the client (system) and the authentication chip. InBit and InBitValid provide a means for clients to send commands and data to the authentication chip. OutBit and OutBitValid provide a means for the client to obtain information from the authentication chip. Both InBitValid and OutBitValid are cleared when the chip is reset. The client sends the command and parameter bits to the authentication chip one bit at a time. From the authentication chip perspective:
Reading from InBit hangs while InBitValid is cleared. InBitValid remains cleared until the client writes the next input bit to InBit. Reading InBit clears the InBitValid bit so that the next InBit can be read from the client. The client cannot write bits to the authentication chip until the InBitValid bit is cleared;
Writing to OutBit hangs while OutBitValid is set. OutBitValid remains set until the client reads a bit from OutBit. Writing OutBit sets the OutBitValid bit so that the client can read the next OutBit. The client cannot read the bit from the authentication chip until the OutBitValid bit is set.

The actual command extension is managed by the state machine, but various communication registers and circuits exist in the I / O unit.

FIG. 188 is a diagram showing the data flow and relationship between components of the I / O unit.

シリアルＩ／Ｏユニットは、データピンを使って外界と外部的に通信する回路を含む。InBitＵｓｅｄ制御信号は、所定のクロックサイクル中にInBitを使用する何れかのユニットによってセットされるべきである（サイクルのあらゆる状態になり得る）。２個のＶＡＬユニットは、タンパー防止及び検出回路に接続された検証ユニットであり、各ＶＡＬはＯＫビットを含む。ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー防止及び検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットはそのユニットを通過する各データビットとＡＮＤ演算される。

The serial I / O unit includes a circuit that communicates with the outside world using data pins. The InBitUsed control signal should be set by any unit that uses InBit during a given clock cycle (it can be any state in the cycle). The two VAL units are verification units connected to the tamper prevention and detection circuit, and each VAL includes an OK bit. The OK bit is set to 1 at reset and ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit.

For VAL ₁ , the effective bit output from the chip is always 0 if the chip has been tampered with. In this way, an attacker cannot produce useful output. For VAL ₂ , the effective bit input to the chip is always 0 if the chip has been tampered with. In this way, the attacker cannot select useful input. There is no need to verify the registers of the I / O unit. This is because attackers do not benefit from destroying or changing them.

ALU
FIG. 189 is a schematic block diagram of the arithmetic logic unit. The arithmetic logic unit (ALU) includes a 32-bit Acc (accumulator) register and circuitry for simple arithmetic and logic operations. The ALU and all subunits should be implemented in non-flashing CMOS. Because the key passes through it. In addition, the accumulator is parity checked. The logic and registers housed in the ALU are covered by both tamper detection lines. This ensures that the key and intermediate calculation values are not changed by the attacker. The 1-bit Z register stores the accumulator zero. Both Z and accumulator registers are cleared after reset. The Z register is updated when the accumulator is updated, and the accumulator is updated for any of the LD, LDK, LOG, XOR, ROR, RPL, and ADD commands. Each arithmetic and logic block operates based on two 32-bit inputs: the current value of the accumulator and the current 32-bit output of the MU. here,

である。

It is.

WriteEnables of Acc and Z, because (Logic 1 by) considering the CMD ₇ and cycle, these two bits are not required by multiplexer MX ₁ to select the output. The output selection of MX ₁ is simpler as a result as it requires only CMD bits 6-3.

２個のＶＡＬユニットは、タンパー防止及び検出ラインに接続された検証ユニットであり、各ＶＡＬはＯＫビットを含む。ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー防止及び検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットはそのユニットを通過する各データビットとＡＮＤ演算される。ＶＡＬ_１の場合、チップが改ざんされたならば、アキュムレータから出力された実効ビットは常に０である。これは、攻撃者がアキュムレータに関する何かを処理することを妨げる。ＶＡＬ_１はアキュムレータに対するパリティチェックを実行し、チェックが失敗したならば、消去タンパー検出ラインをセットする。ＶＡＬ_２の場合、アキュムレータの実効Ｚ状態は、チップ改ざんされたならば、常に真である。このようにして、攻撃者はループ構造を作成できない。ＡＬＵの残りの機能ブロックは後述する。すべてがフラッシュしないＣＭＯＳで実現されるべきである。

The two VAL units are verification units connected to the tamper prevention and detection lines, and each VAL includes an OK bit. The OK bit is set to 1 at reset and ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. For VAL ₁ , the effective bit output from the accumulator is always 0 if the chip has been tampered with. This prevents an attacker from processing anything related to the accumulator. VAL ₁ performs a parity check on the accumulator and sets the erase tamper detection line if the check fails. For VAL ₂ , the effective Z state of the accumulator is always true if the chip has been tampered with. In this way, an attacker cannot create a loop structure. The remaining functional blocks of the ALU will be described later. Everything should be realized in non-flash CMOS.

ＲＰＬ
図１９０はＲＰＬユニットの概略ブロック図である。ＲＰＬユニットはＡＬＵ内のコンポーネントである。これは、認証チップのＲＰＬ
ＣＭＰ機能を実現するため設計されている。ＲＰＬ ＣＭＰコマンドは、ＡｃｃｅｓｓＭｏｄｅの値に基づいて、フラッシュメモリＭへの安全な書き込みに使用するため特に設計された。ＲＰＬユニットは、シフトパルス毎に２ビット右シフトを実行するＡＭＴ（ＡｃｃｅｓｓＭｏｄｅＴｅｍｐ）と呼ばれる３２ビットシフトレジスタと、ＷＲ擬似コードのＥｑＥｎｃｏｕｎｔｅｒｅｄフラグ及びＤｅｃＥｎｃｏｕｎｔｅｒｅｄフラグに直接基づいているＥＥ及びＤＥと呼ばれる１ビットレジスタと、を含む。全てのレジスタはリセット後に０にクリアされる。ＡＭＴは、ＰＲＬ
ＩＮＩＴコマンドを用いて（アキュムレータを介して）３２ビットＡＭ値がロードされ、ＥＥ及びＤＥは、ＲＰＬ
ＭＨＩへの呼び出し及びＲＰＬ
ＭＬＯへの呼び出しを用いて汎用書き込みアルゴリズムに従ってセットされる。ＥＱブロック及びＬＴブロックはＷＲコマンド擬似コードに記載されている通りの機能を備えている。ＥＱブロックは、２個の１６ビット入力がビット一致であるならば、１を出力し、ビット一致でなければ、０を出力する。ＬＴブロックは、アキュムレータから入力された上位１６ビットがＭＸ_２によってＭＵから選択された１６ビット値よりも小さい場合に、１を出力する。比較は符号無しである。オペランドのビットパターンは組み合わせロジックがより簡単化されるように特に選択される。オペランドのビットパターンを利用するので、そのパターンをもう一度列挙する。

RPL
FIG. 190 is a schematic block diagram of the RPL unit. The RPL unit is a component in the ALU. This is the authentication chip RPL
It is designed to realize the CMP function. The RPL CMP command was specifically designed for use in secure writing to the flash memory M based on the value of AccessMode. The RPL unit includes a 32-bit shift register called AMT (AccessModeTemp) that performs a 2-bit right shift for each shift pulse, and a 1-bit register called EE and DE that are directly based on the EqEncountered flag and the DecEncountered flag of the WR pseudo code. ,including. All registers are cleared to 0 after reset. AMT is PRL
A 32-bit AM value is loaded (via the accumulator) using the INIT command, and EE and DE are RPL
Call to MHI and RPL
Set according to the general write algorithm using a call to MLO. The EQ block and the LT block have functions as described in the WR command pseudo code. The EQ block outputs 1 if the two 16-bit inputs match, and outputs 0 if it does not match. LT block, the upper 16 bits input from the accumulator is smaller than the 16-bit value selected from MU by MX _2, and outputs a 1. The comparison is unsigned. The bit pattern of the operands is specifically selected so that the combinational logic is simplified. Since the bit pattern of the operand is used, the pattern is enumerated again.

ＭＨＩ及びＭＬＯは、ハイビットがセットされているので、Ｉｎｉｔのビットパターンと簡単に区別することができ、最下位ビットはＭＨＩとＭＬＯを区別するために使用できる。ＥＥフラグ及びＤＥフラグは、ＲＰＬコマンドが発行されるたびに更新される。Ｉｎｉｔステージのため、２個の値を設定し、ＭＨＩ及びＭＬＯのため、ＥＥ及びＤＥの値を適切に更新する。したがって、ＥＥ及びＤＥのためのＷｒｉｔｅＥｎａｂｌｅは以下の通りである。

Since the high bit is set in MHI and MLO, it can be easily distinguished from the bit pattern of Init, and the least significant bit can be used to distinguish between MHI and MLO. The EE flag and the DE flag are updated each time an RPL command is issued. Two values are set for the Init stage, and the EE and DE values are updated appropriately for the MHI and MLO. Thus, WriteEnable for EE and DE is as follows:

３２ビットのＡＭＴレジスタに関して、このレジスタには、ＲＰＬ Ｉｎｉｔコマンドのときに、（ＭＵから読み出された）ＡＭの内容をロードし、ＲＰＬ ＭＬＯコマンド及びＲＰＬ
ＭＨＩコマンドのためにＡＭＴレジスタを右へ２ビット位置シフトさせる。これは、ＲＰＬオペランドの最上位ビット（ＣＭＤ_３）に対して簡単にテストできる。したがって、ＡＭＴレジスタのためのＷｒｉｔｅＥｎａｂｌｅ及びＳｈｉｆｔＥｎａｂｌｅは、以下の通りである。

For the 32-bit AMT register, this register is loaded with the contents of the AM (read from the MU) during the RPL Init command, and the RPL MLO command and RPL
Shift the AMT register 2 bits to the right for the MHI command. This can be easily tested against the most significant bit (CMD ₃ ) of the RPL operand. Thus, WriteEnable and ShiftEnable for the AMT register are as follows:

Ｌｏｇｉｃ_３からの出力は、マルチプレクサＭＸ_１への入力としても有用である。なぜならば、それは、現在の２個のアクセスモードビット、又は００（これは、アクセスモードＲＷを表すので、ＤＥ及びＥＥレジスタをリセットさせる）の何れかを通すために使用されるからである。したがって、ＭＸ_１は次の通りである。

Output from Logic ₃ is also useful as an input to multiplexer MX _1. This is because it is used to pass either the current two access mode bits, or 00 (which represents the access mode RW, which causes the DE and EE registers to be reset). Therefore, MX ₁ is as follows.

ＲＰＬロジックは、アキュムレータの上位１６ビットだけを置換する。下位１６ビットはそのまま通過する。しかし、（Ｍ［０−１５］のうちの一つに対応する）ＭＵからの３２ビットのうちの、上位又は下位１６ビットだけが使用される。したがって、ＭＸ_２はＭＨＩとＭＬＯを区別するためにＣＭＤ_０をテストする。

The RPL logic only replaces the upper 16 bits of the accumulator. The lower 16 bits pass through as they are. However, only the upper or lower 16 bits of the 32 bits from the MU (corresponding to one of M [0-15]) are used. Therefore, MX ₂ tests CMD ₀ to distinguish between MHI and MLO.

ＤＥレジスタ及びＥＥレジスタを更新するロジックは、ＷＲコマンドの擬似コードと一致する。ＡｃｃｅｓｓＭｏｄｅ値が００（＝ＲＷ、これはＲＰＬ
ＩＮＩＴ中に出現する）である入力は、ＤＥとＥＥの両方に０（正しい初期値）をロードさせることに注意する必要がある。ＥＥはＬｏｇｉｃ_４からの結果がロードされ、ＤＥはＬｏｇｉｃ_５からの結果がロードされる。

The logic for updating the DE and EE registers matches the WR command pseudo code. The AccessMode value is 00 (= RW, which is RPL
Note that an input that appears in INIT) causes both DE and EE to load 0 (the correct initial value). The EE is loaded with the result from Logic ₄ , and the DE is loaded with the result from Logic ₅ .

アキュムレータの上位１６ビットはＭへ書き込まれるべき値によって置き換えられる。したがって、Ｌｏｇｉｃ_６はＷＲコマンド擬似コードからのＷＥフラグと一致する。

The upper 16 bits of the accumulator are replaced by the value to be written to M. Therefore, Logic ₆ matches the WE flag from the WR command pseudo code.

Ｌｏｇｉｃ_６からの出力は、マルチプレクサＭＸ_３による、アキュムレータからの元の１６ビットと、Ｍ［０−１５］からの値との間の選択を駆動するためそのまま使用される。アキュムレータからの１６ビットが選択された場合（アキュムレータは変更されないで残される場合）、これは、アキュムレータ値がＭ［ｎ］に書き込まれることを意味する。Ｍからの１６ビット値が選択された場合（アキュムレータの上位１６ビットを変更する場合）、これは、Ｍの１６ビット値が変更されないことを意味する。したがって、ＭＸ_３は以下の形式をとる。

The output from the Logic ₆ is due to the multiplexer MX _3, the original 16 bits from the accumulator, as it is used to drive the selection between the values from M [0-15]. If 16 bits from the accumulator are selected (if the accumulator is left unchanged), this means that the accumulator value is written to M [n]. If the 16-bit value from M is selected (when changing the upper 16 bits of the accumulator), this means that the 16-bit value of M is not changed. Thus, MX ₃ takes the following form:

ＡＭＴをパリティチェックする意味はない。なぜならば、攻撃者はＭＸ_３への入力を強制的に０にする方がよいからである（これにより、攻撃者は任意の値をＭに書き込めるようになる）。しかし、攻撃者がチップ（全てタンパー検出テスト及び回路を含む）をレーザー切断する手間を選ぶならば、ＭＸ_３の入力を固定することにより制限選択テキスト攻撃を可能にさせるよりも優れたターゲットがある。

There is no point in parity checking the AMT. This is because it is better for the attacker to force the input to MX ₃ to be 0 (this allows the attacker to write an arbitrary value to M). However, if an attacker chooses to laser-cut the chip (all including tamper detection tests and circuitry), there is a better target than allowing limited choice text attacks by fixing the MX ₃ input. .

ROR
FIG. 191 is a schematic block diagram of the ROR block of the ALU. The ROR unit is a component in the ALU. The ROR unit is designed to realize the ROR function of the authentication chip. A 1-bit register named RTMP is stored in the ROR unit. RTMP is cleared to 0 by reset and ROR
Set in RB command and ROR XRB command. The RTMP register allows the implementation of a linear feedback shift register with an arbitrary tap structure. The XOR block is an XOR of two 1-bit inputs and 1-bit outputs. RORn is blocks are shown for convenience, in practice, are wired to multiplexer MX _3. Because each block is simply a 32-bit rewrite, shifted N bits to the right. All three multiplexers (MX ₁ , MX ₂ and MX ₃ ) depend on an 8-bit CMD value. However, the bit pattern of the ROR opcode is aligned for logic optimization purposes. Since the bit pattern of the operand is used, this pattern is enumerated again.

Ｌｏｇｉｃ_１は、ＲＴＭＰにＷｒｉｔｅＥｎａｂｌｅ信号を与えるために使用される。ＲＴＭＰレジスタは、ＲＯＲ
ＲＢ及びＲＯＲ ＸＯＲコマンドの間だけ書き込まれるべきである。Ｌｏｇｉｃ_２は、InBitが使われたときならいつでも、制御信号を与えるために使用される。２個の組み合わせロジックブロックは次の通りである。

Logic ₁ is used to provide a Write Enable signal to RTMP. RTMP register is ROR
Should only be written during RB and ROR XOR commands. Logic ₂ is used to provide a control signal whenever InBit is used. The two combinational logic blocks are as follows.

マルチプレクサＭＸ_１を用いて、ＲＴＭＰに格納されるビットを選択する。Ｌｏｇｉｃ_１は、予めＲＢとＸＲＢの一方へのＣＭＤ入力の範囲を限定する。したがって、これらの二つを区別するため、ＣＭＤ_０をテストするだけでよい。以下の表は、ＣＭＤ_０とＭＸ_１から出力される値との間の関係を表現する。

The multiplexer MX ₁ is used to select the bits stored in the RTMP. Logic ₁ limits the range of CMD input to one of RB and XRB in advance. Therefore, it is only necessary to test CMD ₀ to distinguish these two. The following table represents the relationship between CMD ₀ and the value output from MX ₁ .

マルチプレクサＭＸ_２を用いて、アキュムレータ入力のビット０を置換する予定の入力ビットを選択する。ここでは、少しの量の最適化を行う。なぜならば、個別の入力ビットは、典型的に、特定のオペランドに関係するからである。以下御表は、ＣＭＤ_３−０とＭＸ_２から出力される値との間の関係を表現する。

Using multiplexer MX _2, selects the input bit plan to replace the bit 0 of the accumulator input. Here, a small amount of optimization is performed. This is because individual input bits are typically related to a particular operand. The table below represents the relationship between CMD _3-0 and the value output from MX ₂ .

最後のマルチプレクサＭＸ_３は３２ビット値の最終的な回転を行う。再度、ＣＭＤオペランドのビットパターンが利用される。

Last multiplexer MX ₃ performs the final rotation of 32-bit values. Again, the bit pattern of the CMD operand is used.

ＭｉｎＴｉｃｋｓユニット
図１９２は、ＭｉｎＴｉｃｋｓユニットのコンポーネント間のデータフロー及び関係を示す図である。ＭｉｎＴｉｃｋｓユニットは、認証チップ内の鍵付きオペレーションの間のプログラマブル最小遅延（カウントダウンを利用）の役目を担う。ＭｉｎＴｉｃｋｓユニットに収容されるロジック及びレジスタは、両方のタンパー検出ラインによってカバーする必要がある。これは、攻撃者が鍵付きの関数への呼び出しの間の時間を変更できないことを保証する。殆ど全てのＭｉｎＴｉｃｋｓユニットは通常のＣＭＯＳで実現可能である。なぜならば、鍵は、殆どのこのユニットを通らないからである。しかし、アキュムレータは、ＳＥＴ
ＭＴＲ命令で使用される。その結果として、この回路のこの小さい区域は、ノンフラッシングＣＭＯＳで実現しなければならない。ＭｉｎＴｉｃｋｓユニットの残りの部分はノンフラッシングＣＭＯＳで実現しなくてもよい。しかし、ＭＴＲＺラッチ（以下を参照）はパリティチェックしなければならない。

MinTicks Unit FIG. 192 is a diagram showing the data flow and relationship between the components of the MinTicks unit. The MinTicks unit is responsible for the programmable minimum delay (using countdown) between keyed operations in the authentication chip. The logic and registers housed in the MinTicks unit need to be covered by both tamper detection lines. This ensures that the attacker cannot change the time between calls to the keyed function. Almost all MinTicks units can be realized with ordinary CMOS. Because the key does not go through most of this unit. However, the accumulator is
Used in MTR instruction. Consequently, this small area of the circuit must be realized with non-flashing CMOS. The remaining part of the MinTicks unit may not be realized by non-flashing CMOS. However, the MTRZ latch (see below) must perform a parity check.

The MinTicks unit includes a 32-bit register named MTR (MinTicks Remaining). The MTR register stores the number of clock ticks remaining until the next keyed function can be read. In each cycle, the value of MTR is decreased by 1 until the value becomes 0. When MTR reaches 0, it does not decrease further. An auxiliary 1-bit register named MTRZ (MinTicks Remaining Register Zero MinTicksRegisterZero) reflects the current zeroness of the MTR register. If the MTRZ register is 0, the MTRZ is 1; if the MTRZ register is not 0, the MTRZ is 0. The MTR register is cleared by reset and SET
A new count is set using the MTR command, and the current value of the accumulator is transferred to the MTR register.

here,

であり、且つ

And

である。

It is.

Since the cycle is connected to the MTR and MTRZ WriteEnable, these registers are updated during the execution cycle, ie, only when cycle = 1. The two VAL units are verification units connected to the tamper prevention and detection lines, and each VAL includes an OK bit. The OK bit is set to 1 at reset and is ORed with the ChipOK values from both tamper detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. In the case of VAL ₁ , the effective output from the MTR is 0, indicating that the output from the decrementer unit is all 1, so that the MTRZ remains 0 and the attacker is locked Use of the function is prevented. VAL ₁ verifies the parity of the MTR register. If the parity check fails, the erase tamper detection line is triggered. In the case of VAL ₂ , if the chip has been tampered with, the effective output from MTRZ is 0, indicating that the MinTicksRemaining register has not yet reached 0, so that the attacker can use the keyed function. Be blocked.

Program Count Unit FIG. 192 is a block diagram of the program count unit. The program count unit (PCU) includes a 9-bit PC (program counter) and logic for branch and subroutine control. The program count unit can be realized by a normal CMOS. Because the key does not pass through this unit. This unit may not be realized by non-flashing CMOS. However, the latch is parity checked. Moreover, the logic and registers housed in the memory unit need to be covered by both tamper detection lines to ensure that the PC cannot be changed by the attacker. The PC is actually implemented as a 6 level × 9 bit PCA (PC array) and indexed by a 3 bit SP (stack pointer) register. The PC and SP registers are all cleared by reset and updated during the program control flow according to the opcode. The current value of PC is output to the MU during cycle 0 (fetch cycle). The PC is updated during cycle 1 (execution cycle) based on the command to be executed. In most cases, the PC simply increases by one. However, when a branch appears (by a subroutine or other form of jump), the PC is replaced with a probable value. The mechanism for calculating the new PC value depends on the opcode being processed.

ADD block is a simple adder of modulo ^{2 9.} Input is either PC value and 1 (to increase PC by 1) or 9 bit offset (high bit is set and lower 8 bits are from MU). The “+1” block uses a 3-bit input and increments by 1 (with wrap). The “−1” block uses a 3-bit input and is decremented by 1 (wrapped). Various types of PC control are as follows.

ＪＳＲとＪＳＩ（ＡＣＣ）に対して同じ動作がとられるので、Ｌｏｇｉｃ_１でそのケースを具体的に検出する。同じ考え方によって、Ｌｏｇｉｃ_２でＪＳＩ
ＲＴＳを具体的にテストする。

Since the same operation is performed for JSR and JSI (ACC), the case is specifically detected by Logic ₁ . With the same concept, JSI in Logic ₂
RTS is specifically tested.

ＰＣを更新するとき、ＰＣが完全に新しい項目によって置き換えられるべきか、又はアダーの結果によって置き換えられるべきかを決定しなければならない。これは、ＪＳＲ及びＪＳＩ（ＡＣＣ）の場合に成り立つが、テストビットがアキュムレータの状態と一致する限り、ＴＢＲにも成り立つ。ＴＢＲ以外の全ては、Ｌｏｇｉｃ_１によってテストされるので、Ｌｏｇｉｃ_３はＬｏｇｉｃ_１の出力を入力として含む。Ｌｏｇｉｃ_３からの出力は、次に、新しいＰＣ値を得るため、マルチプレクサＭＸ_２によって使用される。

When updating a PC, it must be determined whether the PC should be replaced by a completely new item or by the result of an adder. This is true for JSR and JSI (ACC), but for TBR as long as the test bit matches the accumulator state. Everything except TBR is tested by Logic ₁ , so Logic ₃ includes the output of Logic ₁ as an input. The output from Logic ₃ is then used by multiplexer MX ₂ to obtain a new PC value.

９ビットアダーへの入力は、１ずつ増加するか（普通のケース）、又はＭＵから読み出されたオフセットを加算するか（ＤＢＲコマンド）に依存する。Ｌｏｇｉｃ_４はｔｅｓｔを生成する。Ｌｏｇｉｃ_４からの出力は、これに応じてマルチプレクサＭＸ_３によってそのまま使用される。

The input to the 9-bit adder depends on whether it increments by 1 (normal case) or adds the offset read from the MU (DBR command). Logic ₄ generates a test. The output from Logic ₄ is directly used by multiplexer MX ₃ accordingly.

最後に、どのＰＣエントリーを使用するかについての選択は、ＳＰの現在値に依存する。サブルーチンに入るとき、ＳＰインデックス値はインクリメントされ、サブルーチンから戻るとき、ＳＰインデックス値はデクリメントされる。全てのケースで、且つコマンドをフェッチしたいとき（サイクル０）、ＳＰの現在値を使用する。Ｌｏｇｉｃ_１は、サブルーチンに入るときを通知し、Ｌｏｇｉｃ_２はサブルーチンから戻るときを通知する。マルチプレクサ選択は、したがって、以下のように定義される。

Finally, the choice of which PC entry to use depends on the current value of the SP. When entering a subroutine, the SP index value is incremented, and when returning from the subroutine, the SP index value is decremented. In all cases and when you want to fetch a command (cycle 0), use the current value of SP. Logic ₁ notifies when entering the subroutine, and Logic ₂ notifies when returning from the subroutine. The multiplexer selection is therefore defined as follows:

２個のＶＡＬユニットは、タンパー防止及び検出ラインに接続された検証ユニットであり、各ＶＡＬはＯＫビットを含む。ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットはそのユニットを通過する各データビットとＡＮＤ演算される。両方のＶＡＬユニットは、データビットをパリティチェックし、それらが有効であることを保証する。パリティチェックが失敗した場合、消去タンパー検出ラインがトリガーされる。ＶＡＬ_１の場合、ＳＰレジスタからの実効出力は、チップが改ざんされたならば、常に０である。これにより、攻撃者がサブルーチンを実行することは阻止される。ＶＡＬ_２の場合、実効ＰＣ出力は、チップが改ざんされたならば、常に０である。これにより、攻撃者がプログラムコードを実行することは阻止される。

The two VAL units are verification units connected to the tamper prevention and detection lines, and each VAL includes an OK bit. The OK bit is set to 1 at reset and is ORed with the ChipOK values from both tamper detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. Both VAL units parity check the data bits to ensure that they are valid. If the parity check fails, the erase tamper detection line is triggered. For VAL ₁ , the effective output from the SP register is always 0 if the chip has been tampered with. This prevents the attacker from executing the subroutine. For VAL ₂ , the effective PC output is always 0 if the chip has been tampered with. This prevents an attacker from executing the program code.

Memory unit The memory unit (MU) contains the internal memory of the authentication chip. The internal memory is addressed by a 9-bit address, which is passed from the address generator unit. The memory unit outputs appropriate 32-bit and 8-bit values depending on the address. The memory unit also has a special programming mode, whereby a program is input to the flash memory. The contents of the entire memory unit should be protected from tampering. Therefore, the logic and registers housed in the memory unit should be covered by both tamper detection lines. This ensures that the program code, key, and intermediate data values cannot be changed by an attacker. All flash memories are multi-state and must be checked when read with an invalid voltage. The 32-bit RAM should also be parity checked. A 32-bit data path through the memory unit can be realized with non-flashing CMOS. This is because the key is sent along the data path. The 8-bit data path can be realized by a normal CMOS. This is because keys are not sent along the data path.

The address range of the constant constant memory area is 000000000000-00000111. Therefore, the range is 00000xxxx. However, if the next 48 addresses are reserved, this can be used during decoding. The constant memory area is selected by the upper 3 bits (Adr _8-6 = 000) of the address, the lower 4 bits are supplied to the combinational logic, and the 4 bits are mapped to a 32-bit output value as follows.

ＲＡＭ
３２エントリーの３２ビットＲＡＭのアドレス空間は、００１００００００−００１０１１１１１である。したがって、レンジは００１０ｘｘｘｘｘである。ＲＡＭメモリ領域は、したがって、アドレスの上位４ビット（Adr_８−５＝００１０）によって選択され、下位５ビットはアドレス指定する値を選択する。連続的な３２エントリーのアドレス空間が与えられた場合、ＲＡＭは、単純な３２×３２ビットＲＡＭとして簡単に実現できる。ＣＰＵは、特殊な方法でレンジ０００００−１１１１１からの各アドレスを取り扱うが、ＲＡＭアドレスデコーダ自体は、アドレスを特別に取り扱うわけではない。全てのＲＡＭ値はリセットで０にクリアされるが、プログラムコードはこのことを当てにしてはならない。

RAM
The address space of a 32-entry 32-bit RAM is 001000000-001011111. Therefore, the range is 0010xxxx. The RAM memory area is therefore selected by the upper 4 bits of the address (Adr _8-5 = 0010), and the lower 5 bits select the value to be addressed. Given a continuous 32-entry address space, the RAM can be easily implemented as a simple 32 × 32 bit RAM. The CPU handles each address from the range 00000-11111 in a special way, but the RAM address decoder itself does not handle the address specially. All RAM values are cleared to 0 on reset, but the program code must not rely on this.

The address space of the flash memory-variable 32-bit width flash memory is 001100000-001111111. Therefore, the range is 0011xxxx. The flash memory area is therefore selected by the upper 4 bits of the address (Adr _8-5 = 0111), and the lower 5 bits select the value to be addressed. Flash memory requires special erasure. It takes a considerably long time to complete the erase of the flash memory. Therefore, the Wait signal is set inside the flash controller when the CLR command is received, and is cleared only when the requested memory is erased. Internally, the erase lines for a particular memory range are combined together, so that only 2 bits are required, as shown in the table below.

フラッシュ値はリセットによって変更されないが、プログラムコードは、ガーベッジ以外では、（製造後に）フラッシュの初期値をとるべきではない。フラッシュアドレスを利用するオペレーションは、ＬＤ、ＳＴ、ＡＤＤ、ＲＰＬ、ＲＯＲ、ＣＬＲ及びＳＥＴである。全てのケースで、オペランド及びメモリ配置は、アドレス生成及び復号化を最小限に抑えるため、密接に連結される。フラッシュメモリの全変数区域は、プログラミングモードへ入ったとき、及び明確な物理的攻撃を検出したときに消去される。

The flash value is not changed by reset, but the program code should not take the initial value of the flash (after manufacturing) except for garbage. Operations that use the flash address are LD, ST, ADD, RPL, ROR, CLR, and SET. In all cases, the operands and memory placement are tightly coupled to minimize address generation and decoding. All variable areas of flash memory are erased when entering programming mode and when a clear physical attack is detected.

The address range of the flash memory-program 384 entry 8-bit wide program flash memory is 010000000-111111111. Therefore, the range is 01xxxxxxxx-11xxxxxxxx. Decoding is simple if the ROM start address and address range are given. Although the CPU handles a part of the address range specially, the address decoder itself does not handle the address specially. The flash value is not changed by reset and is cleared only by entering the program mode. After manufacture, the flush content should be considered garbage. 384 bytes are loaded by the state machine only when in programming mode.

Block Diagram of MU FIG. 193 is a block diagram of a memory unit. The illustrated logic takes advantage of the fact that 32-bit data and 8-bit data are required by separate commands, resulting in fewer bits required for decoding. As shown, 32-bit output and 8-bit output are always generated. The remaining appropriate components of the authentication chip simply use a 32-bit value or an 8-bit value, depending on the command being executed. Multiplexer MX ₁ selects truth table constant, RAM, and a 32 bits output from the selection of the flash memory. To select between 3 outputs, only 2 bits are required, namely Adr ₆ and Adr ₅ . Thus, the format of MX ₂ is as follows.

３２ビットフラッシュメモリの特定の部分を消去するロジックはＬｏｇｉｃ_１によって満たされる。消去部分制御信号は、サイクル＝１の間に、メモリの正しい部分へのＣＬＲコマンド中に限りセットされるべきである。単一のＣＬＲコマンドがあるレンジのフラッシュメモリをクリアすることに注意する必要がある。Adr_６はＣＬＲのためのアドレスレンジとして十分である。なぜならば、レンジは、有効オペランドに対して常にフラッシュの範囲内に治まり、有効ではないオペランドに対して０である。３２ビット幅フラッシュメモリのレンジ全体は、（攻撃者によって、又は意図的にプログラミングモードに入ることによって）消去検出ラインがトリガーされたときに消去される。

The logic for erasing a specific part of the 32-bit flash memory is satisfied by Logic ₁ . The erase portion control signal should be set only during a CLR command to the correct portion of memory during cycle = 1. Note that a range of flash memory is cleared by a single CLR command. Adr ₆ is sufficient as an address range for CLR. This is because the range always stays within the flash range for valid operands and is zero for non-valid operands. The entire range of 32-bit wide flash memory is erased when the erase detection line is triggered (by an attacker or intentionally entering a programming mode).

フラッシュメモリの特定の部分に書き込むためのロジックは、Ｌｏｇｉｃ_２によって満たされる。ＷｒｉｔｅＥｎａｂｌｅ制御信号は、サイクル＝１の間に、フラッシュメモリレンジへの適切なＳＴコマンド中に限りセットされるべきである。Adr_６−５だけのテストが許容される。なぜならば、ＳＴコマンドだけがフラッシュまたはＲＡＭへ有効に書き込むからである（Adr_６−５が００であるならば、Ｋ２ＭＸは０でなければならない。）。

The logic for writing to specific parts of the flash memory is satisfied by Logic ₂ . The WriteEnable control signal should be set only during the appropriate ST command to the flash memory range during cycle = 1. Only tests Adr _6-5 are allowed. This is because only the ST command effectively writes to flash or RAM (if Adr _6-5 is 00, K2MX must be 0).

ＷＥ（ＷｒｉｔｅＥｎａｂｌｅ）フラグは、ＳＥＴ ＷＥコマンドとＣＬＲ
ＷＥコマンドの実行中にセットされる。Ｌｏｇｉｃ_３はこれらの二つのケースをテストする。ＷＥに書き込まれる実際のビットはＣＭＤ_４である。

The WE (Write Enable) flag indicates the SET WE command and the CLR.
Set during execution of a WE command. Logic ₃ tests these two cases. The actual bits written in WE is CMD _4.

メモリのＲＡＭ領域へ書き込むためのロジックは、Ｌｏｇｉｃ_４によって満たされる。ＷｒｉｔｅＥｎａｂｌｅ制御信号は、サイクル＝１の間に、ＲＡＭメモリレンジへの適切なＳＴコマンド中に限りセットされる。しかし、これは、Ｘ［Ｎ］への書き込みが許可されるかどうかを管理するＷＥフラグによって改ざんされる。Ｘ［Ｎ］レンジはＲＡＭの上位半分であり、これは、Adr_４を使用してテストできる。ＲＡＭのフルアドレスレンジとしてAdr_６−５だけをテストすることが容認できる。なぜならば、ＳＴコマンドはフラッシュ又はＲＡＭだけに書き込むからである。

The logic for writing to the RAM area of the memory is satisfied by Logic ₄ . The WriteEnable control signal is set only during the appropriate ST command to the RAM memory range during cycle = 1. However, this is tampered with by the WE flag that manages whether writing to X [N] is allowed. The X [N] range is the upper half of the RAM, which can be tested using Adr ₄ . It is acceptable to test only Adr _6-5 as the full address range of the RAM. This is because the ST command writes only to flash or RAM.

３個のＶＡＬユニットは、タンパー防止及び検出ラインに接続された検証ユニットであり、各ＶＡＬはＯＫビットを含む。ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー防止及び検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットはそのユニットを通過する各データビットとＡＮＤ演算される。ＶＡＬユニットは、データビットをパリティチェックし、それらが有効であることを保証する。ＶＡＬ_１及びＶＡＬ_２は、各データビットの状態をチェックすることにより検証し、ＶＡＬ_３はパリティチェックを実行する。妥当性テストが失敗したとき、消去タンパー検出ラインがトリガーされる。ＶＡＬ_１の場合、プログラムフラッシュからの実効出力は、チップが改ざんされているならば、常に０である（ＴＢＲ
０として解釈される）。これは攻撃者が有用な命令を実行することを阻止する。ＶＡＬ_２の場合、実効３２ビットは、チップが改ざんされたならば、常に０である。これにより、攻撃者は、鍵、又は中間記憶値を入手できない。８ビットフラッシュメモリは、プログラムコード、ジャンプテーブル、及びその他のプログラム情報を保持するため使用される。３８４バイトのプログラムフラッシュメモリは、（アドレスレンジ０１ｘｘｘｘｘｘｘ−１１ｘｘｘｘｘｘｘを使用して）完全な９ビットのアドレスにより選択される。プログラムフラッシュメモリは、（攻撃者によって、又はプログラミングモード検出ユニットが原因でプログラミングモードに入ることによって）消去検出ラインがトリガーされたときに限り消去される。消去検出ラインがトリガーされたとき、プログラムフラッシュメモリユニット内の小型状態機械は、８ビットフラッシュメモリを消去し、消去を検証し、シリアル入力から新しい内容（３８４バイト）にロードする。以下の擬似コードは、消去検出ラインがトリガーされたときに実効される状態機械ロジックを説明する：

プログラミングモード状態機械実行中に、０が８ビット出力にセットされる。０コマンドは、認証チップの残りの部分に、コマンドをＴＢＲ
０として解釈させる。チップが全部で３８４バイトをプログラムフラッシュメモリへ読み込んだとき、チップはハングする（無限ループに入る）。次に、認証チップはリセットされ、プログラムは正常に使用される。消去は、８ビットプログラムフラッシュメモリの新しい内容をロードするために使用される８ビットレジスタによって検証されることに注意すべきである。これは、攻撃が成功する見込みを低下させるために役立つ。なぜならば、プログラムコードは、消去を認証するため使用されるレジスタが攻撃者によって破壊された場合に、適切にロードされ得ないからである。更に、状態機械全体は、両方のタンパー検出ラインによって保護される。

The three VAL units are verification units connected to the tamper prevention and detection lines, and each VAL includes an OK bit. The OK bit is set to 1 at reset and ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. The VAL unit performs a parity check on the data bits to ensure that they are valid. VAL ₁ and VAL ₂ verify by checking the state of each data bit, and VAL ₃ performs a parity check. When the validity test fails, the erase tamper detection line is triggered. For VAL ₁ , the effective output from the program flash is always 0 if the chip has been tampered with (TBR
Interpreted as 0). This prevents attackers from executing useful instructions. For VAL ₂ , the effective 32 bits are always 0 if the chip has been tampered with. As a result, the attacker cannot obtain the key or the intermediate storage value. The 8-bit flash memory is used to hold program codes, jump tables, and other program information. The 384 byte program flash memory is selected by a complete 9-bit address (using the address range 01xxxxxxxx-11xxxxxxxx). The program flash memory is erased only when the erase detection line is triggered (by the attacker or by entering the programming mode due to the programming mode detection unit). When the erase detect line is triggered, the small state machine in the program flash memory unit erases the 8-bit flash memory, verifies the erase, and loads the new contents (384 bytes) from the serial input. The following pseudo code illustrates the state machine logic that is executed when the erase detect line is triggered:

During programming mode state machine execution, 0 is set to the 8-bit output. The 0 command sends the command to the rest of the authentication chip
Interpret as 0. When the chip reads a total of 384 bytes into the program flash memory, the chip hangs (enters an infinite loop). Next, the authentication chip is reset and the program is used normally. Note that erasure is verified by the 8-bit register used to load the new contents of the 8-bit program flash memory. This helps to reduce the chances of a successful attack. This is because the program code cannot be loaded properly if the register used to authenticate the erasure is destroyed by an attacker. Furthermore, the entire state machine is protected by both tamper detection lines.

Address Generator Unit The address generator unit generates an effective address for accessing the memory unit (MU). In cycle 0, the PC is passed to the MU to fetch the next opcode. The address generator interprets the returned opcode to generate an effective address for cycle 1. In cycle 1, the generated address is passed to the MU. The logic and registers stored in the address generator unit are covered by both tamper detection lines. This ensures that the attacker cannot change the generated address. Almost all address generator units can be implemented in normal CMOS. Because the key does not go through most of this unit. However, the 5 bits of the accumulator are used for JSI address generation. As a result of this, this small area of the circuit is realized with non-flashing CMOS. The remaining portion of the address generator unit may not be implemented with non-flashing CMOS. However, the counter and the calculated address latch should be parity checked. If any of the tamper detection lines are broken, the address generator unit generates address 0 in each cycle and all counters are set to zero. This is effective only when the attacker disables resetting and / or disables the erase circuit. This is because, under normal conditions, a tamper detection line break will cause a reset or erase of all flash memory.

Background of address generation Logic for address generation needs to examine various combinations of opcodes and operands. The relationship between opcode / operand and address is discussed in this section and is used as the basis for the address generator unit.

The constant lower 4 entries are simple constants for general purpose as well as HMAC algorithms. The lower 4 bits of the LDK operand correspond directly to the lower 3 bits of these four values, that is, 0000, 0001, 0010, and 0011 in-memory addresses. The y and h constants are also addressed by the LDK command. However, the address is generated by ORing the lower 3 bits of the operand with the inverse of the C1 counter value and retaining the fourth bit of the operand as it is. In this way, LDK
For y, the y operand is 0100, and for LDK h, the h operand is 1000. The inverted C1 value is in the range of 000-011 for y and in the range of 000-100 for h, and the OR operation result gives an accurate address. For all constants, the upper 5 bits of the final address are always 00000.

RAM
Variables AT have addresses directly related to the lower 3 bits of their operand values. That is, in the case of the operand values 0000-0101 of the LD, ST, ADD, LOG, and XOR commands, and in the case of the operand values 1000-1101 of the LOG command, the lower 3 bits of the operand address bits generate the final address. , 001000 with a constant upper 6-bit address. The remaining register values can be accessed by an indexed mechanism. Variables A-E, B160 and H are only accessible as indexed by the C1 counter value, and X is indexed by N ₁ , N ₂ , N ₃ and N ₄ . For LD, ST and ADD commands, the address of the AE indexed by C1 is generated by taking the lower 3 bits (000) of the operand and ORing with the C1 counter value. However, the addresses for H and B160 cannot be generated in the same way (otherwise the RAM address space will be discontinuous). Thus, simple combinational logic converts AE to 0000, H to 0110, and B160 to 1011. The final address is obtained by adding C1 to the 4-bit value (a 4-bit value is obtained) and prepending the constant 1005 upper 5 bit address. Finally, the X range of the register is only accessible as indexed by N ₁ , N ₂ , N ₃ and N ₄ . For the XOR command, any N _1-4 can be used for indexing, while for LD, ST and ADD, only N ₄ can be used. Since the operand of X in LD, ST and ADD is the same as the _XN4 operand, the lower 2 bits of the operand select N to be used. The address is thus generated as a constant upper 5 bit value of 00101 and lower 4 bits derived from the selected N counter.

The addresses of the flash memory-variable variables MT and AM are generated from the associated command operands. The 4 bits of the operand are used as they are (0110 and 0111) and are prefixed with a constant upper 5 bit address of 00110. The variables R _1-5 , K1 _1-5 , K2 _1-5 , and M _0-7 are only accessible as indexed by the inverse of the C1 counter value (additionally, in the case of R, the actual C1 value). Simple combinational logic converts R and RF to 00000, K to 01000 or 11000, depending on whether K1 or K2 is addressed, and M (including MHI and MLO) to 10,000. The final address is obtained by ORing (or adding) C1 with a 5-bit value (in the case of RF, using C1 as it is) and adding 0011 constant upper 4 bits to the front. Each of the variables IST and ISW is only a 1-bit value, but can be implemented with any number of bits. Data is read and written as either 0x00000000 or 0xFFFFFFFF. They are addressed only by ROR, CLR and SET commands. In the case of ROR, the lower bits of the operand are combined with a constant upper 8 bit value of 00111111 to obtain 0011111110 and 001111111 for IST and ISW, respectively. This is because the other ROR operands do not use memory, and the returned value can be ignored in cases other than IST and ISW. For SET and CLR, the IST and ISW are addressed by combining the constant upper 4 bits of 0011 with the mapping from IST (0100) to 11110 and the mapping from ISW (0101) to 11111. Since IST and ISW share the same operand values as E and T from RAM, it is possible to use the same decoding logic for the lower 5 bits. The final address requires bits 4, 3 and 1 to be set (this is done by ORing the result of the test with operand value 010x).

Flash memory-program program The address to look up in flash memory comes directly from the 9-bit PC (for cycle 0) or 9-bit Adr register (for cycle 1). Commands such as TBR, DBR, JSR and JSI change the PC according to the data stored in the table at specific addresses in the program memory. As a result, address generation utilizes some constant address components, and command operands (or accumulators) form the lower bits of the effective address.

アドレス生成器ユニットのブロック図
図１９４はアドレス生成器ユニットの概略ブロック図である。アドレス生成器ユニットからの一次出力は、以下の表に示されるように、マルチプレクサＭＸ_１によって選択される。

Address Generator Unit Block Diagram FIG. 194 is a schematic block diagram of the address generator unit. Primary output from the address generator unit, as shown in the table below, is selected by multiplexer MX _1.

ＣＭＤデータとＭＵからの８ビットを区別することが重要である：
サイクル０では、８ビットデータラインは、後のサイクル１で実行されるべき次の命令を保持する。この８ビットコマンド値は実効アドレスをデコードするため使用される。これに対して、ＣＭＤの８ビットデータは前の命令を保持するので、無視されるべきである；
サイクル１では、ＣＭＤラインは現在実行中の命令（サイクル０中に８ビットデータラインに保持されていた命令）を保持し、一方、８ビットデータラインは命令からの実効アドレスのデータを保持する。ＣＭＤデータはサイクル１中に実行されなければならない。

It is important to distinguish between 8 bits from CMD data and MU:
In cycle 0, the 8-bit data line holds the next instruction to be executed in later cycle 1. This 8-bit command value is used to decode the effective address. In contrast, CMD 8-bit data holds the previous instruction and should be ignored;
In cycle 1, the CMD line holds the currently executing instruction (the instruction held in the 8-bit data line during cycle 0), while the 8-bit data line holds the effective address data from the instruction. CMD data must be executed during cycle 1.

Therefore, selection of the 9-bit data from the MU or CMD is performed by a multiplexer MX ₃ as shown in the table below.

９ビットAdrレジスタはサイクル０毎に更新されるので、AdrのＷｒｉｔｅＥｎａｂｌｅは〜サイクルに接続される。カウンタユニットはカウンタＣ１、Ｃ２（内部的に使用される）と、選択されたＮインデックスを生成する。その上、カウンタユニットは、プログラムカウンタユニットによって使用されるフラグＣ１Ｚ及びＣ２Ｚを出力する。様々な＊ＧＥＮユニットは、サイクル０中に、特有のコマンドタイプのアドレスを生成し、マルチプレクサＭＸ_２は、ＰＣ（即ち、８ビットデータライン）によってプログラムメモリから読み出されるようなコマンドに基づいてそれらの間で選択を行う。生成された値は以下の通りである。

Since the 9-bit Adr register is updated every cycle 0, Adr's WriteEnable is connected to ~ cycle. The counter unit generates counters C1, C2 (used internally) and the selected N index. In addition, the counter unit outputs flags C1Z and C2Z used by the program counter unit. Various * GEN units, during cycle 0, generates an address of the specific command type, the multiplexer MX ₂ is PC (i.e., 8-bit data lines) thereof on the basis of the command as read from the program memory by Make a selection between. The generated values are as follows:

マルチプレクサＭＸ_２は以下の選択規準をもっている。

Multiplexer MX ₂ has the following selection criteria.

ＶＡＬ_１ユニットは、タンパー防止及び検出回路に接続された検証ユニットである。それはＯＫビットを含み、ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー防止及び検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットは、使用可能前の９ビットの実効アドレスとＡＮＤ演算される。チップが改ざんされているならば、アドレス出力は常に０であり、これにより、攻撃者はメモリの他の部分へのアクセスが妨げられる。ＶＡＬ_１ユニットは、実効アドレスビットにパリティチェックを実行し、それらが改ざんされていないことを保証する。パリティチェックが失敗であるならば、消去タンパー検出ラインがトリガーされる。

The VAL ₁ unit is a verification unit connected to a tamper prevention and detection circuit. It includes an OK bit, which is set to 1 at reset and is ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with the 9-bit effective address before use. If the chip has been tampered with, the address output is always zero, which prevents the attacker from accessing other parts of the memory. The VAL ₁ unit performs a parity check on the effective address bits to ensure that they have not been tampered with. If the parity check is unsuccessful, an erase tamper detection line is triggered.

JSIGEN
FIG. 195 is a schematic block diagram of the JSIGEN unit. JSIGEN unit is JSI
Generate an address for the ACC instruction. The effective address is simply
4-bit upper part of the address for the JSI table (0101);
The lower 5 bits of the accumulator value,
Is a concatenation of

The accumulator may hold the key at other times (when no jump address is generated), so its value must be kept secret. As a result, this unit is realized with non-flushing CMOS. Multiplexer MX ₁ simply selects between the lower 5 bits or 0 from the accumulator based on whether the command is JSIGEN. Multiplexer MX ₁ has the following selection criteria:

ＪＳＲＧＥＮ
図１９６はＪＳＲＧＥＮユニットの概略ブロック図である。ＪＳＲＧＥＮユニットはＪＳＲ及びＴＢＲ命令のためのアドレスを生成する。実効アドレスは、単に、
ＪＳＲテーブル（０１００）用のアドレスの４ビット上位部分と、
オペランドからのテーブル内のオフセット（ＪＳＲコマンド用の３ビットと、ＴＢＲ用の３ビット＋定数０ビット）と、
の連結からもたらされる。ここで、Ｌｏｇｉｃ_１は実効アドレスのビット３を生成する。このビットはＪＳＲの場合にはビット３であり、ＴＢＲの場合には０である。

JSRGEN
FIG. 196 is a schematic block diagram of the JSRGEN unit. The JSRGEN unit generates addresses for JSR and TBR instructions. The effective address is simply
4-bit upper part of the address for the JSR table (0100);
The offset in the table from the operand (3 bits for JSR command, 3 bits for TBR + constant 0 bit),
Resulting from the concatenation of Here, Logic ₁ generates bit 3 of the effective address. This bit is bit 3 for JSR and 0 for TBR.

ＪＳＲ命令はビット５に１があるので（これに対して、ＴＢＲはこのビットが０である）、これをビット３とＡＮＤ演算することにより、ＪＳＲの場合にはビット３を生成し、ＴＢＲの場合には０を生成する。

Since the JSR instruction has 1 in bit 5 (as opposed to TBR, this bit is 0), this is ANDed with bit 3 to generate bit 3 in the case of JSR and the TBR In this case, 0 is generated.

DBRGEN
FIG. 197 is a schematic block diagram of the DBRGEN unit. The DBRGEN unit generates an address for the DBR instruction. Effective address is
A 6-bit upper part of the address for the DBR table (011000);
The lower 3 bits of the operand;
Resulting from the concatenation of

LDKGEN
FIG. 198 is a schematic block diagram of the LDKGEN unit. The LDKGEN unit generates an address for the LDK instruction. Effective address is
A 5-bit upper part of the address for the LDK table (00000);
The high bit of the operand,
Resulting from concatenation of the lower 3 bits of the operand (if the lower order is a constant) or the lower 3 bits of C1 and the ORed operand (for an indexed constant)

The OR ₂ block only ORs the 3 bits of C1 with the lower 3 bits from the 8-bit data output from the MU. Multiplexer MX ₁ is either high order bits of the operand is set, or based on whether not set, the actual data bits, but only a selection is made between the C1 is OR operation data bits. The selector input to the multiplexer is a simple OR gate, and bit ₂ is ORed with bit ₃ . Multiplexer MX ₁ has the following selection criteria:

ＲＰＬＧＥＮ
図１９９は、ＲＰＬＧＥＮユニットの概略ブロック図である。ＲＰＬＧＥＮユニットはＲＰＬ命令のためのアドレスを生成する。Ｋ２ＭＸが０であるとき、実効アドレスは定数０００００００００である。Ｋ２ＭＸが１であるとき（Ｍからの読み出しが有効な値を返すことを示す）、実効アドレスは、
Ｍ（００１１１０）用のアドレスの６ビット上位部分と、
Ｃ１の現在値の３ビットと、
の連結からもたらされる。

RPLGEN
FIG. 199 is a schematic block diagram of the RPLGEN unit. The RPLGEN unit generates an address for the RPL instruction. When K2MX is 0, the effective address is a constant 000000000000. When K2MX is 1 (indicating that reading from M returns a valid value), the effective address is
A 6-bit upper part of the address for M (001110);
3 bits of the current value of C1,
Resulting from the concatenation of

Multiplexer MX ₁ based on the current value of K2MX, selects between the two addresses. Therefore, the multiplexer MX ₁ has the following selection criteria.

ＶＡＲＧＥＮ
図２００は、ＶＡＲＧＥＮユニットの概略ブロック図である。ＶＡＲＧＥＮユニットはＬＤ、ＳＴ、ＡＤＤ、ＬＯＧ及びＸＯＲ命令のためのアドレスを生成する。Ｋ２ＭＸ
１ビットフラグは、Ｍからの読み出しが定数０アドレス（０を返すが、書き込み不可である）にマッピングされるか、オペランドがＫを指定したとき、Ｋ１とＫ２のどちらがアクセスされるか、を決定するため使用される。４ビットアダーブロックは２組の４ビット入力をとり、モジュロー２^４の加法で、４ビットの出力を生成する。シングルビットレジスタＫ２ＭＸは、ＣＬＲ
Ｋ２ＭＸ命令、又はＳＥＴ Ｋ２ＭＸ命令の実行中だけに書き込まれる。Ｌｏｇｉｃ_１は、これらの条件に基づいてＫ２ＭＸ
ＷｒｉｔｅＥｎａｂｌｅをセットする。

VARGEN
FIG. 200 is a schematic block diagram of the VARGEN unit. The VARGEN unit generates addresses for LD, ST, ADD, LOG and XOR instructions. K2MX
The 1-bit flag determines whether a read from M is mapped to a constant 0 address (returns 0 but cannot be written), or when the operand specifies K, which of K1 and K2 is accessed Used to do. 4-bit adder block takes two sets of 4-bit input, in addition of modulo 2 ^4, to produce a 4-bit output. Single bit register K2MX is a CLR
It is written only during execution of the K2MX instruction or the SET K2MX instruction. Logic ₁ is based on these conditions, K2MX
Set WriteEnable.

ＳＥＴ命令中にＫ２ＭＸ変数に書き込まれるビットは１であり、ＣＬＲ命令中は０である。入力ビットのソースとして、オペコードの下位ビット（ｂｉｔ_４）を使用するのが好都合である。アドレス生成中に、組み合わせロジックとして実装された真理表は、以下の通り、ベースアドレスの一部を決定する。

The bit written to the K2MX variable during the SET instruction is 1 and 0 during the CLR instruction. It is convenient to use the lower bit (bit ₄ ) of the opcode as the source of the input bits. The truth table implemented as combinational logic during address generation determines part of the base address as follows:

真理表は５ビットの出力を生成するが、下位４ビットは４ビットアダーへ送られ、そこで、下位４ビットはインデックス値（Ｃ１、Ｎ又はオペランド自体の下位３ビット）に加算される。最上位ビットはアダーへ送られ、５ビット結果を生成するために、アダー結果からの４ビット結果の前に付加される。アダーへの第２の入力はマルチプレクサＭＸ_１に由来し、マルチプレクサＭＸ_１は、Ｃ１、Ｎ、及びオペランド自体の下位３ビットからインデックス値を選択する。Ｃ１は３ビットしかないが、４番目のビットは定数０である。マルチプレクサＭＸ_１は以下の選択規準をもっている。

The truth table produces a 5-bit output, but the lower 4 bits are sent to the 4-bit adder, where the lower 4 bits are added to the index value (C1, N or the lower 3 bits of the operand itself). The most significant bit is sent to the adder and is prepended to the 4-bit result from the adder result to produce a 5-bit result. A second input to adder is derived from multiplexer MX _1, multiplexer MX ₁ selects an index value from the lower 3 bits of the C1, N, and operand itself. C1 has only 3 bits, but the fourth bit is a constant 0. Multiplexer MX ₁ has the following selection criteria:

実効アドレスの第６ビット（ｂｉｔ_５）は、ＲＡＭアドレスの場合に０であり、フラッシュメモリアドレスの場合に１である。フラッシュメモリアドレスは、ＭＴ、ＡＭ、Ｒ、Ｋ、及びＭである。ｂｉｔ_５の計算はＬｏｇｉｃ_２によって与えられる。

The sixth bit (bit ₅ ) of the effective address is 0 for a RAM address and 1 for a flash memory address. The flash memory addresses are MT, AM, R, K, and M. The calculation of bit ₅ is given by Logic ₂ .

定数１ビットが表現され、実効アドレスは全部で７ビットになる。これらのビットは、Ｋ２ＭＸが０でなく、命令がＬＤ、ＡＤＤ又はＳＴ
Ｍ［Ｃ１］ではない限り、実効アドレスを形成する。後者では、実効アドレスは定数アドレス０００００００である。両方のケースで、２個の０ビットが最終的な９ビットアドレスを形成するため前に追加される。この計算は、以下に示され、Ｌｏｇｉｃ_３及びマルチプレクサＭＸ_２によって行われる。

One constant bit is expressed, and the effective address is 7 bits in total. For these bits, K2MX is not 0 and the instruction is LD, ADD or ST
An effective address is formed unless M [C1]. In the latter, the effective address is a constant address 0000000. In both cases, two 0 bits are added before to form the final 9-bit address. This calculation is shown below and is performed by Logic ₃ and multiplexer MX ₂ .

ＣＬＲＧＥＮ
図２０１は、ＣＬＲＧＥＮユニットの概略ブロック図である。ＣＬＲＧＥＮユニットはＣＬＲ命令のためのアドレスを生成する。実効アドレスは常に、有効メモリアクセスオペランド用のフラッシュメモリにあり、無効オペランドに対して０である。ＣＬＲ
Ｍ［Ｃ１］命令は、（ＶＡＲＧＥＮユニットに保持された）Ｋ２ＭＸフラグの状態とは無関係に、常に、Ｍ［Ｃ１］を消去する。真理表は、単純な組み合わせロジックであり、以下の関係を実現する。

CLRGEN
FIG. 201 is a schematic block diagram of the CLRGEN unit. The CLRGEN unit generates an address for the CLR instruction. The effective address is always in flash memory for valid memory access operands and is 0 for invalid operands. CLR
The M [C1] instruction always clears M [C1] regardless of the state of the K2MX flag (held in the VARGEN unit). The truth table is a simple combinational logic and realizes the following relationship.

真理表に必要なロジックを削減することは簡単である。なぜならば、全部で４種類の主要なケースにおいて、実効アドレスの最初の６ビットは００であり、その後にオペランド（ｂｉｔ_３−０）が続く。

It is easy to reduce the logic required for the truth table. This is because, in all four main cases, the first 6 bits of the effective address are 00, followed by the operand (bit _3-0 ).

BITGEN
FIG. 202 is a schematic block diagram of the BITGEN unit. The BITGEN unit generates addresses for ROR and SET instructions. The effective address is always in flash memory for valid memory access operands and is 0 for invalid operands. The ROR and SET instructions access only the IST and ISW flash memory addresses (the rest of the operands access the registers), and a simple combinatorial logic truth table is sufficient for address generation.

カウンタユニット
図Ｙ３７は、カウンタユニットの概略ブロック図である。カウンタユニットは、カウンタＣ１、Ｃ２（内部で使用）と、選択されたＮインデックスと、を生成する。更に、カウンタユニットは、外部で使用するための出力フラグＣ１Ｚ及びＣ２Ｚを出力する。レジスタＣ１及びＣ２は、ＤＢＲ命令又はＳＣ命令のターゲットであるときに更新される。オペランドのハイビット（実効コマンドのｂｉｔ_３）は、Ｃ１とＣ２の間で選択を行う。Ｌｏｇｉｃ_１及びＬｏｇｉｃ_２は、それぞれ、Ｃ１及びＣ２のためのＷｒｉｔｅＥｎａｂｌｅを判定する。

Counter Unit Diagram Y37 is a schematic block diagram of the counter unit. The counter unit generates counters C1, C2 (used internally) and the selected N index. Further, the counter unit outputs output flags C1Z and C2Z for external use. Registers C1 and C2 are updated when they are targets of DBR or SC instructions. The high bit of the operand (bit _{3 of the} effective command) selects between C1 and C2. Logic ₁ and Logic ₂ determine the Write Enable for C1 and C2, respectively.

シングルビットフラグＣ１Ｚ及びＣ２Ｚは、それらのマルチビットＣ１及びＣ２の対応した部分のＮＯＲ演算によって生成される。このようにして、Ｃ１Ｚは、Ｃ１＝０のときに１であり、Ｃ２Ｚは、Ｃ２＝０のときに１である。ＤＢＲ命令中に、Ｃ１又はＣ２の何れかの値が１ずつデクリメントされる（ラップ付き）。デクリメンターユニットへの入力は、以下の通り、マルチプレクサＭＸ_２によって選択される。

The single bit flags C1Z and C2Z are generated by NOR operation of corresponding portions of the multibits C1 and C2. Thus, C1Z is 1 when C1 = 0, and C2Z is 1 when C2 = 0. During the DBR instruction, the value of either C1 or C2 is decremented by 1 (with wrap). The input to decremental Mentor units, as follows, are selected by multiplexer MX _2.

Ｃ１又はＣ２に書き込まれた実際の値は、ＤＢＲ命令又はＳＣ命令が実行されたかどうかに依存する。マルチプレクサＭＸ_１は、デクリメンターからの出力（ＤＢＲ命令用）と、真理表からの出力（ＳＣ命令用）との間で選択を行う。５ビット出力のうちの最下位３ビットだけがＣ１に書き込まれることに注意する必要がある。マルチプレクサＭＸ_１は、したがって、以下の選択規準をもっている。

The actual value written to C1 or C2 depends on whether a DBR or SC instruction has been executed. The multiplexer MX ₁ selects between the output from the decrementer (for the DBR instruction) and the output from the truth table (for the SC instruction). Note that only the least significant 3 bits of the 5-bit output are written to C1. Multiplexer MX ₁ therefore has the following selection criteria:

真理表は、ＳＣ命令を用いて、Ｃ１及びＣ２へロードされるべき値を保持する。真理表は、以下の関係を実現する簡単な組み合わせロジックである。

The truth table holds the values to be loaded into C1 and C2 using the SC instruction. A truth table is a simple combinational logic that achieves the following relationship:

レジスタＮ１、Ｎ２、Ｎ３及びＮ４は、ＸＯＲ命令によって参照されたとき、次の値−１（ラップ付き）によって更新される。レジスタＮ４は、ＳＴ
Ｘ［Ｎ４］命令が実行されるときにも更新される。ＬＤ及びＡＤＤ命令はＮ４を更新しない。その上、全部で４個のレジスタはＳＥＴ
Ｎｘコマンド中に更新される。Ｌｏｇｉｃ_４−７は、レジスタＮ１からＮ４のＷｒｉｔｅＥｎａｂｌｅを生成する。どれもがＬｏｇｉｃ_３を使用し、Ｌｏｇｉｃ_３は、コマンドがＳＥＴ
Ｎｘである場合に１を生成し、そうでなければ０を生成する。

Registers N1, N2, N3 and N4 are updated with the next value -1 (with wrap) when referenced by the XOR instruction. Register N4 is ST
It is also updated when the X [N4] instruction is executed. LD and ADD instructions do not update N4. In addition, all four registers are SET
Updated during Nx command. Logic _4-7 generates Write Enable for registers N1 to N4. None is using the Logic _3, Logic _3, the command SET
If it is Nx, 1 is generated, otherwise 0 is generated.

送出された、又はデクリメンターへの入力として使用された実際のインデックス値は、マルチプレクサＭＸ_４がオペランドの下位２ビットを使用して簡単に選択する。

The actual index value sent or used as input to the decrementer is simply selected by the multiplexer MX ₄ using the lower 2 bits of the operand.

インクリメンターは、（マルチプレクサＭＸ_４によって選択された）入力値のうちの４ビットを取得し、１を加え、４ビット結果を生成する（モジュロー２^４の加法による）。最後に、マルチプレクサＭＸ_３は４時点で、（Ｎ毎に異なり、ＳＥＴ
Ｎｘコマンド中にロードされる）定数値と、（ＸＯＲ又はＳＴ命令中の）デクリメンターの結果と、の間で選択を行う。その値は、適当なＷｒｉｔｅＥｎａｂｌｅフラグがセットされている場合に限り書き込まれ（Ｌｏｇｉｃ_４−Ｌｏｇｉｃ_７を参照せよ）、Ｌｏｇｉｃ_３はマルチプレクサのために安全に使用できる。

The incrementer takes 4 bits of the input value (selected by multiplexer MX ₄ ), adds 1 and produces a 4 bit result (by modulo 2 ⁴ addition). Finally, the multiplexer MX ₃ is at 4 time points (different every N, SET
A choice is made between a constant value (loaded during the Nx command) and the decrementer result (in an XOR or ST instruction). Its value is written only if the appropriate WriteEnable flag is set (see Logic _4- Logic ₇ ), and Logic ₃ can be safely used for the multiplexer.

ＳＥＴ Ｎｘコマンドは、Ｎ１からＮ４に以下の定数をロードする。

The SET Nx command loads the following constants from N1 to N4.

参照される各初期Ｘ［Ｎｎ］は、最適化されたＳＨＡ−１アルゴリズムのインデックスＮ１からＮ４の初期状態と一致する。各インデックス値Ｎｎがデクリメントするとき、実効Ｘ［Ｎ］はインクリメントする。その理由は、Ｘワードは最上位ワードから先にメモリに格納されているからである。３個のＶＡＬユニットは、タンパー防止及び検出回路に接続された検証ユニットであり、各ＶＡＬはＯＫビットを含む。ＯＫビットはリセットで１にセットされ、サイクル毎に両方のタンパー防止及び検出ラインからのＣｈｉｐＯＫ値とＯＲ演算される。ＯＫビットはそのユニットを通過する各データビットとＡＮＤ演算される。全てのＶＡＬユニットは、データビットをパリティチェックし、カウンタが改ざんされていないことを保証する。パリティチェックが失敗であるならば、消去タンパー検出ラインがトリガーされる。ＶＡＬ_１の場合、カウンタＣ１からの実効出力は、チップが改ざんされているならば、常に０である。これは、攻撃者が鍵の最初から最後までインデックス付けをするループ構造を実行することを阻止する。ＶＡＬ_２の場合、カウンタＣ２からの実効出力は、チップが改ざんされたならば、常に０である。これは、攻撃者がループ構造を実行することを阻止する。ＶＡＬ_３の場合、Ｎカウンタ（Ｎ１−Ｎ１４）からの実効出力は、チップが改ざんされているならば、常に０である。これは、攻撃者がＸの最初から最後までインデックス付けをするループ構造を実行することを阻止する。

Each initial X [Nn] referenced matches the initial state of the optimized SHA-1 algorithm indices N1 to N4. As each index value Nn decrements, effective X [N] increments. The reason is that the X word is stored in the memory before the most significant word. The three VAL units are verification units connected to the tamper prevention and detection circuit, and each VAL includes an OK bit. The OK bit is set to 1 at reset and ORed with the ChipOK value from both tamper prevention and detection lines every cycle. The OK bit is ANDed with each data bit passing through the unit. All VAL units parity check the data bits to ensure that the counter has not been tampered with. If the parity check is unsuccessful, an erase tamper detection line is triggered. For VAL ₁ , the effective output from counter C1 is always 0 if the chip has been tampered with. This prevents an attacker from executing a loop structure that indexes from the beginning to the end of the key. In the case of VAL ₂ , the effective output from the counter C2 is always 0 if the chip has been tampered with. This prevents an attacker from executing the loop structure. For VAL ₃ , the effective output from the N counter (N1-N14) is always 0 if the chip has been tampered with. This prevents an attacker from executing a loop structure that indexes from the beginning to the end of X.

Next, referring to FIG. 203, information 705 stored in the flash memory storage device 701 is shown. This data includes:

Factory code The factory code is a 16-bit code indicating the factory where the print roll is manufactured. This identifies the factory that belongs to the owner of the print roll technology or that manufactures the print roll with permission. The purpose of this number is to be able to track the factory from which the print roll originates if there is a quality problem.

Batch number The batch number is a 32-bit number indicating a print roll manufacturing batch. The purpose of this number is to track the batch from which the print roll comes if there is a quality problem.

The serial number 48-bit serial number is provided so that each print roll can be uniquely identified up to 28 trillion.

Manufacturing Date A 16-bit manufacturing date is incorporated to track the age of the print roll when the expiration date is limited.

The length of the print media remaining on the media length roll is represented by this number. This length is expressed in small units, such as millimeters or the minimum dot pitch of printers using print rolls, and the remaining photos in the well-known C, H and P formats, as well as other formats to be printed. Allows calculation of numbers. Using this small unit ensures that a high resolution can be used to keep in sync with the pre-printed media.

Media type Media type data lists the media stored in the print roll:
(1) Transparent (2) Milky white (3) Opaque coloring (4) Three-dimensional lenticular (5) Pre-printing: Length limited (6) Pre-printing: Non-length limited (7) Metal foil (8) Holographic / Optical Variable device foil.

Pre-print media length For example, the length of the repetitive pattern of the pre-print media accommodated on the back surface of the print roll is stored here.

Ink viscosity The viscosity of each ink color is stored as an 8-bit numerical value. The ink viscosity number is used in adjusting printhead actuator characteristics to compensate for viscosity (typically, higher viscosity requires longer actuator pulses to achieve the same drop volume) .

Recommended drop volume for 1200 dpi The recommended drop volume for each ink color is stored as an 8-bit numerical value. The most appropriate drop volume depends on the ink and print media characteristics. For example, the required drop volume decreases as the dye concentration or absorbency increases. In addition, transparent media requires about twice the drop volume of milky white media. This is because light passes through the dye layer only once for transparent media.

Since the print roll contains both ink and media, customer adaptation can be achieved. Drop volume is only a recommended drop match. This is because the printer may not be 1200 dpi and the printer may be adjusted for the lightness of the print.

Each color of the ink color dye color is stored and can be used to fine tune the digital halftone applied to the image before printing.

The length of the print medium remaining in the remaining media length indicator roll is expressed by a numerical value and updated by the camera device. The length is expressed in small units (eg 1200 dpi pixels) so that the number of remaining pictures can be calculated in C, H and P formats, as well as other formats to be printed. High resolution may be used to keep in sync with the pre-print media.

Copyright or Bit Pattern This 512 bit pattern represents an ASCII string sufficient to allow copyright protection of the stored flash memory contents.

Referring to FIG. 204, an Artcam authentication chip storage table 730 is shown. The table contains the manufacturing code, batch number, serial number, and date, and is the same format as described above. Table 730 further includes information 731 on the print engine in the Artcam device. The accumulated information includes the printer engine type, the DPI resolution of the printer, and the printer count of the number of prints created by the printer device.

In addition, an authentication test key 710 is provided, which varies randomly between chips and is also used as an Artcam random identification code in the above algorithm. A 128-bit print roll authentication key 713 is also provided, which is equivalent to a key stored in the print roll. The 512-bit pattern is then saved, followed by a 120-bit spare area suitable for Artcam usage.

As described above, Artcam preferably includes a liquid crystal display 15, which indicates the number of prints left on the print roll stored in Artcam. In addition, Artcam includes a tri-state switch 17, which allows the user to switch between three standard formats C, H and P (Classic, HDTV and Panorama). After switching between the three states, the liquid crystal display 15 is updated to reflect the number of images remaining on the print roll if the particular format selected is used.

In order for the liquid crystal display to operate correctly, the Artcam processor inserts the print roll and passes the authentication test, and then reads from the flash memory storage device of the print roll chip 53 to determine the remaining amount of paper. Next, the value of the output format selection switch 17 is determined by the Artcam processor. By dividing the print length by the corresponding length of the selected output format, the Artcam processor determines the printable number and updates the liquid crystal display 15 with the remaining print number. When the user changes the output format selection switch 17, the Artcam processor 31 recalculates the number of output pictures according to the format, and updates the LCD display 15 again.

By storing process information in the print roll table 705 (FIG. 165), the Artcam device can take advantage of process changes and print roll print characteristic changes.

In particular, the pulse characteristics applied to each nozzle in the print head can be changed to account for changes in process characteristics. Referring to FIG. 205, the Artcam processor may be adapted to run software programs stored in the auxiliary memory ROM chip. A pulse profile characterizer 771, which is a software program, can read a number of variables from the print roll. These variables include the remaining roll media on print roll 772, printer media type 773, ink color viscosity 774, ink color drop volume 775, and ink color 776. Each of these variables is read by a pulse profile characterizer and the corresponding most suitable pulse profile is determined by conventional trial and error methods. The parameter changes the printer pulse received at each printer nozzle to increase the stability of the ink output.

As is apparent, the authentication chip has made significant progress in that important and valuable information is stored in a printer chip integral with the printer roll. This information includes the process characteristics of the target print roll, along with the type of print roll and the amount of paper remaining on the print roll.

Furthermore, the print roll interface chip can provide valuable authentication information and can be constructed in a tamper resistant manner. Also provided is a tamper resistant method utilizing this chip. The use of a print roll chip can provide a convenient and efficient user interface for the quick output form of Artcam devices, can output multiple photo formats, and at the same time the number of photos remaining on the printing device An indicator can be provided.

Printhead Unit FIG. 206 shows an exploded perspective partial cross-sectional view of the printhead unit 615 of FIG. 162.

The print head unit 615 is centered on a print head 44 that ejects ink drops onto the print media 611 on demand to form an image. The print medium 611 is sandwiched between two sets of rollers including a first set 618 and 616 and a second set 617 and 619.

The print head 44 operates under the control of the power supply, ground, and signal line 810, and the power supply, ground, and signal line 810 provides power and control for the print head 44, and print heads by tape automatic bonding (TAB). It is joined to the surface of 44.

Importantly, the printhead 44, which can be constructed from a suitably separated silicon wafer device, relies on a series of anisotropic etches 812 and has substantially vertical sidewalls through the wafer. Etch 812 through the wafer can supply ink directly to the printhead surface from the backside of the wafer for later removal.

Ink is supplied to the back side of the inkjet print head 44 using an ink head supply unit 814. Inkjet printhead 44 has three separate rows along its surface to supply inks of different colors. The ink head supply unit 814 includes a lid 815 that seals the ink channel.

A number of perspective views of the ink head supply unit 814 are shown in FIGS. Each of FIGS. 207 to 210 shows only a portion of an ink head supply unit that is not limited in length, and the portions shown show typical details. 207 shows a bottom perspective view, FIG. 148 shows an upper perspective view, FIG. 209 shows an enlarged perspective partial sectional view, FIG. 210 shows an upper perspective view showing details of the ink channel, and FIG. Is an upper perspective view and FIG. 212 is the same.

For example, the ink head supply unit 814 is considerably advantageous in terms of cost when formed from injection-molded plastic rather than from micromachined silicon. The manufacturing cost of the plastic ink channel is very low and the manufacturing is sufficiently easy. The design shown in the accompanying drawings assumes a 1600 dpi three-color monolithic printhead of a predetermined length. The given flow rate calculation is for a 100 mm photo printer.

The ink head supply unit 814 includes all necessary fine portions. A lid 815 (FIG. 206) is permanently bonded or ultrasonically welded to the ink head supply unit 814 to provide a seal for the ink channel.

Referring to FIG. 209, cyan, magenta, and yellow ink flow into ink inlets 820-822, and the magenta ink flows through through holes 824 and 825, along the magenta main channels 826 and 827 (FIG. 141). Flowing. The cyan ink flows along the cyan main channel 830, and the yellow ink flows along the yellow main channel 831. As best shown in FIG. 209, the cyan ink in the cyan ink main channel flows into the cyan sub-channel 833. Similarly, the yellow sub-channel 834 receives yellow ink from the yellow main channel 831.

As best shown in FIG. 210, magenta ink also flows from magenta main channels 826, 827 to magenta through holes 836, 837. Returning to FIG. 209, the magenta ink flows out from the through holes 836 and 837. Magenta ink flows along the magenta subchannel, eg, 838, then flows along the second magenta subchannel, eg, 839, and then flows into the magenta trough 840. The magenta ink then flows through a magenta via, eg, 842, which is aligned with a corresponding inkjet head through hole (eg, 812 in FIG. 166) that then prints the ink. To the inkjet nozzle.

Similarly, the cyan ink in the cyan subchannel 833 flows into the cyan pit area 849, which supplies the ink to the two cyan vias 843, 844. Similarly, the yellow sub-channel 834 supplies to the yellow pit area 46, and the yellow pit area supplies to the yellow vias 847 and 848.

As shown in FIG. 210, the printhead is designed to be accommodated in a printhead slot 850, and various vias, eg, 851, are aligned with corresponding through-holes, eg, 851, in the printhead wafer.

Returning to FIG. 206, care should be taken to ensure that the appropriate ink flows throughout the printhead chip 44 and at the same time meets the constraints of the injection molding process. The size of the ink through wafer hole 812 on the back side of the print head chip is about 100 μm × 50 μm, and the gap between the through holes carrying different color inks is about 170 μm. Shapes of this size can be easily plastic molded (compact discs are of micron size) and ideally the wall height is more than several times the wall thickness to maintain proper stiffness Must not. The preferred embodiment solves this problem by using a progressively smaller ink channel hierarchy.

In FIG. 211, a small portion 870 of the surface of the print head 44 is shown. The surface is divided into three series of nozzles composed of a cyan series 871, a magenta series 872, and a yellow series 873. Each series of nozzles is further divided into two rows, eg, 875, 876, and the printhead 44 includes a series of bond pads 878 that bond power and control signals.

The printhead is preferably constructed according to a very large number of different forms of inkjet created for applications involving Artcam devices. These inkjet devices are described in detail below.

The printhead nozzle includes an ink supply channel 880 that is equivalent to the anisotropic etch hole 812 of FIG. Ink flows from the back side of the wafer through supply channel 881 and then through filter grill 882 to an ink nozzle chamber, eg, 883. The operation of the nozzle chamber 883 and the print head 44 (FIG. 1) is as described above and is described in the above-mentioned patent specification.

Ink Channel Fluid Flow Analysis Next, the ink flow analysis will be described. The main ink channels 826, 827, 830, and 831 (FIGS. 207 and 141) are about 1 mm × 1 mm and are supplied to all nozzles of one color. To do. The subchannels 833, 834, 838, and 839 (FIG. 209) are about 200 μm × 100 μm, and one feeds to about 25 inkjet nozzles. Print head through holes 843, 844, 847, 848 and wafer through holes, for example 881 (FIG. 221), are 100 μm × 50 μm and feed to three nozzles on each side of the print head through hole. Each nozzle filter 882 has 8 slits, and each slit has an area of 20 μm × 2 μm and is supplied to one nozzle.

The pressure requirements of the inkjet printer configured as described above were analyzed. The analysis is for a 1600 dpi 3-color process printhead for photo printing. The print width is 100 mm, there are 6250 nozzles for each color, and a total of 18750 nozzles.

For full black printing, the maximum ink flow rate required for the various channels is important. This determines the pressure drop along the ink channel and, therefore, whether or not the printhead is filled with surface tension alone, and if not, the ink needed to keep the printhead full Determine the pressure.

To calculate the pressure drop, a 2.5 pI drop volume was utilized for 1600 dpi operation. The nozzle has the ability to operate faster, but the selected drop repetition rate is 5 kHz, which is appropriate for printing 150 mm long photos within 2 seconds. Thus, in extreme cases, the printhead has 18750 nozzles, all printing at a maximum of 5000 drops per second. This ink flow spreads throughout the ink channel hierarchy. Each ink channel effectively supplies a fixed number of nozzles when all nozzles are printing.

The pressure drop Δρ was calculated according to the Darcy-Weisbach equation:
Δρ = ρU ² fL / 2D
Where ρ is the density of the ink, U is the average flow velocity, L is the length, D is the hydraulic diameter, f is the dimensionless coefficient of friction, calculated with the following formula:
f = κ / Re
Where Re is the Reynolds number and κ is a dimensionless friction coefficient that depends on the cross-sectional area of the channel, calculated with the following formula:
Re = UD / ν
Where ν is the kinematic viscosity of the ink.

For rectangular sections, κ is
κ = 64 / (2/3 + (11b / 24a) (11b / 24a) (2-b / a))
Can be approximated by

In the formula, a is the long side of the rectangular cross section, and b is the short side. The hydraulic diameter D for a rectangular cross section is
D = 2ab / (a + b)
Given by.

Ink was removed from the main ink channel at 250 points along the length of the channel. Since the ink viscosity drops linearly from the tip of the channel and drops to zero at the end of the channel, the average flow rate U is half the maximum flow rate. Thus, the pressure drop along the main ink channel is half of the pressure drop calculated using the maximum flow rate.

The following table was obtained by calculating the pressure drop using these equations.

Ink channel dimensions and pressure drop table

インクインレットからノズルまでの総圧力降下は、したがって、シアン及びイエローに関して約７０１Ｐａであり、マゼンタに関して８４５Ｐａである。これは、大気圧の１％未満である。勿論、印刷された画像がフルブラック未満の場合、インクフロー（したがって、圧力降下）はこれらの値よりも低下するであろう。

The total pressure drop from the ink inlet to the nozzle is therefore about 701 Pa for cyan and yellow and 845 Pa for magenta. This is less than 1% of atmospheric pressure. Of course, if the printed image is less than full black, the ink flow (and hence the pressure drop) will drop below these values.

Mold Production for Ink Head Supply Unit The ink head supply unit 14 (FIG. 1) has an outer shape of 50 μm and a length of 106 mm. It is not feasible to process the injection molding tool in a conventional manner. However, even if the overall shape is complex, complex curves are not necessary. Injection molding tools can be manufactured using conventional milling for main ink channels and other millimeter scale shapes, and fine shapes use lithographically produced insets. The LIGA process can be used for inset.

A single injection molding tool simply includes more than 50 cavities. Most of the complexity of the tool is inset.

Referring to FIG. 206, the printing system integrally molds ink supply unit 814 and lid 815 and seals them together as described above. Next, the print head 44 is installed in the corresponding slot 850. Adhesive sealing strips 852, 853 are placed on the magenta main channel to ensure that they are properly sealed. A tape automatic bonding (TAB) strip 810 is connected to the inkjet printhead 44 and the tab bonding wire passes into the cavity 855. As can be seen from FIGS. 206, 207 and 212, aperture slots 855 to 862 are provided for snapping upon roller insertion. The slot “clipping in” the roller with some play and then easily rotates the roller.

213 to 217 show various perspective views of the interior of the finally assembled Artcam device, with the devices numbered appropriately:
FIG. 213 is a top perspective view of the interior of the Artcam camera showing the flattened parts;
FIG. 214 is a perspective view of the bottom side inside the Artcam camera showing the flattened parts;
FIG. 215 is a perspective view of the first top side inside the Artcam camera, showing the components housed in the Artcam;
FIG. 216 is a perspective view of the second top side inside the Artcam camera, showing the components housed in the Artcam;
FIG. 217 is a perspective view of the second upper surface side inside the Artcam camera, and shows components housed in the Artcam.

Postcard Print Roll Referring to FIG. 218, in one preferred embodiment, the output printer paper 11 includes a number of pre-printed “postcard” type endorsing portions 885 on the non-image printed side. The postcard-type section 885 includes a prepaid postage “stamp” 886, which consists of a printed certificate from the relevant postal authority and within its jurisdiction its print roll. Are sold or used. Depending on the agreement with the relevant jurisdiction's postal authorities, print rolls can use various postage charges. This is particularly useful when a foreign traveler is within the jurisdiction of the country and sends a large number of postcards to the home country. In addition, an address form portion 887 is provided to provide details of the destination shipment in the form of a normal postcard. Finally, a message area 887 is provided where a personal message can be entered.

The operation of the camera apparatus will be described with reference to FIGS. 218 and 219. FIG. 218 shows the corresponding end face when a series of images 890 to 892 are printed on the first side of the print roll. Yes. Thus, as each image, eg 891, is printed by the camera, the back side of the image is already a postcard 885 and can be immediately posted to the nearest postbox in the jurisdiction. In this way, a personalized postcard can be created.

Obviously, only a predetermined image size is possible when utilizing a postcard system as shown in FIGS. 219 and 220. This is because the correspondence between the postcard portion 885 on the back surface and the image 891 on the front surface must be maintained. This is accomplished by utilizing the memory portion of the authentication chip housed in the print roll to store details of the length of each postcard endorsement form sheet 885. This can also be achieved by making each postcard the same size, or storing each size in the on-board print chip memory of the print roll.

When the Artcam camera control system utilizes pre-formatted postcards, the print roll can ensure that each image is only used to print images so that they fit within the postcard boundaries. . Of course, a certain amount of “play” is provided by providing a border region at the edge of each photo so that some alignment errors can be taken into account.

As is apparent from FIG. 220, the postcard roll can be pre-purchased by the camera user when traveling within the specific jurisdiction where it is available. The postcard roll has on its outer surface the country of purchase, the postage of each postcard, the format of each postcard (eg C, H, P, or a combination of these image modes), the country name suitable for use, And information including postage deadlines that may result in a postage shortage after that date.

Thus, the user of the camera device uses his handheld camera directed to the relevant scene, takes a picture, the image is posted on one side, and the prepaid postcard on the other side It is possible to produce a postcard for shipping by mail with the details of. The postcard can then be addressed, a short message can be written, and then mailed out immediately.

Various software designs are possible with respect to the software operation of the Artcam device, but in one design, each Artcam device is composed of a set of loosely coupled functional modules, which can be represented by a single embedded application. Made available in linked form and serves for the central purpose of the device. Functional modules are reused in different combinations on different classes of Artcam devices, and applications are specific to the Artcam class.

Most functional modules include both software and hardware components. The software is shielded from hardware details by the hardware abstraction layer, while the user of the module is shielded from the software implementation by the abstract software interface. Since the system as a whole is driven by user-initiated and hardware-initiated events, most modules can run one or more synchronous event-driven processes.

The most important modules that make up a typical Artcam device are shown in FIG. In this figure and subsequent figures, the software component is shown on the left side and separated from the right side hardware component by a vertical dotted line 901. The software aspects of these modules are described next.

Software Module-Artcam Application 902
The Artcam application implements the higher level functionality of the Artcam device. This usually involves taking an image, applying an artistic effect to the image, and printing the image. In the case of a camera-oriented Artcam device, the image is captured by the camera manager 903. In the case of a printer-oriented Artcam device, the image is captured by the network manager 904, which is probably the result sent by another device.

The artistic effect takes place within an integrated file system managed by the file manager 905. An artistic effect consists of a script file and a set of resources. The script is translated by the image processing manager 906 and applied to the image. Scripts are typically shipped at ArtCards, known as Artcard. As a default, the system uses a script stored in the currently installed Artcard.

The image is printed by the print manager 908.

When the Artcam device starts up, the bootstrap process starts various manager processes before starting the application. This allows applications to request services from various managers as soon as they start.

After initialization, the application 902 registers itself as a handler for the events listed below. When an application receives an event, it performs the actions listed in the table.

カメラがディスプレイを含むとき、アプリケーションはユーザインタフェースマネージャ９１０によってグラフィカルユーザインタフェースを構築し、ユーザは現在日時、及びその他の編集可能なカメラパラメータを編集できるようになる。アプリケーションは、全ての常駐パラメータをフラッシュメモリに保存する。

When the camera includes a display, the application builds a graphical user interface with the user interface manager 910, allowing the user to edit the current date and time and other editable camera parameters. The application saves all resident parameters in flash memory.

Real-time microkernel 911
The real-time microkernel schedules processes in advance based on interrupts and process priorities. It provides an integrated interprocess communication and timer service that is very close to process scheduling. All other operating system functions are performed outside the microkernel.

Camera manager 903
The camera manager provides an image capture (imaging) service. It controls the camera hardware built into Artcam. It provides an abstract camera control interface and allows querying and setting of camera parameters and image capture. This abstract interface separates the application from the details of other camera implementations. The camera manager uses the following input / output parameters and commands.

カメラマネージャは、非同期イベント駆動型プロセスとして動く。それは、連結された状態機械の組を含み、各状態機械は一つの非同期オペレーションに対応する。それらは、オートフォーカス、フラッシュ充電、セルフタイマのカウントダウン、及び像の撮影を含む。初期化時、カメラマネージャはカメラハードウェアを既知状態にセットする。これは、通常の焦点距離の設定、及びズームの戻しを含む。カメラマネージャのソフトウェア構造は、図２２２に示されている。ソフトウェアコンポーネントは以下のサブセクションに記載されている。

The camera manager runs as an asynchronous event driven process. It includes a set of linked state machines, each state machine corresponding to one asynchronous operation. They include autofocus, flash charging, self-timer countdown, and image taking. At initialization, the camera manager sets the camera hardware to a known state. This includes normal focal length setting and zoom back. The software structure of the camera manager is shown in FIG. Software components are described in the following subsections.

Focus fixed 913
Lock focus automatically adjusts the focus and exposure for the current scene according to the focus control mode, exposure control mode, and flash mode, and makes the flash operable when necessary. The lock focus is usually started when the user presses the shooting button halfway. This is part of the normal image capture sequence, but is separated from actual image capture in time if the user keeps the capture button pressed halfway. This allows the user to perform spot forcing and spot ranging.

Shooting 914
Shooting (image capture) captures an image of the current scene. When the flash mode includes red eye removal, the red eye lamp is turned on, the shutter is controlled, and if it is operable, the flash is triggered and the image is detected by the image sensor. It determines the orientation of the camera, that is, the orientation of the captured image, so that the image can be properly oriented in subsequent image processing. It also detects the presence or absence of camera movement during imaging in order to remove blur during subsequent image processing.

Self-time shooting 915
In self-time shooting, an image of the current scene is shot after a countdown of a 20-second timer. This provides feedback to the user using a self-timer LED during the countdown. In the first 15 seconds it can turn on the LED. The LED blinks in the last 5 seconds.

View scene 917
The view scene periodically senses the current scene through an image sensor, displays the scene on a color LCD, and provides the user with an LCD-based viewfinder.

Autofocus 918
Autofocus changes the focal length until the selected area of the image is sharp enough to be said to be in focus. It presupposes that the region is in focus when the image sharpness metric extracted from the specified region of the image sensor exceeds a fixed threshold. It performs a gradient descent method on the sharpness differentiation and finds the optimal focal length by changing direction and step size as needed. If the focus control mode is multi-point auto, three areas arranged horizontally across the field of view are used. If the focus control mode is single point auto, one region in the center of the field of view is used. Autofocus functions within the effective focal length specified by the focus controller. In the case of fixed focus devices, this is effectively disabled.

Auto flash 919
Autoflash determines if the scene illuminance is dim enough to require a flash. This assumes that the illuminance is sufficiently dim if the illuminance of the scene is below a predetermined threshold. The illuminance of the scene is obtained from the illuminance sensor, which extracts the illuminance metric from the central area of the image sensor. If a flash is needed, charge the flash.

Automatic exposure 920
The scene illuminance, aperture, and shutter speed determine the exposure of the captured image. The desired exposure is a fixed value. If the exposure control mode is auto, automatic exposure determines the combination of aperture and shutter speed that produces the desired exposure for a given scene illuminance. If the exposure control mode is aperture priority, automatic exposure determines the shutter speed that produces the desired exposure for a given scene illuminance and current aperture. If the exposure control mode is shutter priority, automatic exposure determines the aperture that produces the desired exposure for a given scene illuminance and current shutter speed. The scene illuminance is acquired from the illuminance sensor, and the illuminance sensor extracts an illuminance metric from the central area of the image sensor.

Automatic exposure functions within the effective aperture and shutter speed ranges specified by the aperture controller and shutter speed controller. The shutter speed controller and shutter controller hide the lack of a mechanical shutter in most Artcam devices.

If the flash is enabled manually or auto, the effective shutter speed is the flash interval, which is typically 1/1000 second to 1/10000 second.

Image processing manager 906 (FIG. 221)
The image processing manager provides image processing and artistic effect services. This utilizes a VLIW vector processor built into Artcam to perform high speed image processing. The image processing manager includes an interpreter for a script written in the Vark image processing language. Thus, artistic effects include Vark script files and associated resources such as fonts and clip images. The image processing manager software structure is shown in more detail in FIG. 223 and includes the following modules:

Image conversion and enhancement 921
The image processing manager executes image processing in a CIE LAB color space that is device independent, at a resolution suitable for the generation capability of the Artcam printer hardware. The captured image is first enhanced by filtering the noise. It is optionally done to remove the blur effect caused by the movement. The image is then extracted from the device dependent RGB color space from the CIE.
Converted to LAB color space. The image is rotated to cancel the effects of camera rotation during shooting and scaled to work image resolution. The image is enhanced by scaling its dynamic range to an available dynamic range.

Face detection 923
Faces are detected in images taken based on hue and local feature analysis. The list of detected face regions is used by the Vark script to apply face-only effects such as warping and balloon placement.

Vark image processing language interpreter 924
Vark includes a general purpose programming language that is extended with a rich set of image processing. It includes a range of primitive data types (integer, real, Boolean, character), a range of aggregate data types (arrays, strings, records) to build more complex types, a rich set of arithmetic and logical operations, Provides conditional and iterative control flows (if-then-else, while-do), and recursive functions and procedures. It also covers a range of image processing data types (image, clip image, matte, color, color lookup table, palette, dither matrix, convolution kernel, etc.), graphic data type (font, text, path), image processing It provides a set of functions (color conversion, composite processing, filtering, spatial conversion and warping, illumination, text setting and rendering), and a set of high-level artistic functions (tiling, painting, and stroking).

The Vark program is portable in two ways. Since it is interpreted, it does not depend on the CPU and host image processing engine. Since a device-independent model space and device-independent color space are used, it is independent of input color characteristics, host input device resolution, output color properties, and host output device resolution.

The Vark interpreter 924 analyzes the source statements constituting the Vark script and generates a parse tree that expresses the meaning of the script. Parse tree nodes correspond to program statements, expressions, sub-expressions, variables, and constants. The root node corresponds to the main procedure statement list. The interpreter executes the program by executing the root statement of the parse tree. Each node in the parse tree requires its children to evaluate or execute itself appropriately. For example, the if statement node has three children: a conditional expression node, a then statement node, and an else statement node. The if statement causes the conditional expression node to evaluate itself and cause the then or else statement to execute itself based on the returned Boolean value. It does not know the actual conditional expression or the actual statement.

Most data type operations are performed during the parse tree execution, but image data type operations are postponed until after the parse tree execution. This optimizes the image operation and only the intermediate pixels that contribute to the final image are calculated. This allows the final image to be calculated in multiple passes with spatial subdivision, thereby reducing the amount of memory required.

During the execution of the parse tree, each image function only returns an image graph, where the node represents the image operator, the leaf represents the image, the corresponding image operator is the root, and the image Constructed by making a parameter a child of the root. Of course, the image parameters are themselves image graphs. In this way, each sequential image function returns a deeper image graph.

After executing the parse tree, an image graph corresponding to the final image is obtained. This image graph is then executed in a depth-first manner (similar to an arbitrary expression tree) and is optimized in two ways: (1) Only pixels that contribute to the final image are calculated at a given node. (2) Node children are executed in an order that minimizes memory requirements. Image operators in the image graph are executed in an optimized order to produce the final image. A computationally intensive image operator is accelerated using a VLIW processor embedded in the Artcam device. When the amount of memory required to execute the image graph exceeds available memory, the final image area is subdivided until the required memory does not exceed available memory.

In the case of a well-constructed Vark program, the initial optimization does not seem to be particularly effective. However, if the final image area is subdivided, the optimization can have a significant effect. It is precisely this optimization that subdivision can be used as an effective technique to reduce memory requirements. One result of delaying execution of image operations is that the program control flow cannot be made dependent on image content. This is because the image content is not known during the execution of the parse tree. In practice, this is not so restrictive, but nevertheless, care must be taken during language design.

For the idea of postponing execution (or delay evaluation) of image operations, see Guibas
and Stolfi (Guibas,
L. J,. , And J.H. Stolfi, '&apos; A Language for Bitmap Manufacture'&apos;, ACM Transactions
on Graphics, Vol. 1,
No. 3, July 1982
pp. 191-214). They build an image graph as well during the execution of the program and back propagate the result region during subsequent graph evaluation to avoid computing pixels that do not contribute to the final image. Shantzi further propagates available pixels forward during image graph evaluation (Shantzis,
M.M. A. , '&Apos; A Model for Efficient and
Flexible
Image Computing '', Computer Graphics Proceedings, Annual
Conference
Series, 1994, pp. 147-154). The Vark interpreter uses the more sophisticated multipath bit directional region propagation scheme proposed by Cameron (Cameron,
S. , '&Apos; Efficient
Bounds in
Constructive
Solid Geometry '&pos;, IEEE Computer Graphics & Applications,
Vol. 11, no. 3,
May
1991, pp. 68-74). The method for optimizing the execution order to minimize memory usage is derived from Shantzis, but is based on standard compiler theory (Aho,
A. V. ,
R. Sehi,
and J. et al. D.
Ullman,
''Generating Code from DAGs'&apos;, in Compilers: Principles, Technologies,
and Tools,
Addison-Wesley, 1986, pp. 557-567). The Vark interpreter uses a more sophisticated scheme than Shantzi, but supports variable size image buffers. Subdividing the resulting region in combination with region propagation to reduce memory usage stems from Shantzis.

Printer manager 908 (FIG. 221)
The printer manager provides an image printing service. It controls the inkjet printer hardware embedded in Artcam. It provides an abstract printer control interface, allows querying and setting of printer parameters, and printing images. This abstract interface separates the application from the details of the printer implementation and includes the following variables:

プリンタマネージャは非同期イベント駆動型プロセスとして実行される。それは、連結された状態機械の組を含み、各状態機械は一つの非同期オペレーションに対応している。これらは、画像を印刷し、プリントロールを自動マウントする。プリンタマネージャのソフトウェア構造は図２２４に示されている。ソフトウェアコマンドは後述される。

The printer manager is executed as an asynchronous event driven process. It includes a set of linked state machines, each state machine corresponding to one asynchronous operation. They print images and automatically mount print rolls. The software structure of the printer manager is shown in FIG. The software command will be described later.

Print image 930
The print image prints the supplied image. This uses a VLIW processor circuit to prepare an image for printing. The VLIW processor circuit converts the image color space to device specific CMY and generates halftone bi-level data in the format expected by the printhead.

During printing, the paper can be pulled into the lip of the print roll and the nozzles can be capped to prevent ink leakage and drying so that the print roll can be removed. Before the actual printing starts, the nozzles are therefore uncapped and cleaned, and the paper is advanced to the printhead. Printing itself transfers line data from the VLIW processor, prints the line data, and advances the paper until the image is completely printed. After the printing is finished, the paper is cut by a cutting machine, drawn into a print roll, and a cap is put on the nozzle. The remaining media length is updated by the print roll.

Print roll automatic mount 131
The print roll automatic mount responds to the installation and removal of the print roll. It generates print roll loading and unloading events that are processed by the application and used to update the status display. The print roll is authenticated according to a protocol between an authentication chip incorporated in the print roll and an authentication chip incorporated in Artcam. If the print roll fails authentication, the print roll is rejected. Various information is extracted from the print roll. Paper and ink properties are used during the printing process. In any case, the remaining media length and the fixed page size of the media are published by the print manager and used by the application.

User interface manager 910 (FIG. 221)
The user interface manager is shown in more detail in FIG. 225 and provides user interface management services. It includes a physical user interface manager 911 that controls status display and input hardware and a graphical user interface manager 912 that manages a virtual graphical user interface on a color display. The user interface manager translates virtual and physical inputs into events. Each event is queued to the event queue of the process registered for that event.

File manager 905 (FIG. 222)
The file manager provides a file management service. It provides an integrated hierarchical file system, in which the file systems of all mounted volumes appear. The primary removable storage medium used in Artcam is ArtCard. Artcard is printed at high resolution by binary dot blocks, which directly represent Reed-Solomon encoded binary data that allows for errors. The block structure supports append and append-rewrite in a suitable read-write ArtCard device (not initially used in Artcam). At a higher level, ArtCard may include an extended write-once-rewritable ISO9660 compliant CD-ROM file system. The file manager software structure, and in particular the ArtCard device controller, is shown in FIG.

Network manager 904 (FIG. 222)
The network manager provides home appliance networking services on various interfaces including infrared (IrDA) and universal serial bus (USB). This allows the Artcam system to share the captured image and receive the image for printing.

Clock manager 907 (FIG. 222)
The clock manager provides date and time clock services. It utilizes a battery-backed real-time clock built into the Artcam and controls it to the extent that it automatically adjusts the clock variation based on an automatic calibration performed when the user sets the time.

When the power management system is not in use, the system enters a sleep power state where periodic scanning of input events takes place. Input events include button presses or ArtCard insertions. As soon as an input event is detected, the Artcam device returns to the operating power state. The system then handles input events as usual.

Even when the system is in the operating power state, the hardware associated with the individual modules is typically in a slightly power state. This reduces overall power consumption and allows particularly wasting hardware components such as printer paper cutters to monopolize the power supply during operation. Camera-oriented Artcam devices are in image capture mode by default. That is, the camera is in operation and other modules such as printers are in a dormant state. That is, the application must explicitly pause the camera module when a function other than the camera is activated. When other modules enter a standby state, they pause spontaneously.

The watchdog timer system generates a periodic high priority watchdog timer interrupt. The interrupt handler resets the system if it determines that the system has not progressed since the last interrupt, ie, the system has failed.

In another embodiment of an alternative print roll, an improved form of print roll is provided that can be constructed in large part by injection molded plastic parts that are suitably and easily fitted together. The improved form of the print roll is somewhat simplified in structure and has an increased ink storage capacity. The print media on which the image is to be printed is wrapped around a plastic sleeve former to simplify the structure. The ink media reservoir has a series of vents structured to minimize the possibility of ink flowing out. In addition, a rubber seal is provided in the ink outlet hole, and the rubber seal is punctured when the print roll is inserted into the camera system. In addition, the print roll includes a print media discharge slot, the discharge slot having a circumferentially shaped surface that is positioned so that the print media discharge slot is accurately positioned relative to the print head in the printing or camera system. To be placed in.

Next, referring to FIGS. 227 to 231, FIG. 227 shows a single point roll unit 1001 in an assembled form, which is partially cut to show the inside of the print roll. 228 and 229 are exploded perspective views of the left side and the right side, respectively. 230 and 231 are enlarged perspective views of the inner core portion 1007 of FIGS. 227 to 229.

The print roll 1001 is built around an inner core portion 1007 that includes an inner ink replenishment portion. A former 1008 is provided outside the core portion 1007, and a paper or film supply product 1009 is wound around the former. Two cover parts 1010 and 1011 are provided around the paper supply, and remain snapped together around the print roll to form a cover unit as shown in FIG. The lower cover part 1011 includes a slot 1012 through which the output of the print media 1004 interconnects with the camera system.

Since two pinch rollers 1038, 1039 are provided to press the paper against the drive pinch roller 1040, they together remove the curl of the paper around the roller 1040. The removal of the curl acts so as to cancel the strong curl applied to the paper stored in the form of a print roll over a long period of time. Rollers 1038 and 1039 are provided to snap fit with the end of the cover base 1077 and the roller, and the roller 1040 includes a geared end 1043 for driving and snap fit with the upper cover part 1010. Then, the paper 1004 is firmly tightened between them.

Cover component 1011 includes an end ridge or lip 1042. An end lip 1042 is provided to accurately align the paper exit hole with a corresponding printing thermal platen structure in the camera system. In this way, the existing paper is accurately aligned or positioned with respect to the nearby print head and fully guides the paper to the print head.

Referring to FIGS. 230 and 231, there is shown an exploded perspective view of an inner core portion that can be formed from an injection molded part centered on three core ink cylinders having inner sponge portions 1034 to 1036. It has been placed.

One end of the core is provided with a series of vent channels, for example 1014 to 1016. Each vent channel 1014 to 1016 interconnects a first hole, eg, 1018, with an external contact 1019, which is interconnected to the atmosphere. The passage followed by the vent channel, eg 1014, is preferably of a tortuous nature and is tortuous around. The ventilation channel is sealed by a portion of the sealing tape 1020, and the sealing tape is placed over the end of the core. The surface of the sealing tape 1020 is preferably hydrophobically treated to increase hydrophobicity, thus preventing fluid portions from entering the vent channel.

A rubber sealing cap 1023 including three thickened portions 1024, 1025, and 1026 is provided at the second end of the core portion 1007, and each thickened portion has a series of sparse holes. For example, portion 1024 includes sparse holes 1029, 1030 and 1031. The sparse holes are arranged so that one hole from each of the separate thickened portions is aligned with one straight line. For example, the sparse holes 1031, 1032 and 1033 (FIG. 230) all line up in a single line, and each hole comes from a separate thickened portion. Each thickened portion is paired with a corresponding ink supply reservoir and is in fluid communication with the corresponding reservoir when three holes are pierced.

The end cap unit 1044 is attached to the core portion 1007. The end cap 1044 includes an aperture 1046 for inserting the authentication chip 1033, and further includes an extended adapter (not shown) including three protrusions, the three protrusions corresponding to the corresponding holes ( For example, through 1048), a thickened portion (eg, 1033) of seal 1023 is punctured and interconnected with a corresponding ink chamber (eg, 1035).

Further, an authentication chip 1033 is inserted into the end 1044, and the authentication chip is provided for authenticating access to the camera system of the print roll. Thus, the core is divided into three separate chambers, each chamber containing a different color ink and an internal sponge. Each chamber includes an ink outlet at a first end and a vent at a second end. A cover of the sealing tape 1020 is provided to cover the ventilation channel, and a rubber seal 1023 is provided to seal the second end of the ink chamber.

The internal ink chamber sponge and hydrophobic channel make the print roll available in a movable environment and in a wide variety of orientations. Furthermore, since the sponge can be hydrophobized by itself, the ink can be ejected in order from the core.

A series of ribs (eg, 1027) can be provided on the surface of the core portion to minimize frictional contact between the core portion 1007 and the print roll former 1008.

Most parts of the print roll can be composed of injection molded plastic, which has a large internal ink storage capacity. The simplified structure includes a mechanism for removing paper curl and an ink chamber vent that minimizes leakage. The rubber seal provides an efficient passage between the ink supply chamber and increases the operational capability.

Artcard is, of course, used in various other environments. For example, Artcard can be used in both embedded and personal computer (PC) applications and provides an easy-to-use interface to large amounts of data or configuration information.

As a result, many applications are expected. For example, an ArtCard reader may be attached to a PC. There are a wide variety of PC applications. The simplest application is a low-cost read-only distribution medium. Since ArtCard is printed, it provides an audit trail if used for in-house data distribution.

Further, although PCs are often used as the basis for closed systems, there are numerous configuration options. Rather than relying on a complex operating system user interface, all configuration requirements can be met by simply inserting ArtCard into the ArtCard reader.

The back side of ArtCard has the same visual appearance regardless of the application (because it stores data), but the surface of ArtCard is application dependent. The surface is understood by the user in the context of the application.

Accordingly, the apparatus of FIG. Z35 efficiently distributes information in the form of books, newspapers, magazines, technical manuals, and the like.

In a further application, as shown in FIG. Z36, the surface of Artcard 80 displays an image containing artistic effects applied to the sample image. It is possible to prepare a camera system 81 including a card reader 82 that reads programmed data on the back side of the card 80 and apply algorithmic data to the sample image 83 to generate an output image 84. The camera unit 81 includes an on-board inkjet printer and processing means sufficient to process sample image data. A further application of the ArtCard concept is called BizCard, which stores company information on a business card. BizCard is a new concept of company information. The surface of bizCard is exactly the same as the current ordinary business card in terms of appearance and function, as shown in FIG. Z37. It includes photos and contact information, and there are as many different card styles as business cards. However, the back side of each biZCard contains a printed array of black and white dots and holds 1 to 2 megabytes of information about the company. The result is the same as a 3.5 inch diskette storage device attached to each business card.

The information may be any information desired by the owner of the bizCard, such as company information, a specific product sheet, a website pointer, an e-mail address, and a career. BizCard can be read by any ArtCard reader, such as a PC-attached card reader that can be connected to a standard PC using a USB port. BizCard may be displayed on a dedicated embedded device as a document. In the case of a PC, the user only has to insert the bizCard into the reader. The bizCar is preferably navigated using a normal web browser, just like a website.

By accommodating the owner's photo and digital signature along with a pointer to the company's public key, each biZCard is the person who is really the person himself and who is actually working for the specified company. Can be used to verify electronically. Moreover, by pointing to the company's public key, biZCard can easily initiate secure communication.

A further application called TourCard is an ArtCard application that stores information for travelers and visitors to a city. When tourCard is inserted into an Artcard book reader, the information can take the following form:
* Maps * Public transport timetables * Famous points * Local history * Events and events * Restaurant guidance * Shopping center TourCard is a low-cost alternative to tourist brochures, guidebooks, and street corners. If the manufacturing cost is only 1 cent per card, tourCard can be distributed free of charge at tourist information centers, hotels, and tourist destinations with minimum cost or financial support through advertising. The portability of the book reader makes this a perfect solution for travelers. TourCard can also be used in an information kiosk that allows a computer equipped with an ArtCard reader to decode information encoded in tourCard on any web browser.

The interactivity of the book reader makes tourCard very versatile. For example, hypertext links stored on a map can be selected to display historical stories of featured buildings. In this way, travelers can go on a guided tour of the city, and relevant transportation route maps and timetables are available at any time. tourCard eliminates the need for separate maps, guidebooks, timetables, and restaurant guides and creates a simple solution for each individual traveler. Of course, there are many other possibilities for data cards. For example, newspapers, study guides, pop group cards, baseball cards, timetables, music data files, product parts, advertisements, TV guides, movie guides, trade show information, magazine coupon cards, recipes, job advertisements, medical information, programs And software, horse racing newspapers, electronic newspapers, annual reports, restaurant guides, hotel guides, vacation guides, translation programs, golf course information, news broadcasts, comics, weather details, etc.

For example, ArtCard may contain book content or newspaper content. An example of such a system is shown in FIG. Z35, where ArtCard 70 includes the title of the book on one side and the encoded content of the book is printed on the other side. The card 70 is inserted into a reader 72, which can include a flexible display 73 that allows the card reader 72 to be folded. The card reader 72 includes a display control unit 74 that allows movement to the previous page, movement to the next page, and other control of the card reader 72.
Art card special use technology explanation technology 04 explanation

  In a further improvement, the art cam device is suitably modified to be equipped with a microphone device and associated recording technology. When a picture is taken, you are given the opportunity to record either the ambient sound environment or a message related to the image. The recorded audio is printed on the back side of the output photo in the encoding format. The encoding is desirably highly recoverable. The recorded audio provides a permanent audio recording related to the photo. A playback device is then provided to scan the encoded audio and decode this information.

  Referring now to FIG. 233, a further improvement of the art cam array 1601 is shown in schematic form, where the image 1602 is detected by the CCD sensor 1603 and forwarded to the art cam central processor 1604, which has significant computational resources. It has come to be. The art cam central processor 1604 can store images in a memory 1605, which preferably comprises a high speed RAMBUS (R) coupled to the memory. The art cam central processor 1604 is also adapted to control the drive of the print head 1606 for printing full color photos, eg, 1607, to provide instant photos on demand.

  In a further improvement, the camera organization 1601 also includes a sound chip 1610 that is connected to the memory 1605 via the RAMBUS 1611 under the control of the ACP processor 1604. The sound chip 1610 can be in a standard or dedicated format and can include, for example, a DSP processor that captures the analog input 1612 from the sound microphone 1613. Alternatively, as chip complexity increases (Moore's Law), the functionality of sound chip 1610 can be incorporated into ACP chip 1604, which includes a state-of-the-art CMOS integrated circuit chip. Is preferred. It will be readily apparent that many other types of organization falling within the scope of the present invention can be provided.

  The sound chip 1610 converts the analog input 1612 to the corresponding digital format and forwards it for storage in the memory 1605. The recording process can be activated by pressing a button (not shown) on the camera device; otherwise, the button is under the control of the ACP processor 1604, and the recording process is effective when taking a picture. Can be automated. The recorded data is stored in the memory 1605.

  Referring to FIG. 234, the camera organization preferably has a printer device 1606 consisting of two print heads 1615, 1616, the first print head 1615 being used to print an image on print media 1617. The second print head 1616 is used to print information on the back side of the print media.

  Referring to FIG. 235, a sample output to the back side of photographic media 1617 is shown. The output is printed by the print head 1616 in FIG. The information can include location, date, and time data 1620. The location data is then provided by keyboard input or by using an attached positioning system such as GPS or the like. The information 1620 is presented in a viewable format for the user to maintain a record of the printed image. What is important is that audio information 1622 in an encoded format is also printed on the back surface of the image 1617. The encoding format can vary widely, but encoding is preferably performed in a highly fault-tolerant manner to allow scratches, dirt, writing, wear, rotation, fading, etc. In FIG. 236, a portion of data 1625 is shown in schematic form, and the data has dot rows printed by the print head 1616 of FIG. The disclosed technique stores about 2.5 megabytes of arbitrary data on the back of the photo, and in this special case can include audio data. The encoding format is highly dependent on the use of Reed-Solomon encoding of data and provides high fault tolerance. In addition, a high frequency “checkerboard-like” pattern is described to be added to the data, as shown in FIG. 236, to assist in image detection. A data portion 1625 is shown in schematic form in FIG. 236 such that the data includes dot rows printed by the printhead of FIG.

  When it is desired to “reproduce” the recorded voice, the photograph 1617 is moved so as to pass through the reading device 1626. The reading device 1626 has a pinch roller, and the pinch roller rotates while sandwiching the photograph so that the photograph passes through the CCD linear sensor device 1627.

  Referring to FIG. 238, a schematic form showing the operation of the voice reader 1626 of FIG. 237 is shown. The CCD linear sensor 1627 is connected to a second artcam central processor 1628, which is suitably adapted to read and decode data stored on the back side of the photograph. The decoded audio information is stored in the memory 1632 for reproduction by the speaker 1629 via the sound processing chip 1633. The sound processing chip 1633 can operate under the control of an ACP decoder 1628, which operates under various user input controls 1633, which can include volume control, rewind, playback, fast-forward control, and the like.

  Importantly, as described above, the CCD linear sensor 1627 and the ACP decoder 1628 perform the reading process as if the information was printed on the back of the art card.

As can be seen from the above description of the preferred embodiment, a system for automatic recording of audio associated with an output image is provided to provide audio recording associated with a photograph. An audio reading system for reading the image output on the back side of a photograph is also disclosed.
Explanation of Technology 06

  In a further improvement, the art cam described above includes a device for automatically determining the orientation of the camera that is used for subsequent digital image generation. The art cam is then properly programmed to use orientation information for image generation.

  By the way, many examples using orientation information will be described with reference to the accompanying drawings. Referring to FIG. 239, a first portrait photograph 1801 is shown. In the photograph 1801, information related to the portrait 1801, such as date and shooting location information input to the camera system, is arranged. An area 1802 is prepared for this purpose. It is further anticipated that the portrait 1801 may be processed by an image processing technique that deforms certain visible portions of the image, such as a face. These deformations depend on the orientation determination of the camera that takes the photograph 1801. FIG. 239 shows a second portrait mode image 1803 used in the portrait mode in which the camera is rotated. Unfortunately, if the image data 1802 of the image 1801 in landscape mode is in the same position, the data appears in an unwanted area 1805 in the last image. Ideally, when a picture is taken in portrait mode, specific information about the image should appear at location 1806. It should be noted that the image processing effects that are imparted to the image in a specific manner of image content that are perpendicular, anisotropic, and image content will be wasted when the image is rotated.

  The orientation sensor 1846 detects the orientation, and the detected orientation is converted into a corresponding digital orientation angle by the ACP 1831. The area image sensor 1802 detects an image captured by the camera, and the image is transferred to the ACP 1831 and stored in the memory storage device 1833. The detected orientation is then used to determine whether the captured image is in portrait mode or landscape mode. The detected image is read from the memory storage device 1833 and rotated before being output to the memory storage device 1833 again. The final result is a rotated image 1808. Other techniques for using detected orientation information can also be mentioned. For example, the detected image stored in the memory storage device 1833 can be stored in association with the orientation information, and the orientation information is used for conventional rotation matrix mapping, world coordinate system, and local coordinate system mapping. . Subsequently, the computer graphics algorithm that processes the information can use the coordinate system while processing to manipulate the image in a rotational orientation. If position sensing information (eg, 1806 in FIG. 239) is incorporated into the image, the text can be placed in the correct orientation. Furthermore, the camera orientation can be used for algorithmic techniques that identify the image applied to the image, such as face detection. Since the position of the face is roughly, the use of orientation information greatly simplifies the face detection algorithm.

The orientation sensor can include a simple and inexpensive gravity switch, such as a Mercury sensor, or it can be a more complex small scale device, such as a micromachined silicon accelerometer.
Explanation of Technology 07

  In addition, other forms of print rollers can be configured. Referring to FIG. 241, there is schematically shown another arrangement of print rolls such as that utilized in a modified version of the apparatus described above. Importantly, disposable print rolls (eg, 1910) include an “extended leaf portion 1911” having a sponge portion 1912 that allows ink to be collected. Accordingly, during operation, the sponge portion 1912 is disposed below the print head in order to allow printing by the print head to the edge of the film or paper conveyed across the print head.

The sponge portion 1912 forms part of the disposable print roll 1910 and is therefore removed from the camera when the print roll is used up, thereby removing excess waste ink from the print head.
Explanation of Technology 08

  In a further embodiment, autofocus is performed by processing a CCD data stream that ensures maximum contrast. Techniques for determining the focal position based on the CCD data stream are known. For example, “The Encyclopedia of Photographic” published in 1993 by Butterworth-Heinemann and edited by Leslie Stroebel and Richard Zakia, and “Applied Photographic Boston” Yes. These techniques first measure the contrast between adjacent pixels covering a part of the input image. The image is processed by ACP to determine the correct autofocus setting.

  This autofocus information is used by ACP in certain modes, such as when trying to find the position of a face in an image, and simplifies the face position detection process by guiding the approximate size of the face in the image. It has become.

  Referring to FIG. 242, a sample method 2001 is shown that is used to determine approximate image features for investigation by a face detection algorithm.

  For example, various images such as 2002, 2003, and 2004 are acquired by the camera device. Details of the focus setting are stored in the ACP as a by-product of the autofocus operation. In addition, the current position of the zoom motor is also used (2006). Any of these settings are determined by the ACP. Next, before outputting an image including an object in the image (an object of interest) having a size corresponding to the approximate depth location, the ACP applies the analysis technique 2008 to the detected value. .

Next, in order to find the position of the object in the image, the depth value (depth value) is used in the face detection algorithm 2010 of the ACP 2031 in addition to the input detection image 2011. A short distance value 2009 indicates a high possibility of a portrait image, a medium distance value indicates a high possibility of a group photograph, and a long distance value indicates a high possibility of a landscape image. This possibility information can also be used for the purpose of face detection, along with other techniques and clip art applied depending on the distance to the object, creating an effect that “draws” in the image Or when drawing an image, it can be used to select various parameters.
Explanation of Technology 09

  In a further embodiment, the art cam device can be modified to have an eye position sensor that detects the current eye position. The detected eye position information is used to process a digital image captured by the camera in order to perform correction or deformation according to the detected eye position.

  The configuration of the eye position sensor is known to those skilled in the art and is used in many product cameras (particularly Canon cameras). The eye position sensor relies on infrared illumination from the viewfinder to the viewer's eyes and the reflections detected and used to determine the approximate eye position.

  Referring to FIG. 243, the eye position information 2110 and the image 2111 are stored in the art cam memory, processed by the ACP (2112), and the processed image is output for printing as a photograph by the print head. The The form of the image processing 2112 can vary considerably depending on the eye position information 2110. For example, in the first form of image processing, to detect the position of the face in the image and apply various graphic objects (eg, speech balloons that are generally offset relative to the face) A face detection algorithm is applied to the image 2111. An example of such a process is shown in FIG. 244, where the first image 2115 shows three people. By applying the face detection algorithm, three faces 2116, 2117, and 2118 are detected. The eye position information is then used to select the face closest to the eye estimated during the frame. In the first example, the balloon is positioned relative to the head 2116. In the second example 2120, the balloon is disposed with respect to the head 2117, and in the third example 2121, the balloon is disposed with respect to the head 2118. Thus, to place a balloon character above a given face in the image, an art card is provided that includes a coded form of the balloon application algorithm and the processed image.

It will be readily appreciated that the eye position information can be used to process the image 2111 in many different ways. Focusing effects can be applied locally or in specific ways to give specific deformations to faces and objects. Further, the image processing performed includes an artistic rendering of the image, which includes applying artistic paintbrush techniques. The artistic brushing method can be applied in a specific way according to the eye position information 2110. The final processed image 2113 can be printed on demand. Further images are taken and different eye positions are detected and used to generate different output images.
Explanation of Technology 10

  In a further improvement, the art cam is reconfigured to have an automatic exposure sensor to determine the brightness level of the captured image. The automatic exposure sensor is used to process an image according to a set exposure amount in order to enhance a part of the image.

  Preferably, the area image sensor has a device for determining a light state when taking an image. The area image sensor adjusts the value of the dynamic range acquired by the CCD according to the detected level. The acquired image is sent to the art cam central processor and stored in the memory store. The intensity information measured by the area image sensor is also transferred to the ACP. This information is used by the ACP to have a specific effect on the stored image.

  Referring to FIG. 245, automatic exposure setting information 2201 is used in connection with the stored image 2202 to process the image by ACP. The processed image is returned to memory for later printing 2204 on the printer.

  A number of processing steps can be performed according to a predetermined light condition. When the automatic exposure setting information 2201 indicates that an image was taken in a dark place, the pixel color of the image is selectively remapped to make the image color stronger, deeper and more vivid. .

  If the automatic exposure information indicates that the photo is taken in a bright state, the image color is processed to be brighter and more saturated. Image re-coloring can be done by converting the image to Hue / Saturation / Brightness (HSV) format and changing the pixel values as required. The pixel value can be converted into an output color format for printing and output.

  Of course, many different recoloring techniques may be used. These techniques are preferably clearly shown on an art card inserted into the reading device. Alternatively, image processing algorithms can be automatically applied and incorporated into the camera for use in certain situations.

Alternatively, the inserted art card can have a lot of handling, limited to the automatic exposure setting applied to the image. For example, a clip art containing candles etc. can be inserted into a dark image and a large sun can be inserted into a bright image.
Explanation of technology 11

  Removing the “red-eye” effects that occur in the acquired image as a result of using a flash is one of the important processes. Referring to FIG. 246, in a further improvement, the image acquired by the CCD device is subjected to processing step 3 to form an output image processed for printing if a flash was used. Referring to FIG. 247, one special image processing algorithm 2310 that can be used if a flash was used during image acquisition by the CCD device is shown in more detail. The algorithm is preferably used only when the flash is used to acquire an image with the CCD (2311). The purpose of the algorithm is to reduce the effect on the image due to the use of flash. Such image effects can include the well-known “red-eye” effect in which individual eyes appear red in a photographic image. In addition, other influences such as reflection of the flash light from the reflecting surface can be separately processed using other algorithms. The first step to eliminate the red-eye effect in the image is to determine the face in the image (2312). After performing some other image enhancement or color correction, the face detection process is performed by determining a close range in a map of HSV values for human face colors under normal light. The detected range is subjected to various simple tests including determination of eyes, mouth, overall shape and overlap. That simple test can produce a result about the likelihood of a face, and if it is above a threshold, the position of the face in the image can be determined.

  When a face is determined in the image (2312), the position of the eye is found in the face (2313). Then, in order to determine whether the red-eye removal process is necessary, a process for determining a special range of colors is performed for each eye individually. If red-eye removal processing is required, correct for the eye area to reduce red while avoiding any discontinuities or possible unnaturalness in the output image. The algorithm is applied. Of course, many different techniques are used, including a Gaussian degree format around the center point of the eye. Finally, the image is written in the updated format behind the memory store 2333 (2316).

Preferably, other correction algorithms are used at this point, including remapping colors that are affected by the spectral characteristics of the flash light. Instead, the inserted art card may perform many processes applied to the image (process for specifying flash settings). For example, a clip art such as a candle or a light globe may have a flash or a large sun inserted into an image without any flash.
Explanation of Technology 12

In a further improvement, a second ink jet is provided for printing on the back side of the photograph. In FIG. 248, a schematic representation of a standard arrangement is shown. A print head 2402 is provided for printing a photograph on paper 2403 sandwiched between rollers (eg, 2404). According to an embodiment with two inkjet printheads 2402, 2411, an alternative 2410 is shown in FIG. The print head 2411 is prepared for printing important information related to the photograph on the back side of the photograph 2403. For example, the time and date when the photo was taken can be printed on the back side by the inkjet printhead 2411. In addition, the art cam device may be equipped with a global positioning sensor. The sensor may be used to determine where the data was taken and the picture was taken by the Artcam central processor. This additional information may also be printed on the back side of the photographic output 2403.
Explanation of Technology 13

  The print roll is provided in a tightly wound form. Unfortunately, as a result of tightly rolling the print roll, the characteristics of the print media may result in an excessive amount of curl in the photos output by the art cam system.

Referring to FIG. 250, another arrangement of print rolls is shown that provides a system having three roller arrangements 2511-2513. The roller array provides a firm anti-curl to the print media 2514 that has exited the print roll. These three rollers 2511-2513 sandwich the media 2514 and provide anti-curl to the paper before being conveyed to the print head 2516 by a pinch roller (eg, 2517). Those three rollers 2511-2513 can be arranged inside a print roll case or an art cam device.
Explanation of Technology 15

  Referring to FIG. 251, in a further improvement, it is desired to create a photographic image that can be shown to the viewer 2703 with a stereoscopic or pseudo three-dimensional effect. These effects can be obtained by recording two images in close proximity simultaneously, giving the first image to the viewer's right eye 2704 and the second image to the viewer's left eye 2705. it can. 3D stereoscopic effects can be created by recording images for the left and right eyes, and showing the left stereoscopic image to the left eye and the right stereoscopic image to the right eye .

  In a further improvement, with a series of lenticular transparent columns that capture left and right eye images, the photo or stereo image 2702 will be constructed to be more distinct as follows.

  Referring to FIG. 252, a cross-sectional view of the surface portion of photographic paper 2702 is shown. As is apparent in cross-sectional view 2710, the photographic paper has a series of lenticular rows designed to allow left and right stereoscopic viewing. The image is printed on the bottom surface 2712 of the transparent photographic paper 2710.

  The pitch 2713 of one pixel is 80 μm, and the dot arrangement pitch 2714 is 20 μm. Therefore, four dots 2714-2717 are arranged per pixel. Two dots 2714-2715 are prepared for left stereoscopic viewing, and two dots 2716-2717 are prepared for right stereoscopic viewing. The right eye looking at the photographic paper obtains the image of the right pixel dots 2716, 2717 (2720, 2721), and the left eye obtains the image of the left dots 2725, 2726 (2723, 2724). Therefore, the two images are shown separately for each eye, resulting in a stereographic effect. Entity photo effect mainly due to the lens-shaped outer shape of each column such as 2711.

  Referring now to FIG. 253, a perspective view of the bottom surface of paper 2710 is shown. The illustration of FIG. 253 includes a construction line to indicate the dot pitch of each dot 2730 in addition to the pitch of the pixels having four dots 2731-2734. As is apparent from FIG. 253, each pixel is composed of four dots, two dots 2731-2732 prepared for right stereoscopic viewing and two dots 2733-2734 prepared for left stereoscopic viewing. Is done. Since the dots, for example, 2730, are arranged on the print medium 2710, when the medium is turned over, correct stereoscopic viewing is performed.

  Referring now to FIG. 254, one method for capturing a stereoscopic image will be discussed. Desirably, two images are picked up by CCD couplers 2740 and 2741 arranged in the portable camera device. Each CCD device 2740, 2741 acquires a separate image, and these separate images should be combined in the stereo processing unit 2742 in a left-right alternating manner as shown in FIG. When the images are correctly interleaved in the stereo processing unit 2742, the images are sent to a print engine 2743 and a print head 2744 that prints the images on the surface 2746 of the photographic paper 2710.

  The current position of the photographic paper 2710 is detected by the alignment unit 2750, and the current position is fed back by the print engine 2743 for control of the print head 2744. Its position is determined by LED type device 2780 illuminating the lenticular surface of print media 2710 and photoconductor 2781 on the opposite side of print media 2710 measuring periodic brightness changes.

  Referring to FIG. 255, the operation of the alignment unit indicated by dashed line 2750 will now be further described. The photographic paper 2710 is sandwiched between rollers 2751 and 2752, and the roller 2752 is configured to have a surface that coincides with the lens-like surface 2754 of the photographic paper (print media) 2710. Since the lens period of the photographic paper 2710 is on the order of 2780 mm, the surface of the roller 2752 is shown exaggerated like the lenticular surface of the photographic paper 2710. The precise period of the lenticular photographic paper means that the roller 2752 can be used for other forms of print media as well.

  The surface of the roller 2752 ensures that the print media 2710 maintains a certain spatial relationship with the roller 2752. Accordingly, the print rollers 2751 and 2752 are monitored by the alignment unit 2750 under the control of the print engine 2743 to rotate the roller 2752 on demand and thereby transport the print media 2710 through the print head 2744. It is driven while being. As a result, the alignment of the dots printed by the print head 2744 and the lenticular row is maintained.

  A three-dimensional image can be printed by the print head 2744 when the print medium 2710 is conveyed so as to pass through the print head. The print head 2744 is preferably a full page width print head.

  Of course, the current location of the print media 2710 relative to the print head 2744 can be determined by other devices.

  Referring to FIG. 256, a camera device is provided according to the principles of the proposed embodiment by combining a printer roll 2760 with a suitably shaped lenticular paper 2761 and a corresponding ink supply (not shown). obtain. The paper 2761 is taken out from the print roller 2760 to the printing device 2764 by the pinch rollers 2763 and 2763. The printing device 2764 further includes pinch rollers 2765 and 2766, a cutter 2767, a platen 2769, pinch rollers 2751 and 2752, and a print head 2744. The three-dimensional image is printed by the print head 2744, and the paper 2761 is taken out after being cut by the cutter 2767. Its output 2770 is in a transparent form.

Referring to FIG. 257, a flat white surface 2771 can be deposited on the image output 2770. This is accomplished through the use of an adhesive surface to which transparent 2770 is applied. Subsequently, the left and right stereo images are generated, and the image is visually recognized through the lens system as shown in FIG. 251 so as to give a three-dimensional effect to the captured image. The camera system depicted in FIG. 256 is ideally portable so that a photograph can be taken and taken immediately to a place where a stereoscopic photograph is desired.
Explanation of Technology 17 Explanation of Technology 21

  In a further improvement, image media that are tightly rolled and stored are processed to have an anisotropic rib structure, suitable for carrying in rolls, and viewing images printed on the print media Sometimes the curl degree of the print media is reduced.

  Referring now to FIG. 259, there is shown print media 3311 that is handled according to the nature of this improvement (3310). The print medium 3311 is made of a plastic flat film, although other forms are suitable. As shown in the enlarged view of FIG. 259, the print media 3311 is pretreated to have a polymer rib-like structure 3312, and the actual row pitch is about 200 m. Thus, one surface of the print media 331 is treated to have a series of rows 3312 that run up and down the entire length of the print media 331. The cross section of row 3312 is shown in an enlarged view in the figure.

  The advantage of using a series of rows 3312 is apparent when a force 3313 is applied to the surface portion of the print media 3311. The row 3312 allows the print media 3311 to be tightly wound while resisting the print media 3311 from being wound in the 3316 orientation. Thus, any applied force 3313 is likely to be transmitted in both directions 3315 and 3316. The anisotropic nature of the media 3311 is provided by ribs 3312 that function to provide support in both directions 3315 and 3316, thereby limiting the curl of the media 3311. Further, the anisotropic strength of the material is allowed to be stored with the strength along the central axis of the roll.

  Referring to FIG. 260, by simply holding the image 3311 in the user's hand 3320, 3321 and applying a slight pressure on the points 3324, 3325, the effect of curling is reduced and the image on the media 3310 is displayed. Can see.

  Of course, anisotropic media can be manufactured using a number of techniques when the media includes a plastic material. For example, it can be manufactured by an extrusion process. Alternatively, other techniques can be utilized. For example, one manufacturing method is schematically shown in FIG. 261 (3340). In this method, the medium 3341 is pressed between the rollers 3342 and 3343. As shown in FIG. 261 (a), the roller 3343 has a flat surface, and the roller 3342 has a jagged surface 3345. For illustration purposes, the saw-shaped outer shape is enlarged in FIG.

  If the media is not suitable for use in such a process 3340, anisotropic media is provided by utilizing the surfaces of two films that are laminated together. Such a process 3350 is shown in FIG. The first surface 3351 for which printing of an image is desired is affixed to the second surface 3352 as required by a method such as an adhesive or heat fusion.

  In addition, other methods of manufacturing print media are possible. For example, referring to FIGS. 263 and 264, the structure of print media 3360 is shown in cross-section. The structure of the print media 3360 is formed from a first fiber material 3361 comprising a reinforced polymer made of soft polyethylene naphthalate drawn into a fiber mold. A second heat-flowable polymer 3362 including polyethylene or the like is used to provide a "paper" substrate for the ink / image chemical carriage. The media polymer 3362 is pressed against the fiber material 3361 while the adhesive force remains (eg, after heating) so that the fibers 3361 and the print base 3362 form an integral unit as shown in FIG. The media arrangement of FIGS. 263 and 264 can be made according to a number of techniques. Referring now to FIG. 265, a first such technique using a fibrous material molding spool 3380 is shown. In that technique, many spools and corresponding fibers 3381 correspond to the desired print roll length. The spools 3380 are arranged such that the corresponding fibers 3381 are supplied to the first roller 3382. The fibers 3381 are then drawn through a print media application unit 3384 that supplies a heated adhesive print media layer to the fibers. Thereafter, rollers 3385 and 3386 provide the necessary pressure to fuse the fibers and print media together, and finally flatten the print media surface to form an anisotropic surface 3388. Thereafter, the surface 3388 is cut and wound to form an anisotropic print media as described above.

Of course, it is possible to form the fibrous material in other forms as desired. For example, FIG. 266 shows one form in which fibrous material is extruded and generated by a method of pulling fiber 3391 from tank 3390. The apparatus of FIG. 266 can be replaced with the spool 3380 of FIG.
Explanation of Technology 24

  In a further refinement, an algorithm used for the art cam system is described. The algorithm automatically converts a group of pixels in an input image into a “drawn” effect by replacing a “brush stroke” in the output image. The algorithm works to automatically detect major edges in the flat part of the image and propagate information on the direction of the edges so that the brush stroke approaches the Van Gogh style. The algorithm is suitable for implementation on the art cam device described above.

  Referring first to FIG. 267, the algorithm includes a number of steps 3601. These steps include a first step (3602) of filtering the image to detect edges. These edges are then thresholded or “skeletonized (3604)” prior to the final edge determination (3605). Then, a Bezier curve is fitted to these edges. The curve is then offset (3607) and a brush stroke is drawn (3608) in the final image. These processes 3607 and 3608 are repeated until the image is sufficiently covered by the brush stroke. Thereafter, the last “correction” to the image is performed (3609).

  Next, each step will be described in more detail. In a first step (3602) of filtering to detect edges, a Sobel 3 × 3 filter having a set of coefficients 3612 and 3613 as shown in FIG. 268 is applied to the image. The Sobel filter is a well-known filter used in digital image processing, and its characteristics are the normative textbook “Digital Image Processing” published in 1992 by the Addison-Wesley publisher in Massachusetts. Is fully discussed by Gonzalez and Woods on pages 197-201. The Sobel differential filter can be applied by converting the image to grayscale before filtering, or by filtering the image into each color channel and taking the maximum value. By Sobel filtering, a grayscale image indicating the edge strength of each pixel of the image is created.

  Next, a threshold process is performed to create a corresponding binary image (3603). The threshold can be varied, but a value of 50% of the maximum intensity value is appropriate. Each pixel in the edge strength image is compared with the threshold, and if it is greater than the threshold, “1” is output, and if it is less than the threshold, “0” is output. As a result of this processing, a threshold edge map is created.

  Next, in step 3604 of FIG. 267, the threshold edge map is “skeletonized”. The process of skeletonizing an image is described completely and clearly in pages 491 to 494 of the above reference textbooks and other standard textbooks. The skeleton process creates a “thin” skeletonized edge map while maintaining a significant number of features in the threshold edge map.

  In the next step, an edge in the skeletonized edge map is determined that yields a data structure containing a list of points in the image. Preferably, only edges that are longer than a predetermined minimum value are added to the list.

Since the skeletonized image has only one-pixel-wide edges that can be multi-branched, the following algorithm represented by a C ++ code fragment can be applied to unbroken edges in the skeletonized image. One way to determine and identify the points to which it belongs is presented. It breaks the bifurcated edge into separate edges and chooses to extend along the edge in a direction that minimizes the curvature of each bifurcation. For example, at the bifurcation point, help the bifurcation that produces minimal curvature. The code is as follows: :

  By using this algorithm on the skeletonized edge map, a list of edges having at least a predetermined size is generated. A suitable size has been found to be 20 pixels long.

  In the next step 3606 of FIG. 267, a Bezier curve is fitted to each of the edge lists obtained in step 3605. For each of the edge lists, a piecewise Bezier curve is fitted to the list of points. A suitable algorithm for fitting piecewise Bezier curves is "Algorithm for automatically fitting digitized curves" Schneider, P. et al. J. et al. Written by Glassner, A .; S. Schneider curve fitting algorithm as described in Edit, Graphics Gems, Academic Publishing, 1990. This algorithm is intended only for geometric continuity, not parametric continuity, so that it converges quickly. The Schneider algorithm is functional, and if the fitness is poor, the curve is subdivided at the maximum error part, and the curve is fitted to the divided one. Next, the estimation of the tangent at the divided portion is performed using only one of the dividing points. Due to the dense point set, local noise tends to be amplified. The improved quality of fitting the curve is obtained by using points away from the division point on which the tangent is based.

  In the next step 3607 of FIG. 267, the curve is offset from the initial curve list by half the desired “brush stroke width”. The offset results in two curves that are approximately one stroke apart from each other so that they are parallel to both sides of the first main curve.

To generate a piecewise Bezier curve that is approximately parallel to a particular piecewise Bezier curve, an algorithm is used that includes the following steps.
i. Creating an empty point list
ii. Creating an empty tangent (vector) list
iii. Evaluate the selected points for each of the curve portions that form a piecewise Bezier curve and offset them by a specific offset value. An offset point is added to the point list, and a corresponding tangent is added to the tangent list.
iv. Fit a piecewise Bezier curve to the resulting list of points. Rather than predicting the tangent in the process of fitting the curve, use the exact tangent associated with the offset point.

Each curve segment is offset as follows:
i. A curve value, a standardized tangent, and a standardized normal are obtained. The size of the normal is standardized by parameter values (for example, an interval of 0.25) arranged at equal intervals between 0.0 and 1.0.
ii. The normal is scaled by the specific offset value.
iii. Assemble the line segment using the points of the curve and the reduced normal.
iv. If two line segments intersect, delete the point associated with one line segment.
v. Add surviving points to the point list and add corresponding tangents to the tangent list. If the line segment in question is the last piece of a piecewise curve, only the point associated with parameter value 1.0 is added, and the point associated with parameter value 0.0 of the next line segment is copied.

The process of offsetting each curve portion is performed as follows. That is,
1. First, for parameter values that are 0.0 to 1.0 and are equally spaced on the Bezier curve, for each parameter value Pn (FIG. 269), a curve value 3630 and a standardized tangent 3631 , A normalized normal 3632 is calculated.
2. Next, the normal 3632 is scaled with a specific offset value.
3. Next, the line segment from point 3630 to point 3636 (the end point of the scaled normal 3634) is calculated.
4). The line segments 3630-3639 are then checked for all other points on the curve (eg, 3638, 3639). If any two line segments intersect, one of the points 3636 is discarded. Add surviving points to the point list and add corresponding tangents to the tangent list. If the line segment in question is the last piece of a piecewise curve, only the point associated with parameter value 1.0 is added, and the point associated with parameter value 0.0 of the next line segment is copied.

  Referring to FIG. 270, the final result of the curve offset according to step 3607 of FIG. 267 to create a series of Bezier curve segments (eg, C1, C2, etc.) is shown. First, a series of parameter points P1-P5 arranged at regular intervals. Next, normal points N1-N5 (corresponding to point 3636 in FIG. 269) are created for each point P1-P5. Next, a piecewise Bezier curve 3640 is created using these points N1-N5. This process also applies to the opposite curve 3641 that includes points N6-N10. This process is also applied to the following piecewise curve C2 and the like. This results in two curves 3640 and 3641 that are parallel to each other and are spaced apart by approximately one brush stroke width.

  A series of brush strokes is then placed in the output image along the curve. Those strokes are directed in the tangential direction of the curve. Each brush stroke is defined by a footprint that defines where it does not overlap other brush strokes. If the footprint does not overlap with the footprint already displayed in the output image, the brush stroke is only placed along the curve. Curves without brush strokes are discarded and steps 3607 and 3608 are repeated until there are no more curves (with some modifications). A slight modification in step 3607 is to offset the curve by more than half the brush stroke required to offset the curve from curve C1 in FIG.

  The method described above can also be used to cause a series of brush strokes to be emitted from the object of interest in the image.

Following the image being covered with a brush stroke of a given size, further processing is done with a smaller brush stroke, and the threshold level is increased so as to further correct the important features of the brush stroke in the image. Such a technique approximates the technique Van Gogh uses for specific parts that require fineness.
Explanation of Technology 26

  It is desirable to be able to determine a region of interest (for example, the position of a face or the like) in the photograph or the like for some image (for example, a photograph). Once the position of the face in the image is specified, as shown in FIG. 271, image processing that performs many unusual effects such as placing a speech balloon on the face area can be simplified. Once the face area is detected, other effects such as deforming the face can be implemented, such as placing objects related to the face such as glasses.

  As described above, detection of a region in an image can be applied to region-specific processing such as applying an artistic effect only to a region. For example, the background sky area is drawn with a wide brush stroke, and the foreground face area is drawn with a fine brush stroke. Moreover, it is the determination of the feature in an image. The detection of the area based on the color is extremely useful when detecting the human face in the image because the detected color is a natural tone of the skin.

  When lighting is changed, objects of a specific spectral color may appear in the image with varying brightness and saturation. Since the hue evaluates the dominant frequency of the object (objective frequency), the color of the object tends to differ greatly in luminance and saturation than the hue. As shown in FIG. 272, the further improvement method 3801 includes a first step of first converting the acquired image into a color space that distinguishes hue from luminance and saturation. As such, two well-known color spaces include an HLS color space and an HSV color space. Various color spaces including these color spaces are described in Foley, J. et al. D. Van Dam, A.C. Feiner, S. K. , Hughes, J.A. F. It is well described in standard text, such as Computer Graphics, Principles and Practice, Second Edition, Addison Wesley, 1990, pages 584-599.

The next step is to enter a number of input color seeds (3804). Each seed 3805 is defined as follows. :
(1) Seed color range of color values (2) Local color difference limit (a
local color difference limit)
(3) Overall difference limit (a
global color difference limit)

  The next step 3808 is to process the seed to build the bitmap.

  The last step 3807 in the method of FIG. 271 is to output a bitmap indicating that the pixels in the input image are of the same capacity as the input image and belong to the color gamut defined by the seed list 3805. . The method 3801 includes processing each seed in a separate pass 3808 and combining the output bitmap of each pass with the entire output bitmap. Any set bit of an individual bitmap is written in the entire bitmap.

  Referring to FIG. 273, step 3808 of FIG. 272 is shown in more detail. In a first step 3820, a seed bitmap is determined for all colors that have a seed in the seed color range where the seed in question is defined. In the next step 3821, for each seed in the seed bitmap, it is used as a seed position for a "seed filling" algorithm that detects 4 connect or 8 connect regions in the input image connected to the seed position. . The seed filling algorithm flags pixels belonging to the region of the seed output bitmap. The seeds flagged in the seed output bitmap at step 3821 are erased from the seed bitmap created at step 3820 so that they are not used as a starting point for the next seed fill.

  When all seeds are used for the seed filling process, the seed output bitmap is combined with the overall output bitmap for all lists of seeds.

  As disclosed on pages 979-986 of the above-mentioned reference, the seed filling algorithm is a standard type. However, the seed filling algorithm has a specific pixel relationship test.

  If the color differs to the same extent as the pixel color described above and the local color difference limit, and the seed color differs to the same extent as the global color difference limit, the pixel becomes part of the current region. The color difference limit is determined separately for each color component. Since the span is detected or processed from the left and right, the aforementioned pixel is the pixel to the left of the current pixel. At the beginning of the new span, the color of the aforementioned pixel is made to be the average color of the original span. The seed color is set to be an intermediate color in the seed color range.

  Rather than using a single seed, a list of seeds is used to identify distinguishable color features but well-known facial-like regions.

  The use of local and global chrominance limits is used to detect as much of the region as possible without crossing discontinuities in the image. An area may contain a variety of colors, but these vary gently over the area. The overall color change in the region is generally greater than the local color change. The use of local color difference limits reduces the risk of crossing image discontinuities. It also allows the overall color difference limit not to be conservative. The detection of the area can be made more complete, for example, even in areas that gradually become brighter or darker.

  If the color space of the input image is based on hue, the hue difference limit is preferably reduced by luminance (or luminance value) and saturation, since the hue is somewhat unreliable due to low luminance or low saturation. Therefore, when the hue is not reliable in a dark area or an unsaturated area in the area, the effect is reduced so that those areas are included in the detected area. This is of course premised on a difference limitation on brightness and saturation.

  Also, the hue difference is accurately calculated in accordance with the characteristics of the hue hexagonal circle.

It has been found that very good results are often obtained when using the above algorithm for detecting objects in an image. For example, the above method was used for images with many solids, where the color seed was set to a standard skin tone. As a result, the position of each face in the image can be accurately searched.
Explanation of Technology 27

  In a further improvement, as shown in FIG. 274, the source image 3901 is converted into a corresponding “tile” color image 3902. The conversion process will be described later. In the first example, each tile 3094 takes a standard arrangement in the output image 3902. Further, the color of each tile 3904 is drawn from the color of the corresponding pixel 3905 in the input image 3901. The tiling process in the output image is performed at a higher resolution than the input image 3901, and thus the tiling process is actually used to make a low resolution image.

  Of course, many different tile shapes are possible. The shape of the tile that tiles the image is preferably combined with a “pattern unit” when the entire image is tiled and duplicated. For example, as shown in FIG. 276, tiles 3908 and 3909 are combined as a pattern unit 3907 with the image tiled when duplicated.

  In a further embodiment, each tile is defined by an image with two channels. The first channel 3970 defines the shape and opacity of the tile, as shown in FIG. A non-zero opaque region 3971 defines pixels that are part of the tile. The value of each pixel 3971 indicates the opacity of that pixel. Referring to FIGS. 277 and 278, the second channel 3980 defines the pixel surface texture in the tile as a height field. FIG. 278 shows a cross-sectional view along the line A-A ′ in FIG. 277, showing the intensity, peak 3991, and slopes 3992 and 3993.

The inputs to the algorithm are as follows: That is,
Color image to be played back as a tile Tile pattern output resolution as described above

  The added optical input parameters are as follows.

  The output from the algorithm is to perform a tiled effect on the input image.

A repeating tile “pattern unit” can be defined as follows:
(1) Pattern unit size (number of rows, number of columns)
(2) Offset to the start point in the pattern unit (row and column offset) for the first pixel of the image
(3) Vertical offset for the arrangement of consecutive pattern units
(4) List of tiles that make up the pattern unit
(5) Horizontally continuous tile layout offset in one pattern unit
(6) Resolution of the pattern (for example, resolution of tile image of the pattern)

  The width of the pattern is defined by the sum of the column offsets of tiles that are continuously arranged in the horizontal direction.

  All offsets are within consecutive pixel combinations (eg, real-valued row and column offsets). This allows tiles to be placed with sub-pixel accuracy. This allows the resolution of the input image, output image and independent tile pattern.

The basic tiling algorithm is implemented with the following C ++ code fragment. That is:

  As described above, each tile has a form characterized by its shape / opacity channel. The shape of the tile as well as the placement of the tile in the image together determine the series of pixels in the input image that are covered by the tile. The tile color in the output image is based on the series of pixels. The function of giving a color is not limited, but copying the input color pixel by pixel, averaging the input color, taking the center of the input color, etc. It may be included.

  Tiles colored in this way may be drawn directly on the output image, ignoring opacity, or may be composited into the output image according to the opacity of the tile. In the latter case, the output image must first be initialized to some brightness (eg, black) or some arbitrary image (eg, the input image itself). Composite images using opaque masks and channels are standard, for example, as disclosed in general articles such as Thomas Porter and Tom Duff (1984) “Composite Digital Image” SIGGRAPH Report 1984, pages 253-259. Any synthetic technique can be used.

  Each tile has a surface texture map defined by a texture channel. The texture channel defines the relative surface height of each pixel in the tile. This height field is later used to determine the surface normal used to determine the reflection angle of the simulated incident light. When simulated light is applied to an image of a tile or a portion where the tile is placed, the surface texture becomes visually more apparent.

  Whether the tiles are drawn directly on the output image or composited on the output image, in the simplest case, the tile texture is written directly to the texture channel of the output image.

  The opaque channel of each tile controls the composition of the tile into the background. In addition, an image-sized opaque channel may control the composition of the entire tile to the background. In the simplest case, the opacity of a tile is measured by the global opacity that gives rise to the opacity used to synthesize the tile. Referring to FIG. 279, an example of a process that uses global opacity is shown. In the first plot 4000, an example of the cross-sectional profile of each tile with a simple shape of a folded opacity profile (eg, 4001, 4002, etc.) is shown. Next, a preferred global opacity channel 4004 consisting of a gradual increase in opacity 4005 is shown. In the simple case described above, the tile opacity profile 4001 is measured with a global opacity profile 4004 to produce an opacity 4006 with an increasing opacity area (eg, 4007).

  Of course, a scheme of combining local tile opacity with a global opaque channel is possible. For example, for each tile, pixel opacity is first averaged, and opacity per pixel is measured by this averaged opacity rather than global opacity per pixel.

  It has been surprisingly discovered that the tiling algorithm described above helps to produce the effect as drawn by making a few simple modifications. By treating the pattern unit as a brush stroke stack and making each tile image look like a brush stroke (eg, a layer of brush hair or one having a thickness at the stroke end), the strokes are overlapped and drawn. Such effects are performed in tiles and pattern units. The algorithm is then modified to randomly select strokes from the stroke collection, rather than continuously repeating tiles in the pattern. In addition, many of the texturing, placement, and coloring effects depicted below can be provided as alternatives.

  First, there are typically various levels of detail in the input image. The input image requires different levels of detail when reproduced via tiling or coloring algorithms. This can be achieved by processing the image within various changes, using different degrees of stroke collection within each change, and combining each change and (cumulative) background. An image-sized detail map composed of detailed dimensions in pixels can be used to control the image portion of each changing process. Each change is given a “detail boundary” over which the corresponding part of the image is processed. Continuous changes are given an increased level of detail and a reduced stroke collection rate. The first change thus processes the entire image, for example, the second change adds a defined level of detail, the third change still adds more detail, and so on. This is somewhat similar to how a painter refines a painting by adding more and more details. This detail map itself can be generated easily and automatically by placing edges in the image, in other words by passing the image between the derived filters.

  Tile is conceptually inflexible. The surface texture is not affected by the surface texture of the background or the surface texture of other overlapping tiles (usually not overlapping with any other tiles). On the other hand, the brush stroke is conceptually flexible. Its thickness and surface texture are affected by the surface texture of the background and the surface structure of other overlapping brush strokes. In particular, the brush stroke tends to fill the recesses in the background, but tends to be thin at the background peak. (The background includes the effect of the brush stroke preceding the stroke.)

  In another embodiment, when a brush stroke is drawn, the texture is combined with the background texture in one of several ways. First, to create a new background height, the stroke height is added by a fixed percentage of the background height (eg, 25%), but the height is not reduced. The effect of this texture will be explained with reference to FIGS. 280 (a) to (c). FIG. 280 (a) shows an example of a cross-sectional profile of a single tile brush. In FIG. 280 (b), a cross section of the image portion covering the brush pattern 4010 in FIG. 280 (a) is shown as an example. FIG. 280 (c) shows the result of the join process using the above-described rule. Line segments 4013, 4014, 4016 and 4017 are taken directly from the image 4011 so that the image value 4011 exceeds the corresponding brush value 4010. However, the portion 4015 is calculated by adding the stroke height of the recording 4010 to 25% of the corresponding portion of the image 4011. The background under the stroke is calculated. For an individual pixel, if the background height is lower than average, the stroke height is simply added to the background height. In this way, a mimicry is performed in which the stroke is “filled” into the recesses in the background. If the background height is equal to or higher than the average, the stroke height is added to the average. In this way, the stroke is lightened by the peak of the background. To prevent the background from substantially breaking the stroke (eg, not thinning to zero), its height is made to increase with a minimum amount (of course, including the zero, minimum stroke thickness) Yes.

  Furthermore, stroke opacity may be measured to calculate the net thinness of the stroke, which is the result of the texture combining algorithm. In this way, the place where the stroke becomes lighter due to the peak of the background becomes more transparent.

  Once the tile / stroke position is determined, it may be further processed in a number of ways. For example, to mimic natural variations in tile and stroke positioning by hand, tile / stroke positions may be arbitrarily jittered at specific ranges and specific speeds. When a tile is imitated rather than a stroke, the range of jitter of the tile is usually limited so that the tiles do not overlap. For example, only the width of the “inner tile gap” is changed. When mimicking a brush stroke, the brush strokes usually overlap, so the stroke jitter range is usually narrower and limited.

  Furthermore, the position of each tile is replaced according to a replacement map to mimic a more standard variation of tile and stroke positions. The replacement map consists of two channels. The first includes row replacement and the second includes column replacement.

The tile / stroke color itself may be further processed for artistic effects. For example,
(1) To mimic a restricted tile or restricted paint palette, the color may be mapped to any palette.
(2) To mimic the imperfection of tile colors and variations that mix paints, the colors may be arbitrarily jittered at specific ranges and at specific speeds.
(3) The chrominance of the color may be arbitrarily converted at a specific speed to mimic an impressionist style using the opposite color.

  Once the average global opacity of the tile / stroke is determined, it may be further processed for artistic effects. For example, the averaged opacity may be binarized to zero opacity or 100% opacity. This creates the absence or presence of tiles. If the averaged opacity is binarized, global opacity reproduction may be improved by performing partial error diffusion on the input image. This has been found to produce important effects in the image.

  A further embodiment is implemented by appropriate programming of the art cam device. A painter who “draws” an image on a canvas usually works with a palette with a limited color of paint and a brush that is of a limited set and forms a characteristic brushwork, They are drawn on a canvas with a unique color and texture. By recognizing that, further examples are built. To draw a scene closer to the truth or away from the truth, the painter uses different colors to draw the artist's vision of the scene. Several picture styles are tried to reproduce scene features and colors. Other styles are tried to get more impressions about scene features and colors, and are characterized by simpler and more uniform brush strokes and by not mixing or overpainting the paint. These Impressionist styles achieve a brighter and clearer color than the Realistic style, but at the expense of detailed depiction.

  A computer system that automatically converts a photographic image to produce a “drawn” effect typically places a brush stroke of a color obtained in some way from pixels in the input image in the output image. The problem with such a system is to incorporate the mode of action that a person draws as described above. A partial solution to create an impressionist presentation is the construction of a color palette based on the color of the actual artist's paint, the selection of a brush stroke color from the palette that matches the input image as closely as possible, and the input image And color error diffusion to maintain the overall color matching.

  In a further embodiment, an algorithm is constructed that error diffuses an image based on the final color of the brush stroke, rather than only on the raw color. This final color is determined by the color of the paint, the composition of the brush, the color of the canvas and the fabric.

The input of the algorithm is as follows. That is,
* Palette with limited paint colors * Each is an opacity map
map) and bump map (bump
A series of brush strokes defined in map). The use of bump maps is well known to those skilled in the art of 3D graphic rendering and is described, for example, in Graphic Gems, Vol. 4, pages 433-437.
* Canvas defined by color images and bump maps * Color images reproduced as paintings

  The output of the algorithm is to “draw” on the canvas with the rendering of the input image, ie, the brush stroke colored by the paint palette. Since the canvas and brush strokes are textured and these textures are combined during the painting process, the final painting may be illuminated in one direction.

In order to achieve the effect as depicted, a basic tiling algorithm is used to achieve better color matching between the output image and the input image. The first process is a brush stroke palette feedback algorithm. This process is performed by creating a table of stroke color palettes. A table 4025 shown in FIG. 281 is divided by brush strokes B1, B2, B3, and the like. For each brush stroke Bn in the brush stroke set and for each paint color C1-CN in the paint palette set, the stroke colors S1-S4 are as follows, as described with reference to the flowchart of FIG. To be determined. That is,
1. In step 4021 the paint color C1 is used to color the corresponding brush stroke B1.
2. Then, using the opaque map and the bump map, a colored brush stroke is composited on an empty canvas.
3. In step 4023, the average color of the drawn stroke is calculated. This average color is the stroke color S1 in FIG. A pointer (not shown) is then set to point from the stroke color palette input S1 to the corresponding paint color palette input C1.
4). The stroke color table is then sorted by color.

  When drawing an image, each brush stroke is selected in the normal way, as shown in FIG. The desired color of each brush stroke is then determined from the input image in the usual way. However, the closest paint color Cn in the paint color palette is not used. Instead, because of the brush stroke, the closest stroke color Sn in the stroke color palette is used. The corresponding paint color Cn is then used as the paint color for the brush stroke.

  Since the range of colors available in the paint palette is limited, there is usually some difference between the desired color and the selectable color Cn. This difference, or error, can be error diffused to adjacent parts of the input image in the usual way of error diffusion. The difference error between the desired color and the paint color Cn is not used. Instead, an error that is the difference between the desired color and the stroke color Sn is used for this purpose as the average color across the stroke. While the actual brush strokes overlap with other strokes, the strokes in the stroke palette are drawn on an empty canvas, so you will notice that this average color Sn may be different from the average color in the stroke palette.

  This method can be extended so that the effects of other painting parameters can be incorporated in the same way. This includes the type of painting medium (pine oil, chalk, charcoal, etc.) and the type of canvas (texture paper, glass, etc.). Each parameter adds a dimension to the multidimensional table of the stroke color palette, but the essential algorithm does not change.

  Artists sometimes mix two colors or more in one brush stroke. In the Impressionist style, these colors are often not well mixed, and unmixed colors are visible as separate components of the stroke.

  In the algorithm described above, multiple color brush strokes can be generalized to single color brush strokes. It can be done as appropriate with multiple opacity maps. These opacity maps define non-zero opacity because of the common exclusive region of the brush stroke. They define the overall brush stroke shape. By introducing the following modifications to the algorithm, multiple color brush strokes are provided.

When a stroke color palette is constructed for each brush stroke, the brush stroke is colored with possible combinations of pliers colors from the paint color palette. This is obviously helpful for a small paint color palette. The input of each stroke color palette is constructed in the usual way, and each stroke color palette has a corresponding number of paint color palettes. When drawing an image, find the closest stroke color in the stroke color palette in the usual way. However, the stroke is colored with a number of paint colors.
Explanation of Technology 30

  Referring to FIG. 284, further improvement steps of the art cam device program are shown. As with standard techniques, it is assumed that the brush stroke is characterized as a piecewise Bezier curve. The brush stroke is assumed to have a predetermined thickness. The brush stroke consists of a series of Bezier curves. The first step 4211 is to process each piecewise Bezier curve. Each Bezier curve is first converted to a polyline using standard techniques (4212). Next, the “fastest” edge (whose meaning will become clear below) is used to “step along” the fastest edge in a given increment. Used to do.

  Referring to FIG. 285, a first Bezier curve 4220 that forms the basis of the brush stroke 4221 is shown. The process of converting the Bezier curve 4220 into a corresponding series of line segments (eg, 4222). The process of linear approximation to line segmentation is standard and well known and is covered in ordinary textbooks. As part of the linear approximation process, the normal of each line endpoint (eg, 4224) is also determined. Those normals can be determined from lines adjacent to a point (eg, 4225), if necessary, by a method that interpolates the slope of the line.

  Referring to FIG. 286, an enlarged view of two line segments 4233 and 4234 is shown. For each point (eg, 4232), a normal (eg, 4233, 4234) protrudes on each side a distance of the width of the brush stroke. Point 4236 has protruding normals 4237 and 4238. Next, the distance 4239 between the normal 4233 and the normal 4237 is measured, and the distance 4240 between the normal 4234 and the normal 4238 is measured. The “fastest” edge is defined on the side with the longest distance in the sense that the body moving along edge 4239 must move faster than along edge 4240. Therefore, the fastest edge is the convex edge of the curve.

  The fastest edge is used to define a predetermined number of points as parameters separated by an equal amount. The example of FIG. 286 shows three points 4243-4245 defined at equal intervals along line 4239. Most of the intervals are longer than those shown in FIG.

  Next, the parameter position of each point is determined and used to determine the parameter points along line 4230. The position is a position where the brush stroke is arranged.

  Referring to FIG. 287, a series of brush stamps 4246, 4247, 4248 are shown. Two stamps 4246, 4248 are used for the end portion of the line, and portion 4247 is used continuously along the central portion of the line.

  The brush stamp is defined by three separate channels: a matte channel, a bump map channel, and a footprint channel. In the example of FIG. 287, a brush matte channel is shown. In the first technique, known as the “maximum” technique, the matte and bump map channels are used to build brush strokes in a separate brushing buffer. In this synthesis technique, the brush is built by taking maximum opacity. If the new opacity exceeds the old opacity in the buffer, the opacity is replaced with the new opacity. The result produced is close to an airbrush or painting in Photoshop.

  In the second synthesis technique, it is known as the “footprint” technique. The footprint channel is used so that the mat value is changed only if the brush stamp described above for the current line has not been written to the brush buffer for the pixel. In the third technique, a mat channel and a bump map channel are used. At this time, the mat channel and the bump map channel are minimum.

技術３１の説明 The above technique has been found to give good results, especially when mimicking the effect of a watercolor painting. The following code segment illustrates the processing of a further improved embodiment. :

Explanation of Technology 31

  Referring to FIG. 288, a further improvement arrangement (4310) is shown, including an art cam device 4302, which is connected to a text input device 4303 comprising a touchpad LCD capable of proper character recognition. Yes. Alternatively, a keyboard input device (eg, 4304) can be a text input device. A preferred form of text input device may be configured with an Apple Newton® device adapted and programmed to be connected to the art cam device 4302. Also, other forms of the text input device 4303 can be used. In addition, an art card device 4305 is provided that is inserted into the art cam device 4302 to process the sensed image (as fully discussed in advance) in accordance with the figures drawn on the surface of the art card.

  Referring to FIG. 289, a preferred form for further improved operation is shown. In this form of operation, the art card 4305 is encoded with a Var script that includes a font defined in Roman letters and a description of how to create special characters in the font. For example, the description includes a method for processing an outline to create a new character in the font.

  The input devices 4303 and 4304 have input device fonts. The input device font is used by text input devices 4303 and 4304 to display information. In particular, non-Roman characters are used. Therefore, the input devices 4303 and 4304 are used for inputting text requested by the art card 4305. Based on the input, the font outline is downloaded to the art cam unit 4302 which processes the outline according to the instructions encoded in the art card 4305 for the creation of special characters. These characters are created by the art cam device 4302 and displayed as part of the printed output image.

By using this operation method, the flexibility of the art cam device 4302 is expanded without storing the font arrangement in the art cam device 4302 or the art card device 4305. This method only requires that the text input devices (eg, 4303, 4304) be country specific to reduce the complexity of the model used to operate the art cam device 4302 in a non-Roman language format. It is.
Explanation of Technology 33

  A further improvement is the reproduction of the camera image by taking a picture with a digital image camera and printing with a printing device such as an inkjet printer having a predetermined dpi (eg 1600 dpi) as disclosed in the aforementioned patent application. It is a premise that it is desired.

  Referring to FIG. 290, such a digital image camera device 4501 is designed to print out a standard output resolution image 4502. The image 4502 is a multi-color output (cyan, magenta, yellow), and is printed by an ink jet printer apparatus that includes an array of dots for each color component of the input image. If a copy of the output photo 4502 is desired, the image is first scanned and appropriate color components are extracted.

A further improved method is shown in FIG. In this method, a photographic image 4502 is scanned for each color component by a linear CCD 4510. Suitable linear CCDs are known in the art. For a description of the configuration and operation of a linear CCD device, “CCD array, camera, display” Gerald
C Holst, 1996, SPIE
Standard literature such as “Optical Engineering Publishing” is consulted. In addition, suitable sensor devices are regularly described in the Institute of Electrical and Electronic Engineers report on home appliances.

  The CCD array 4510 scans an image passing under the CCD head to generate a signal, and converts an analog signal into a digital signal. The CCD array 4510 has 4800 dots per inch, which is three times the dot resolution of photographs. The CCD array 4510 has three primary colors, filters, and CCDs, one for each color. 4800 dpi is achieved by using a series of staggered CCD arrays.

  The data value is transferred to the frame buffer controller 4512 and stored in the frame buffer 4513 for each color component. The stored image is printed on the photograph, and as described above, when it is scanned by the CCD at a CCD scanning resolution that is about three times the resolution at the time of printing, it is arranged at the original dot position. To be processed. Unfortunately, there may be many defects in the scanned image. These defects include defects associated with the scanning process, including the effects of scratches and distortion on the photo 4502 in addition to the slight rotation of the photo 4502 when the photo 4502 is sent to the CCD scanner 4510.

  The procedure for determining the pattern and dots depends on the use of a process that extracts the rotation of the photograph 4502. Many methods can be used to determine such rotation.

  For example, a scanned pattern of ink dots has fundamental characteristic frequencies based on the rotation. Since photographs are printed with ink in a general arrangement of dots, the Fourier transform of the image is used to determine possible rotations. As an alternative, the edge of the card can be determined from the photo boundary or the underneath scanning surface of the CCD. In addition, the desired dot pitch of the pixel is determined (1600 dpi).

  From one end edge of the scanned image of the photograph, a process for determining whether or not a dot is arranged at each output dot position is performed. Importantly, a method of maintaining synchronization across the card is used to make the output dot determination. It is clear that the local change in the gap from one column to the next is very small, and many changes exceed the length of the image.

  One form of processing to determine rotation and the spacing between adjacent dots is to record the row of dot centers (known as the “centroid”) and rewrite the centroid in the column using the column method (update). It is. Referring to FIG. 292, one form of gravity center processing is shown, and the image area 4520 has sample ink dots 4521 on the surface portion of the card. The size of each ink dot in each pixel is about three times the corresponding pixel sampling rate. A series of center of gravity marks (eg, 4522) is kept in each column. In order to determine the initial rotation of the center of gravity in the center of gravity mark of the column Cn + 1, the center of gravity mark calculated previously or the center of gravity mark of the adjacent column Cn is used. For example, centroid marks 4522, 4524, and 4525 are used to determine the initial position of the centroid mark 4523. Once the initial position is determined, pixel values around the center of gravity mark 4523 are examined to determine if further adjustment is necessary.

  The determination that the centroid 4523 is moving from the expected position is derived by examining pixel values around the point 4523. The survey is done separately in the X and Y directions, and the movement is done separately in those directions.

  Many different methods can be used. One method for determining whether a minor correction of the centroid 4523 is necessary is described according to FIG. It will be noted that according to this method, only a limited number of ink dot arrangements are possible. These possible arrangements are as shown at 4530 in FIG. 293 and show the current pixel 4531 and two adjacent pixels 4532 and 4533. Each ink dot 4531-4533 has approximately three pixel values as detected by the CCD sensor. An example of the CCD output value 4535 after digital conversion is shown for each pixel pattern 4530 in FIG. Due to the sampling effect of the CCD and other mechanical and brightness effects, the CCD value 4535 of the dot pattern often includes an equivalent to an AD-converted Gaussian curve (or part thereof). Therefore, a sharp edge is usually not given. However, the expected cross-section can be examined for what was obtained to determine the closest cross-section, and the error is not very large and is adjusted within limits to better fit. According to this method, a new center of gravity position is obtained in the first dimension. Of course, the task of adjusting the center of gravity is symmetric in the two directions, and the same processing steps apply in other dimensions. Many other techniques may be used, such as more advanced techniques such as neural networks trained from sample images.

  When the adjusted position is determined, the center of gravity 4523 (FIG. 292) is slightly adjusted, and the center of gravity determination process is continued. From that new centroid position, the value of the surrounding swatch is examined to determine whether an on or off ink dot value should be recorded for a particular centroid position.

  Returning to FIG. 291, the recorded ink dot values are stored in the ink dot array 4516 and are printed at the same resolution (1600 dpi) or different resolutions of the printer (eg, as shown in 4517 and equal to the original photo 4502. Used to print a photo 4518). According to this method, it is possible to obtain a copy of a high-quality photograph that is a copy from the original photograph without using a negative or digitally dividing and storing the image.

Of course, many improvements may be possible, including the use of a customized real-time pipeline basic design concept to speed up processing and eliminate the need to store large data sets in frame buffers etc. Absent.
Explanation of Technology 34

  In a further improvement, the art cam device is modified to include a 2D motion sensor. The motion sensor may comprise a small micro electro mechanical system (MEMS) device that detects motion in two axes, or other suitable device. The motion sensor can be attached to the camera device and its output is monitored by the Artcam central processor.

Referring to FIG. 294, a schematic diagram of a preferred arrangement for further improvement is shown. The accelerometer 4601 outputs the blurred detected image from the CCD device to the art cam central processor 4602. The art cam central processor 4602 uses the acceleration watch measurements to determine the angular velocity of the camera when the picture was taken. The measured value (velocity factor) is used by a suitably programmed art card processor 4602, blur correction is performed on a blurred detection image 4603, and an output image 4604 without blur is output. As the blur correction program in the art card processor 4602, a standard algorithm known by those skilled in the art of computer programming and digital image restoration is used. For example, “Selected Papers on Digital Image Restoration”, M.M. Reference is made to Ibrahim Sezan edited, SPIE Milestone series, volume 74, especially reprinted pages 167-175. Furthermore, a simplified technique is described in “Image Processing Handbook” 2nd edition, John C. Published by Russ, CRC publication, pages 336-341.
Explanation of Technology 38

  Referring to FIG. 295, the configuration of the attached printer 5010 is shown as a configuration according to the direction of further improvement. The printer 5010 is designed to have a print roll 5011 that is inserted into the cavity 5012. The print roll 5011 is inserted into the cavity 5012 and has rollers (for example, 5013 and 5014). The roller sandwiches the internal paper and conveys the paper to the print head 5015 as required for printing an image. Ideally, the print roll 5011 includes a print medium in addition to a device that supplies ink to be consumed. Ink is consumed by the print head 5015 during image printing.

  The printer 5010 can be constructed by appropriate reworking of the art cam embodiment. The printer communicates with the computer 5024 via the USB port. The printer unit 5018 is a plastic molded case and has an internal print head 5015 for printing an image.

  The print head 5015 is connected to the print head chip 5020 by an automatic bonding technique. In order to reduce the size of the printer unit 5018, the print head chip 5020 is attached to a flexible circuit board. In order to generate an image from a description transferred to the printer unit 5018 via the USB communication cable 5022, the print head chip is made by the latest semiconductor manufacturing technology and has an associated memory and the like. Of course, many other interface formats can be utilized. The printer head chip can be described in advance.

Accordingly, when it is desired to print an image, the computer system 5024 connected to the printer unit 5018 transfers a page description language (page description) to the print unit 5018 via the cable 5022. The print chip 5020 prepares an image to be printed later by the printer head 5015. A print roll 5011 supplies print media and ink to be consumed in an easy-to-use form. When the print roll 5011 is used by the printer unit 5018, the print roll is replaced with a new print roll.
Explanation of Technology 40

  In a further adaptation, the camera system and printing system are exempted and replaced with a large screen reader. The art card is set so that information is written on the back side and the display of the information is printed on the other side. For example, an art card can include book content and newspaper content. An example of such a system is shown in FIG. 296, where art card 5210 has a book title on one side and printed coded data on the other side. The card 5210 is inserted into the reading device 5212. The reading device 5212 includes a flexible display 5213 that can be folded. The card reading device 5212 has a display control device 5214 for turning pages forward and backward and performing other controls.

  Therefore, it can be seen that the configuration of FIG. 296 can efficiently distribute information in the form of books, newspapers, magazines, technical manuals, and the like.

It will be understood by those skilled in the art that numerous changes and / or modifications may be made to the invention as illustrated in the specific embodiments without departing from the spirit or scope of the invention as broadly described. Let's do it. Accordingly, this embodiment should be considered in all respects, whether or not illustrated.
Explanation of Technology 47

  Further improvements are discussed in the process of “drawing” an image on the canvas using a pseudo brush stroke. It is assumed that bump map technology is used, and in particular, it is assumed that each image used has a bump map that defines the texture of the image surface.

  FIG. 297 shows a round brush / bump map 5901 as a sample. In FIG. 298, a cross-section along line A-A ′ of FIG. 297 is shown, which has a general outline 5903 of the surface of a general round brush stroke 5901. Brush stroke with bump map of FIG. 297 is applied to a typical Hessian “canvas” image (the bump map is shown in FIG. 299 and the height of the BB ′ line is shown in FIG. 300). That is the premise. Its height consists of wavy peaks (eg 5906).

  In a further improvement, the bump map of FIG. 298 and the bump map of FIG. 300 are combined in various ways according to the supplied stiffness factor to form a final bump map. The stiffness factor is designed to produce an effect using “rigid” paint or “flexible” paint. In FIG. 301, an example is shown where the two bump maps of FIGS. 298 and 300 are combined for a low stiffness brush paint. In FIG. 302, an example is shown in which the two bump maps of FIGS. 298 and 300 are combined for a highly rigid brush paint.

  In a further improvement, the bump map is not simply added when joining the bump maps, since the brush stroke made of paint has a certain plasticity and fills the recesses in the background. Therefore, in order to achieve the effects of FIGS. 301 and 302, the brush map of the brush stroke is added to the background bump map which has been low-pass filtered. The four-pass filter has a radius determined by the stiffness factor of the paint. The higher the stiffness factor, the wider the filter range and the less visible surface texture of the background. The range of the low-pass filter so that the filtered background surface texture converges to the actual background surface texture where the brush strokes are light or empty (mostly along the edges) Varies depending on the brush stroke thickness.

Referring to FIGS. 303 and 304, an example of processing using FIG. 303 showing an initial background bump map is shown. FIG. 304 shows a high-rigidity paint 5912 in a state of covering the bump map 5911 with a low-pass filter applied, and FIG. 305 shows a low-rigidity paint 5914 in a state of covering the bump map 5913 with a low-pass filter applied. Shows.
Explanation of Technology 48

In a further improvement, the color gamut of the input image is the color gamut of the output image.
“morphed” by gamut. The color gamut of the output image is determined arbitrarily by a method of trial of a plurality of different color gamuts. In a further improvement, pre-processing steps are ideally used for further artistic processing, such as replacing an image with a particular style of brush stroke. The brush stroke extracts a color from the image, and the brush stroke has a color associated with the desired color gamut.

  The step of creating the final output image is indicated by reference numeral 6010 in FIG. The first step 6011 is to input an arbitrary input image that is preferably detected by the art cam device. The color gamut of the input image is “Morphed” or “Warped” to create an output image with a predetermined range of color gamut (6012). The morphing process will be described later. Once the output color gamut is created, the next step is to apply the appropriate brush stroke technique. The brushstroke technique is a process that can bring about great changes. In the actual form of brush stroke used, the technique described above is not absolutely necessary for the present invention. The brush stroke technique is applied to produce an output image by performing an effect with a limited color gamut on the input image.

  Referring to FIG. 307, an example of a gamut mapping and morphing problem is shown. It is assumed that a single color space is used by defining many possible color values in the universal color space 6021 which is the L * a * b * color space. Within this color space, the input sensor can detect a range of brightness 6022. Further, it is assumed that the output printing apparatus can output a specific range of colors in the printer color gamut 6023. The printer color gamut may be smaller or larger than the input sensor color gamut 6022. The artistic color gamut 6024 is defined according to a specific style. The artistic color gamut 6024 can be carefully constructed through the use of various techniques. For example, a desired output image is scanned and a color histogram is constructed such that the desired output image includes a specific range within the L * a * b * color space. It can be understood by those skilled in the computer graphic arts that FIG. 307 can be interpreted as a three-dimensional L * a * b * color space of a given brightness.

  Therefore, it is necessary to map the colors in the input sensor gamut 6022 to the artistic gamut 6024. Moreover, unfortunately, artistic color gamut 6024 includes colors that are “protruding” for a particular output printer color gamut 6023. In such a case, it may be necessary to map the artistic color gamut 6024 because the output color is in the printer color gamut 6023.

  Referring to FIG. 308, the main requirement is that the art card has a three-dimensional lookup table (eg, 6030) for moving the L * a * b * value 6031 to the L * a * b * output value 6032. It is. Preferably, the 3D lookup table 6030 is a compact having specific points defined for 3D mapping and cubic interpolation used for mapping intermediate values between the defined points. Provided in form.

Referring to FIG. 309, a very general example of a process for applying gamut morphing from one substantially arbitrary input gamut space 6040 to a second desired output gamut target space 6041 is shown. Yes. Potentially, the gamut mapping process must process points that are inside the input gamut space 6040 and outside the output gamut target space 6041 (eg, 6042). The three-dimensional color lookup table 6030 has a size of (2 ⁿ +1) ³ . n ranges from 1 to the maximum color component precision (eg, typically 8 bits). Any warp is encoded in the lookup table. The use of the table also results in an algorithm that makes the separately prepared encoding warp functions independent. The algorithm has predictable performance for any warp function. It is not required that the warp function is continuous. Therefore, the algorithm is robust for any warp function.

  The lookup table is a 3D array of integers, and real output color coordinates are arranged in the order of input coordinates. In this way, the forward warp function is encoded.

  The requested image color warp is calculated pixel by pixel in the order of the output image. Although the algorithm requires random access to the 3D color look-up table, random access is generally coherent because the input color changes smoothly in space. Sequential access to input and output images is required.

The essential trilinear warp algorithm is implemented with the following pseudo C ++ code:

  The gamut compression process attempts to map source gamut colors to smaller target gamut colors in a manner that minimizes perceptible color changes while maintaining color differences in the source gamut. Since the calculation of gamut compression is decoupled from color warping, effective gamut compression is performed using the look-up table driven process described above.

The gamut compression algorithm involves the construction of a three-dimensional lookup table and is implemented with the following pseudo code as described with reference to FIG.

  If either the source or target gamut is known in the pallet form, the gamut polyhedron is calculated from the convex hull of pallet points.

  Referring to FIG. 310, a process commonly used in the creation of artistic effects, which is referred to as gamut morphing as will be described later, is used to convert some source gamut 6050 to the target gamut 6051. The above process can be used to map. Gamut morphing is used to directly control the mapping of colors in the source gamut to colors in the target gamut.

  Similar to gamut compression, gamut morphing may be used to resemble the gamut of an image to a particular artistic style. Since gamut morph computation is decoupled from the color warping process, as with gamut compression, effective gamut morphing can be implemented by an algorithm that drives a lookup table.

  Along with gamut morphing, many source gamut colors (eg, 6053) are directly mapped to the same number of corresponding target gamut colors (eg, 6054). The remaining target color of the mapped target in the lookup table is calculated as a weighted sum of the specific target color (eg, 6054). The target color is preferably weighted by the reciprocal of the square of the distance from each lookup table point to the corresponding source color.

The gamut morphing algorithm is embodied by the following pseudo code.

It will be apparent to those skilled in the art of computer graphics that the techniques described above can be used to initially limit an image gamut to a predetermined area. Subsequently, a brushstroke filter is applied to the restricted gamut image, for example, to produce a similar effect as given by the “point drawing” technique.
Explanation of Technology 53

  The basics of the art cam system described above are indicated by reference numeral 6501 in FIG. The art cam system 6501 relies on an art cam 6502 that requires an art card 6503 as input. The art card 6503 has coded information (instruction information) for processing an image scene 6504 to create an output photo 6505, and the output photo contains instructions for the art card 6503. The image is greatly processed according to the information. The art card 6503 is made extremely inexpensive, one side contains coded information, and the other side has effects that would be generated when inserted into the art cam 6502. Includes a depiction.

According to a further improvement method, a number of art cards 6510 are prepared and sold in packs as shown in FIG. Each pack is associated with wearing a specific size garment, and has an image of a model (eg, 6511) with the garment 6512 mapped onto an image taken by the camera. The mapping is ha with different art cards 6510
It is also possible to have different clothing items. One form of mapping algorithm is shown at 6520 in FIG. In the algorithm, the input image is first warped (6521) using a warp map that maps the image to an iterative tiling pattern that produces an attractive warping effect. Of course, many other types of algorithms, given by art card 6503 (FIG. 311), are prepared to generate attractive forms of materials.

  Next, a second warp 6522 is prepared to warp the output of the first warp map 6521 to an image of a particular model in the art card. Thus, warp 6522 is unique to art cards. The result is output for printing as art cam photo 6505 (6523). Thus, the user can show the desired image 6504 on the art cam 6502 and can create an art cam photo 6505 having a processed version of the image with a fashionable clothing model. This process can continue until the desired result is obtained.

  Next, when the final selection of the cloth is made, as shown in FIG. 314, the art cam 6502 is connected to the cloth printer 6534 having an inkjet cloth printer and related drive control electronic devices via the USB port. Connected (6503). Either the art cam 6502 or the inkjet printer 6534 is programmed to print a garment piece (eg, 6536) on the cloth 6535. The garment piece has an original warp image on its surface to create the same garment conveyed by the art card model.

  The output fabric has tabs (eg, 6538) for alignment and boundary areas (eg, 6539) in addition to instructions 6540 for joining garment pieces. Preferably, the output program includes provision for edge warp matching to generate continuously on the crossed seam of the garment.

Furthermore, the user interface is prepared for the use of the same card with many different output sizes to take into account differently shaped bodies. Based on the use of art cam technology, the use of art cam mechanism 6502 can provide a system for the production of customized clothing and rapid rendering of appropriate results.
Explanation of Technology 54 **
Explanation of Technology 56

  In a further improvement, a plurality of art cam devices as described above are connected via a USB port in order to transfer the effect of the image. The transfer of image effects is accomplished by an appropriate program in the computer section inside each art cam.

  A preferred arrangement is that a series of art cams (eg, 6802, 6803, 6804) are connected via their USB ports (6805) as shown in FIG. Each art cam 6802, 6803, 6804 is provided with an art card 6807, 6808, 6809 in which a suitable image processing program is stored. Further, an instruction to use the network environment can be given by the art card 6807, 6808, 6809. The image 6810 detected by the art cam 6802 is processed by the processing program on the art card 6807. The result is transferred to an art cam device (next art cam device in a series of art cam devices) 6803 to which the image processing function is applied by the art card 6808 (6805). For this purpose, each art cam device has been modified to have two USB ports. The last art cam device 6804 applies the processing stored in the art card 6809 in order to create the output 6812 which is a collection of the image processing described above.

Therefore, the configuration 6801 in FIG. 315 can give various effects to one detected image. Of course, many further improvements are possible. For example, each art cam may print an image it has processed in addition to transferring the image to the next art cam. In addition, the paths are separated so that one art cam is output to two different downstream art cams and different final images are output. In addition, a loop or the like is also used.
Explanation of Technology 57

  In a further improvement, the disclosed technique is adapted to be incorporated into a binocular set so that images can be printed upon request from a binocular user. Referring first to FIG. 316, an example of a binocular system is disclosed. The binoculars 6901 are of a known type and are modified in accordance with the principles of the present invention. The binocular has a conventional optical lens system 6902, 6902 with a lens input. The two lens systems are attached to the connector plate 6906 via hinge systems 6904, 6905. The arrangement is largely conventional.

  The first optical lens system 6902 has a beam splitter device 6908 that splits the optical path 6909 of light incident on the optical system 6902 into two optical paths. The first optical path 6910 is directed to the side on which the CCD system 6914 is attached. The second optical path 6911 is directed toward the eyepiece portion 6915. The beam splitter device 6908 weakens the output intensity of the light beam 6911. Accordingly, an attenuation filter 6918 may be arranged to match the attenuation by the beam splitter device 6908 in the optical lens system 6902.

  Accordingly, the CCD image capturing device 6914 captures the same scene as viewed with the optical lens system 6902. The CCD device 6914 is connected to a processing system mounted on a printed circuit board 6920 having an art cam central processor and a memory device. The art cam central processor device has an important image processing function and controls the CCD device to capture an image and prints the image using a print head device 6923. As the printer, an inkjet printing apparatus that discharges ink onto a print medium is used. The print media consists of a “paper” film (actually a suitable polymer) supplied from a removable disposable print roll 6925 containing the print media and printing ink. The print roll 6925 is accommodated in a housing 6925, and a battery 6927 is detachably disposed in the housing.

  When the user depresses button 6919, the art cam central processor activates CCD 6914 to capture the scene. The image captured by the CCD device 6914 is processed by the ACP processor and transferred to the printhead for immediate printing in order to immediately and permanently record the captured scene. The printhead is again as described above.

Many improvements can achieve further improvements. For example, a USB port connected to the ArtCam central processor may be provided so that the image processing algorithm can be preloaded into the binoculars 6901. A series of buttons for operating the routine may be arranged. The routine includes various image enhancement operations and the like. In addition, the output image may be modified in many different ways to enhance the image characteristics. The image may be further characterized by automatically placing relevant information about the date and place in the image. In addition, the binoculars may be connected to the GPS so that the observer's coordinates are also immediately printed on the output image. In addition, other options such as providing a distance measuring device for specifying the location of the subject and displaying information about the subject on the output image may be performed.
Explanation of technique 58 Explanation of technique 59

  In a further improvement, the disclosed technique is utilized to provide a fault tolerant data array on the card surface. The data is updated by rewriting to another fault-tolerant array.

  Referring first to FIG. 317, the card 1701 is shown after data (eg, 7102) has been written on the card surface and the data array has been completely written. Each data array (for example, 7102) is an array of 64 data blocks shown on the card surface, and about 20 KB of information is encoded. Of course, other arrays can reduce or increase the amount of data written into each data block (eg, 7102). In the initial state, the card 7101 has no data block. Instead, perhaps one usage data block is written on the surface of the card.

  Referring to FIG. 318, a second card 7105 is shown, which has been “updated” 12 times with the latest data block 7106, which is the last data block written. Thus, each card (eg, 7105) is updated according to the number of data blocks given to the card and used many times.

  Referring to FIG. 319, the configuration of one data block (eg, 7108) is shown. The block is a reduced version of the “Art Cam” technology. The data block 7108 consists of a data area 7109, which has a printed dot array that has a one pixel wide border in addition to a series of clock marks along the edge. A target 7110 is arranged along the edge of the data area 7109 to assist in positioning and reading the data area 7109. The configuration of each target 7110 is as shown in FIG. 320, and consists of a large black area surrounding only one white dot. Of course, other configurations are possible. The target 7110 is provided to accurately position the data area 7109. In addition, those targets 7110 are further provided to accurately sense and derive accurate location information.

  Referring to FIG. 321, a card reading and writing device 7120 is shown. An enlargement of the main part of FIG. 321 is shown in FIG. The device 7120 has an elongated gap 7121 for inserting a card 7122 on which information is printed. Many pinch rollers (eg, 7123, 7124) control the movement of the card across the print head 7125 and linear CCD scanner 7126. The scanner 7126 scans data passing below. FIG. 322 shows an enlarged view of the arrangement of the print head 7125 and CCD scanner 7126 for the inserted card 722.

  Referring to FIG. 323, how the card 7122 is inserted into the CCD scanner 7126 is schematically shown. The CCD scanner 7126 conveys the card 7122 so as to pass through the scanner, and information stored in the card 7122 is decoded by an art cam central processor unit attached to the CCD scanner 7126. If it is desired to remove the card 7122 from the card reader, a determination is made whether writing a new block to the card is necessary. The position of the new block is known from the previously scanned CCD data. The card is taken out as shown in FIG. 324 so that the card passes through the print head 7125 and the CCD scanner 7126. The scanner 7126 monitors the current position of the card 7122. As shown in FIG. 325, when it is desired to update the square area 7128 with new data, the print head 7125 ejects ink droplets. Then, the card is taken out from the apparatus (FIG. 321) by a method similar to a floppy disk or the like. The card is then used continuously until all data space is filled and a new card is created.

  Therefore, the use of the card system described above provides an effective and inexpensive form of information placement in that the card can be made inexpensively and used in an elastic manner for information placement. It will be obvious.

Referring to FIG. 326, a block diagram of different functions utilized in the art card reader is shown. The print head 7125 and linear CCD 7126 are operated under the control of an appropriately programmed artcam central processor (ACP) chip (ACP) 7130. The ACP 7130 includes a memory 7131 for storing scanned data and other data and programs. In addition, the ACP controls various motors 7132 that actuate the pinch rollers. Of course, other control buttons and the like can be arranged as required. The use of such an organization provides a system for monitoring and updating information stored on the card surface.
Explanation of Technology 60

  In a further improvement, an art card photo vending machine is provided that can create its own art card upon user request. The vending machine is configured in the same way as a normal photo vending machine, but only the request for computational resources configured to automatically create an art card within a given time is essential. Absent.

  Referring first to FIG. 327, functional elements of a further improvement 7201 are illustrated schematically. A further improvement 7201 includes a high resolution scanner 7202 for scanning a user sample photo 7203. On the other hand, the scanner 7202 may optionally be allowed to allow the user to process their photos in order to give the art card creation system an added degree of realism. The photograph 7203 scanned by the scanner 7202 is transferred to a backbone computer system 7204 which is a high-end PC type computer having an appropriate operating system and program. The computer 7204 controls the storage of scanned photos and the photographic printer and art card printer 7205. The photographic printer and art card printer 7205 are preferably capable of printing on both sides of the output print media and can use printing techniques such as those disclosed previously. The printer 7205 outputs a processed version of the user's photograph 7206 in addition to the art card 7207. Art cards are printed on both sides. In order to generate the photo 7206, the encoded instructions for image processing of the photo 7203 are included. The encoded instructions are output in an art card format.

  The computer system 7204 also has a user interface 7209 such as a standard touch screen type. The computer system 7204 also controls or incorporates a payment system 7210. The payment system 7210 comprises a standard coin or bill payment system or a credit payment system with appropriate network connection with a credit card service provider for transaction authorization. In addition, EFTPOS facilities may be provided.

  When the user inserts their photo 7203 into the scanner slot 7202, the photo is scanned, stored, and subsequently ejected. Next, the user operates the user interface 7209 located in the “photo booth”. The user interface 7209 first instructs the payment system 7210 to pay money. However, the core of the user interface 7209 is the creation of various art cards with a touch screen.

  An example of a suitable user interface is shown in FIG. The core of the user interface consists of showing the user a number of sample thumbnail images 7213 that are processed in the manner described below. First, for example, image processing 7213 such as a company event, a birthday event, a seasonal event, and a processing type is arranged in the subject area. The user selects a favorite image (for example, 7213) using the touch screen. If the preferred image does not exist, the user can press the arrow button 7214 to display other processing. When the user selects an image 7213, that image is used to generate an alternative that presents a series of processes with a “theme” similar to that image. The user can continue selecting images from the selected type.

  A favorite image can be saved by using a save button 7217. The user interface then shows the selection 7218 being saved. The navigation button 7215 is for returning to the previous screen, and the navigation button 7216h is for returning to the upper level. This method allows users to search various art cards to make their own unique customized requests. According to this method, unique artistic creation activities encourage the creation of unique art cards.

  The creation of such various art cards relies on genetic algorithm technology that gives the user the role of the creator in the creation process. Referring now to FIG. 329, there is shown an example of a software layout of an application executed in the computer system of FIG. The software layout 7220 includes a genetic pool 7221 of image processing that can be performed, and theme processing adapted to specific images. This genetic pool is used by the genetic algorithm core for the generation of new types (genus). Introducing the standard texts of genetic algorithms, for example, is “Genetic Algorithm” by Golberge, who has added the latest processing in this field. Alternatively, the field of genetic programming can be utilized, and in this regard, reference is made to the general work by Koza entitled “Gene Programming”.

  The core of the genetic algorithm can also use the user's selection 7223 in creating new and preferred images. The output of the genetic algorithm core includes the appropriate new image that is transferred to the user interface module 7225 for display on the touch screen display.

  Returning again to FIG. 328, if the user finds a favorite selection, the accept button 7219 can be pressed. Thereby, payment by the user is requested and accepted. In addition to a series of image processing, the art card 7207 has an instruction to output an image to the printer 7205 so that the user can use it with another device that accepts the art card 7207 (for example, the above-described art cam device). 7204 (FIG. 327).

  Thus, it will be apparent to those skilled in the art that further improvements provide a system for creating complex, customized images to fill in their names that can be used separately by the user. The use of many images leads to a sudden change in the art card's important connections that can lead to important personalization as well.

  Of course, many other types of core technologies can be used for image construction. For example, other non-genetic techniques may be suitable. In the worst case, the selection of each image is prepared manually. In addition, other user interface mechanisms may be provided. In addition, the present invention can be implemented in an environment that is not a vending machine, such as a standard computer system.

Further refinement operations provide for an important expression of personal creativity. And it is thought that individual creation by an artist may present important value in providing an art card “series” of individual artists.
Explanation of Technology 61 Explanation of Technology 62

  The basic art cam arrangement described above is shown in FIG. 330 in schematic form. This array has a CCD sensor 7402 for detecting images and scenes. In addition, an art card reading sensor 7403 is provided to read the art card 7408 for an encoded image processing algorithm for processing the sensed image. Both the CCD sensor 7402 and the art card reading sensor 7403 are connected to an art card central processing unit (ACP) 7404 that provides complex computational power for processing the detected image. In addition, a memory unit 7405 is provided for storing images, detected data, programs, and the like. Connected to the ACP 7404 is a print head 7406 for printing a final photograph 7404 on a print medium supplied from an internal print roll.

  In a further improvement, a unique series of art cards 7408 are inserted into the art card reader 7403 to provide unique control of the art cam central processor 7404. A first example, shown in FIG. X2 or FIG. 332, provides the use of multiple art cards to provide a superimposed or augmented image effect. A suitable duplicate art card is shown at 7410 in FIG. 332, with instructions on one side regarding how to operate the camera device to produce an effect. An instruction for operating the art cam device is provided on the other surface to produce the effect. Referring to FIG. 331, an example of a repetition card operation for producing a combined effect is shown. The art cam system has a detected image or a stored image of a certain scene. The first step is to insert a repetition card 7413 with a code that changes the operation of the art cam system to enter the repetition mode. Subsequently, the first art card 7414 is inserted into the art card reading sensor 7403, and the first effect is given to the image according to the instruction of the card 7414 (see reference numeral 7415). Subsequently, the repetition card is inserted again (reference numeral 7416), and then the second art card 7417 is inserted, giving a second effect of placing a text message on the image, for example (reference numeral 7418). Next, the repetition card 7419 is inserted again, and then a third art card 7420 is inserted to give further effects to the image 7421. The process of FIG. 331 is repeated as required to form the desired output image. According to this method, the above-described apparatus is used with flexibility in order to create a combined effect from an art card having only one effect. Furthermore, the provided user interface is simple and effective in creating a combination effect. Of course, many changes can be made. For example, in another embodiment, a repetition card is inserted only once, and a series of art cards is inserted following the repetition card.

Referring to FIG. 333, another art card 7430 is shown for internal testing of the art cam system. Each ArtCam system has many internal test routines in its internal ROM. The test is accessed by a special function call in the translation language provided in the Artcam central processor. The routine is the specification of the art cam device, for example,
Printing a test pattern to determine the operating state of the printhead;
Printouts of test patterns (eg, all-black prints) that perform printhead manipulation processes for nozzle cleaning and for nozzle arrays that improve printhead operation;
A test pattern is printed for later analysis to show the effectiveness of the printhead operation;
Is included.

  Referring to FIG. 334, there is shown a sample test output 7435 having various internal data to be referenced in addition to the printout of the test pattern 7437. The test pattern 7437 can later be investigated by an automatic or lead method that identifies any problems that may exist in the camera system. Further improvements are performed through the use of software routines programmed into the ArtCam device and stored in ROM memory.

Of course, many improvements can be expected as the routine is updated or changed. Many tests are not really limited. According to this method, the operation of the camera device can be changed according to the inserted card.
Explanation of Technology 63 Explanation of Technology 64

  In a further improvement, magnetically sensitive print media material is used to record a voice message on the back side of the output photo. The art cam device is modified to include a magnetic recording device comprising a magnetic strip, such as an array of magnetic recorders covering the entire surface of the photograph or, for example, a magnetically sensitive center of the photograph. The art cam device is further provided with the ability to record voice messages for later playback.

  In a further improvement, the art cam device is suitably modified to be equipped with a microphone device and associated recording technology. When taking a photo, you are given the opportunity to record either the surrounding sound environment or a message related to the image. The print media or film has been pretreated to have magnetic sensitivity, similar to tape media. Recordings are made on the entire backside of the output photo or on the magnetic sensitivity band. The recorded audio is retained on the back of the output photo in coded form. The encoding is preferably in a highly digitally reproducible format. The recorded audio provides a permanent audio recording related to the photo. A playback device is then provided to scan the encoded speech and decode this information.

  Referring to FIG. 335, a further improvement 7601 is shown in schematic form, including the arrangement described above, that is, an arrangement in which an image 7602 is detected by a CCD sensor 7603 and transferred to an art cam central processor 7604. The art cam central processor 7604 stores the image in a memory 7605 which preferably comprises a high speed RAMBUS® interface memory. The Artcam central processor 7604 also includes a printhead 7606 (for printing full-color photos 7607 to provide instant images upon request) in addition to a magnetic recording head 7616 for recording on the back of a photo. The operation is controlled.

  In a further improvement, the camera organization 7601 comprises a sound chip 7610 that is interfaced to the memory 7605 via the RAMBUS bus 7611 under the control of the ACP processor 7604. The sound chip 7610 can be in a standard format or a dedicated format, and can include, for example, a DSP processor that obtains an analog input 7612 from a sound microphone 7613. Alternatively, with the complexity of the chip (Moore's Law), the functionality of the sound chip 7610 can be incorporated into an ACP chip 7604, which preferably comprises a state-of-the-art CMOS integrated circuit chip. It will be readily apparent that many other types of organization falling within the scope of the present invention can be provided.

  The sound chip 7610 converts the analog input 7612 to the corresponding digital format and forwards it for storage in the memory 7605. Such a recording process can be activated by pressing a button (not shown) on the camera device. The button is under the control of the ACP processor 7604 or otherwise can be substantially automated when taking a picture. The recorded data is stored in the memory 7605.

  Referring to FIG. 336, the camera organization preferably includes a printer device 7606 (with a print head used to print an image on print media 7617) and a magnetic recording print head 7616 (information on the back of the print media 7617). Used for printing). Referring to FIG. 337, an output example of a magnetic strip 7618 formed on the back surface of the photo media 7617 is shown. The magnetic band is recorded by the recording head 7616 in FIG. The recorded information includes location, date, and time data, which is provided by keyboard input or by using an attached positioning system such as GPS or the like. Importantly, coded audio information is recorded on the back side of the image 7617. The method of encoding can be many different, but is preferably encoded in a highly fault tolerant manner to allow for errors. The form of sophistication may utilize Reed-Solomon encoding of data to provide a high degree of fault tolerance.

  Referring to FIG. 338, when the recorded sound is to be “reproduced”, the reading device 7626 is used, and the magnetic sensor device 7627 is passed while the photograph 7617 is sandwiched between the pinch rollers.

  Referring to FIG. 339, the operation of the voice reader 7626 of FIG. 338 is shown in schematic form. The magnetic sensor 7627 is connected to the second art cam central processor 7628 so as to read and decode data stored on the back side of the photograph. The decoded audio information is stored in the memory 7632 and reproduced by the speaker 7629 via the sound processing chip 7633. The sound processing chip 7633 is operated under the control of the ACP decoder 7628, and sequentially operated based on various user input controls 7633 such as volume control, rewinding, reproduction, and fast-forwarding.

As can be appreciated from the above description of further improvements, a system for automatic recording of audio related to the output image is provided, and audio recording related to the photo is made. An audio reading system for reading an image recorded on the back side of a photograph is also disclosed. It will be understood by those skilled in the art that numerous changes and / or modifications may be made to the invention as illustrated in the specific embodiments without departing from the spirit or scope of the invention as broadly described. Let's do it. For example, it is used for more complicated audio recording and reproducing technologies such as stereo and B format technologies.
Explanation of Technology 65

  In a further improvement, the art cam apparatus described above is properly programmed to print a “negative” using the printhead. The negative conforms to a standard art card format so that it can be inserted into an art card sensor device for use. The art card sensor device detects the information stored on the “negative” art card and prints a photo.

  Referring first to FIG. 340, a standard art cam device 7701 comprising a sensor device 7702 for detecting images upon request and a card reading sensor 7703 for detecting art card information is shown in schematic form. ing. These sensors 7702, 7703 are operated under the control of an art cam central processor 7704, which is an important calculation means for the operation of the art cam device. The art cam central processor 7704 can include an on-board CPU in addition to a fully vectorized very long instruction word (VLIW) central processor. The ACP 7704 performs processing and decryption of the art card (information) read by the art card reading device 7703 in addition to the processing of the acquired image 7702. The ACP 7704 performs processing with the memory 7705 (interact) in order to form a processed image to be printed by the print head 7706 on the print medium supplied by the internal print roll and to output a photograph 7707 (interact). The art cam system 7701 is designed to build a complete portable camera system and is intended to print a photohead 7706 on demand under the control of an ACP processor 7704. A print roll (built-in and disposable, for supplying print media and ink).

  Referring to FIG. 341, there is shown 7710 operational steps of the art cam system of FIG. 340 according to further improvements. In a further improvement, the desired image is first taken using the first art cam device (7710). The captured image is then processed to provide a highly fault tolerant output data format that is recovered. The process can include compression of Reed-Solomon encoded data in addition to pseudo-random expansion of the data into an array of images, replicated data, and data values. The output format can be the same as provided by an “art card” device as described above, in addition to image compression, if desired. In addition to the output data of the art card, a thumbnail image of the detected image is printed along with the encoded data.

  As shown in FIG. 342, the art card organization includes an art cam device 7701 that acquires an image 7720 by a CCD sensor 7702 and outputs an art cam “negative” 7721. The “negative” format includes a thumbnail 7724 of the input image and a highly encoded 7722 of the detected image. Although not shown in FIG. 342, the output is usually cut along line 7725 by a cutting device in art cam device 7701.

  Data area 7722 is printed with a series of indicia (eg, 7727, 7728) along the side edges of area 7722. Those indicia 7727, 7728 give the exact location of the data area 7722 when scanned. By printing “negative” 7721, a semi-permanent record of the image is obtained. This “negative” 7721 is stored in the same manner as a photographic negative is stored. If a further high quality copy of the acquired image is required, the art card “negative” is inserted into the art card reader 7703 of the art cam device of FIG. 342 (7713 of FIG. 341), The art card data is decrypted by a normal method to obtain an original image. The encoded data can include the original image and instructions necessary to print the image with the art cam device. The detected card is decrypted (7714 in FIG. 341) and printed many times as required (7715) to obtain the original image. That is, the detected image is output. Therefore, a system for copying the detected image with high resolution and high resilience can be obtained.

Other embodiments can also be configured. For example, an organization that automatically prints the “negative” data on the back side of the photographic surface may be used. Such an organization is shown in FIG. What is shown in the figure is the back surface of the photograph 7730, and in addition to the thumbnail portion 7732, a data encoding area 7731 is printed. Accordingly, the data area 7731 can provide a semi-permanent and highly accurate recording of images on the surface of the card 7730.
Explanation of Technology 66

  In a further improvement, an art card printer is provided that can output multiple art cards in response to random output requirements. The printer of the product is connected to the printers of other products via a network, and the integrated computer system controls the network to distribute the art card according to a predetermined request. In this way, the production of art cards is controlled to give high flexibility.

  Referring to FIG. 344, only one art card printer 7801 is shown in schematic form. The printer has two print heads 7802, a print head 7802 for printing an image on the upper surface of the art card and a print head 7803 for printing an image instruction manual (encoded) on the lower surface of the art card. , 7803. The print head 7802 can be a three-color print head having an ink supply unit 7806. The bottom printer 7803 can be a single color print head having one ink supply 7807. The printer 7801 is operated under the control of a computer system to print out an art card in response to a request. After printing, a punching device 7809 is arranged to punch out the art card from the base plate 7810 (a number of art cards arranged for distribution purposes). Distribution according to the requirements of a given area.

  Referring to FIG. 345, a general arrangement of an art card delivery system control network 7820 is shown. A series of printer computer devices (eg, 7821) are arranged and connected together to a network 7822, which can take a variety of forms upon request. Central integrated computer 7823 controls network 7822 and printer computer (eg, 7821) to send instructions to print a combination of art cards by printer computer (eg, 7821). A printer computer (e.g., 7821) can be located locally upon request. Alternatively, the printer computers may be centrally located, but may be sent to different regions or ordered to meet different requirements. Each printer computer can have a large cache of art cards. The cache can be provided in a memory, a disk, or a CD-ROM appropriately distributed to each printer computer. The central integrated computer distributes a list of art cards to be printed by the printer computer. In addition, a new art card is read into the central integrated computer 7832 and sent to the associated printer computer (eg, 7821) upon request. The distribution is in the form of a cached distribution array.

  Referring to FIG. 346, one form of a preferred organization 7830 of a printer computer is shown. The organization 7830 includes a backbone computer system 7831, which is a high-end PC type computer with large memory and disk capacity. The computer system 7831 is connected to an art card printer 7832 arranged according to the organization of FIG. The computer system 7831 includes a large store of art cards 7833 to be cached in addition to the network 7834. Thus, a list of art cards to be printed out is transferred from the central computer to each printer computer for art card printing.

The organization of FIG. 345 overcomes the large combination of complex distribution problems and reveals an improved form of the distribution system.
Inkjet technology

    The embodiment of the present invention uses an ink jet printer type device. Of course, many different devices can be used. However, currently popular inkjet printing technology is unlikely to be suitable.

    The most important problem with thermal inkjet is power consumption. Due to the poor energy efficiency of ink drop ejection, the power required for high speed is about 100 times. This is because rapid boiling of water is required to generate vapor bubbles for ejecting ink. Water has a very high heat capacity and is overheated in the use of thermal ink jets. This requires an efficiency of about 0.02% to convert the electrical input to the motion output.

    The most important issues with piezo electric ink jets are size and cost. Piezoelectric crystals have very little deflection at the appropriate drive voltage, and therefore require a large area for each nozzle. Also, each piezoelectric actuator must be connected to a separate substrate drive circuit. This is not an important problem in limiting the current of about 300 nozzles, but is a major obstacle to the manufacture of page width printheads with 19,200 nozzles.

Ideally, the inkjet technology used requires private digital color printing and other high-quality, high-speed, low-cost print applications. New inkjet technologies have been created to meet the demands of digital photography. The objective features are: : That is,
Low power (less than 10 watts)
High resolution performance (1,600 dpi or higher)
Photo quality output Low manufacturing cost Small size (Adjust page width to minimize cross-cut width)
High speed (<2 seconds per page)

    All of these features can be overcome with different levels of difficulty by the inkjet system described below. Forty-five different inkjet technologies have been developed by successors to give a wide choice for high volume manufacturing. These techniques form isolated applications designated by the applicant, as shown in the tables described below.

    The inkjet design shown here is suitable for a wide range of digital printing systems, from battery-powered single-use digital cameras to desktop network printers and commercial printing systems.

In order to be easily manufactured using standard equipment, the printhead is designed into a monolithic 0.5 micron CMOS chip by a MEMS post-processing method. For color photography applications, the printhead is 100 mm long and has a width that depends on the type of ink jet. The smallest printhead is IJ38, the width is 0.35 mm and has a chip area of 35 mm ² . The print head has 19,200 nozzles and data and control circuitry.

The ink is supplied to the back surface of the print head via an injection-molded plastic ink passage. The molding requires 50 micro features. The features can be formed using lithographically micromachined inserts in standard injection molding tools. Ink flows through a hole formed through the wafer to a nozzle chamber formed in the front of the wafer. The print head is connected to the camera circuit by TAB.

Cross-referenced application

ドロップ・オン・デマンド方式のインクジェット The following table is a guide for recently filed US patent applications. These applications are filed together with this, and are discussed using the references used in the subsequent tables when referring to special cases. These applications are also filed as Australian provisional patent applications (as shown in the preceding table) with corresponding reference numbers.

Drop-on-demand inkjet

    Eleven important features relating to the basic operation of individual inkjet nozzles have been identified. These features are roughly orthogonal and can therefore be solved as an 11-dimensional matrix. Most of the eleven axes of this matrix contain entries developed by the applicant.

The following table forms the axis of an ink jet type 11-dimensional table.
Actuator mechanism (18 types)
Basic operation mode (7 types)
Auxiliary actuator (8 types)
Actuator amplification and improvement method (17 types)
Actuator movement (19 types)
Nozzle replenishment method (4 types)
Method to limit backflow to the inlet (10 types)
Nozzle cleaning method (9 types)
Nozzle plate structure (9 types)
Drop ejection direction (5 types)
Ink type (7 types)

    The complete 11-dimensional table displayed by these axes contains 36.9 billion possible forms for inkjet nozzles. Millions are viable, although not all of them are feasible in the various inkjet technologies. It is clearly impractical to describe all possible forms. Instead, several ink jet types have been examined in detail. These are the nominated IJ01 to IJ45 described above.

    Other ink jet forms can be readily derived from these 45 examples by replacing them with alternative forms along one or more of the 11 axes. Most of IJ01 through IJ45 can incorporate features that are superior to any currently available inkjet technology into inkjet printheads.

    Where there are prior art examples known to the inventor, one or more of these are shown in the example column of the table below. The series IJ01 to IJ45 is also shown in the example column. In some cases, if a printer shares more than one feature, it may appear more than once in a table.

    Suitable applications include: That is, home printers, office network printers, short-term digital printers, business printing systems, fabric printers, pocket printers, Internet www printers, video printers, medical images, large format printers, notebook computer printers, fax machines , Industrial printing systems, photocopiers, photo development stores, etc.

    Information associated with the 11-dimensional matrix described above is shown in the following table.

インクジェットプリンティング

Inkjet printing

インクジェットの生産 A considerable number of new ink jet printers have evolved to replace ink jet technology as the preferred embodiment. Various combinations of inkjet devices can be included in the printer device disclosed as part of the present invention. There is a specification of Australian provisional patents related to these ink jets, specifically indicated by a cross-reference table. That is,

Inkjet production

流体の供給 In addition, the present application may use advanced semiconductor manufacturing techniques in the construction of multiple inkjet printers. Suitable manufacturing techniques are disclosed in the Australian provisional patent specification shown in the following cross-reference table.

Fluid supply

ＭＥＭＳ技術 Furthermore, this embodiment may use an ink delivery system to the inkjet head. A delivery system for supplying ink to a series of inkjet nozzles is disclosed in the Australian provisional patent specification shown in the following cross-reference table.

MEMS technology

ＩＲ技術 Furthermore, the present application may use advanced semiconductor microelectromechanical technology (MEMS technology) when arranging an inkjet printer. Suitable microelectromechanical techniques are disclosed in the Australian provisional patent specification shown in the following cross-reference table.

IR technology

ドットカード技術 In addition, the present application may utilize a disposable camera system as disclosed in the Australian provisional patent specification shown in the following cross-reference table.

Dot card technology

アートカム技術 Further, the present application may use a data distribution system as disclosed in the Australian provisional patent specification shown in the following cross-reference table.

Art cam technology

In addition, the present application may use a camera and data processing technology such as the art cam type device disclosed in the Australian provisional patent specification shown in the following cross-reference table.

  It will be apparent to those skilled in the art that various modifications and variations can be made to the invention shown in the specific embodiments without departing from the scope and spirit of the invention as broadly described. This embodiment is therefore exceptional and should be considered in all respects as non-limiting.

It is explanatory drawing of the art cam apparatus comprised by the preferable Example. FIG. 2 is a schematic block diagram of the main artcam electronic components. It is a schematic block diagram of an art cam central processor. It is a detailed view of a VLIW vector processor. It is detail drawing of a processing unit. It is a detailed view of ALU188. It is a detailed view of an input block. It is a detailed view of an output block. It is detail drawing of a register block. 2 is a detailed view of a crossbar 1. FIG. 3 is a detailed view of a crossbar 2. FIG. It is a detailed view of a read process block. It is a detailed view of a read process block. It is detail drawing of a barrel shifter block. It is a detailed view of an adder / logic block. It is a detailed view of a multiplication block. It is a detailed view of an I / O address generator. It is explanatory drawing of a pixel accumulation | storage format. It is explanatory drawing of a sequential read iterator process. It is explanatory drawing of a box read iterator process. It is explanatory drawing of a box write iterator process. FIG. 10 is an illustration of a vertical strip read / write iterator process. FIG. 10 is an illustration of a vertical strip read / write iterator process. It is explanatory drawing of a sequential production | generation process. It is explanatory drawing of a sequential production | generation process. It is explanatory drawing of a vertical strip production | generation process. It is explanatory drawing of a vertical strip production | generation process. It is explanatory drawing of a pixel data structure. It is explanatory drawing of a pixel processing process. It is a schematic block diagram of a display controller. It is explanatory drawing of CCD image organization. It is explanatory drawing of the accumulation | storage format for logical images. It is explanatory drawing of an internal image memory storage format. It is explanatory drawing of an image pyramid accumulation | storage format. It is explanatory drawing of the time series of the process which samples an art card. It is explanatory drawing of a supersampling process. It is explanatory drawing of the process which reads the rotated art card. Fig. 4 is a flowchart of the steps necessary to decrypt an art card. It is an enlarged view of the left corner of a single art card. A single target for detection. FIG. 6 is an illustration of a method used to detect a target. It is explanatory drawing of the method of calculating the distance between two targets. It is explanatory drawing of the process of a gravity center drift. It is a figure which shows one form of a gravity center look-up table. It is explanatory drawing of a gravity center update process. It is explanatory drawing of the look-up table for delta processes utilized in a preferable Example. It is explanatory drawing of the process which cancels | scrambles an art card. It is an enlarged view of a series of dots. It is explanatory drawing of the data surface of a dot card. It is a schematic explanatory drawing of a single data block. It is explanatory drawing of a single data block. It is the elements on larger scale of the data block of FIG. It is the elements on larger scale of the data block of FIG. It is explanatory drawing of a single target structure. It is explanatory drawing of the target structure of a data block. It is explanatory drawing of the positional relationship of the target with respect to the area | region for border clocks of a data area. It is explanatory drawing of the orientation column of a data block. It is explanatory drawing of the array of the dot of a data block. It is a schematic explanatory drawing of the structure of the data of Reed-Solomon encoding. FIG. 6 is an illustration of an exemplary Reed-Solomon encoding process. It is explanatory drawing of a Reed-Solomon encoding process. It is explanatory drawing of the layout of the encoded data in a data block. It is explanatory drawing of the sampling process at the time of sampling an alternative art card | curd. FIG. 6 is an enlarged view of an example of sampling a rotated alternative art card. It is explanatory drawing of a scanning process. It is explanatory drawing of the scanning distribution which can occur a scanning process. FIG. 6 is a diagram illustrating the relationship between the probability of symbol error and Reed-Solomon block error. 3 is a flowchart of a decryption process. It is a process operation | movement figure of a decoding process. It is explanatory drawing of the data flow step in decoding. It is a detailed view of the read process. FIG. 5 is a detailed view of a process for detecting the start of an alternative art card. It is a detailed view of a bit data extraction process. It is explanatory drawing of the segmentation process utilized by a decoding process. FIG. 4 is a detailed view of a decoding process for finding a target. It is explanatory drawing of the data structure utilized when finding a target. It is explanatory drawing of Lancos3 function structure. It is the elements on larger scale of the data block which shows a clock mark and a border area | region. It is explanatory drawing of the processing step at the time of decoding a bit image. It is explanatory drawing of the data flow step at the time of decoding a bit image. FIG. 6 is an illustration of the descrambling process of the preferred embodiment. It is explanatory drawing of one Embodiment of a convolver. It is explanatory drawing of a convolution process. It is explanatory drawing of a composite process. It is a detailed view of a normal composite process. It is explanatory drawing of the warp process which uses a warp map. It is explanatory drawing of a warp bilinear interpolation process. It is explanatory drawing of the process of span calculation. It is explanatory drawing of a basic span calculation process. It is explanatory drawing of one detailed embodiment of a span calculation process. It is explanatory drawing of the process which reads an image pyramid level. It is explanatory drawing of the method of using a pyramid table for bilinear interpolation. It is explanatory drawing of a histogram collection process. It is explanatory drawing of a color conversion process. It is explanatory drawing of a color conversion process. FIG. 4 is a detailed view of a color space conversion process. It is explanatory drawing of the process which calculates an input coordinate. It is explanatory drawing of the process of performing a composite process with a feedback. It is explanatory drawing of a general-purpose scaling process. It is explanatory drawing of the scale in X scaling process. It is explanatory drawing of the scale in a Y scaling process. It is explanatory drawing of a mosaic process. It is explanatory drawing of a subpixel movement process. It is explanatory drawing of a composite process. It is explanatory drawing of the process of compositing with a feedback. It is explanatory drawing of the process of tiling with the color from an input image. It is explanatory drawing of the process of tiling with a feedback. It is explanatory drawing of the process of tiled by texture replacement. It is explanatory drawing of the process of tiling with the color from an input image. It is explanatory drawing of the process of tiling with the color from an input image. It is explanatory drawing of the process which provides a texture without feedback. It is explanatory drawing of the process which provides a texture with a feedback. It is explanatory drawing of the process of rotation of a CCD pixel. It is explanatory drawing of the process of interpolation of a green subpixel. It is explanatory drawing of the process of the blue subpixel interpolation. It is explanatory drawing of the process of interpolation of a red subpixel. It is explanatory drawing of the process of CCD pixel interpolation of 0 degree rotation with respect to an odd pixel row. It is explanatory drawing of the process of CCD pixel interpolation of 0 degree rotation with respect to an even-numbered pixel row. It is explanatory drawing of the process of the color conversion to Lab color space. It is explanatory drawing of the process of calculation of 1 / √X. It is detailed explanatory drawing of the Example of calculation of 1 / √X. It is explanatory drawing of the process of normal calculation by a bump map. It is explanatory drawing of the process of the illumination intensity calculation by a bump map. It is detailed explanatory drawing of the process of the illumination intensity calculation by a bump map. It is explanatory drawing of the process of the calculation of L which uses a directional light. It is explanatory drawing of the process of calculation of L using an omni light and a spotlight. FIG. 6 is an illustration of an embodiment of calculating L using omni lights and spotlights. N. It is explanatory drawing of the process which calculates L inner product. N. It is detailed explanatory drawing of the process which calculates L inner product. R. It is explanatory drawing of the process which calculates V inner product. R. It is detailed explanatory drawing of the process which calculates V inner product. It is explanatory drawing of the input and output of attenuation | damping calculation. It is explanatory drawing of actual embodiment of attenuation calculation. It is explanatory drawing of the graph of a cone factor. It is explanatory drawing of the process of penumbra calculation. It is explanatory drawing of the corner utilized for penumbra calculation. It is explanatory drawing of the input and output to penumbra calculation. It is explanatory drawing of actual embodiment of penumbra calculation. It is explanatory drawing of the input and output to ambient light calculation. It is explanatory drawing of actual embodiment of ambient light calculation. It is explanatory drawing of actual embodiment of scattered light calculation. It is explanatory drawing of the input and output to scattered light calculation. It is explanatory drawing of actual embodiment of scattered light calculation. It is explanatory drawing of the input and output to reflection calculation. It is explanatory drawing of actual embodiment of reflection calculation. It is explanatory drawing of the input and output to reflection calculation. It is explanatory drawing of actual embodiment of reflection calculation. It is explanatory drawing of actual embodiment of the German calculation only of ambient light. It is explanatory drawing of the process outline | summary of optical calculation. It is explanatory drawing of an example of the illumination calculation for single infinite light sources. It is explanatory drawing of an example of the illumination calculation for omni light sources which does not use a bump map. It is explanatory drawing of an example of the illumination calculation for omni light sources using a bump map. It is explanatory drawing of an example of the illumination calculation for spotlight light sources which do not use a bump map. FIG. 6 is an explanatory diagram of a process of applying a single spotlight to an image using an associated bump map. It is an explanatory diagram of a logical layout of a single print head. It is explanatory drawing of the structure of a print head interface. It is explanatory drawing of the process of rotation of a Lab image. It is explanatory drawing of the format of the pixel of a printing image. It is explanatory drawing of a dithering process. It is explanatory drawing of the process which produces | generates 8-bit dot output. It is a perspective view of a card reader. It is a development perspective view of a card reader. It is a close-up figure of an art card reader. It is a perspective view of a print roll and a print head. It is a 1st expansion | deployment perspective view of a print roll. It is a 2nd expansion perspective view of a print roll. It is explanatory drawing of a print roll authentication chip. It is an enlarged view of a print roll authentication chip. It is explanatory drawing of a one-step authentication chip | tip data protocol. It is explanatory drawing of a two-step authentication chip | tip data protocol. It is explanatory drawing of a 1st presence only protocol. It is explanatory drawing of a 2nd presence only protocol. It is explanatory drawing of a 3rd data protocol. It is explanatory drawing of a 4th data protocol. It is a schematic block diagram of the longest period LFSR. It is a schematic block diagram of a clock limiting filter. It is a schematic block diagram of a tamper detection line. It is a schematic block diagram of an oversized nMOS transistor. It is a figure explaining taking-out of many XOR from a tamper detection line. It is explanatory drawing of the state which a tamper line covers a noise generation circuit. It is explanatory drawing of embodiment of normal FET. FIG. 4 is an illustration of a modified FET embodiment of the preferred example. It is an abbreviated block of an authentication chip. It is explanatory drawing of an example of a memory map. It is explanatory drawing of an example of a constant memory map. It is explanatory drawing of an example of RAM memory map. It is explanatory drawing of an example of a flash memory variable memory map. It is explanatory drawing of an example of a flash memory program memory map. FIG. 4 is a diagram illustrating data flow and relationships between components of a state machine. It is a figure which shows the data flow and relationship between the components of an I / O unit. It is a schematic block diagram of an arithmetic logic unit (ALU). It is a schematic block diagram of an RPL unit. It is a schematic block diagram of the ROR block of ALU. It is a block diagram of a program counter unit. It is a schematic block diagram of a memory unit. It is a schematic block diagram of an address generation unit. It is a schematic block diagram of a JSIGEN unit. It is a schematic block diagram of a JSRGEN unit. It is a schematic block diagram of a DBRGEN unit. It is a schematic block diagram of a LDKGEN unit. It is a schematic block diagram of a RPLGEN unit. It is a schematic block diagram of a VARGEN unit. It is a schematic block diagram of a CLRGEN unit. It is a schematic block diagram of a BITGEN unit. It is a figure which shows the information memorize | stored in the print roll authentication chip. It is explanatory drawing of the data memorize | stored in the art cam authentication chip | tip. FIG. 6 is an illustration of a printhead pulse characterization process. FIG. 4 is an enlarged cross-sectional perspective view of a print head ink supply mechanism. It is a bottom perspective view of the ink head supply unit. It is sectional drawing by the side of the bottom face of an ink head supply unit. It is a top perspective view of an ink head supply unit. It is sectional drawing of the upper surface side of an ink head supply unit. 2 is a perspective view of a portion of a print head. FIG. FIG. 3 is an enlarged perspective view of a print head unit. FIG. 3 is a perspective view of the top side of the interior of the art cam camera, showing the flattened parts. It is a perspective view of the bottom side inside the art cam camera, showing the flattened parts. FIG. 2 is a perspective view of a first upper surface side inside the art cam camera, and shows components housed in the art cam. FIG. 6 is a perspective view of a second upper surface side inside the art cam camera, and shows components housed in the art cam. FIG. 6 is a perspective view of a second upper surface side inside the art cam camera, and shows components housed in the art cam. It is explanatory drawing of the endorsement part of a postcard print roll. It is explanatory drawing of the front image corresponding to the postcard print roll after image printing. It is explanatory drawing of the form of the print roll of the state which a customer can purchase. It is explanatory drawing of the layout of the software / hardware module of the whole art cam application. It is explanatory drawing of the layout of the software / hardware module of a camera manager. It is explanatory drawing of the layout of the software / hardware module of an image processing manager. FIG. 6 is an explanatory diagram of a layout of a software / hardware module of a printer manager. It is explanatory drawing of the layout of the software / hardware module of an image processing manager. It is explanatory drawing of the layout of the software / hardware module of a file manager. FIG. 6 is a partial cross-sectional perspective view of an alternative form of print roll. 227 is an enlarged perspective view of the left side of the print roll of FIG. It is an enlarged perspective view of the right side of a single print roll. It is a partial section expansion perspective view of the core part of a print roll. It is a 2nd expansion perspective view of the core part of a print roll. It is a front view of 'Bizcard'. 1 schematically shows a camera system configured according to a further improvement. 1 schematically shows a printer device for printing on the front and back surfaces of an output medium. The format of the output data on the back of the photograph is shown. Indicates an enlarged portion of the output media. Fig. 2 shows a reader used to read data from the back of a photograph. The use of a further improved device is shown. A schematic example of specific processing of image orientation is shown. Fig. 4 shows how to create a unique effect of an image when using a camera configured according to further improvements. Fig. 4 shows a print roll according to a further improvement. A further improved example method is shown. The operation method of the further improvement is shown. Fig. 4 shows a form of image processing according to a further improvement. The operation method of the further improvement is shown. Fig. 4 shows the process of image capture and output. The red-eye removal process is shown. Figure 2 shows a photo print organization configured according to a standard art cam device. Fig. 4 shows a dual printhead knitting constructed according to a further improvement. Figure 2 shows a further improved decurler. Fig. 3 shows a process of viewing a stereoscopic image. Fig. 4 shows a further improved lens system designed to create a stereographic effect. It is a fragmentary perspective view which shows creation of a three-dimensional photographic image. Fig. 2 shows an apparatus for producing a stereographic image according to a further improvement. FIG. 254 shows the positioning unit of FIG. 254 in more detail. 1 shows a camera device suitable for creating a stereoscopic photo image. Fig. 4 illustrates a process using an opaque backing to improve the stereoscopic effect. 1 schematically illustrates a method for generating an image on a print medium. 1 illustrates the structure of a print media constructed in accordance with the present invention. Fig. 4 illustrates the use of a print media configured according to further improvements. 1 shows a first form of a print media structure according to a further improvement; Fig. 4 shows a further form of the structure of the print media according to the present invention. Fig. 4 shows a schematic cross-sectional view of a further form of structure of the print media according to the present invention. Fig. 4 shows a schematic cross-sectional view of a further form of structure of the print media according to the present invention. FIG. 26 shows one form of manufacturing a print media configuration according to FIGS. 264 shows another form of manufacturing by extruding fiber material to utilize the knitting of FIG. The main steps for further improvement are shown. The effect of the Sobel filter used in further refinement is shown. Fig. 4 shows a curve offset method used in further refinement. Fig. 4 shows a curve offset method used in further refinement. A standard image including a recognized face is shown. 1 shows a first flow chart for determining a region of interest. 2 shows a second flowchart for determining a region of interest. The first step of tiling the image is shown. Fig. 5 shows another tile pattern for image tiling. Shows the opaque mask of the tile. 276 shows the corresponding surface texture height for the tile of FIG. 276 shows the corresponding surface texture height for the tile of FIG. Fig. 4 illustrates the process of using globalopacigty to modify an image. Shows how to modify the stroking effect to mimic the effect of mixing actual paint. A process of creating a brush stroke table will be described. The process of creating a stroke color palette is shown. The resulting painting process is shown. Fig. 5 shows a flow chart of the steps in a further embodiment for composing a brush stroke. A method for converting a Bezier curve to a piecewise line segment is shown. The brush stroke process is shown. Figure 3 shows a suitable brush stamp for use in further examples. 1 shows a first organization of further improvements. The flowchart of the operation | movement of the further improvement is shown. The structure of the image printed by the high resolution is shown. Fig. 2 shows an apparatus for processing the image of Fig. 290 for making a copy. The processing steps of centroid are shown. A series of detected patterns is shown. A diagram of the implementation of further improvements is shown. Further improvements are shown. Fig. 4 shows a card reader of a further embodiment. Shows the brush / bump map. Shows the brush / bump map. Shows the background canvas bump map. Shows the background canvas bump map. Fig. 4 illustrates the process of combining bump maps. Fig. 4 illustrates the process of combining bump maps. Indicates the map background. Fig. 4 illustrates a process for combining bump maps according to further improvements. Fig. 4 illustrates a process for combining bump maps according to further improvements. Fig. 4 shows steps in a further refinement method of artistic image generation. Fig. 4 illustrates the process of mapping one gamut to a second gamut. 1 shows an embodiment of a further improvement. A preferred form of gamut remapping is shown. Shows gamut morphing used in further improvements. The basic operation of the art cam device is shown. Shows a series of art cards used in further refinement. FIG. 6 is a flow chart diagram of an algorithm used in further refinement. FIG. 2 is a schematic diagram of printed fabric output made in accordance with the present invention. A further improved connection method is shown. Figure 3 shows a perspective view of a further improvement, partly in section. Fig. 4 shows a card having a write data area array. A card having only a limited number of write data areas is shown. The structure of the data area is shown. The target configuration is shown. A further improved device is shown. The state which looked at FIG. 321 closely is shown. Fig. 4 shows the process of inserting a card into the reader. The process of removing a card is shown. Demonstrate the process of writing the data area on the card. It is the schematic of the structure of a card reader. A schematic arrangement of further improvements is shown. An example of an interface for further improvement is shown. Fig. 3 shows one form of the configuration of a software module for further improvement. It is the schematic of operation of an art cam system. A first example of a modified operation of the art cam system is shown. A repetition card that changes the operation of the art cam is shown. An art card test card for changing the operation of the art cam is shown. The output test result of the art cam device is shown. 1 schematically shows a camera system configured according to a further improvement. Fig. 2 schematically shows a further improved printer device and recording device. The structure of the magnetic strip on the back of the photo is shown. Fig. 2 shows a reading device used for reading data recorded on the back side of a photograph. The use of a further improved device is shown. The schematic of the functional part of an art cam apparatus is shown. The steps of using further improvements are shown. Fig. 4 shows the operation of the art cam device according to a further improvement. Another embodiment printed on the back side of the outputted “photo” is shown. Fig. 3 shows the interior of a printer device configured according to a further embodiment. Fig. 3 shows a network distribution system configured according to a further embodiment. The operation of the printer computer of FIG. 345 is schematically shown.

Claims (4)

An area image sensor adapted to detect images;
A memory storage device adapted to store the detected image;
An orientation sensor adapted to detect the orientation of the camera with respect to gravity;
Adapted to process the detected image using the detected camera orientation by applying an orientation dependent conversion to the detected image to form a converted image A processor;
A built-in printer for printing the converted image on a print medium;
With
The orientation-dependent transformation is
(I) using the sensed camera orientation to detect at least one face in the sensed image and relate to the detected at least one face to form the transformed image; And changing the detected image,
(Ii) selected from the group of providing text information in the detected image to form the converted image in an orientation that harmonizes with the orientation of the detected camera;
Camera device. The orientation dependent transformation includes transforming features in the sensed image to form the transformed image.
The camera device according to claim 1. An area image sensor adapted to detect images;
A memory storage device adapted to store the detected image;
An orientation sensor adapted to detect the orientation of the camera with respect to gravity;
A processor adapted to process the detected image to form a transformed image;
A built-in printer adapted to print the converted image;
With
(A) detecting the orientation of the camera with respect to gravity by the orientation sensor; (b) applying a direction-dependent transformation to the detected image to form the transformed image by the processor. Using the detected camera orientation to process the detected image (c) printing the converted image with the built-in printer, the orientation-dependent conversion comprising:
(I) using the sensed camera orientation to detect at least one face in the sensed image and relate to the detected at least one face to form the transformed image; And changing the detected image,
(Ii) selected from the group of providing text information in the detected image to form the converted image in an orientation that harmonizes with the orientation of the detected camera;
A method of processing images captured by a digital camera device. Further, the orientation dependency conversion is
(Iii) distorting features in the detected image to form the converted image;
4. A method according to claim 3, comprising (iv) either or both of arranging information relating to features of animals or human faces appearing in the image to form the transformed image. .

Images