Failure loading soft font from printer hard disk into printer ram, for use within PCL print job

[Hugh_Hood]

=========================================
USING FLASH AND HARD DRIVE PJL RESIDENT TRUETYPE FONTS

                              ON HP PCL 5 PRINTERS
=========================================
                           by Hugh Hood
                  (revision 1.2 - September, 2018)

       [heartfelt thanks to Chris Hutchinson (aka 'DansDadUK')
          for his encouragement and guidance in helping me
            discover and share this useful information]
    

------    
Intro
------
This short note describes the procedure for (i) converting, (ii) installing and (iii) using PJL Resident scalable TrueType fonts on HP PCL 5 Printers that support and use a properly initialized PJL File System Architecture (FSA) present on either a Flash SIMM, Flash DIMM, EIO Hard Drive or on other supported media attached to the printer.

By enabling the user to store additional TrueType fonts on the printer's internal file system, and then allowing those additional fonts to be selected by attribute 'Escape Sequence' [e.g. - <esc>(10U<esc>(s1p12v0s0b4101T for a plain, solid 12 point CG Times typeface with the PC-8 symbol set], functionality, efficiency and flexibility in text-based applications development is greatly improved.


--------------
Background
--------------
In addition to the fonts included in the firmware of PCL 5 printers as shipped (called 'Internal Fonts'), or fonts downloaded as part of a PCL print job (called PCL 5 'Soft Fonts'), many PCL 5 printers also support file system based user-supplied additional fonts (called 'PJL Resident Fonts'), in both scalable TrueType as well as fixed-size PCL 5 bit-mapped forms.

This note concentrates on using the former, rather than the latter, type of file system font, although the general procedure is the same for both.

First and foremost, it is important to make this distinction --  'PJL Resident Fonts' are NOT in the same format as downloaded 'PCL 5 Soft Fonts'. Moreover, 'PJL Resident Fonts' reside on the PJL file system storage device of the printer (e.g. - flash drive or hard drive) and remain there for use at any time, even across power-on/power-off cycles.

'PCL 5 Soft Fonts' must be downloaded to the printer's RAM prior to use, and are lost whenever the printer's power is turned off, even if made 'permanent'.

'PCL 5 Soft Fonts' also have a unique soft font format that is different from that of 'PJL Resident Fonts', even though both types of fonts may be created from the same donor TrueType font.

While the format for 'PCL 5 Soft Fonts' is detailed in the 'PCL 5 Technical Reference Manual', the special format required for 'PJL Resident Fonts' is not publicly documented.

Fortunately, HP's own 'Web JetAdmin 10' software, which is a free download, performs among its many functions the task of converting a standard TrueType font (.ttf) into the format required to be a valid 'PJL Resident Font'. This capability is found in the 'Device Management' navigation pane in the 'Storage/Repository/Import Fonts and Macros' function, where TrueType fonts are imported into the 'Storage Repository'.

After an imported and converted TrueType file is in the Storage Repository, Web JetAdmin 10 can either 'Install' the font in the printer's PJL File System (for storage in the "0:\fonts\" directory) or can 'Save to File' the font (as part of a complete PJL Job file) to the host computer's file system. That saved file can then later be sent to the printer via several methods, including over the same channel that normal print jobs are sent, provided the method used permits binary file transfers, since font data will invariably include both high ASCII and also control characters.

In addition to the converted font, also either 'Installed' or 'Saved' is a small bit of information that will be appended to a very important file called the 'pclResourceFile.'  

The purpose of the pclResourceFile is to associate an installed font with an attribute 'Escape Sequence', enabling the font to be selectable by attribute, just as the printer's 'Internal Fonts' are so selectable.


------------------------------------------------
Selecting by Attribute - the pclResourceFile
------------------------------------------------
Unlike PCL 5 Soft Fonts, which contain a font header containing PCL-specific information, PJL Resident Fonts (e.g. - a TrueType font converted by Web JetAdmin 10 and then installed) do not internally contain sufficient PCL-specific information that can be used by the printer to generate the attribute 'Escape Sequence' (e.g. - <esc>(s1p12v0s0b8775T) by which they may be selected and used from within a PCL 5 print job. Accordingly, some index or 'key' external to the font file itself must be provided in order to associate a particular installed PJL Resident font with a specific 'Escape Sequence'. That is where the pclResourceFile comes in.

The pclResourceFile is stored in the printer's "0:\pcl\macros\" directory, and associates font and macro files with particular ID #s in the case of macros and fonts, and with particular PCL-specific attributes in the case of fonts. This association is made whenever the printer is powered 'on' from an 'off' condition, and also whenever the '<esc>&n16w4pclResourceFile' association file command is sent to the printer. By having this PCL-specific attribute information, the printer can generate a proper 'Escape Sequence' that can be used to select a particular font.


-----------------------------------------------
Web JetAdmin 10 - pclResourceFile Errors
-----------------------------------------------
Web JetAdmin 10, when constructing the information to be appended to the pclResourceFile after importing and converting a TrueType font, attempts to determine the PCL-specific attributes needed in order for the printer to generate a proper 'Escape Sequence' for the particular font. Unfortunately, unless the donor TrueType font contains the optional PCLT table (which contains much the same information as contained in a PCL 5 Soft Font header table), and most TrueType fonts do NOT contain this, Web JetAdmin 10 has no way of determining the proper font attributes and constructing an accurate font attribute entry to be placed in the pclResourceFile, and thus makes a 'guess'. Unfortunately, that guess appears to be very much off the mark.

For example, let's say we want to make the Bitstream TrueType font 'Cheltenham' a PJL Resident Font to be installed and used by the printer, and have it available for selection by its attribute 'Escape Sequence'. Furthermore, let's assume that no PCLT table is present in the donor TrueType font.

After importing and converting the donor Cheltenham TrueType font to the PJL Resident Font format needed for PCL 5, Web JetAdmin 10 then produces the following entry to be appended to the pclResourceFile:

02 301 EBD04F539A2F32 Cheltenham


Note that there are four (4) parts to this entry -- the Descriptor Field, the ID # Field, the Attributes Field and the Filename Field.

In the Descriptor Field, the '02' denotes that this is a font descriptor entry, as opposed to '01' which is a macro descriptor entry, or a '00' which is just a comment entry. This part, namely the '02', is correct.

In the ID # Field, the '301' denotes the pre-specified font ID # that can be used to select the font by ID #. This also is correct, as it was requested by the user from within Web JetAdmin 10.

In the Filename Field, 'Cheltenham' is the actual name of the font file to be installed  (after conversion by Web Jet Admin 10) in the "0:\fonts\" directory. This is the correct filename.


---------------------------
Now for the problem ...
---------------------------
In the Attributes Field, the 'EBD04F539A2F32', containing (7) hex bytes, is the heart of the entry constructed by Web JetAdmin 10. This entry actually contains four (4) different Attributes sub-fields, all of them expressed in hexadecimal bytes, and each of which contains one (1) of four (4) attribute specifiers for the font -- Symbol Set, Style, Stroke Weight and Typeface, in that order.

This is where Web JetAdmin 10 makes its mistakes, directly in its construction of the pclResourceFile entry, and indirectly in the attribute 'Escape Sequence' generated by the printer when a font listing is printed.

Most importantly, even if one attempts to use the erroneous attribute 'Escape Sequence' provided in a font listing printout to select the installed PJL Resident Cheltenham font, the font selection will fail, and the desired PJL Resident font will not be selected.


Examining the Web JetAdmin 10 constructed Attributes Field entry in more detail:

Sub-Field 1 containing (2) hex bytes  - Symbol Set = 0xEBD0 / decimal 60368 / 1886P [IN ERROR]

Sub-Field 2 containing (2) hex bytes  - Style = 0x4F53 / decimal 20307 [IN ERROR]

Sub-Field 3 containing (1) hex byte  - Stroke Weight = 0x9A / twos compliment hex in decimal -102 [IN ERROR]
(Note: negative Stroke Weight attribute values are expressed in twos compliment hexadecimal, which is obtained by inverting all the binary bits of the absolute value and then adding '1' - see the 'Stroke Weight' chart below.)

Sub-Field 4 containing (2) hex bytes  - Typeface = 0x2F32 / decimal 12082 [IN ERROR]


After the pclResourceFile (residing in the "0:\pcl\macros\" directory) is updated with that entry 'as is' and then re-associated, printing a font listing from the LJ control panel shows the following as the 'Escape Sequence' to use in order to select Cheltenham by attribute:

<esc>(1886P<esc>(s1p[ ]v20307s-102b12082T


Unfortunately, this is an invalid and nonfunctional entry. Let's also examine it in more detail:


Symbol Set
-------------
Obviously, the Symbol Set '1886P' is out-of-range (64Z/2073 is maximum permitted value per HP) and invalid. Since imported TrueType fonts are generally not bound to an HP PCL 5 Symbol Set of any sort, the 'Escape Sequence' shown in the font listing printout should not include any Symbol Set descriptor at all, and should appear blank.

[Very Important Note: When specifying an unbound TrueType font in the 'Attribute Fields' of the pclResourceFile entry, use the accepted method of denoting a font as being unbound to a Symbol Set, which is to show it as having a Symbol Set of '1X'. This is expressed as (2) hex bytes [e.g. 0038] in Sub-Field 1 of the Attributes Field in the pclResourceFile entry (Symbol Set = 0x0038 / decimal 56 / 1X)]  

{Side Note - the numerical value for a Symbol Set ID is determined by multiplying the first place digit by 32, and then adding to it the ordinal value (within the English alphabet) of the second place letter. In the case of Symbol Set ID of '1X', this is: (1 x 32) + 24 = 56}


Style
-------
Likewise the Style '20307' is also out-of-range (1023 is maximum permitted value per HP) and invalid. The Style attribute sub-field encompasses three (3) sub-attributes of its own -- Structure, Width and Posture.

Structure includes values such as 'Solid', 'Outline', 'Shadow', 'Inverse' and others.

Width includes values such as 'Normal', 'Condensed', 'Expanded' and others.

Posture includes the values 'Upright', 'Oblique/Italic' and 'Alternate Italic'.

Without detailing all the possible values for these sub-attribute combinations, know the following values for these common Styles:

Solid/Normal/Upright = 0x0000 / decimal 0
Solid/Normal/Italic = 0x0001 / decimal 1
Solid/Condensed/Upright = 0x0004 / decimal 4
Solid/Condensed/Italic = 0x0005 / decimal 5


Stroke Weight
----------------
Next, the Stroke Weight '-102' is also out-of-range (-7 to 7 per HP) and invalid.

Stroke Weights conform to the following table:


Ultra Thin  = 0xF9 / decimal -7
Extra Thin  = 0xFA / decimal -6
Thin        = 0xFB / decimal -5
Extra Light = 0xFC / decimal -4
Light       = 0xFD / decimal -3
Demilight   = 0xFE / decimal -2
Semilight   = 0xFF / decimal -1
Book/Text   = 0x00 / decimal  0
SemiBold    = 0x01 / decimal  1
DemiBold    = 0x02 / decimal  2
Bold        = 0x03 / decimal  3
Extra Bold  = 0x04 / decimal  4
Black       = 0x05 / decimal  5
Extra Black = 0x06 / decimal  6
Ultra Black = 0x07 / decimal  7


Typeface
-----------
Finally, the Typeface '12082' is incorrect for the Bitstream Cheltenham font, according to Tables C-2 and C-3 in HP's PCL 5 Comparison Guide.

The correct Typeface value is the sum of two (2) numbers, the Typeface Vendor Value (from Table C-2), and the Typeface Base Value (from Table C-3)

Table C-2 Lists the following for Typeface Vendor Values:

AGFA = 4096 (1 x 4096)
Bitstream Inc. = 8192 (2 x 4096)
Linotype Company = 12288 (3 x 4096)
Monotype Corporation = 16384 (4 x 4096)
Adobe Systems = 20480 (5 x 4096)
Bigelow & Holmes = 28672 (7 x 4096)


Table C-3 Lists over 900 Typeface Base Values, and specifically lists the 'Cheltenham' font of having a base value of '583'.

So, the Bitstream Cheltenham Font should have a Typeface number of '8775' (hexadecimal 0x2247), obtained by adding the Vendor Value '8192' and the Base Value '583'.

As I have been unable to determine whether HP ever updated tables C-2 and C-3 to support fonts from unlisted font vendors and/or those with unlisted font names, I suggest you 'make up' your own Typeface number for those fonts, but start with Typeface Vendor Values of at least 32768 (8 x 4096), but note that the maximum Typeface Vendor Value is 61440 (15 x 4096) per HP.


---------------
Proper Entry
---------------
Given all that, here is how the 'Attribute Field' in the pclResourceFile SHOULD appear for our sample Bitstream Cheltenham font:

02 301 00380000002247 Cheltenham


And the 'Escape Sequence' in the LJ control panel font listing printout, SHOULD appear as:

<esc>([ ]<esc>(s1p[ ]v0s0b8775T


-----------------------
Suggested Solution
-----------------------
Barring an update to Web JetAdmin 10 adding the improvements I mention below in 'Final Note', here is how I suggest adding and successfully using new TrueType fonts installed on a PCL 5 capable printer containing either Flash or Hard Drive storage:

1. Use HP's Web Jet Admin 10 to import your TrueType font into its 'Repository'
    [Device Management/Storage/Repository/Import];

2. After Web Jet Admin 10 imports (and converts) the font, save a PJL job file (.pjl) containing the converted font and the constructed pclResourceFile entry to your host computer's (not printer's) hard drive.
    [Device Management / Storage / Repository / Save to File / Select from List / {select your font} / Next / Save];
 
3. Load the PJL job file you just saved into a hex editor program. There are many such programs available, including free offerings, such as wxHexEditor    <https://www.wxhexeditor.org/>
[Note - as a PJL job file will contain binary data, most text editors will be confused by some of the characters, and some may mistake the type of line end conversion required, which will result in a corruption of data when the file is saved.]

4. Within the hex editor, locate, toward the end of the PJL job file, the entry for the pclResourceFile.
[e.g. - 02 301 EBD04F539A2F32 Cheltenham]
There will be other types of information in this area that you can safely ignore. You are looking for the specific line in this specific format: 02 <space> <font id> <space> <14-character attributes field> <space> <font name>

5. Place the cursor (using the ASCII entry side of the editor, not the HEX entry side) and replace the errant characters in the Attributes Field with the correct characters, in hexadecimal.

If you are unsure of the correct values for your particular font, and are not a purist, you can use:

003800000084D0

which will specify (i) an unbound font, (ii) a solid/normal/upright font, (iii) a Book/Text weight font, and (iv) a Typeface number of '34000', which is '84D0' in hex.

6. After those characters have been replaced (and none other) save the file.

7. Now, the modified PJL job file can be sent to your printer in any of several ways, including by sending over a binary enabled 'raw' channel to your printer. Also, Chris Hutchinson's 'PCL Paraphernalia' has the 'PRN File Print' tool that can send the PJL job file to the printer.   <http://www.pclparaphernalia.eu/>

8. After these steps have been completed, printing a font listing from the LJ control panel will show whether you had success, or failure.

In case of the latter, please feel free to post your questions to this Forum.

------------
Final Note
------------
If any developers of Web JetAdmin 10 happen upon this note, I would request that you add the capability, when preparing an imported TrueType font in the repository to be installed on the printer (or saved to a file),  to allow the user to specify (i) the 'style' attributes, (ii) the stroke weight attribute, and (iii) the typeface attribute that will be used in the construction of the entry for the pclResourceFile.  I would also suggest that the symbol set attribute default to be unbound (e.g. - 1X / 56 / 0x0038), as imported TrueType fonts can contain many glyphs spanning several HP Symbol Sets, and there is no need to limit the possible glyph selection. In any case, a user requiring that a certain HP Symbol Set be respected when sending single-byte 8-bit data to the printer can always specify the use of that particular Symbol Set in the PCL stream sent to the printer as part of a print job. (e.g. - <esc>(10U to specify the PC-8 Symbol Set)

Since 'Edit Properties for Storage' in the 'Repository' area on the 'Storage' tab of Web JetAdmin 10 already allows the user to specify the desired font id to be inserted into the pclResourceFile, allowing the user to specify these three (3) other elements should not take much additional coding, particularly when one considers that Web JetAdmin 10, at over half of a gigabyte in size, is already a huge program, and this would be a very minor addition.