[NM] Basic Technical Matters

Librarian: HexCode
Post Reply
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

[NM] Basic Technical Matters

Post by HexCode »

CONTENT LINKS
==============

Introduction
viewtopic.php?f=100&t=535#p8923

Filename Neatness
viewtopic.php?f=100&t=535#p8924

Tab-Separated Unicode Text Files
viewtopic.php?f=100&t=535#p8925

Character Set Evolution
viewtopic.php?f=100&t=535#p11090

Don't Overwrite, Just Append !
viewtopic.php?f=100&t=535#p8926

The AI Never Does...
viewtopic.php?f=100&t=535#p8927

Unit Icon Orientation
viewtopic.php?f=100&t=535#p8928


===================================================================

The topic's contents may be modified or progressively added upon as time goes by.

===================================================================

INTRODUCTION
==============

This topic should be of interest to Novice Modders (NMs). To boot, some Veteran Modders (VMs) may benefit from the topic's contents as well. In any case, it is assumed that readers are already intimately familiar with the information featured here:

[NP] Introductory Documentation
viewtopic.php?f=100&t=531

The term User Interface (UI) may be more accurately recast as Player Interface (PI). PGF's PI constitutes what normally passes for a software's UI; namely, whatever is, well, readily visible to a Novice or Veteran Player and which directly supports play. PGF's PI combines and visually integrates many Textual and Pictorial Display elements. Players take obvious advantage of PGF's PI.

Novice Modders (NMs) are reasonably assisted by PGF's Novice Modder Interface (NMI). In addition to PGF's PI, PGF's NMI comprises certain important aspects invisible to a Player. These aspects, when sufficiently mastered, allow NMs to do their "modding thing".

More often than not, NMs exclusively depend on support utilities such as Fred's Panzer General Editor (FPGE). Depending on how lucky or persevering a NM is, such utilities may (or not) do his requisite "trick(s)".

As a rule, NMs focus on:

a) Content thematically revolving around WWII conflicts between the Axis and the Allies.

b) Editing external support files sporting plain-text formats.

From an intended play quality perspective, it is very important for NMs to clearly indicate whether their custom content should be viewed as playable:

1) In single- or dual- purpose fashion. Single-purpose play should either be H2H or H2AI / AI2H but not both; dual-purpose play can entail both.

2) Against PGF's AI module and, if so, whether in single- or dual-sided fashion. Single-sided play should either be H2AI or AI2H but not both; dual-sided play can entail both.
Last edited by HexCode on 2021-09-16 14:56, Thursday, edited 10 times in total.
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

FILENAME NEATNESS

Post by HexCode »

FILENAME NEATNESS
==================

This post is about a couple of hints that may help Novice Modders (NMs) better organize their PGF scenario / campaign definition files.

Anyone who takes a cursory look at the contents of the \SCENARIO sub-folder will be served with a visual... mess of sorts ! Specifically:

1) xxx.PGSCN files are rather well... behaved. "xxx" usefully and efficiently stands for a 3-digit descriptor sporting leading zeroes as well. Thus "xxx" may span all integers in the range "000" to "999" inclusive (yes, 000.PGSCN is quite acceptable). One thousand (1,000) scenario definition slots should be quite adequate, I should think...

2) Companion files MAPxx.SET and MAPxx.STM hark back to the days of PG1-DOS. Fortunately, PGF has not hard-coded this precise format. Thus, renaming such files to MAPxxx.SET and MAPxxx.STM so as to match the indexing of the companion xxx.PGSCN file is a very good idea provided one:

a) Appropriately updates the "SET file" designation in the xxx.PGSCN file.

b) Appropriately updates the name of the companion MAPxxx.STM file by ever so lightly hex editing the MAPxxx.SET file, starting at decimal offset 048.

Incidentally, the suggested procedure under point (b) goes back to the days of PG1-DOS. Many, many years ago it was discovered and conclusively demonstrated that only the format of MAPxx.SET was hardcoded. To this effect, the companion *.STM file was defined within MAPxx.SET. Well, this "tradition" has been heeded to and emulated by PGF as well. This is the reason why the AG \SCENARIO sub-folder contains files with such wonderfully descriptive names as MOSCOW.STM and OVERLORD.STM...

3) In my opinion, xxxx.PGBRF files are quite... dysfunctional. They are all over the place ! Luckily, the format Xxxx.PGBRF seems to be a life saver. "X" stands for any capital letter (Z is such an obvious choice). "xxx" can span all (leading zero) integers in the range "000" to "999" inclusive (yes, Z000.PGBRF is quite acceptable). One thousand (1,000) campaign briefing slots should be more than adequate, I should think... Thus:

a) The campaign definition file PG.PGCAM is perfectly happy to accommodate the suggested format.

b) All briefing files will be neatly presentable at the very end of the \SCENARIO sub-folder file listing, no longer cluttering the... landscape !
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

TAB-SEPARATED UNICODE TEXT FILES

Post by HexCode »

TAB-SEPARATED UNICODE TEXT FILES
=================================

Introductory Documentation

Here is a somewhat modified version of what PGF's Developer / Programmer had to say on the subject:
PGF uses a number of different external support data files: Equipment Spec File (EQUIPMENT.PGEQP), Campaign Definition File (PG.PGCAM) and Scenario Parameters Files (xxx.PGSCN) ... Although PGF's external support data files are diverse in substance, they all use the same data organization scheme: they are Tab-Separated Unicode Text Files.

a) Each such file contains a number of tables separated by one or more empty lines.

b) All columns in a table are separated by a single tab character. Multiple tabs, spaces or any other delimiters are not supported.

c) Lines starting with the character # are treated as comments and are completely ignored by the program. Often, the comments are used to provide identifier labels for the columns in the tables.

d) The files use UTF-16 encoding. Among other things, this allows for easy language localization of the information contained in the game's external support data files.

Since the above files are "plain text", they can be edited in any text editor, including MS-Notepad. However, a better way to work with such files is to open them in MS-Excel. MS-Excel understands tab delimiters and presents the file contents in a visually convenient tabular format. While working with such a file in MS-Excel, one can change the widths of any columns at will - this, per se, will not affect the file in any adverse way, when ultimately saved.

Please keep in mind that empty lines are very important: they specify where a table ends and the next one begins. So, do not remove empty lines or insert new empty lines where they were not there before. If one does so, chances are that he may render the whole file unreadable by PGF's engine !
Typical... Headaches

Typical Headache #1 ==>
Help please ! My desktop crashed so now I'm using my laptop to play PGF. I've been unable to find my MS-Office disk. I've tried to use the spreadsheet functions in MS-Works and Open Office to modify PGF's Equipment File. For whatever reason neither seems to work. Open Office allows me to save the file with the correct file name and extension but, once saved, PGF doesn't "see" the file. Any suggestions ?
Typical Headache #2 ==>
There's a problem with spreadsheet programs adding quotation marks here and there ... PGF ignores quotation marks; consequently, a reference such as 3" (Gun) is displayed as 3 (Gun).
Headache... Remedies

For Typical Headache #1, the following "medication" has proven quite effective:

Make sure the modified PGF Efile is saved with UNICODE encoding - otherwise it won't work ! I sometimes forget to do it when I edit PGF's Efiles and then I get crashes when I try to run the game (just like you do). To repeat, all you need is to re-save the file selecting the UNICODE encoding option and it will work !

For Typical Headache #2, the following "medication" has proven quite effective:

Don't expect perfection out of automatism. Once any heavy duty modifications are in place, just use MS-Notepad or MS-Wordpad to effect a light edit of undesired present or desired absent quotation marks !

Caveats

1) Not every spreadsheet program version is capable of saving text files with UTF-16 encoding. Even somewhat dated MS-Excel versions cannot do that !

2) UTF-8 encoding just will not do. It has to be UTF-16 encoding !

3) More precisely, UNICODE has to be UTF-16LE TSV (i.e., Little Endian Tab Separated Version) and not CSV (i.e., Comma Separated Version). The text file's first two bytes always show as hexadecimal "FF FE". Moreover, the file sports Windows-style "carriage returns / line feeds" (CR/LF).

Hints

A) To create a UTF-16LE TSV text file from scratch, just run a relatively recent version of MS-Notepad and save the file with Encoding: UNICODE. This will automatically generate the appropriate Unicode byte signature. To add data, use the TAB key to insert tabs (column separators) and the ENTER key to end a line (insert "carriage return-line feed" pair). After one adds a couple of lines in MS-Notepad, the rest can be done in a capable spreadsheet program like some relatively recent version of MS-Excel.

B) Sometimes, it is easier to use an original PGF external support data file as a template / shell. To this effect, one can use a capable text editor to paste spreadsheet tab-separated data into the template file and save the latter.

Additional Commentary

MS-Notepad

For editing purposes, I still use MS-WIN XP. When I open a PGF *.PGSCN file, the file's contents are displayed in all their glory. If I, then, turn around and attempt to invoke the "Save As" dialog box, the "Encoding" option displayed at the very bottom of the dialog box which is automatically chosen for the file is "Unicode".

Now, my Notepad version gives me 4 "Encoding" choices:

"ANSI"
"Unicode"
"Unicode big endian"
"UTF-8"

CRITICAL CLARIFICATION : When Microsoft tells me "Unicode", they actually mean "UNICODE: UTF-16LE TSV (i.e., Little Endian Tab Separated Version)"... By the way, the Notepad version that's native to MS-Windows 10 explicitly lists a UTF-16 LE encoding option.

Spreadsheets

Technical realities here can be a bit challenging...

With respect to MS-Office Excel versions, the "big picture" appears to be like this: No version later than 2011 allows the user to choose some specific encoding option... Presumably, all such spreadsheets assume that the encoding is and, in any case, should be, UTF-16LE...

With respect to OpenOffice Calc versions, encoding designations such as "Unicode" and / or UTF-16 aren't UTF-16LE. Rather,they are UTF-16BE (Big Endian). Calc does preserve PGF's UTF-16LE encoding but, for some reason, "normalizes" row length by inserting a bunch of trailing tabs at the row's end... For example, If one row is 9 cells long and all other rows 5 cells long, Calc makes all rows 9 cells long. Every row in the file will have the same length - that of the longest row. When this happens, PGF's engine is unable to properly "read" such a "padded" file and crashes. The extra tabs must be manually removed. Files where all rows have the same length (e.g., EQUIPMENT.PGEQP) are treated rather kindly by Calc.
Last edited by HexCode on 2021-06-01 02:06, Tuesday, edited 1 time in total.
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

DON'T OVERWRITE, JUST APPEND !

Post by HexCode »

DON'T OVERWRITE, JUST APPEND !
===============================

Elsewhere in these forums:
Backwards compatibility is important, if for no other reason than to already have some working content for your modded stuff. If backward compatibility is preserved, at least you'll have the stock SSI PG1 / AG scenarios and campaigns done and ready to play, and then you can expand from there. Keep adding extra stuff leaving the originals in place (you can adjust the original stuff, even replace like for like, but don't replace a battleship with a tank or infantry with an airplane, if you know what i mean) and it's a great shortcut to give you a head start to wherever you're planning to go.
A VM has given "us" invaluable, practical advice which, by the way, has stood the test of time. Notice that the above has absolutely nothing to do with any romantic notion of hobbyist "camaraderie", not even "enlightened" self-interest. It squarely addresses the modder's very own, almost tangible, practical interest(s); period.

By the way, the kinds of content which immediately come to mind are:

a) BMP files (e.g., flags, map terrain tiles, unit images) containing icons;
b) File EQUIPMENT.PGEQP containing unit specs;
c) File MAPNAMES.STR containing geographical location names.

Such content governs play involving multiple scenarios. PGF's engine does not impose any explicit limits on how many items / records may be usefully accommodated within each one of the above files.
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

THE AI NEVER DOES...

Post by HexCode »

THE AI NEVER DOES...
===================

Intent

The intent here is not to tell modders how to mod. Far from it. Some modders design custom scenarios intending for PGF's AI to lead a specific Side (e.g., Alliance) and no other. Other modders are quite comfortable with symmetrical play where the AI may be called upon to lead either one of the two Sides (e.g., Alliances). Finally, there are modders who do not mind viewing their creations as dual-purpose (i.e., fighting against PGF's AI or playing H2H).

Focus

The focus herein is somewhat narrow. Namely, quite a few actions routinely available to human players are identified which, nevertheless, PGF's AI never engages in. In so doing, modders are urged to ask themselves:

Given that PGF's AI never engages in action X, why is it that I have designed my custom scenarios as if it may still do so ?

Sorry, Can't Do...

In-game, PGF's AI never:

1) Disbands any of its units.

2) Upgrades any of its units.

3) Embarks any of its units.

4) Disembarks any of its air-transported units.

5) Repairs any of its naval units in some friendly port(s).

6) Purchases any new air units.

7) Refuels or repairs any of its air units on some friendly aircraft carrier(s).

In addition:

A) Absent some friendly airfield on the map, PGF's AI keeps all of its air units stationary.

B) PGF's AI never purposively situates a friendly bridging unit on underlying river terrain and immediately utilizes it to allow other friendly units to cross.
Last edited by HexCode on 2021-06-01 02:09, Tuesday, edited 1 time in total.
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

UNIT ICON ORIENTATION

Post by HexCode »

UNIT ICON ORIENTATION
======================

The present post focuses on the orientation of unit icons contained in graphics file TACICONS.BMP. The "default" orientation is characterized by Side 0 (e.g., "Axis") unit icons facing towards the east and Side 1 (e.g., "Allied") unit icons facing towards the west. A "symmetrically flipped" orientation is also employed in certain scenarios. In this latter case, Side 0 unit icons face towards the west and Side 1 unit icons face towards the east.

PGF's code implements the above feature in a rather unorthodox way. Instead of sticking to the generally intuitive way of single data entry reference (or, even, horizontally adjacent data entry references) the code embraces a counter-intuitive data structure based on... vertical adjacency ! The "problem" with this implementation is that a prospective modder may get the impression that there are more degrees of freedom in tinkering with the relevant entries in file xxx.PGSCN than is actually the case...

It turns out that the "default" orientation is triggered by the following "vertically adjacent" data entries:

Code: Select all

# Sides
# Index	Orientation
    0 	    0	
    1 	    1
On the other hand, the "symmetrically flipped" orientation is triggered by the following "vertically adjacent" data entries:

Code: Select all

# Sides
# Index	Orientation
    0 	    0	
    1 	   -1
User avatar
HexCode
First Lieutenant
First Lieutenant
Posts: 923
Joined: 2019-09-30 18:54, Monday

CHARACTER SET EVOLUTION

Post by HexCode »

CHARACTER SET EVOLUTION
=========================

ASCII Standard & Extended Character Sets

The ANSI Standard character set comprises 256 characters. Collectively, the first 128 characters (0 through 127) form the ASCII Standard character set. The remaining 128 characters (128 through 255) are collectively referred to as the ASCII Extended character set.

The ASCII Standard character set addresses the need to represent lowercase letters (a-z), uppercase letters (A-Z), digits (0–9) and a number of symbols such as punctuation marks within the usual context of the English language. The ASCII Extended character set contains a variety of characters primarily addressing the need to represent non-English alphabet (Latin-based) characters sporting diacritics (e.g., Umlaut).

Each ASCII Standard character is represented by a distinct, universally recognized and accepted, single byte value. On the other hand, ASCII Extended character set representation has never been truly standardized...

Unicode Character Sets

Enter the Unicode character set which, practically speaking, could represent every single character in any imaginable language. This has necessitated a paradigm shift in how to think about characters. In this new paradigm, each character is an idealized, abstract entity represented as a code-point. The code-points are written as, say, U+039B, where U stands for "Unicode", and the number is hexadecimal.

One should kindly consult

Engine Hexadecimal Character Strings
viewtopic.php?f=100&t=536#p8935

to familiarize himself with the basic concept underlying UTF-16 character encoding. UTF stands for "Unicode Transformation Format". It is important to note that UTF-16 character encoding is NOT backwards compatible with the ASCII Extended one.

PGF has been coded to be natively supportive of the Little Endian variant of UTF-16 (UTF-16LE) character encoding. To this effect, say, character U+039B would appear as byte pair value "9B 03" in a binary file's hexadecimal code.
Post Reply