Emulating VT102 Hardware in Perl - Part 4: Support Hardware

This fourth "chapter" in the VT102 emulator series will deal with the VT100/102-specific hardware, the NVRAM, the keyboard and serial I/O (but not the video hardware, which will be dealt with in part 5).

The NVRAM

Being the modern terminal that it is, the VT100 comes with an NVRAM that stores set-up information even when the terminal is switched off.

The specific NVRAM chip used inside the VT100 is a ER1400 chip, which contains 100 14 bit words of "EAROM" ("electrically alterable read-only memory") in MNOS technology (metal nitride oxide semiconductor). If that wouldn't be weird enough, the protocol it uses and the way it is wired into the VT100 is a thing of beauty: Anybody explaining it to me before I would have seen it myself I would consider a liar.

The NVR needs an external clock input, which is wired to the horizontal line rate of 15.734 kHz. On each (falling) clock cycle, the NVR executes one of seven commands, which are selected by bits 1..3 (values 2..8) in the I/O port 0x62. The lowest bit in port 0x62 is a data bit used by some of the commands.

The seven commands are:

0 accept data - shift the data bit into the data register

1 accept addr - shift the data bit into the address register

2 shift out - shift a data bit out (bit value 0x20 in port 0x42)

4 write - store a word (NVRAM[addr_register] = data_register)

5 erase - erase a word (NVRAM[addr_register] = 0x3fff)

6 read - read a cell and put the data into the data register

7 standby - do nothing ("corrupt data on poweroff" - not used :)

Since the "shift data into register" commands shift only a single bit into the registers, the protocol becomes a bit clearer - to read a cell, you first give 20 "accept addr" commands (why 20? stay tuned!), which shift 20 bits into the address register. Then you issue a "read" command to read it, followed by "shift out" commands to actually shift out the 14 data bits so you can read them (only 8 bits of each cell are ever being used in the VT100).

And all that at a fixed frequency on 15.734kHz - that's one reason why the VT100 displays "Wait" on power up - that's when it copies the NVRAM to RAM, a slow process. And also rather tedious.

If that wouldn't be enough, the chip wants the address as a two digit decimal(!) number with each digit encoded in one-out-of-ten code(!), which means you transfer 10 bits, 9 of which are 0 and one of which is 1, indicating the actual digit.

Urgs.

The resulting code isn't actually as convoluted as the description might make you suspect, but that owes more to the succinctness and versatility of Perl, rather then this being a simple or reasonable protocol.

Anyway, let's start looking at the code for this mess:

my @NVR = (0x3fff) x 100; # vt102: 214e accum, 214f only lower 8 bit used, first 44 bytes
my $NVRADDR;
my $NVRDATA;
my $NVRLATCH;

my @NVR_BITIDX; $NVR_BITIDX[1 << $_] = 9 - $_ for 0..9;

This declares the NVRAM contents as an array @NVR of 100 values, all 0x3fff (14 one bits). It also declares the $NVRADDR and $NVRDATA registers, and the $NVRLATCH variable, which contains the last command given to the NVR by an out 0x62 instruction.

And then it creates a lookup table for the one-out-of-ten encoding. Which is backwards, too, of course. And wasteful. But since it is actually used very often (the VT100 drives the NVR in almost permanent "accept data" mode to avoid data corruption at power off) it's a useful optimisation. And it simplifies the code enormously.

Here is the NVR state machine. It is called on each in 0x42 instruction, which turns out to be a good way to clock it.

sub nvr() {
   my $a1 = $NVR_BITIDX[(~$NVRADDR      ) & 0x3ff];
   my $a0 = $NVR_BITIDX[(~$NVRADDR >> 10) & 0x3ff];

   $NVRCMD[($NVRLATCH >> 1) & 7]($a1 * 10 + $a0, $NVRLATCH & 1)
}

It decodes the current address in the $NVRADDR register and then calls the command function according to the $NVRLATCH, passing the address as first argument, and the current data bit as second.

The commands themselves are stored in an array indexed by command number:

my @NVRCMD = (
   sub { $NVRDATA = ($NVRDATA << 1) + $_[1]; }, # 0 accept data
   sub { $NVRADDR = ($NVRADDR << 1) + $_[1]; }, # 1 accept addr
   sub { ($NVRDATA <<= 1) & 0x4000 },           # 2 shift out
   undef,                                       # 3 not used, will barf
   sub { $NVR[$_[0]] = $NVRDATA & 0x3fff; },    # 4 write
   sub { $NVR[$_[0]] = 0x3fff;            },    # 5 erase
   sub { $NVRDATA = $NVR[$_[0]];          },    # 6 read
   sub { },                                     # 7 standby
);

As you can see, an accept data or address command shifts the corresponding register and adds the current data bit value. The other commands are also pretty straight-forward translations of the commands described earlier.

The return value of these functions becomes the current bit in port 0x42.

To tie this NVR hardware into the actual emulator we need to add logic to the ports 0x62 and 0x42:

sub out_62 { # nvr latch register (4 bits)
   $NVRLATCH = shift;
}

sub in_42 { # flag buffer
   ++$LBA6;

   $NVRBIT = nvr ? 0x20 : 0x00 if ($LBA6 & 0x3) == 0x2;

   # KBD_XMITEMPTY LBA7 NVRDATA ODDFIELD - OPTION !GFX !AVO PUSART_TXRDY

   my $f = 0x85 | $NVRBIT;

   $f |= 0x02 unless $AVO;
   $f |= 0x40 if $LBA6 & 0x2;

   $f
}

out_62 is easy, it only has to store the value in the $NVRLATCH. in_42 is a bit more complicated because it contains a number of other flags, but basically, when the vertical line retrace happens (emulated by $LBA6), the NVR state machine gets invoked and provides a single bit.

The reason for the expression ($LBA6 & 0x3) == 0x2 is to call the state machine only on the falling edge if the signal.

And that's it for the NVRAM.

The Serial Line

While the VT100 terminal can be expanded with a serial line printer interface, this section is only about the "main" serial line interface, with which the terminal connects to the outside world (for RS-232 typically within a few meters, for the 20mA current loop, much longer distances are possible).

The serial line is driven by an Intel 8251A chip, which is a no-thrills serial interface capable of a whopping 19200 Baud (if you need a comparison, that's less than 2KB per second, or just about a single screenful of data per second). The synchronous mode is a bit faster but is not used in the VT100.

In vt102 it is simulated only to the extent necessary to exchange data between the VT100 firmware and a pseudo terminal (pty).

Part of the interface was already described in the previous episode - the mainloop accepts data from the pty and queues it up into @PUSARTRECV, generating interrupt signals as long as there is data to be read by the VT100. Data transmitted by the VT100 is accepted at infinite speed, which doesn't bother the VT100 firmware at all. Data received is slowed down, because the VT100 can't handle infinite speeds here :) The slowdown is implemented by the mainloop "occasionally" asserting the receive interrupt when there is receive data.

Early VT100 models had a 64 character software FIFO (or "SILO" as DEC calls it), later models had 128 characters. The receive interrupt just puts the received character into this FIFO, so if you interrupt it too often, the VT100 has no time to handle these characters, and, worse, not even time to send an xOFF signal to tell vt102 to temporarily pause sending it data.

When the VT100 receives an interrupt, it will check the hardware ports, which is where vt102 delivers it's data. The simplest port is port 0x00, which returns the current read data:

sub in_00 { # pusart data
   # interrupt not generated here, because infinite
   # speed does not go well with the vt102.

   shift @PUSARTRECV
}

Since the simulated serial line is much faster than a real line, we deliver one character for every read. In theory, we could instantly post the next interrupt, as if the next character had already been received, but the firmware doesn't cope well with this.

The next port of importance is port 0x01, which is the status register for the serial chip. The comment shows the meaning of the bits. The emulator always asserts DSR, TXEMPTY and TXRDY (meaning a modem is connected and the chip is ready to receive serial send data) and additionally asserts RXREADY whenever there is data to be received by the VT100.

sub in_01 { # pusart status
   # DSR SYNDET FE OE | PE TXEMPTY RXRDY TXRDY
   0x85 + (@PUSARTRECV && 0x02)
}

Port 0x42 ("flags buffer") also has the TXRDY bit (called XMIT). This reflects the TXRDY pin of the 8251A, which works a bit differently than the TXRDY status bit: the former is affected by CTS ("clear-to-send", a modem control line for flow control) and the TXEN setting, while the latter is not. That means the TXRDY status bit is set when the chip can send, while the XMIT flag bit in the flags buffer is set when the serial line is in a condition to send out data.

The modem control lines, to the extend they are implemented, are provided in port 0x22, whose specific values are a bit of a guess from a cursory glance at the firmware. vt102 always asserts CTS, SPDI and CD, meaning the modem is always ready to receive data ("clear-to-send"), at high speed ("speed indicator", a pretty meaningless signal), and a carrier has been detected, but the phone is not ringing.

sub in_22 { # modem buffer
   # wild guess: -CTS -SPDI -RI -CD 0 0 0 0
   0x20
}

The output side of things is a bit more interesting, mostly because we need to slow down when the VT100 says so:

sub out_00 { # pusartdata
   # handle xon/xoff, but also pass it through
   if ($_[0] == 0x13) {
      $XON = 0;
      return;#d#
   } elsif ($_[0] == 0x11) {
      $XON = 1;
      return;#d#
   }

   syswrite $PTY, chr $_[0];

   $INTPEND |= 1;
}

Writing a byte to port 0x00 means sending the character over the serial line. vt102 does so, except for xOFF/xON commands, which it uses to pause and resume, respectively, providing receive data to the VT100. Sending xOFF/xON to tell the other side to pause and resume sending data is called "software flow control" as opposed to "hardware flow control" provided by modem control lines. Software flow control is a bit more finicky because it has a higher delay, but it does work end-to-end, unlike hardware flow control, and is the only form of flow control supported by the VT100.

The vt102 could pass xOFF/xON to the PTY as well, but doesn't do so because it doesn't seem to be particularly necessary or efficient.

The remaining ports would be difficult to implement if vt102 didn't chose to fake them completely. First, the command register, which is also the mode register:

sub out_01 { # pusartcmd
   $PUSARTCMD = shift;

   $INTPEND |= 1 if $PUSARTCMD & 0x01; # VT102, 5.5 txrdy
   $INTPEND |= 2 if $PUSARTCMD & 0x04 && !@PUSARTRECV; # VT102, 6.5 rxrdy, needed for some reason
}

We simply ignore (or even misinterpret) the mode byte written first, and only interpret the "transmit enable" (0x01) and "receive enable" (0x04) status bits, which seemingly are only used on the VT102 and it's variants.

When receive or transmit is enabled, vt102 instantly asserts the associated interrupts.

One more port is missing, the baud rate generator:

sub out_02 { } # baudrate generator

Indeed, nobody cares, because the simulated serial line is exactly as fast as it needs to be.

The Keyboard

Next in the line of miscellaneous hardware is the keyboard. The keyboard itself does 2-key-rollover in the worst case and also uses a serial protocol. It is not a RS-232 interface (which would be overkill) but the hardware is very similar, except for a twist - a single wire is used both for receiving and sending and can be used simultaneously - the four states needed to do so are represented by 0V, 12V, 6V with current flowing to the keyboard, and 6V with current flowing to the mainboard, which is quite ingenious, IMHO.

The VT100 mainboard can send data to control the LEDs and start a keyscan. The keyscan is initiated by the mainboard, but otherwise the keyboard scans the key matrix by itself using a simple counter, which enables one of the 16 rows and 8 columns of the matrix. When a keyscan is initiated, the counter counts from 0x00 to 0x7f. When the corresponding switch is pressed, it will transmit the current counter value. The "key" with number 0x7f does not actually exist but is hardwired to be pressed, which serves as an indication to the mainboard that they keyboard matrix has been scanned.

The code that decodes these keycodes and converts them into either actions or key sequences to be sent to the host is quite marvellous and well worth a deeper look (consider it homework...).

As for the simulation side, again a few shortcuts have been made. On the reading side, port 0x42 (the flags buffer) has the KBD_XMITEMPTY bit, which is always on in vt102, meaning the emulator is always ready to receive data, unlike the real keyboard.

Keyboard data (the keycodes) are read from port 0x82:

sub in_82 { # tbmt keyboard uart
   return 0x7f unless @KXMIT;

   $RST |= 1;
   shift @KXMIT
}

Like the real keyboard, each time a scan is initiated the virtual keyboard matrix (a has called %KXMIT) is scanned and all currently pressed keys are transmitted. This is done by sending all codes in @KXMIT, which is populated elsewhere. If no keys are remaining, the port always returns a 0x7f to indicate end of scan.

All the real work is done in out_82, which is the transmit register for the keyboard. And it is a fair amount of work indeed:

sub out_82 { # keyboard txmit
   # CLICK STARTSCAN ONLINE LOCKED | L1  L2  L3  L4 (vt100)
   # CLICK STARTSCAN ONLINE LOCKED | CTS DSR INS L1 (vt102)
   $KSTATUS = $_[0];

   # start new scan unless scan is in progress
   if (($_[0] & 0x40) && !@KXMIT) {

The easy part is the status LEDs - quite simply, some bits enable some LEDs. The CLICK signal creates an audible click, which, cutely enough, can be "defeated" in the Set-Up, according to the manual.

The fun starts when a keyboard scan is initiated by transmitting STARTSCAN. In theory, scanning the keyboard matrix involves looking at which keys are set in the %KXMIT hash and putting these into @KXMIT. In practise this is not enough, because anything pressing and releasing keys must do so in a synchronised (to the keyscan) way, and keys also need to be held for a minimum amount of time to "defeat" (ehem) the debouncing done by the firmware.

   # do not reply with keys in locked mode
   # or during post (0xff),
   # mostly to skip init and not fail POST,
   # and to send startup keys only when terminal is ready
   unless (($_[0] & 0x10) || ($_[0] == 0xff) || ($VT102 && $INTMASK == 0x07)) {
      if ($KXCNT <= 0 && @KQUEUE) {
         my $c = shift @KQUEUE;

         if ($c < 0) { # key up
            delete $KXMIT{-$c};
            $KXCNT = 10;
         } elsif ($c > 0) { # key down
            undef $KXMIT{$c};
            $KXCNT = 10;
         } else { # delay
            $KXCNT = 100;
         }
      }

      --$KXCNT;
      @KXMIT = sort keys %KXMIT;
   }

   $RST |= 1;

Ok, first of all, the keyscan is infinitely fast in the emulator, so when initiated, it instantly finishes and asserts the relevant interrupt ($RST |= 1), leaving the firmware to read all the keycodes.

The first if condition is documented in the comments - basically, the emulator is not allowed to generate keycodes under a variety of conditions, and not too early after powering up.

To ensure a minimum key press time, the function uses the counter $KXCNT - it is initialised to 10 for key changes, and only when it reaches 0 is a new key press (or release) allowed. When a change is allowed and there are commands in @KQUEUE, the next command is dequeued and used.

The commands in @KQUEUE are very simple: If it is a strictly positive number, then the key with the given code is pressed down (added to %KXMIT). If it is negative, the key will be released (deleted from %KXMIT). And as a special hack for automatically entering Set-Up mode, a value of 0 indicates a delay, where vt102 will not allow any key changes for 100 keyscan cycles.

Keycodes? What keycodes?

That is all that is to the hardware emulation side. The real fun, however, is translating key sequences received from the outside to key press/release events.

The previous chapter in this saga (on the mainloop) explained how STDIN is being read into $STDIN_BUF, followed by calling the magic stdin_parse function to make sense of it.

This function is quite short:

sub stdin_parse {
   key $KEYMAP{$1}
      while $STDIN_BUF =~ s/$KEYMATCH//;

   # skip input we can't decipher
   substr $STDIN_BUF, 0, 1, "";
}

It makes use of a regex that greedily matches key sequences. If none can be matched, the function simply skips an input octet. This is, strictly speaking wrong, because at high throughputs and over very slow connections, this could lose keys. Neither condition is likely to appear in reality, though. This is a problem with all programs parsing key sequences, and the modern solution is to use this method, while older (and more stable, but slower) solutions employ a timeout.

Once it has identified a key sequence, it looks up its corresponding key code in %KEYMAP and generates a key press/key release using the key helper function:

my %KMOD; # currently pressed modifier keys

sub key {
   my ($key) = @_;

   push @KQUEUE, -0x7c if !($key & 0x100) && delete $KMOD{0x7c}; # ctrl-up
   push @KQUEUE, -0x7d if !($key & 0x080) && delete $KMOD{0x7d}; # shift-up

   push @KQUEUE,  0x7c if   $key & 0x100  && !$KMOD{0x7c}++; # ctrl-down
   push @KQUEUE,  0x7d if   $key & 0x080  && !$KMOD{0x7d}++; # shift-down

   $key &= 0x7f;
   push @KQUEUE, $key, -$key;
}

It's surprisingly complicated to press a key. The reason is that some "keys" are actually a combination of multiple physical keys (uppercase A for example is actually shift + A). The key map is encoded using a single 9 bit integer per key. The low 7 bits are the keycode/key number used by the VT100. Bit value 0x100 means control must be pressed together with the key, and value 0x80 is the same but for the shift key.

This is implemented by releasing or pressing modifier keys that have changed before the actual key is pressed, that means that entering A might press Shift, then A, then release A, while keeping Shift pressed until a sequence arrives that needs it released. The current modifier state is stored in %KMOD, and you can see nicely how presses/releases are implemented via positive or negative keycodes.

The heart of this is the %KEYMAP hash, which is used both to map key sequences to keycodes, and to generate the $KEYMATCH regex.

Many of the special keys are hardcoded for rxvt-unicode (and only some are shown here, refer to the source for the complete list):

# 0x080 shift, 0x100 ctrl
my %KEYMAP = (
   "\t"     => 0x3a,
   "\r"     => 0x64,
   "\n"     => 0x44,

   "\x00"   => 0x77 | 0x100, # CTRL-SPACE
   "\x1c"   => 0x45 | 0x100, # CTRL-\

   # hardcoded rxvt keys
   "\e"     => 0x2a, # ESC
   "\e[2~"  => 0x79 | 0x100, # CTRL-C (insert)
   "\e[5~"  => 0x7e, # CAPS LOCK (prior)
   "\e[6~"  => 0x6a, # NO SCROLL (next)
   "\e[c"   => 0x10 | 0x080, # RIGHT
   "\e[7~"  => 0x7b, # SETUP (home)
   "\x7f"   => 0x33, # BACKSPACE

   "\e[11~" => 0x32, # PF1
);

Some are generated from an obscure table (compactness was a goal for the script):

@KEYMAP{map chr, 0x20 .. 0x40, 0x5b .. 0x7e} = unpack "C*", pack "H*",
   "779ad5a9a8b8a755a6b5b6b466256575"
 . "351a3929283837273626d656e634e5f5"
 . "b9"                              # 20..40
 .                       "154514b7a5"
 . "244a6879591949485816574746766706"
 . "050a185a0817780969077a95c594a4"; # 5b..7e

And Ctrl and Shift variations are generated programmatically:

$KEYMAP{"\x1f" & $_} ||= $KEYMAP{$_} | 0x100 for "a" .. "z"; # ctrl
$KEYMAP{"\x20" ^ $_} ||= $KEYMAP{$_} | 0x080 for "a" .. "z"; # shift

The regex is then generated from the keys of %KEYMAP:

my $KEYMATCH = join "|", map quotemeta, reverse sort keys %KEYMAP;
$KEYMATCH = qr{^($KEYMATCH)}s;

The only complication here is that some keys (for example escape) are prefixes of others, so we sort the keys in reverse order, which puts shorter keys after longer keys with the same prefix, and thus matches the longest sequence first.

And this concludes the keyboard processing, except for a tiny detail: The NVRAM defaults of the VT100 do not actually allow it to talk to the outside world, so the emulator first needs to configure it. This is where the delay code comes in handy, as toggling Set-Up mode and its pages takes a fair amount of time during which the VT100 will ignore key presses.

# initial key input, to set up online mode etc.
# could be done via nvram defaults

@KQUEUE = (
   0x7b, -0x7b,                 # setup
   0,                           # delay
   0x28, -0x28,                 # 4, toggle local/online
   0x38, -0x38,                 # 5, setup b
   0,                           # delay
   (0x10, -0x10) x 2,           # cursor right
   0x37, -0x37,                 # 6 toggle soft scroll
   (0x10, -0x10) x 1,           # cursor right
   0x37, -0x37,                 # 6 toggle autorepeat off
   (0x10, -0x10) x 8,           # cursor right
   0x37, -0x37,                 # 6 toggle keyclick
   (0x10, -0x10) x 1,           # cursor right
   $VT102 ? () : (0x37, -0x37), # 6 toggle ansi/vt52
   (0x10, -0x10) x 7,           # cursor right
   0x37, -0x37,                 # 6 toggle wrap around
   0x7b, -0x7b,                 # leave setup
);

The comments hopefully speak for themselves. Apart from toggling online mode, some other settings are toggled as well, such as smooth scroll, which is ignored by the emulator and would only slow things down.

Next Time?

This has been a lengthy episode in our quest and explained most of the hardware of the VT100. The next article in this series will be about the video processor, and possibly other things. Stay tuned!