Subject: Archiver format

From: "Michael Zapf" <[email-masked]>
Date: Fri, 28 Oct 2011 08:51:59 -0000

Is the Archiver format documented somewhere? I'd like to include a viewer in my TIImageTool in a future version.

Michael

---

From: "Klaus L" <[email-masked]>
Date: Fri, 28 Oct 2011 10:34:03 -0000

Hi Michael!

Sent you an email a minute ago about Archiver format.

Klaus

---

From: "Paolo Bagnaresi" <[email-masked]>
Date: Fri, 28 Oct 2011 21:39:53 +0200

Hi Michael!

I had to deal with Archiver format when I was developing TI99-PC, a QBasic 
program that handles TI files on DOS based PC.

I was lucky enough because I found on the Internet some 80x86 assembler 
source code that quite probably Barry Boone used to develop his Archiver.

It was written by Tom Pfau (Digital Equipment Corporation), later modified 
(v1.2) by Toad Hall. Barry Boone and his Archiver was next on the line. 
Either Barry further modified the code (basing the LZW algorithm on a 
different seed value) or he was inspired by a slightly different source 
code.

At any rate, that Tom Pfau code, coupled with some disassembling of 
Archiver, made translating the program to a 80x86 MASM source code 
possible.

The amazing thing is that I did not need to understand how compressing and 
decompressing work in Archiver. In fact, I didn't! For any TMS9900 
instruction forming the core of Barry Boone Archiver, I worked out the 
exact translation in 80x86 MASM source code. To my big surprise, that was 
enough.

Some years ago, Fred Kaal asked me for some help while developing his 
TiDir. I sent him my 80x86 MASM source code of Archiver (Ark-TI.ASM and 
Deark-TI.ASM). He never said if my code was useful to him or not, but sure 
enough his TiDir program (supposedly written in MS Visual C++ 6.0) handles 
Archiver files pretty well. So, he must have found somehow a solution to 
the problem, in a more advanced language too.

I think he is the right person to ask if you do not want to get involved 
with old pesky DOS MASM code :-)

Paolo Bagnaresi

>----- Original Message ----- 
>From: "Michael Zapf" <[email-masked]>
>To: <[email-masked]>
>Sent: Friday, October 28, 2011 10:51 AM
>Subject: [TI994A] Archiver format
>
>
>Is the Archiver format documented somewhere? I'd like to include
>a viewer in my TIImageTool in a future version.
>
>Michael

---

From: Barry Boone <[email-masked]>
Date: Fri, 28 Oct 2011 20:16:39 +0000

The compression piece is a variable 9-12 bit LZW table that resets back to 9 bits once it fills up.  The limit on the upper end of 12 bits was needed to deal with the memory constraints of a minimally expanded TI.  If you work out the decompression part, the datastructure part is pretty simple, as the first part of the file contains a directory of all the content pieces (like the filename and the TIFILES bytes needed to recreate the file accurately), and the rest of the archive file contains the actual archived contents.  If you create an uncompressed archive, you can just look at the layout it creates to get the feel for how it works, as all I am doing is running that content verbatim through a compressor.

If memory serves me correctly, if you encounter a code 256 in the compression stream (a 1 with 8 zeros following), that indicates the need to reset back to 9 bits and clear the tables and start over.  Code 257 is the end of stream code, and signifies EOF.  This is BTW the reason that the beginning of all archived files starts with >80 as it is the first 8 bits of the 9 bit string 100000000.  In the case where you encounter code 000100000000 in the datastream (this is a 12 bit version of code 256) you'd need to reset back to 9 bits and start over.

While I did look at both the theory and some implementation samples of LZW, I tried to stick to a textbook (theory-like) implementation, which is why my code differs slightly from those x86 samples you found.

With some minor tweaking I was able to use the same code to decompress GIF files as well.  The main tweaks dealt with being able to handle a smaller variable bit size, so instead of starting with 9 bits I could start with 3 (for example) if needed.  IIRC, GIF files max at 12 bits thankfully, so that the TI was able to decompress them.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Paolo Bagnaresi
Sent: Friday, October 28, 2011 2:40 PM
To: [email-masked]
Subject: Re: [TI994A] Archiver format

Hi Michael!

Paolo Bagnaresi

>From: "Michael Zapf" <[email-masked]<mailto:forums11%40mizapf.de>>
>To: <[email-masked]<mailto:TI994A%40yahoogroups.com>>

---

From: "Paolo Bagnaresi" <[email-masked]>
Date: Fri, 28 Oct 2011 23:11:40 +0200

Hi Barry!

Thanks for stepping in and clarifying the basic structure of your 
Archiver. The Archiver was quite an accomplishment, I would say. And 
rightly got a widespread use in our great community.

Thanks, Barry.

Paolo

>From: "Barry Boone" <[email-masked]>
>Sent: Friday, October 28, 2011 10:16 PM
>Subject: RE: [TI994A] Archiver format
>
>
>The compression piece is a variable 9-12 bit LZW
>table that resets back to 9 bits once it fills up.
>The limit on the upper end of 12 bits was needed to deal
>with the memory constraints of a minimally expanded TI.
>If you work out the decompression part, the datastructure
>part is pretty simple, as the first part of the file
>contains a directory of all the content pieces (like the
>filename and the TIFILES bytes needed to recreate the file
>accurately), and the rest of the archive file contains
>the actual archived contents.
>If you create an uncompressed archive, you can just look at
>the layout it creates to get the feel for how it works, as
>all I am doing is running that content verbatim through a compressor.
>
>If memory serves me correctly, if you encounter a code 256
>in the compression stream (a 1 with 8 zeros following), that
>indicates the need to reset back to 9 bits and clear the tables
>and start over.  Code 257 is the end of stream code, and signifies EOF.
>This is BTW the reason that the beginning of all archived files
>starts with >80 as it is the first 8 bits of the 9 bit string
>100000000.  In the case where you encounter code 000100000000
>in the datastream (this is a 12 bit version of code 256)
>you'd need to reset back to 9 bits and start over.
>
>While I did look at both the theory and some implementation...
>With some minor tweaking I was able to use the same code...
>
8<
>Barry Boone and his Archiver was next on the line.
8<

---

From: Jeffrey White <[email-masked]>
Date: Sat, 29 Oct 2011 10:15:32 -0700 (PDT)

Barry Boone released the source for Archiver 3 several years ago at 
dealhunting.com.  The source to Archiver 3 can be found in a zip file at:

ftp://ftp.whtech.com/source/Archiver%203%20source/

Jeff White

________________________________
From: Paolo Bagnaresi <[email-masked]>
To: [email-masked]
Sent: Fri, October 28, 2011 5:11:40 PM
Subject: Re: [TI994A] Archiver format

Hi Barry!

Thanks, Barry.

Paolo

<snip>

---

From: "Paolo Bagnaresi" <[email-masked]>
Date: Sat, 29 Oct 2011 22:13:26 +0200

Thanks a lot Jeff. I did not know Barry had released the source code: it 
could have saved me hours of disassembling... :-)

Very interesting stuff indeed.

Paolo Bagnaresi

----- Original Message ----- 
From: "Jeffrey White" <[email-masked]>
To: <[email-masked]>
Sent: Saturday, October 29, 2011 7:15 PM
Subject: Re: [TI994A] Archiver format

Jeff White

Hi Barry!

Thanks, Barry.

Paolo

<snip>