Be Newsletter

Volume III, Issue 44; November 3, 1999

Table of Contents

• BE EVENTS: Comdex Fall '99
• BE DEVELOPER CLASSIFIEDS:
  I. Computer Inspirations Announces Release of BeCalc
  II. Le Buzz announces"Studiolab"
• BUSINESS & MARKETING: Major Chain Stores Carrying BeOS Software
• BE ENGINEERING INSIGHTS: IMAP for the Masses, By Adam Haberlach
• BE ENGINEERING INSIGHTS: I Wrote a Media Node, and It Was OK, By Victor Tsou
• DEVELOPERS' WORKSHOP: Translation Kit Add-ons, By Daniel Switkin
• More on FAT By Jean-Louis Gassée

Comdex
November 15 - 19
Las Vegas, Nevada
Booths S1753 and NS89

Register for a FREE exhibits guest pass compliments of Be Inc. - A $100 value. Just enter the guest ticket number 34065 in the Priority Code box on this page:

<http://www.comdex.com/comdex/owa/register0?v_event_id=289>

I. Computer Inspirations Announces Release of BeCalc

BeCalc, a desktop scientific calculator with support for programmer functions, plots, mortgages, statistics, complex numbers, history lists, drag 'n drop, and much more was released October 30, 1999 by Computer Inspirations. For details visit <http://www.comp-inspirations.com>.

II. Le Buzz announces "Studiolab"

A new test facility will soon be releasing regular product reviews. "Le Buzz StudioLab" will combine dedicated BeOS test computers, provided by BeComputing, with a fully-equipped production studio, to test new hardware and software products. The results will be reported on a new "StudioLab" web page <http://www.thestudiolab.com/>.

Details on how to submit products to StudioLab for review can be obtained by e-mailing Dane Scott at dsuden@netnet.net.

Will your commercial product be ready soon?

Contact reseller@be.com when your boxed product is ready for distribution.

I get a lot of mail -- work related, private mail, automated bug reports, correspondence with old friends, and those unsolicited advertisements that we all love so much. I separate work and private mail by using two different accounts. At work, I use BeMail and break it down with queries to break it down (usually the breakdown is between "Unread" and everything else). I confess that at home I store my mail on my headless Linux machine, filter it with procmail, and read it with mutt (via telnet from my BeOS machine, of course). Why? Because I can connect to the machine from anywhere on the planet (at least anywhere that I can use ssh to get through Brian's gestapo firewall) and read my mail remotely.

All this is a result of the way that mail is usually handled. One of the older mailbox protocols on the net is POP, or Post Office Protocol. It's simple and reliable, but it's really only designed to let you store mail on a server temporarily, until you can download and read it. There is no provision for storing messages on the server, much less sorting, categorizing, or shuffling them.

Enter IMAP -- the Internet Message Access Protocol. Designed and proposed in late December of 1996, it provides an interface for remotely storing and retrieving e-mail messages. However, since it's complicated compared to POP, it doesn't seem to be widely implemented. At least not on BeOS, the only operating system that really matters in this context.

An Internet protocol is usually defined in a "Request for Comment," or RFC. This document is a proposal by someone that explains the goals of their protocol and methods for implementation. IMAP version 4 is documented in RFC 2060, available at <http://sunsite.auc.dk/RFC/rfc/rfc2060.html>, and finer RFC mirrors everywhere. It may help if you read along.

I started this project with the goal of creating a set of objects that I could use to represent the bits of data and processes that need to be shuffled around in order to deal with IMAP. I looked through the specification to find the best way to break down the process into units of information and the processes that can be done to them. After scanning the RFC, I came up with the following group of objects. I'll list them in simplest to most complex order, because many of them are defined in terms of each other and I wish to avoid forward declarations.

** Flag

IMAP messages and mailboxes can have a set of flags associated with them. These can be used both by the user and by the system. In addition, flags can either be "permanent," in which case they'll be preserved from session to session, or "temporary," in which case they'll only be valid during the current session. Other than that, there isn't much to do with a flag object, except own it.

** Message

IMAP messages consist of an RFC-822 (you know where to look it up) or MIME message, with an associated 32-bit unique identifier. The server must be capable of providing several different representations of the message. For now, this system just deals with the raw message. Somewhere around here is a set of classes for parsing messages, but they don't belong to me. I'll pressure the author or something.

** Mailbox

IMAP mailboxes contain lists of messages and a unique "validity ID" used to avoid collisions. The server also provides a "Next UID" which can be used by clients to create new messages. I simply use a BList to keep track of the messages (or at least the Message IDs) in the mailbox. In addition, mailboxes can be heirarchial -- but I've chosen not to implement this feature of IMAP for the time being.

** Connection

This is where the magic happens. A Connection is responsible for connecting to the server, logging in, handling the listing of mailboxes and their messages, and eventually the downloading message bodies. It needs to know information like the hostname, username, and password. It also maintains lists of mailboxes, as well as an API for dealing with them. This is the most complicated object, containing functions for opening a network stream, composing requests, and parsing the results. It also uses a BList of Mailbox objects to track the boxes for the current session.

An IMAP transaction consists of a request tagged with an ID, followed by one or more reponses from the server, terminated with a final response which is tagged with the original tag from the request. The tag can be any set of alphabetic or numeric characters, and should be different for every transaction. The IMAPConnection class has a private function to generate tags based on a counter, by appending the counter's value to the word "KRIMEZ" (for historical reasons) and then incrementing the counter. A typical transaction works like this:

Client sends a request tagged with a string: KRIMEZ3 LIST "" * Server responds with an untagged response: * LIST (\NoInferiors) NIL INBOX Server ends the transaction with the tagged response KRIMEZ3 OK LIST completed

IMAP servers are allowed to respond with any set of valid responses they wish. This means that the parser must be able to deal with just about anything, and figure out quite a bit based on its idea of the current state of the connection. I've chosen to have a single function, HandleResponse(), called after every command is sent. The function reads lines from the network stream and deals with them until it detects a tagged response. Each line is identified and dealt with accordingly. Many commands are simply ignored, as they require no acknowledgement and we are not interested in them. Some commands contain information, which we grab, parse, and store.

After that, this project is pretty much just a parser and a network wrapper. I'm running out of space, so I'll hope that the code speaks for itself. You'll find it at <ftp://ftp.be.com/pub/samples/network_kit/IMAP.zip>

This is a minimal video producer node. It reminds me of Whack.

My contribution to society: <ftp://ftp.be.com/pub/samples/media_kit/VideoProducer.zip>

The life cycle of the node, in terms of the hooks that are called, looks something like this:

1. VideoProducer()
2. NodeRegistered()
3. Preroll()
4. HandleStart()
5. PrepareToConnect()
6. Connect()
7. Disconnect()
8. HandleStop()
9. ~VideoProducer()

You should not rely on steps 3 through 8 being called in any particular order (or at all).

The binary belongs in /boot/home/config/add-ons/media. You can see its output with CodyCam.

---

"Developers' Workshop" is a weekly feature that provides answers to developers' questions or topic requests. To submit a question or suggestion, visit: <http://www.be.com/developers/suggestion box.html>

---

Writing an add-on for the Translation Kit, like writing most software, involves laying some groundwork before you get to the good stuff. For that reason, I've written a framework into which you only need drop code which is specific to the format you want to support. The rest is done for you. Pick up the archive here:

<ftp://ftp.be.com/pub/samples/translation_kit /TranslatorTemplate.zip>

If you're eager to get started, you can pretty much skip the rest of this article. Just get the code, and search for !!!, which will indicate where you need to make changes. Please note that the code as is will compile and run, but not do anything useful, since the encoding and decoding routines have been left blank (they're up to you).

For the purpose of illustration, I've picked an up and coming image format, MNG. This should not be construed as a hint that we're going to support it.

The main steps you need to take are:

1. Define some strings and constants, as follows:

   #define NATIVE TRANSLATOR ACRONYM "MNG"
     #define NATIVE TRANSLATOR FORMAT 'MNG '
     #define NATIVE TRANSLATOR MIME STRING "image/mng"
     #define NATIVE TRANSLATOR DESCRIPTION "MNG image"
     #define copyright string "© 1999 Be Incorporated"
  

2. Add controls to the TranslatorView if you have any user selectable options.

3. Modify the Identify() function to recognize images in your format.

4. Write the encode and decode routines.

5. Rename the executable as appropriate (MNGTranslator, for instance).

That's all there is to it. A couple of things to note:

1. As a good translator, this framework provides an app and a window should the user run the add-on directly. This is not much extra work, but is useful if there are options to set.

2. Translating from and to the same format is done using a dumb copy between the input and output streams. This was the source of a bug in two of the Be-supplied translators, and has been fixed.

3. Invoking this translator with a type constant of 0 will output a B TRANSLATOR BITMAP, as explained in my previous article.

4. The layout and sizing of configuration views has been standardized in the next release. This code conforms to that look.

Last week's column on peaceful coexistence brought more e-mail than I expected. I thought the amusing situation I reported was just that, an anecdote illustrating how well we managed to exist on the hard disks of PC clones next to the Windows partition. Your Windows system refuses to work, so boot BeOS and we might help you retrieve your data and/or edit system files. The usual caveats are hereby stipulated, as counsel recommends, and we have no plans at this time to start a Windows utilities business.

After several readers and BeOS users reported similar experiences, I went to the in-house gurus and asked why we succeeded where our worthy elders failed. What seemed initially like a happy accident now looked like a systematic feature our FAT file system extension; I wanted to know where and why our implementation was superior.

Victor Tsou, the engineer who implemented this file system extension, put it more modestly. He thought it might be imprudent to call our FAT module "better." We certainly don't have (yet) the benefit of the same large scale testing as the Windows FAT implementation does. Ours is a different implementation. We use a different approach to achieve the same goal of safely reading and writing FAT files. The tests we perform might exhibit subtle differences here and there. As a result, in a small number of cases, we might be able to read FAT files that Windows can't read. Conversely, in a similarly small number of instances, Windows might be able to read FAT files that BeOS can't read.

In other words, either system might fail to read .02 percent of FAT files, but not the same .02 percent. (I use .02 percent as a place holder; the actual number is much smaller.) Statistically, Windows is called upon to read and write FAT files much more often than BeOS. As a result, Windows' "bad" .02 percent is more frequently seen than our own "bad" .02 percent.

None of this diminishes the merit of our ability to coexist nicely with a FAT file system. It's a very pleasant feature, one that I personally enjoy greatly, but we don't need to harm a good piece of engineering by exaggerating its reach. And while we're on the subject of reach and file systems, our readers know that we support more than FAT; see <http://www-classic.be.com /support/qandas/faqs/faq-0259.html> for more. We might even claim to have both a very robust 64-bit journaling native file system with BeFS and a very (the most?) extensive set of external file system plug-ins. This is consistent with a philosophy we share with many: facilitate the movement of data, avoid roach motels.

Lastly, a note I received from Russia, from Max Bazarov, a.k.a. Baza, as he calls himself:

If you like Norton Commander...I think...final release of BeFar will be interesting for you :)

http://www.beos.ru/projects/befar/
http://www.beos.ru/projects/befar/shot_0_0_6.gif

gl, Max Bazarov ( Baza )
BeNews.Ru Editor,
http://www.benews.ru/
baza@benews.ru

I used the URL and, yes indeed, a BeOS version of Norton Commander appeared on my screen. Now I'm fond of Norton Commander. Not the least reason for this affection concerns the time in 1995, when I upgraded my home PC from Windows 3.11, the one that really worked, to Windows 95. I'd heard all the stories about the end of the "Windows on top of DOS" era. With 3.11, you booted DOS or, in my case, launched Norton Commander at the end of the AUTOEXEC.BAT file. Then, you could forage around, or type WIN to launch Windows. With Windows 95, Microsoft declared this ancient history, presenting Windows 95 as a totally integrated product.

So -- I upgrade my system and, after a number of restarts, it reboots for good and.... I see the familiar Norton Commander. I type WIN and Windows 95 starts. I restart, same story. This time, I type VER, for the DOS version, and get a Version 7.0 reply. The upgrade process preserved my AUTOEXEC.BAT file, complete with NC as the last two (visible) characters. This, in turn, exposed the Win 95 marketecture.