Be Newsletter

Volume III, Issue 20; May 19, 1999

Table of Contents

• Be Developer Classifieds
• BE ENGINEERING INSIGHTS: Network File Systems for BeOS -- The Art of Being Paranoid, By Alfred Landrum
• DEVELOPERS' WORKSHOP: Big Bad Voodoo Teapot, By Mikol Ryon
• PC Expo in Five Weeks, By Jean-Louis Gassée

Be Developer Classifieds

Be Developer Classifieds are an opportunity for Be developers to tell each other about projects they're working on, are thinking about working on, or would like to work on. Here's your chance to request assistance on a project, or make your talents known to other projects.

For more information, check out the Classified Ads section in the Registered Be Developer area.

BeOS Job Openings: GUI and AUDIO Developers

Are you a BeOS developer who likes games? Wouldn't it be cool if you could do something to help bring more commercial games to BeOS? Help Next Generation Entertainment bring new gaming excitement to BeOS?

NGE is looking for BeOS developers to fill out its team; we'll be producing professional BeOS versions of new commercial games (and even some that aren't released yet). Here's what we're looking for:

• programmer/GUI specialist
• programmer/audio specialist
• programmer (several of these)

Note that these are part-time (evenings and weekends) positions, not full-time office jobs. Luckily, you can do some sweet work on BeOS in your spare time because of the nice APIs.

Our new team members will have experience with some (or all if you're a real winner). They are listed in order of importance:

• programming BeOS (duh!)
• POSIX/standard ANSI C library
• high-performance graphics
• realtime audio
• TCP/IP networking
• porting software to BeOS
• the Standard C++ Library

You'll also have to be able to work independently with a small team over the Internet.

Email your resume to me (Chris Herborth; arcanedragon@home.com), along with a cover letter telling me what unique experiences and skills you'll bring to NGE. Please send these in HTML, plain text, or PostScript (if you send all three I'll laugh at you).

BeOS Job Opening: Graphical and UI Applications Developer

Full-time help needed in developing 3D, graphics, and streaming media applications. Looking for experience in graphical and UI applications development, and of course, interest and experience with the BeOS. Located in Issaquah, in the Seattle area. Want to be more than an enthusiast?, let's talk!

Send resume to rodneys@zipcon.net

Rodney Smith
Distributed Objects Technology
425.392.7962

Writing a network file system for the BeOS is an interesting challenge. Our network stack is in user space, and file systems are handled in kernel space. This will change in the as-yet uncode-named Release 5, when the network stack will go into the kernel. But until then, you must straddle this fence between kernel and user space. How do you do this? Since you'll have code in both places, you'll need a way to communicate between them. One way to do this is with ports, which I'll use as an illustration. With your code separated, you must pay special attention to various types of failures and be able to handle them.

I've written a trivial network file system to show how to make network calls from the kernel (or anywhere, as you'll see). This file system, the "one file network file system" (OFNFS), consists of one file. The OFNFS listens to a specified UDP port for broadcasts, and changes its name and modification time. OFNFS is just simple enough for me to write quickly (and get back to writing CIFS -- more later), and just complex enough to show the extra worries that network file systems bring. Oh, and I'd also like to say that the communication package OFNFS uses is from one of our Be developers, Andreas Huber. Andreas has just made his NFS source code available on Beware, so grab that as well if you're into networked file systems.

So, download the source code from <ftp://ftp.be.com/pub/samples/add-ons/ofnfs.tgz>, and follow along -- I've tried to make copious comments in all the right places. Notice that the file system consists of two programs: the file system add-on and the user-level stub that makes socket calls. The user level stub is called ksocketd, and it must be running for the file system to communicate with the network. A named port is used to transfer data between the add-on and ksocketd. The add-on makes calls like 'ksocket' and 'krecvfrom', which behave functionally identically to 'socket' and 'recvfrom'. We use the 'param' parameter with the mount system call to pass in the port to use. We start up our network connection, and then spawn a kernel thread that will listen for packets on our UDP port. When we get a packet, we rename our file to whatever string we find in the packet. When we're told to unmount, we signal our listening thread to die, and then exit.

Simple, eh? Ah grasshopper, but what do you do when things go wrong? The number one priority of a net fs developer should be handling error conditions. So -- what could go wrong? Well, in the case of OFNFS, a great deal. Surprised? Let me count the ways: What happens if you get a packet that contains '\\\\\\'? This is definitely not a legal file name. What happens if the net server (our blessed network stack) gets restarted? You'd need to recreate your socket, and rebind to it. Hmm -- what happens if it crashes? You'll have to be prepared for your socket calls to start returning error messages. And what if ksocketd gets killed? In that case, you're effectively cut off from communicating with the network. No problem for OFNFS, you just don't get name changes anymore. But what if you're a not-so trivial file system, like... CIFS!

Allow me tell you about the current work on CIFS. In case you don't know, CIFS stands for "Common Internet File System." It's the fancy name for the protocol that Microsoft products use to share files, locate network resources, and other things. You may also know that a software package called Samba allows Unix machines to communicate with Microsoft boxes. (The name "Samba" comes from the fact that the protocol has traditionally been known as SMB, or "Server Message Block.") There will be a CIFS add-on in the experimental folder of Genki, which will let you use files from Microsoft or Samba-enhanced Unix machines. Use and abuse.

Getting back to the point, how do you handle errors in a "real" network file system? In short, "as gracefully as possible." Let's start with a specific example. Say your NT box dies a horrible, bluishly tinted death while you're in the middle of transferring a file. First off, how do you know when the server has crashed? You'll to need to set some timeouts on all your network transactions, so that you'll notice when your file server isn't responding. Meanwhile, you can start passing out errors as if they were fruitcakes. The user-level application should close all file descriptors to you, allowing the user to unmount your now worthless file system. Maybe you're optimistic; you'll think about reconnecting to the server. You'll have to log in again, so you need to whatever authentication info you were given at mount time. You also need to think about a reconnect policy -- do you want to try to reconnect every time you think there's a serious error? Since reconnections have large latencies, this could prove extremely aggravating to the user. Think about limiting your number of retries, or limiting the number of retries you'll make in some limited time frame. (That is, you can retry three times in a 10-second period, then wait ten seconds. Insert your heuristic here.)

In conclusion, here are some tips on network file systems:

1. Since you'll probably cache some kind of information on the server, keep an eye on your memory usage. The kernel exports some useful memory calls: smalloc, sfree, scalloc, etc. The extra "s" here is for "swappable"; normally, memory allocated in the kernel is locked down and never paged out. Memory returned by these calls can be swapped out.
2. The kernel debugger is your friend. If you write a network file system (or any file system for that matter), you'll develop a very intimate relationship with it. For info on the kernel debugger, please read "Welcome to the Cow...Debugging Device Drivers," by Victor Tsou <http://www.be.com/aboutbe/benewsletter/volume II/Issue21.html>.
3. Be careful reading in packets. Perform sanity checks on data you receive. (Example: I had a bug in CIFS, caused by reading data from the wrong place in a packet, that led to attempts to malloc huge pieces of memory. It is unwise to blindly malloc 2GB.)
4. Be militantly paranoid about reading strings. Do not use any of the normal string functions; always use the number-limited versions (i.e., strncpy, strncat...). If you walk past the end of your string, you will crash the kernel. B DONT DO THAT.
5. Be prepared for everything to die. ksocketd could die. The net server could restart, or it could just up and quit. If you end up waiting for a response from the network without having some kind of timeout, you will block forever in the kernel. This is a Bad ThingÂ[TM]. Your file server could die, and never get back to you. Or, it could die, and come back on-line, and start claiming it doesn't know you. You'll have to reauthenticate to it. Your cache information is probably stale now.
6. Think ahead about what kind of failure policy you want to enforce; a little bit of design here goes a long way.

Since this is my first newsletter article, I'd appreciate it if you'd drop me a line telling me if it was at all helpful to you. Thinking about writing a network file system? Let me know. (Please understand that time is a precious commodity between now and golden master day, though.)

"Developers' Workshop" is a weekly feature that provides answers to our developers' questions, or topic requests. To submit a question, visit

http://www.be.com/developers/suggestion_box.html.

One of the questions most asked often of DTS lately is, "How do I make my OpenGL app use hardware acceleration?" -- or some variation thereof. OK, here's how. If you're running Genki 5 or 6, with your trusty Voodoo2 card installed, along with the glide2 package from the optional folder of the Genki CD, you're ready to get started.

I have a pair of simple examples here, which allow you to test your setup and which show you the basics of hardware acceleration. To do this, I've taken the GLTeapot example code straight off the Release 4 CD and made minimal changes, accelerating from ~40 fps to ~200 fps. That's cool. You'll find the code at <ftp://ftp.be.com/pub/samples/open gl/GLTeapot-Voodoo.zip>. All changes are labeled "voodoo" for easy viewing.

The two big alterations to GLTeapot are adding full screen mode for hardware acceleration and changing the resize mode of our view from B FOLLOW ALL to B FOLLOW NONE. This is so that our view resizes to the size of the window. With Voodoo hardware, we must be in 800x600 or less to get hardware acceleration. You'll see that most of the code changes simply involve my adding the fancy Alt+F switch and menu option to go from window mode to full screen mode and back.

Another quick example is converting Michael Morrissey's 3DLife <http://www.be.com/aboutbe/benewsletter/volume_II/Issue34.html> to make use of Voodoo hardware. The code for this is at <ftp://ftp.be.com/pub/samples/open_gl/3Dlife-Voodoo.zip>. The changes here are a bit more extensive, but they're still relatively minor. In lifeView.cpp, we again change the resize mode of our view to B FOLLOW NONE. In lifeWin.cpp we change our main window to a BDirectWindow, and enable DirectMode with mv->EnableDirectMode(true). Also in lifeWin.cpp we implement lifeWin::DirectConnected and pass them along to our view's implementation of DirectConnected.

The only other changes are adding the menu option and hot switch to switch between full screen and window mode.

This has been a short article, but I hope it answers one of DTS's most frequently asked questions.

A quick look at my calendar tells me that PC Expo opens in less than five weeks. I love trade shows -- they're tiring, expensive, and, at times, bewildering. But how else can we get to see so many customers, developers, business partners, members of the commentariat, and competitors large and nimble in so little time? Combine all that with New York City and its robust culture, and you realize how unreal life can be in Silicon Valley.

For us, though, the main point of PC Expo will be to show how the BeOS incarnates the MediaOS concept. The classical trade show booth usually revolves around one larger-than-life demo, plus some smaller stations for showing particular products or aspects of what the company offers to individuals and smaller groups. For this PC Expo we have more and more spectacular demos of new applications from noted and soon-to-be-notable developers, along with some new twists (perhaps literally) on our own apps.

I still remember what an executive of a respected microprocessor company said when seeing a BeOS demo for the first time: I didn't know my PC could do that. That's one of the messages we want to broadcast as loud and on as large a screen as show regulations and our budget will permit -- within the limits of good taste, of course. As our departed Sales and Marketing VP Alex Osadzinski used to say: No objectified humans and no race cars. I quote him with both license and prudence.

Moving to the rest of the booth, we want to broadcast another message: Here is what you can do. From one station to another you'll be able to use applications to actually create, edit, produce, post-produce, merge, fold, staple, and mutilate digital media. We hope that at the end of the tour you'll go home with a sample of your work using tools made by BeOS developers. Work and fun rolled into one. Speaking of the latter, we'll also have the latest games available on the BeOS.

So, please come along. If you're a developer, we'd love to hear from you and see how we can show off your work. If you're already a BeOS user, please join us in celebrating what's happened since we first demonstrated the x86 version of the BeOS. And if you're not a BeOS user yet, let us give you some good reasons to make room on your hard disk for the Media OS.

Watch our web site for details and/or e-mail us. We look forward to seeing you in NYC.