Be Newsletter

Volume II, Issue 14; April 8, 1998

Table of Contents

• Be Developer Classifieds
• BE ENGINEERING INSIGHTS: How To Be Lazy...., By Dominic Giampaolo
• DEVELOPERS' WORKSHOP: Scripting Power and Wisdom, By Doug Wright
• Where is it?, By Jean-Louis Gassée
• Be Dev Talk Summary By Doug Fulton

Be Developer Classifieds

Be Developer Classified are an opportunity for Be developers to tell each other about projects they're working on, are thinking about working on, or would like to work on. Here's your chance to request assistance on a project, or make your talents available for other projects.

(For more information, check out the Classified Ads section in the Registered Be Developer area.)

Graphics Driver Project Sought

Is there currently a project in development for creating an ATI Mach64 graphics driver for the BeOS? Jeff Weeks is interested in beginning or assisting on such a project. If anybody has any information on this please contact Jeff at info@codex.nu (or pweeks@execulink.com if you have trouble with the first address).

Service Available: BeOS Programmer for Short-Term Contract

BeOS Programmer with BeOS Masters Honorable Mention available for short term contract June 15th to September 1st. Please e-mail Bjorn Tornqvist pt96bto@student.hk-r.se for more information.

BE ENGINEERING INSIGHTS: How To Be Lazy....

By Dominic Giampaolo

The best programmers I know write the least amount of code.

Seriously, being able to bang out thousands upon thousands of lines of code is worthless if you're constantly re-inventing the wheel. In fact, programmers that don't know how to reuse other people's work usually get less done.

This article is about how to be lazy and get other programmers to do your work for you. Aside from being a hustler and duping unsuspecting interns into doing your job, this form of "laziness" can take many forms.

The first form of laziness is to know what tools are available to you as a programmer. That means knowing what functions are in the libraries and how to use them. I have seen programmers re-implement such basic functions as scanf() or strtol() because they didn't know that these functions existed in the libraries. I would have written that off as an anomaly but I've now seen it numerous times. It definitely pays to buy a standard C library and POSIX manual and know about what functions you have at your disposal.

Here are a few of my favorites:

• strerror() - returns a string that explains an error code. Instead of popping up a dialog saying error -2147459069 occurred you can say "No Such File or Directory." There is a string for every single BeOS error code.
• sscanf() is really powerful for parsing strings. It is a little difficult to use but it's worth the effort. scanf() has features to skip over optional parts of a conversion or it can only read in a specific number of characters of a string.
• On a related note, the vsprintf() and vsscanf() forms of sprintf() and scanf() can be very handy if you want to write wrappers functions for printf() and scanf() (for example, to format text that will eventually wind up in a dialog box). Knowing how to use the var-args versions of library functions can save you time.
• strotl(), strotul() and strtod() - these functions convert strings to integers or doubles in any base (decimal, octal, or hex). There are also versions for converting to long longs.
• strftime() is a super-powerful way to format dates and times. The other standard C library date & time routines (localtime, difftime, asctime, ctime, etc.) offer lots of features as well.

The Be Kits also offer you features that can save you time. Nosing around in the BeOS Support Kit turns up a few very useful classes and convenience functions (BStopWatch, UTF conversion functions, error reporting/assertion macros, BAutoLock, etc.). Poke around and see what you find. It's worth it to spend an hour or two reading documentation and header files to know what's available.

Beyond the standard C library and the Be Kits there are large bodies of code available on the 'net that implement various algorithms or protocols. Searching around with a search engine usually turns up something. Sometimes you get what you pay for and other times you really do find a gem that saves you weeks of time.

Now if all my article did was to exhort people to call scanf() and to look on the 'net for code, it would hardly be much of an article (although it is late and ValŽrie is hanging around with the shears). So let's dig in to the real way to be lazy: Getting other programs to do your work for you!

Often times there are command line programs that do something useful but have no UI and are difficult to use. These programs usually languish, crying out for a graphical savior to free them from Terminal hell so that mere mortals will use them. The unenlightened programmer will toil tirelessly to re-implement the entire command line program with a GUI. The lazy programmer will simply write a graphical wrapper that emancipates the poor command line app. The lazy programmer will also be done long before the unenlightened programmer.

Even full-blown Be application programmers can be lazy. For example, there may be a command line app that performs a nifty feature you'd like to include in your application but you don't have the time or inclination to implement it yourself. Knowing how to run a command line program and manipulate its output may save you time and make your product better.

For example, the BeOS Expander app uses popen() to communicate with zip, tar, and friends so that it doesn't have to parse their file formats -- and it will be a snap to add other file formats in the future.

Here are some other ways in which lazy command line program thievery can be useful:

• communicating with the command line ftp program to enable a "save to server" feature
• using a command line tool like "diff" to implement a complex algorithm so that your app can display and modify the results in a pretty manner
• communicating with complex text-filtering scripts in Perl or awk
• filtering images using the NetPBM tools

Now that we've seen why you might want to be lazy, let's talk about how to be lazy. Just as in real life, there are different levels of laziness. The most sloth-like approach is for an app to just call the POSIX function system(). The function system() takes a string and makes the shell execute it. You don't get any direct output from the command and in fact all you do get is a status value as returned by the shell that executed the program. The system() function is handy because you can specify a full command line (including pipes, input and output redirection, etc.). The downside is that system() is synchronous so be sure to spawn a thread to do the dirty work.

The next step up the ladder of indolence is to call popen(). popen() lets you specify a full command line, just as you would with system(), plus it provides you with a pipe to either the input or the output of the program you are running. For example you might ask popen() to give you the input pipe to a program that performs a transformation on data that you write to the pipe. On the other end, you may want the output pipe of a program that prints some information that is needed in your program. The popen() call is very powerful.

Climbing another notch up the scale of torpidity you may find that you need even finer grained control over the program that popen() launches. This may be necessary if you want to have a button in your GUI to pause the operation or to abort it entirely. For this you want the program you launch to be in its own process group and you need to know the thread ID to send the signal to.

The following version of popen() provides the necessary control:

FILE *
my_popen(const char *cmd, const char *type, long *tid)
{
  int p[2];
  FILE *fp;
  char *args[4];

  if (*type != 'r' && *type != 'w')
    return NULL;

  if (pipe(p) < 0)
    return NULL;

  if ((*tid = fork()) > 0) { /* then we are the parent */
    if (*type == 'r') {
      close(p[1]);
      fp = fdopen(p[0], type);
    } else {
      close(p[0]);
      fp = fdopen(p[1], type);
    }

    return fp;
  } else if (*tid == 0) {  /* we're the child */

    /* make our thread id the process group leader */
    setpgid(0, 0);

    if (*type == 'r') {
      fflush(stdout);
      close(1);

      if (dup(p[1]) < 0)
        perror("dup of write side of pipe failed");
    } else {
      close(0);

      if (dup(p[0]) < 0)
        perror("dup of read side of pipe failed");
    }

    close(p[0]); /* close since we dup()'ed what we needed */
    close(p[1]);

    args[0] = "/bin/sh";
    args[1] = "-c";
    args[2] = (char *)cmd;
    args[3] = NULL;


    execve(args[0], args, environ);
  } else {         /* we're having major problems... */
    close(p[0]);
    close(p[1]);
  }

  return NULL;
}

Now if your graphical front end wants to pause the sub-process, it could do:

   kill(-tid, SIGSTOP);

and that will pause all the programs in that process group (which is everything descended from the fork()'ed thread). The kill() function is the (poorly named) way to send signals. Using the negative of a thread id sends the signal to all threads in that process group. A process group is defined when a thread calls setpgid() as our example does above. A process group is a handy way to manipulate entire groups of threads.

If you wanted to continue the operation after pausing it, you would use:

   kill(-tid, SIGCONT);

If you just want to abort the whole process, you can use:

   kill(-tid, SIGINT);

Notice that we use SIGINT so that we give the programs a chance to clean up after themselves. If we used SIGKILL the program would be killed unceremoniously and could leave turds around the system.

The function my_popen() handles quite a number of situations where you're communicating with a command line program. But it's not quite enough in all cases.

Some heavy-duty programs like Eddie and Pe have a worksheet feature that requires controlling both the input and output of a program. The top rung of the lethargy ladder requires a bit more work but provides the most control over the sub-process. With two pipes, your program can feed input to a command-line program and at the same time, read the output >from that program. Setting this up is only moderately more complicated than what my_popen() does and is left as an exercise to the non-lazy reader (those who are smart will go look at pages 442 and 443 of the Stevens book "Advanced Programming in the Unix Environment"). Controlling both the input and output of a command line program lets you run interactive programs such as ftp and manipulate them to do your bidding.

In summary, what I've tried to present in this article are options that are open to you as a BeOS programmer. These options are useful but not always appropriate. There are times when you just want something to work quickly and the methods I've suggested here can help you get there. And then there are other times when simply running a command line app doesn't get you the right granularity of control. You have to make the decision based on your constraints. Regardless of your final decision, it pays to be aware of the options.

DEVELOPERS' WORKSHOP: Scripting Power and Wisdom

By Douglas Wright

"Developers' Workshop" is a new weekly feature that provides answers to our developers' questions. Each week, a Be technical support or documentation professional will choose a question (or two) sent in by an actual developer and provide an answer.

We've created a new section on our web site. Please send us your Newsletter topic suggestions by visiting the web site at: http://www.be.com/developers/suggestion_box.html.

Jan's Deli makes great meatloaf sandwiches. They also have a bunch of random other things lying around like chips and cookies and candy bars. Last time I was there they had a new impulse-buy-product in front of the deli counter: Power and Wisdom bars. The Power and Wisdom bars come in some pretty strange flavors like Orange-Ginger-Jalepeno. This is of course the new-age aware version of the plain old chocolate Powerbar. So what does this have to do with Scripting? Well, in order for scripting to be a really useful tool, it needs to have both power and wisdom, but hopefully with good taste as well.

A few short weeks ago, I gave an overview of our scripting architecture and a code snippet for filling up a list with some useful information about the scriptability of running applications: www.be.com/aboutbe/benewsletter/volume_II/Issue10.html#Workshop

The power of the Be scripting architecture is its ability to control other applications' objects. The wisdom is in the way you can find out about what "suites" any particular object supports so that you know how to talk to it. So as I promised, I've put together a sample app called Scripter that fills an outline list view with running applications: ftp.be.com/pub/samples/application_kit/Scripter.zip

For each app you get a listing of its windows, and for each window a listing of its views. If you double click on one of these objet d'app, a dialog is opened with a list of supported suites for that object and some controls.

When the dialog is opened, a BMessenger is constructed to point at the object that was clicked on. The controls in the dialog allow you to enter specifiers for a message to be sent to the object. It works much like attribute queries in the find panel in Tracker. You start with one specifier, and to add more click on the add button.

The specifiers are added to the message in the order they appear, which is very important to the way scripting works. As a scripting message is passed from one object to the next, specifiers are popped off the "specifier stack."

  BMessage suite_msg(B_GET_PROPERTY);
  suite_msg.AddSpecifier("Suites");
  suite_msg.AddSpecifier("View", "myView");
  suite_msg.AddSpecifier("Window", "myWindow");

This message will get the Supported Suites for a View named myView in a Window named myWindow, but only if you send it to an application with a Window named myWindow. If you send this message directly to a Window, you will get a reply message that says "Specifier not understood," because Windows don't have a specifier called "Window".

In order to make it easy to try this app out, I made a little app called FIR. What is FIR for? Well, it's a simple Finite Impulse Response low-pass filter. It attenuates frequencies above a certain cut-off frequency. FIR has one slider in its window that controls the cut-off frequency. BSliders inherit from BControl and BControls have a supported suite of type suite/x-vnd.Be-control (surprise). So you can use Scripter to send supported messages to FIR to get or set the value of the slider and therefore the high frequency output of the filter. BControls also support messages to change the label, so you can try that out too.

You can easily define your own suites of scripting messages that your objects will understand and respond to. Scripter will come in handy for testing scripting capabilities that you add to your own app. In an upcoming article I will demonstrate the creation of a suite. Until then, look out for those wacky new-age "nutrition bars"!

Where is it?

By Jean-Louis Gassée

The downloadable version of Release 3 for Intel, that is. Wouldn't it be a good idea, since we are born on or of the Web, to deliver the product via the Web just as we take orders for it? After all, many companies -- including Be developers -- do just that.

ESD, Electronic Software Distribution, is an important part of our business model. It's the great equalizer that saves software developers from the ruinous fight for shelf space and lets small start-ups compete on (more) equal terms with the establishment.

So why don't we practice what we preach?

There are two answers to this very sensible question. One has to do with size, the other one with the complexity of the installation process in an ecumenical context -- on a hard disk where valuable OS, programs and even more valuable data already exist.

Let's start with size. On the BeOS Release 3 CD there are two partitions, one for BeOS tools for Windows, about 5 megabytes, the other one for the BeOS installation, about 290 megabytes. If we look at the details, we see seven folders:

  /apps    -- 11.4 MB
  /beos    -- 40.7
  /demos   --  2.6
  /develop --  8.6
  /home    --  1.4
  /preferences -- 0 bytes for 18 files, and...
  /optional -- a mere 202 MB of sounds, movies, and GNU goodies

So why not jettison the optional directory and compress the rest for a manageable 30 MB download? After all, downloading a new 20 MB Web browser doesn't constitute an insurmountable problem.

Unfortunately, there are some hidden but photodegradable assumptions here. A browser, once you install it, immediately does something useful: It feeds off the contents of the Web and needs nothing else. For the BeOS, we need to create a good initial user experience, that's why we include as much material as possible with the operating system itself. We can't (yet) rely on a huge "web" of applications "out there." Not everyone inside Be agrees with me, some of my colleagues think we ought to package a simple trial download for users interested in getting a quick taste of our product and a feel for its speed. We'll keep you posted as we debug our thoughts on this topic.

The other issue is the complexity of the tasks involved in downloading and installing into the Windows world. In the current scheme, you have to use some Be Tools to create a BeOS partition, and then you boot from the floppy packaged with the CD. The computer "bites" on the floppy, the bootloader reads from a BeOS partition on the CD, and the installation procedure is reasonably smooth.

The "off-the-Web" version of the story is more complicated. Again, you have to create a BeOS partition -- so far we're even -- but then you have to download another DOS or Windows program to create a floppy for the bootloading phase. Of course, without a CD the bootloader can't work directly with a BeOS partition. Instead, it must reconstitute the BeOS image by translating a downloaded Windows file (or files) >from the Windows partition into the freshly made BeOS partition.

We feel the CD process is much simpler and safer, and much more conducive to a happy "out-of-the-box" experience. The partitioning exercise is trouble enough.

While we don't yet offer the "instant" (after a few hours on-line) gratification of a Web download, you can order the BeOS on our Web site or get it via snail-mail, try it, and get a cheerful -- really, we can't afford unhappy customers -- refund if we don't meet your needs or expectations.

BeDevTalk Summary

BeDevTalk is a discussion group that's a forum for the exchange of technical information, suggestions, questions, mythology, and suspicions. In this column, we summarize some of the active threads, listed by their subject lines as they appear, verbatim, in the group.

To subscribe to BeDevTalk, visit the mailing list page on our Web site: http://www.be.com/aboutbe/mailinglists.html

Subject: Interrupting real-time processes

High priority threads are necessary and useful -- but what do you do when, for example, you launch a high-priority app accidentally? Peter Morgensen suggests that the OS include some sort of panic switch that lets you kill processes that have taken over the computer.

Chris Herborth rightly points out that a high-priority-process killer would have to run at an even higher priority than its victims, which could lead to problems; he urges, instead, that app developers use real-time priority threads (priority 100+) sparingly and with caution:

"If you're running your whole app at B_REALTIME_PRIORITY, you're a bozo... there's no need for the UI to run at such a high priority, and your apps won't run faster by doing this."

Polite use of system resources is a nice ideal, but not a law. What is a user supposed to about badly written apps? Mr. Morgensen again:

"What if the [Ctrl-Alt-Delete] key combination brought up a shutdown/task manager dialog and suspended all max-priority threads?"

And Mr. Herborth replies:

"I don't think the OS should be wasting cycles babysitting B_REALTIME_PRIORITY threads. Real realtime OSes don't have a way of killing a berserk high-priority thread...they assume that if you're creating a high-priority thread, you know what you're doing."

Mr. Herborth went on to suggest "floating" priorities, imposed by the scheduler, as a way of controlling "berserk" processes. Other listeners offered ways in which developers can detect and correct CPU-greedy processes. Osma Ahvenlampi suggested a CPU-greed registry:

"[A] 100+ thread [would need] to run a benchmark estimating its CPU requirements on the system and register itself to the kernel..."

Marco Nelissen responds:

"Sounds like too much trouble. It would seem much easier to simply have an 'infinite' priority thread somewhere in the system that waits for a certain hotkey, and then 'halts' the system and lets you kill processes. This ability to kill off runaway threads, combined with some simple checking in the applications themselves would be more than sufficient, I think."

Can the system gauge the requirements of each thread and schedule accordingly? Is it sufficient for a developer to monitor CPU use to ensure that a thread never takes more than, say, 90% of the machine? What happens when multiple 90% apps are running at the same time? How can an app accurately check its CPU usage? Should Casey Fleser pack it in and become a farmer?  

Subject: User friendly CTRL-ALT-DELETE
AKA: Successful installation on ASUS P55T2P4

Should Control-Alt-Delete prompt the user to save changes (etc.), or should it immediately pull the plug? Dave Peverley suggested that an initial 3- finger salute could put up a confirmation panel, and a second would confirm. This should be enough to separate the sheep from the chaff.

Eric Shepherd pointed to an anointed fly:

"If someone hits Ctrl-Alt-Delete, it's likely because the system is hosed beyond recovery, and in that case, odds are good that the app server isn't stable enough to present the UI you suggest."

But shouldn't the system at least TRY to be nice (asks Timothy Ritchey)? Should the cache, at least, be saved? Mr. Shepherd thinks not:

"My concern is that if the system is so bad off that you need to Ctrl-Alt-Delete to fix it, nothing on the system is necessarily reliable anyway, so you're better off losing it."

And Jon Watte piles on:

"I agree. Especially when you know that the major part of your file system will still be there -- it's not like UNIX where you can lose your superblock and be out to lunch. I think many of the reactions people have against hard reset come from experiences on previous file systems."

Speaking of the file system, Timothy Ritchey asked...

"Does the BFS, or other [BeOS] systems suffer if the system is shut down improperly?"

And Dominic Giampaolo answeredÉ

"Nope. BFS is a journaled file system which means that you can reboot in the middle of writing to the disk and the disk is not corrupted. Of course you might lose data that was sitting in the cache..."  

Subject: Media OS, a fake?

Dirk Olbertz want to know if "Media OS" is anything more than a white paper buzzword...

"...how does an OS influence Multimedia-Things? Aren't the apps responsible for the functionality? Is speed and a filesystem with 64-bits enough to make an OS a Media OS?"

Dominic Giampaolo pointed out ways in which the OS can help (or, better, get out of the way of) an app:

"...if the OS doesn't offer very low-latency scheduling (like the BeOS does) then you won't be able to respond to events quickly enough when they happen.

...if the OS doesn't offer you the functionality you need, your app may not even be possible.

...if the operating system has dumb limitations about file sizes or the number of processors it supports then your app gets harder to write."

Blake Harris adds...

"Performance is the difference. The same functionality in different OSes does not produce similar results."

And so on.  

Subject: Strange memory problems

While porting GhostScript, Sander Stoks ran into a problem with large chunks of memory. Odder still, he was able to crash "cat" when catting to a printer dev. Anybody want to offer a prognosis? A cure? Commiseration. Willy Langeveld and Olaf Seibert have seen the cat-crasher. Chris Herborth's system "hangs after _heavy_ prolonged swapping."

Fred Fish's suggestion:

"Check to see if it uses alloca. Perhaps it is exceeding the available stack space. If it does use alloca, make sure that it is using the C based alloca that should be included in the source, rather than the stack based one built into the compiler."

In response to Mr. Stoks' assumption that malloc() isn't the culprit, Olaf Seibert offered this:

"Well, I did find a problem with malloc(), in combination with fork() and a BWindow...[T]he thread that calls fork() is instantiated into the new team. That part works great...If the child team then calls exec() to start a new program, also fine. What does *sometimes* not work is calling malloc() in between."

Dominic Giampaolo responded to the bug report:

"The problem you're experiencing is an artifact of how some internal stuff is handled in the C library. We plan on changing this behavior for other reasons and the outcome is that your test program will work."

Mr. Seibert also offered this puzzler:

"Suppose I start the same app once in the background and once in the foreground, as in

bash$ app &
bash$ app

Now I use both of them for a while. Perhaps I get them to fork(), I'm not sure if that is necessary. Then I quit the second (foreground) app. Now the shell remains waiting for dying teams. Only if I quit the first (background) instance of the app the shell is satisfied in its wait."

In the meantime, Sander "Slidey" Stoks posted updates on his situation (no, GS doesn't use alloca, or fork from a thread; the filesystem isn't corrupt and malloc seems to be fine).