Table of Contents
Be Developer ClassifiedsBe Developer Classified are an opportunity for Be developers to tell each other about projects they're working on, are thinking about working on, or would like to work on. Here's your chance to request assistance on a project, or make your talents available for other projects.(For more information, check out the Classified Ads section in the Registered Be Developer area.)
Need Some Guidance Please We are looking at porting a commercial applications environment onto BeOS and we need to store application objects (in their various states) on to disk. Does anyone know of the availability of any objects-storage database (such as Versant or Object-Store) on BeOS? Can anyone suggest any alternates for storage of streaming-objects? (other than sequential files, of course.) Appreciate the help. Alok Arora aa@trilogyworld.com "To Be or not to Be" is no more the question.
Be Developer Conference
BE ENGINEERING INSIGHTS: Be Inc.'s "Swapping Bytes, Part III" Seems as if every other week the Engineer Insight article discusses byte swapping. Must be that time because that's my topic today. Up until now we've always written about what one should do, in theory. Finishing up the first Intel release has given us all more practical experience and I'd like to share some of it with you. There are several sections of the API, including the BMessage class and file attributes, where dealing with endian issues might effect your coding habits. I'll discuss the BMessage class in this article and leave the file attributes case for a later article. The BMessage class does lots of work for you in terms of byte ordering. If you are just reading and writing standard data types to BMessages then you needn't fret about byte ordering. By "standard" I mean all the data types (with three exceptions) defined in support/TypeConstants.h. For example, if you have code like the following: // code that writes your data into a message msg->AddString("label", text); msg->AddInt32("weight", weight); msg->AddMessenger("return_ref", some_messenger); // let's save this message to a file msg->Flatten(&some_file); ... // That file can now be moved to another platform. What // happens isn't under your program's control. ... // let's read that message out of a file msg->Unflatten(&some_file); // code that reads your data out of a message msg->FindString("label", &text); msg->FindInt32("weight", &weight); msg->FindMessenger("return_ref", &some_messenger); In the above code you don't have to worry about byte ordering. If the file is moved across platforms the system (the Flatten/Unflatten calls themselves) will handle any byte ordering issues. The same is true if the message is flattened into any sort of buffer and then moved to another platform. An example of this would be if you flattened a message and saved it in an attribute of some file. The three exceptions mentioned above are B_ANY_TYPE, B_RAW_TYPE, and B_OBJECT_TYPE. The first type, B_ANY_TYPE, isn't really a type. The latter two types have arbitrary and undefined formats. They can be anything, so there is no way that the system can properly swap bytes between Intel and PPC platforms. Also, it is the developers responsibility to swap any custom data types that you add to a message. Here's an example: struct my_info { int32 weight; char bx[2]; float percent_done; }; ... my_info info; // info gets filled in with data // now you add data of this type to a message msg->AddData("data", 'myin', &info, sizeof(my_info)); Now if 'msg' is flattened and moved to platform with differing endianness you'll have to deal with byte swapping: my_info info; void *raw; ssize_t size; msg->FindData("data", 'myin', &raw, &size); // raw points to raw/unswapped data, not necessarily a // valid my_info structure. If endianness of this platform // differs from source platform then work needs to be done. One option is to add a field to the my_info struct specifying the endianness of the data as written. The other option is to redo the data structure so that it is endian insensitive. For more details on dealing with vanilla C structures such as my_info see Brad Taylor's Engineering Insights articles in issues 63 and 99 of the Newsletter: http://www.be.com/aboutbe/benewsletter/Issue63.html Another alternative that I'll discuss in detail is using the BFlattenable class. Using this class can make handling custom data structures somewhat easier. Let's redo the above example of a C structure (my_info) using the BFlattenable class: struct TMyData : public BFlattenable { // here's my data int32 weight; char bx[2]; float percent_done; virtual status_t Flatten(void *buffer, ssize_t size) const; virtual status_t Unflatten(type_code c, const void *buf, ssize_t size); // overrides for other functions are omitted to keep example small }; The BFlattenable class gives you nice bottlenecks for moving data in and out of BMessages, files, or file attributes. These bottlenecks give you a framework for dealing with byte ordering. As always you have two options for dealing with this issue:
Write out data in natural order In this scenario the flatten code will save the flag indicating the endianness of the source platform. The unflatten code will compare that against the current platform's endianness and swap when necessary. status_t TMyData::Flatten(void *buffer, ssize_t size) const { char *p = (char *) buffer; // need to save a flag indicating the current platform. So // we remember if this platform is little endian. The // FlattenSize function must account for this extra space. *p++ = B_HOST_IS_LENDIAN; // now save the rest of the data in natural format *((int32 *)p) = weight; p += sizeof(int32); *p++ = bx[0]; *p++ = bx[1]; *((float *)p) = percent_done; return B_OK; } status_t TMyData::Unflatten(type_code c, const void *buf, ssize_t size) { const char *p = (const char *) buf; uint8 endian; bool must_swap; // read the endian flag saved by the Flatten call endian = *((uint8 *) p++); // compared the saved value with current value must_swap = (endian == B_HOST_IS_LENDIAN); // must_swap will only be true in the source and // destination have different byte ordering // now simply read out the data weight = *((int32 *) p); p += sizeof(int32); bx[0] = *p++; bx[1] = *p++; percent_done = *((float *) p); // now swap the data if needed if (must_swap) { weight = B_SWAP_INT32(weight); percent_done = B_SWAP_FLOAT(percent_done); // don't need to swap the bx chars. } return B_OK; } Write out data in canonical format In this scenario the code always writes out the data in little endian format (choosing between big or little endian is up to the developer). In this case an extra "flag" field isn't needed. status_t TMyData::Flatten(void *buffer, ssize_t size) const { char *p = (char *) buffer; // Decided to always write the data in little endian // format. So we'll write data using the HOST_TO_LENDIAN // macros. *((int32 *)p) = B_HOST_TO_LENDIAN_INT32(weight); p += sizeof(int32); *p++ = bx[0]; *p++ = bx[1]; *((float *)p) = B_HOST_TO_LENDIAN_FLOAT(percent_done); return B_OK; } status_t TMyData::Unflatten(type_code c, const void *buf, ssize_t size) { const char *p = (const char *) buf; uint8 endian; bool must_swap; // We know that the data was saved in little endian // format. So read in the data using the appropriate // macros from support/ByteOrder.h weight = B_LENDIAN_TO_HOST_INT32(*((int32 *) p)); p += sizeof(int32); bx[0] = *p++; bx[1] = *p++; percent_done = B_LENDIAN_TO_HOST_INT32(*((float *) p)); return B_OK; } And there you have it. Hopefully this helps clarify how to use BMessages in a multi-endian world. It's the PC thing to do.
DEVELOPERS' WORKSHOP: Of Indexes And entry_refs "Developers' Workshop" is a new weekly feature that provides answers to our developers' questions. Each week, a Be technical support or documentation professional will choose a question (or two) sent in by an actual developer and provide an answer. We've created a new section on our web site. Please send us your Newsletter topic suggestions by visiting the web site at: http://www.be.com/developers/suggestion_box.html.
One of the great features of the BeOS is the ability to query information about files' attributes. This is both useful in programming and for the general user wanting to find files that meet certain criteria. The mechanism behind queries is a series of attribute indices that list the files that can be searched given a particular attribute and value. Indices are volume-specific, meaning that different volumes connected to your machine can have different indexed attributes. Furthermore, an attribute index is only updated whenever a matching attribute is written. So any files on the volume before the index is created are not listed. The BeOS does not step through and examine all of the files on the volume every time an index is created (that could be a lengthy process on a large volume.) This week's sample code allows you to selectively re-index files to make sure that all of their attributes are up-to-date in the attribute indices. Indexer is comprised of a BApplication class that examines entry_refs, various InfoView classes that process and display information about these refs, and some global indexing functions. There is an InfoView for each type of entry: File, SymLink, Directory and Volume. These classes combine to give you detailed information about each dropped ref in addition to re-indexing any referenced files that need it. When a reference to a file, directory, symbolic link, or
volume enters Indexer, more often than not it arrives in the
form of an entry_ref. Full details about entry_refs can be
found in When items are selected and dropped on Indexer's icon or one of its windows from the Tracker, the refs come bundled in either a B_REFS_RECEIVED or B_SIMPLE_DATA message. Both of these messages are similar: The real meat of the message is bundled into the "refs" members. Parsing the message to get the references is the first task. void Indexer::RefsReceived(BMessage *msg) { uint32 type; int32 count; entry_ref ref; msg->GetInfo("refs", &type, &count); if (type != B_REF_TYPE) return; for (int32 i = --count; i >= 0; i--) { if (msg->FindRef("refs", i, &ref) == B_OK) { EvaluateRef(ref); } } } Each entry_ref pulled out of the message is passed along to
status_t Indexer::EvaluateRef(entry_ref &ref) { struct stat st; BEntry entry; if (entry.SetTo(&ref, false) != B_OK) return B_ERROR; if (entry.GetStat(&st) != B_OK) return B_ERROR; if (S_ISLNK(st.st_mode)) return HandleLink(ref, st); else if (S_ISREG(st.st_mode)) return HandleFile(ref, st); else if (S_ISDIR(st.st_mode)) { BDirectory dir; if (dir.SetTo(&ref) != B_OK) return B_ERROR; if (dir.IsRootDirectory()) return HandleVolume(ref, st, dir); else return HandleDirectory(ref, st, dir); } } The first thing we want to do is determine exactly what type
of entry_ref we have gotten. To do this we need to get a
stat structure. The stat structure provides a host of
information about entries, including the flavor of the node,
and creation and modification times. To get the stat we need
an instance of the BStatable class, and BEntry is an easy
one to get. As we want to be able to tell if we have a
symbolic link, we inform We can then proceed to get the stat and discover what type of entry we have. If we discover a file or a symlink, we simple start to handle them. If we find a directory we need to do a little bit more examination to determine whether we have a normal directory or a volume. Volumes are represented as directories that live in the root of the file system. To determine which type we have we create a BDirectory and ask it if it is a root directory. If it is, we handle it as a volume, if not we handle it as a directory. As Indexer is a fairly large bit of sample code, I won't go into all of the details of processing each entry type. Instead I'll simply point out some of the more interesting bits, and leave you to examine the rest of the sample code in more depth later.
Volumes As I mentioned above, I determine that I have a volume when I receive a reference to a root directory. The next step is to discover exactly what volume I am supposed to examine. This is trickier than you might think, as there is not a
convenient status_t Indexer::HandleVolume(entry_ref &ref, struct stat &st, BDirectory &dir) { BVolumeRoster vol_roster; BVolume vol; BDirectory root_dir; dev_t device; while (vol_roster.GetNextVolume(&vol) == B_NO_ERROR) { vol.GetRootDirectory(&root_dir); if (root_dir == dir) break; } // build the info window and the like ... } I then proceed to create my vInfoView, with gathers up a ton
of information about the volume (essentially every detail
that the BVolume class can muster) and displays it the user.
Among the information presented is a list of all of the
indexed attributes on the volume. I collected this with a
global function I defined called
extern status_t get_attribute_indices(dev_t device, BList &index_list) { DIR *index_dir; struct dirent *index_ent; index_dir = fs_open_index_dir(device); if (!index_dir) return B_ERROR; while (index_ent = fs_read_index_dir(index_dir)) { char *text = strdup(index_ent->d_name); index_list.AddItem(text); } fs_close_index_dir(index_dir); return B_OK; } This function opens the volume's index directory, gets the name of every attribute found there, adds them to the list, and then closes the directory. Note that it is very important to close the index directory, or you will be unable to unmount its volume. You might also notice that I did not traverse the entire volume indexing every file. I could easily do this, but I figured that it made more sense to just display a list of all of the indexed attributes instead. The code could easily be modified to do so by borrowing the appropriate functions from dInfoView.
Directories I do traverse directories and re-index all of the files
found there, as well as collect a bunch of information about
the contents of the directory. The two functions of most
interest in the dInfoView class are the void dInfoView::TraverseDirectory(BDirectory &dir) { entry_ref ref; while(dir.GetNextRef(&ref) != B_ENTRY_NOT_FOUND) { EvaluateRef(ref); } }
void dInfoView::EvaluateRef(entry_ref &ref) { struct stat st; BEntry entry; if (entry.SetTo(&ref, false) != B_OK) return; fEntryCount++; entry.GetStat(&st); if (S_ISLNK(st.st_mode)) fLinkCount++; else if (S_ISREG(st.st_mode)) { if (ref.device != fRef.device) { fInvalidCount++; return; } fFileCount++; BNode node; if (node.SetTo(&ref) != B_NO_ERROR) { fInvalidCount++; return; } status_t status = B_OK; status = reindex_node(node, *fIndexList); if (status == INDEXED) fIndexed++; else if (status == PARTIAL_INDEXED) fPartialIndexed++; else fNotIndexed++; } else if (S_ISDIR(st.st_mode)) { BDirectory dir; if (dir.SetTo(&ref) != B_OK) { fInvalidCount++; return; } if (dir.IsRootDirectory()) fInvalidCount++; else { fSubDirCount++; TraverseDirectory(dir); } } }
Symbolic Links Symlinks are by far the easiest class to deal with. A
BSymLink is created and information about it is displayed.
The main bit of information is whether the link is relative
or absolute, and the actual path to the linked to node. This
information is retrieved with the I'll also note that all of the InfoViews collect some generic information about the entries they receive, namely the creation and modification time of the entry, its path, and its name. Finally I'll note some peculiarities about processing links. When the Tracker bundles up a B_SIMPLE_DATA message to drop on the window of an application, it simply gets the entry_ref of all of the items selected. When dropping onto the icon of an application, however, the Tracker looks at a lot of information about the file, in an attempt to discover whether the application knows how to handle that type. In the process of doing this, the entry_ref for a link is traversed. This means that to get an lInfoView in Indexer you need to reference the link through the command line, or to drop the link onto a window. The current behavior is being looked at internally, and we hope to have some sort of resolution for Release 4.
Files Finally we come to the heart of the original program design,
the re-indexing of a file. To complete this action we need
to get a list of the attributes that are indexed on the
volume (the extern status_t reindex_node(BNode &node, BList &index_list) { attr_info info; status_t status = B_OK; int32 to_be_indexed = 0; int32 indexed = 0; int32 not_indexed = 0; int32 size = 1024; char *value = (char *) malloc(size * sizeof(char)); //rewrite all of the appropriate attributes for (int32 i = 0; i < index_list.CountItems(); i++) { char *attr = (char *) index_list.ItemAt(i); if (node.GetAttrInfo(attr, &info) == B_OK) { to_be_indexed++; // adjust the size of our static buffer if necessary if (info.size > size) { value = (char *) realloc(value, info.size); size = info.size; } if (node.ReadAttr(attr, info.type, 0, value, info.size) > 0) { if (node.WriteAttr(attr, info.type, 0, value, info.size) > 0) indexed++; else not_indexed++; } else not_indexed++; } } free(value); value = NULL; if (to_be_indexed > 0) { if (indexed > 0) { if (not_indexed > 0) return PARTIAL_INDEXED; else return INDEXED; } else return NOT_INDEXED; } else return NOT_INDEXED; }
A running count of the matching attributes and whether they were successfully rewritten is kept, and the function returns a status detailing whether the file was indexed successfully, partially, or not at all. The fInfoView also compiles a list of all of the file's
attributes for display purposes. void fInfoView::GetAttributes(BNode &node, BList &list) { char attr_buf[B_ATTR_NAME_LENGTH]; node.RewindAttrs(); while (node.GetNextAttrName(attr_buf) == B_NO_ERROR) { char *string = strdup(attr_buf); list.AddItem(string); } } As a last note on files, the fInfoView also discovers the
MIME-type of the file through use of
Finally, some last notes, suggestions, caveats and plans. There are a series of volumes that cannot be accessed through the Tracker (such as /dev or /pipe) that information can be gotten about through the command-line. Take a look at a couple of them and see what can be seen (although not much will be seen in the virtual file systems in the way of files, the info windows usually fail.) I also want to note that the code for displaying the information is probably not how the interface guys would want to see it done. Use the sample code to look at the Storage Kit classes, not as a pristine example of Interface Kit code. I'll continue to update Indexer in the future, and I would love to hear of any improvements you make. Upcoming features might include code to remove and create indexes, as well as a look into whether passing BEntrys or entry_refs is more efficient. In the meantime, Indexer can be found at: ftp://ftp.be.com/pub/samples/storage_kit/obsolete/Indexer.zip. I'll be back next week with Victor Tsou from the Doc Team. We'll be filling you in on many of the things you'll want to know about the upcoming Release 3 release of the BeOS for Intel. Until then...
Driver EducationThis isn't about my college-bound son and the education of our auto insurance company. It's an attempt to put some thoughts on electronic paper concerning the sea of driver trouble ahead of us. Once upon a time, we had totally proprietary hardware. This was at the very beginning of the company, when multiprocessor hardware was the province of high-end servers and workstations, not down to the $2000-$3000 level as it is today (see the Ming specials at http://www.be.com/support/guides/ming-specials.html). In the beginning, we were in the position of having to design I/O cards and write the drivers. Now though, we are in the retroactively obvious situation where we can benefit from other people's investment in chipsets, motherboards, and I/O devices, and focus our efforts on the OS. But the abundance of riches comes at a price: how can we get drivers to support this wonderful wacky world of PC add-ons? Put another way, some observers have attributed OS/2's failure to a paucity of drivers. Customers eager for an alternative to Windows were disappointed to find that their interface cards and peripherals weren't always supported by IBM or by the hardware add-on manufacturer. Are we going to disappoint hopeful BeOS users in the same way? Are we going to bleed to death attempting to write drivers or cajoling vendors for support? The hidden pivot in the argument is the comparison to OS/2. If indeed, we come out of nowhere representing ourselves as a general-purpose OS ready for mainstream consumption, then we face an impossible challenge. Put another way, OS/2 positioned itself as an alternative to Windows and failed to deliver what turned out to be an impossible proposition, better DOS than DOS, better Windows than Windows. The BeOS is not a general-purpose OS, it is a complement or a supplement to Windows. The BeOS coexists with the general-purpose Windows as a specialized OS, just as Linux does. Fine, but what does it mean for the sea of drivers issue? First, it means we have to keep setting expectations, just as we did in earlier days with tongue-in-cheek but serious surgeon general warnings: this product is unfit for consumption by normal humans. It still is and, just like Linux, will be for a long time -- if not for ever. As an industry such as ours mature, it will diversify, segment and specialize, and the thought that any OS has to be or do "everything" doesn't have to apply. More specifically, as we represent ourselves as the media OS, best fit for real-time WYSIWYG, the reality-based positioning, a Be, Inc. exclusive, translates into a focus set of hardware targets. This, in turn, translates into a finite set of drivers. That's what we have to define, that's what we have to communicate.
BeDevTalk SummaryBeDevTalk is a discussion group that's a forum for the exchange of technical information, suggestions, questions, mythology, and suspicions. In this column, we summarize some of the active threads, listed by their subject lines as they appear, verbatim, in the group.To subscribe to BeDevTalk, visit the mailing list page on our Web site: http://www.be.com/aboutbe/mailinglists.html
Subject: 128 file descriptors The file descriptor limit (128) was less controversial than the "how many threads should I spawn?" subtopic. A statement by Dominic Giampaolo... "Unless you know what you're doing, creating more threads than there are CPUs in the system is probably not wise." ...was jumped on (and, despite Sander Stoks plea for caution) taken out of context. The specifics of an app (what it's doing, how fast it needs to do it) will temper Dominic's rule of thumb. For example (from Peter Mogensen)... "The reason I have to use lots of threads is that I have a lot of identical objects and I *need* one 'concurrency context' per object. ...and for now I do *not* care about performance." Client management (one thread per client) was also cited as a legitimate use of "too many" threads.
Subject: Interface issues; AKA: /File menu More menu talk: Should all documents have a main menu bar, and should that bar have a reliable set of submenus? (And should those submenus have submenus? etc.) Should the user be able to choose the menu style as an emulation of some other OS (gimme Mac menus... gimme Windows menus... gimme Amiga menus)? Taken out of context, Tyler Riti supplied the quote of the week: "People think that if they can't see something, then it isn't there." (Is this not so?) The discussion teetered on the verge of the old Dionysian/Apollonian argument that pits GUI customizability against consistency. Despite the high noise level, a number of valid suggestions were made, some of them bordering on the interesting. "The global be_app variable could return a pointer to a BMenu or BMenuItem allowing the programmer to add application menu items or submenus to it's Icon/Label in the DeskBar." (Scott Ahten) "Somebody should create a set of standard translations [OK, Cancel, Open, etc.] that everyone can put in their apps to provide at least _some_ internationality." (Sean Gies) "A floating app-level menu is not a terrible idea...we could have application level menus that are only visible when the application is in the foreground." (Claude I. Denton) "Make every toolbar dockable. If it's floating, all of the app's windows use the floating toolbar; if it's docked to a part of the window, each window has its own copy." (Andi Payn) And so on.
Subject: How can I add icons to resources? You make an icon-sized image in IconWorld and add the image file to your BeIDE project expecting to be able to access the data as a resource of the app. But you can't. What's wrong? Brian Stern offered the following IDE guidelines: "You need the resource to be in a file whose target info indicates that it has resources. The default is for any file with an extension of .rsrc to have its resources copied into the executable during link. Just add the resource files to the project and make. You can add as many resource files as you like to a project. If that didn't work then either the target info in the Target prefs panel doesn't indicate that that file kind has resources, or there are no resources in the files." But, as Wendell Beckwith pointed out, IconWorld saves images as attributes, not resources. Mr. Beckwith has created an Attribute Convertor app that converts attributes to resources for just such situations.
Subject: FTP SO_REUSEADDR -- is it necessary? What is it? Scott Andrew says... "[SO_REUSEADDR]... is what allows many users to connect the same IP. It's not necessary for FTP... [but is for a server]." The right spirit, but, according to Howard Berkey, the wrong letter: "You can still have many clients connect() to the same IP address/port without using SO_REUSEADDR by using accept() and listen()." Agreement all around.
Subject: Be Newsletter Volume 2, Issue 7 -- February 18, 1998 Pixel/coordinate confusion. Remember: 0+1 == 2. A rectangle that has a top left at pixel (0,0) and a bottom right at (1,1) touches 2 pixels on each side -- but, nonetheless, it's a 1x1 rectangle. We've seen this discussion before, but never without some objection to the status quo coordination. You're all getting soft.
1997 Be Newsletters | 1995 & 1996 Be Newsletters |