Be Newsletter |
Volume II, Issue 27; July 8, 1998 |
Table of Contents
- BE ENGINEERING INSIGHTS: Porting WinSock Applications to the BeOS, By Russ McMahon
- BE ENGINEERING INSIGHTS: Outsmarting the Scheduler, By Ficus Kirkpatrick
- DEVELOPERS' WORKSHOP: Slumming It, or Making the Most of a db Situation, By Victor Tsou
BE ENGINEERING INSIGHTS: Porting WinSock Applications to the BeOS
By Russ McMahon
The Windows Sockets API, or just WinSock, has become synonymous with Microsoft Windows-based network programming. There are literally thousands of applications that have used WinSock to communicate over TCP/IP networks. If the application is on NT, Win95, or Windows and it's connected to the "net" it's a sure bet the application is using WinSock.
This article is a primer on porting the WinSock portions of Microsoft Windows code over to the BeOS. So now is the time to talk to your Win32 developer friends (admit it -- you have a few) and get that cool Internet application ported to the BeOS.
First, a bit of history. Back in the dark ages of Windows development (3.0) there was a major push to get PCs connected to each other, not just in the local area but also to the larger world outside. TCP/IP and a much smaller Internet already did just that, but not for Microsoft Windows users.
Enter the TCP/IP vendors. Each offered a Windows-based TCP/IP stack, but with no a standard API to program to, developers had to choose which vendor to go with, and often had to support multiple TCP/IP stacks. Then along came the WinSock specifications, a single networking API that all the major stack vendors including Microsoft agreed to. Application writers could now program to one API and have the application run on any vendor's TCP/IP implementation. The dark ages of network programing for Windows were over, sort of.
The WinSock specification was based on the Berkeley Sockets interface popular in the BSD (Berkeley Software Distribution) version of UNIX. But UNIX is a multitasking system, and blocking a process does not totally hog the CPU and keep it from running other processes. When WinSock was created, however, Windows was a non-preemptive, single-tasking OS. Blocking a process completely locked up the system, and the only remedy was the infamous hard reboot.
Developers went to great lengths to yield control to other processes. Asynchronous APIs and message callbacks that would yield the CPU and be good citizens in the Windows world were built into the WinSock specifications. They were given the prefix WSA, for Windows Sockets Asynchronous. Many developers still use the asynchronous calls even now, although NT and WinX are preemptive multitasking systems that support blocking calls without locking up the entire system.
BeOS Networking, like WinSock, modeled the programming interface after Berkeley Sockets to provide developers with a well-defined and understood programming interface. The BeOS, however, is a preemptive multitasking system, so there was no need to include the asynchronous calls provided in WinSock. Developers writing network applications for the BeOS simply used the Sockets interface, as described in many UNIX books, to connect to TCP/IP networks.
It's interesting to note that both BeOS and WinSock provide application support for sockets in dynamic link libraries. For WinSock the calls are in WinSock.dll or WSock32.dll, with WinSock.h being the main header. With BeOS networking the calls are in LibNet.so and the main header file is Socket.h.
Where am I going with this? Well, it should be fairly clear that the two interfaces have a lot in common, so much of the porting will be just copying the network chunks of code and recompiling. The biggest obstacle is getting around a WinSock API beast called WSAAsyncSelect( ). This is the call WinSock developers use to yield CPU control during long network activity. The basic function of WSAAsyncSelect() is to give the TCP/IP stack enough information to notify the network application of some network event of interest. The main parameters given to WSAAsyncSelect() are a callback procedure handle, and flags that define the network events the application is interested in being notified about.
Okay enough background. Now let's do a simple port, nothing fancy or overly ambitious this time -- we'll save porting the sample HTTP server for a future article. For this port I grabbed a WinSock sample application off the MSDN (Microsoft Developers Network) CD. The sample is a simple server that listens and accepts client connections. It receives a small string sent by the client, returns the string, and closes the client connection.
I ported the sample by directly copying much of the networking code into my BeOS application, then used threads instead of WSAAsyncSelect() to handle the client connections. I purposely left out error return codes and much of the "nitty gritty" detail to focus on the porting task at hand. For testing the server I used Telnet to connect to the example server, then sent a small string. For example,
$telnet 10.12.13.45 50000 checking example sever ip: 10.12.13.45 port: 50000
As can be seen below, the code that opens a socket, binds to a specified port, and then listens for incoming client connections is almost identical in both WinSock and the BeOS. I removed the typedefs, but the actual types are the same.
A quick scan of WinSock.h tells all. If the code is built and run at this point, the server will be up and running, and listening for incoming client connections. Port 50000 is just a random listening port number I picked. In the listen call I use a backlog of 5, fine for this example. Getting Windows code running on the BeOS is breeze!
/********** WinSock sample window procedure **********/
LONG APIENTRY MainWndProc(
HWND hWnd, /* window handle */
UINT message, /* type of message */
UINT wParam, /* additional information */
LONG lParam) /* additional information */
{
SOCKADDR_IN local_sin;
SOCKADDR_IN acc_sin;
SOCKET sock;
switch (message) {
case WM_CREATE: {
local_sin.sin.sin_family = AF_INET;
local_sin.sin.sin_addr.s_addr = INADDR_ANY;
/* hard coded port, example only */
local_sin.sin.sin_port = htons(50000);
sock = socket(AF_INET, SOCK_STREAM, 0);
bind(sock, (struct sockaddr*) &local_sin,
sizeof(local_sin));
listen(sock, 5);
/* tell us when a client has connected to our */
/* listening socket. */
WSAAsyncSelect( sock, hWnd, WSA_ACCEPT, FD_ACCEPT);
break;
}
case WSA_ACCEPT:
/* wParam is the listening socket */
int len = sizeof( acc_sin );
sock = accept( (unsigned int) wParam,
(struct sockaddr*) &acc_sin, (int *) &len );
WSAAsyncSelect( sock, hWnd, WSA_READ,
FD_READ | FD_CLOSE );
break;
case WSA_READ:
char recvbuf[ 256];
if (WSAGETSELECTEVENT(lParam) == FD_READ) {
int cnt =
recv((unsigned int) wParam, recvbuf, 256 , 0);
recvbuf[cnt] = '\0';
send((unsigned int) wParam, recvbuf,
strlen(recvbuf), 0 );
closesocket((unsigned int) wParam);
}
else /* FD_CLOSE */
closesocket((unsigned int)wParam);
break;
...
} /* switch */
}
Remember that I said the WSAAsyncSelect() call was an obstacle to our port plans? Well, here is where we climb over it. We need to take the networking functionality from the Windows procedure and map that to the BeOS.
In the Windows sample, multiple client connections are handled in the WSA_ACCEPT message. Accept() returns the client connection and then WSAAsyncSelect() sets up a callback for received data. In WSA_READ the data is received and then sent back to the client.
Threads are used to do the same activity in the BeOS. The accept() call blocks, and when a client wishes to make a connection, a thread is spawned and passed the new connection. The new connection is passed by value to ensure the connection does not get over-written during the spawning of the thread. This means the connection is an integer pointer, and needs to be freed in the thread. Within the thread the client data is received and sent back. After the client connection all client activity is controlled in the thread.
/********** BeOS Networking **********/
...
struct sockaddr_in local_sin;
struct sockaddr_in acc_sin;
int sock;
int* acc_sock;
thread_id new_thread;
local_sin.sin.sin_family = AF_INET;
local_sin.sin.sin_addr.s_addr = INADDR_ANY;
local_sin.sin.sin_port = htons(50000);
sock = socket(AF_INET, SOCK_STREAM, 0);
bind(sock, (struct sockaddr*) &local_sin,
sizeof(local_sin));
listen(sock, 5);
for (;;) {
int len = sizeof(acc_addr);
acc_sock = (int*)malloc(sizeof(int)); // do not overwrite
*acc_sock = accept(sock,
(struct sockaddr*) &acc_addr, &len);
new_thread = spawn_thread(new_accept, "new accept",
B_LOW_PRIORITY, acc_sock);
resume_thread(new_thread);
}
/* thread function */
int32 new_accept(void* data)
{
char recvbuf[256];
int acc_sock;
acc_sock = *((int*)data);
free(data);
int cnt = recv(acc_sock, recvbuf, 256, 0);
recvbuf[cnt] = '\0';
send(acc_sock, recvbuf, strlen(recvbuf), 0);
closesocket(acc_sock);
}
Well, that's it for now, other duties call. I know I went through the information pretty quickly, so if you have any questions, concerns, or ideas about porting WinSock apps to the BeOS, please let me know.
BE ENGINEERING INSIGHTS: Outsmarting the Scheduler
Operating systems that can execute multiple tasks simultaneously generally do it in one of two ways -- "The Your System Freezes When You Do Something As Ridiculously Simple As Starting A Program Way," and "The Right Way," respectively. It was a tough decision, but BeOS implements the latter of the two.
This is done by asking the system to wake up the kernel every so often in order to make scheduling decisions. When the system starts up, we program a timer to deliver an interrupt to the CPU at some interval, so the kernel can check on the system and crack some heads if the situation demands it. We'll call this the clock interrupt.
Under BeOS, the clock interrupt occurs once every millisecond. The scheduler checks to see if any threads are ready to wake up from a snooze() call or if it's time to preempt a thread that's been running too long. So, if you try to snooze for 250 microseconds, you might not actually wake up until 1 ms later, plus any context switching overhead. If you're lucky and you were put on the sleep queue 250 microseconds or so before the next clock interrupt -- and no one else at a higher priority is waiting to run -- you'll wake up at the time you expected. Unfortunately, though, you can't depend on it.
Why doesn't Be increase the frequency of the clock interrupt? Because there's a fine line between more accurate snoozing and spending all your CPU cycles deciding whose turn it is to run, and we'd like to investigate some more before changing the behavior of the scheduler. So for the time being, you're stuck with 1 ms snooze() granularity. That is, of course, unless you have a free spirit, a free evening, and a free timer interrupt.
Well, I had both a timer interrupt and a Newsletter article to write, and so I give you the "wakeup" driver: ftp://ftp.be.com/pub/samples/drivers/obsolete/wakeup.zip
When opened, this driver will program the real-time clock on the ISA bus to begin generating an interrupt every 244 microseconds. Your user-level thread that wants to wake up this often does this:
int fd = open("/dev/misc/wakeup", 0);
set_thread_priority(find_thread(NULL),
B_REAL_TIME_PRIORITY);
while (...) {
read(fd, some_buffer, 0);
do_fun_stuff();
}
close(fd); /* don't forget! */
Despite the misleading name, calling read() on the wakeup device does not actually return any data, but rather blocks the reading thread on a semaphore. When the driver receives a timer interrupt, it releases this semaphore. Then, courtesy of the reading thread's real-time priority, the thread is immediately unblocked and you can do_your_fun_stuff(), after having waited only a quarter of a millisecond.
Unfortunately, there is a problem. Interrupt handlers are supposed to call release_sem_etc() with B_DO_NOT_RESCHEDULE as one of the flags. Normally, releasing a semaphore gets you rescheduled, but it isn't safe to be rescheduled inside an interrupt handler.
Consider the following course of events: Your handler is called by the interrupt dispatcher. It releases a semaphore, causing a reschedule. The user program that has the device open is switched to, and it calls close() on the device's file descriptor.
Since no one else has the device open at this point, the driver is unloaded by the kernel. Some time later, the thread that your interrupt handler was executing in is scheduled. The code it was executing in is now unloaded, leaving it with an invalid instruction pointer. Since it is running in kernel space, the entire system will almost certainly crash.
In order for the driver to work, the thread that handles the interrupt has to be rescheduled when it releases the semaphore. Otherwise, the interrupt handler just returns and, that's right, the reader thread can end up waiting an entire millisecond to be scheduled, which doesn't help.
The only 100%-reliable way to solve this problem is to commit a gross hack. It may not be what you wanted to do, but outsmarting the scheduler is already experimenting on the dark side. So, how do you guarantee that your interrupt handler will have somewhere to land when it returns from release_sem? Well, you have to leak some memory. Here's the solution:
/* globally declared */
static uchar *hack_area;
static uchar springboard[] =
"\xb8\x00\x00\x00\x00" /* mov eax, ... */
"\x50" /* push eax */
"\xe8\x00\x00\x00\x00" /* call ... */
"\x58" /* pop eax */
"\xb8\x01\x00\x00\x00" /* mov eax, 1 */
"\xc3"; /* ret */
/* in wakeup_open() */
if ((aid = find_area("wakeup hack")) < 0) {
/* if the area doesn't exist, create it */
create_area("wakeup hack", &hack_area,
B_ANY_KERNEL_ADDRESS, B_PAGE_SIZE,
B_FULL_LOCK, B_READ_AREA|B_WRITE_AREA);
} else {
/* otherwise just reuse it -- we are only single-open */
get_area_info(aid, &ai);
hack_area = (uchar *)ai.address;
}
/* copy the springboard code into the area */
memcpy(hack_area, springboard, sizeof(springboard));
/* patch the springboard code to have the relocated address
* of release_sem, as well as the correct sem_id to pass to it
*/
*(sem_id *)(hack_area + 1) = wakeup_sem;
*(uint32 *)(hack_area + 7) =
((uint32)release_sem - (uint32)hack_area - 11);
What we've done is copied the machine code for a "springboard" into release_sem to a locked area, and patched that code with the correct sem_id and address of release_sem. The interrupt handler releases the semaphore through the springboard rather than directly to release_sem(), and we have a guaranteed place to return to, even if the driver has been unloaded.
An unfortunate side effect of this is that we have to write the interrupt handler in assembly language:
__declspec(naked) /* don't create a stack frame */
static bool
wakeup_handler(void *data)
{
asm {
/* acknowledge the timer interrupt */
push RTC_REGC
call rtc_read
pop eax
/* jump to the release_sem springboard */
mov eax, hack_area
jmp eax
/* we should never reach this point, as
* the springboard should send us back
* to our caller
*/
ret
}
}
Because we have to restore any registers that were modified inside the interrupt handler in the springboard code, we can't leave it to the compiler's discretion to decide which ones to save and use in the interrupt handler.
Another important aspect of making this work is not creating a new stack frame in the interrupt handler. This will make the "ret" instruction in the springboard code jump back into the tender, loving, and most of all, *safe* hands of the interrupt handler's caller. It's a horrible hack, it leaks a page of locked memory, but desperate times call for desperate measures, and at least we reuse the page on subsequent opens.
This is only one example of outsmarting the normal clock interrupt, though. You could make an ultra-snooze driver that gives you one-off high-granularity sleeping, or you could put more logic into the driver for deciding when to wake up the reader. For example, you could set the RTC interrupt rate to 114 microseconds and only release the semaphore on every third one.
There are also alternate approaches to this periodic wakeup application. In my implementation, the driver makes an attempt to let the reader catch up if it is unable to keep pace with the interrupts. You may want to guarantee the reader a minimum sleep time. This is easily implemented by setting a flag in read() to indicate that someone is waiting, and checking that flag in the interrupt handler before releasing the semaphore.
Careful readers will note that this driver will not work on the PowerPC because of its use of Intel assembly language. However, with some modification, you should be able to make it work on a BeBox. It won't work on a Mac without more modifying, because the Mac has no ISA bus, which is where the real-time clock we use to generate the interrupts sits. But if you know of a timer I haven't found, or you have some hardware that will generate the interrupts for you, the modifications you'll need to make are trivial.
If you're interested in learning more about the scheduler, Cyril Meurillon, Master of the Be Kernel Dominion, wrote about it in Issue 37 of the Be Newsletter:
http://www.be.com/aboutbe/benewsletter/Issue37.html#Insight
The need for better timing without hacks like this is a known deficiency, but things will improve for R4. Look for detailed information about this in a future Newsletter.
Finally, this example is meant to be a mere starting point for exploration. I'd be interested to know what you find out along the way.
DEVELOPERS' WORKSHOP: Slumming It, or Making the Most of a db Situation
By Victor Tsou
http://www.be.com/developers/suggestion_box.html.
Last night, while out taking my daily constitutional, I
found myself face to face with none other than the esteemed
Dr. B. Banner.
"Good evening," I greeted my erstwhile neighbor, who grunted
a curt response. "You're looking a bit green tonight; I hope
nothing's wrong," I added, somewhat alarmed by his bilious
coloring.
"If you must know, I'm having trouble debugging my gamma ray
modelling program," he said irritably. I trembled as his
clothes seemed to shrink. "The lack of a source level
debugger is making me positively ANGRY," he continued.
I quickly administered first aid, giving him a few tips on
debugging user-level code and soon he was once again the
good doctor we all know and love, anxious to return to his
research. In the hope of averting any future developer
transformations, Dr. Banner has generously shared his
personal notes of our conversation.
The debug server provides the default user-level debugger on
the BeOS. The debugger can be called in a number of ways:
The debugger is somewhat more capable than its kernel-level
counterpart; for example, it allows you to set breakpoints
and single-step. For a full list of available commands, type
"help" at the debugger's command prompt.
While it only displays information at the assembly level,
the debugger understands symbols in xMAP files. xMAPs for
the system libraries are included in the /optional folder of
the installation CD and should be copied to /system/lib. To
generate xMAPs for your own images, pass the "-map" option
to the linker followed by the name of the xMAP to generate
(typically the application name followed by .xMAP). If
you're using the BeIDE, you can achieve the same effect by
checking the "Generate Link Map" linker option in the
project settings dialog.
While debugging, you'll typically know the name of the
function you're examining (by looking up the symbol
associated with the current eip/pc) but not the precise
source line. Again, for those who read the article
concerning debugging device drivers:
http://www.be.com/aboutbe/benewsletter/volume_II/Issue21.html#Workshop,
the combination of the -g and -machinecodelist mwcc options
will reward you with an interlaced source/assembly dump of
your code, allowing you to quickly associate eip/pc values
with source lines.
Along similar lines, the pedump (for x86) and pefdump (for
PowerPC) programs disassemble and report import, export, and
dependency information for executables, add-ons, and
libraries. These tools are xMAP-aware and their output is
more readable when the appropriate xMAPs are present. The
import/export information is particularly useful and may be
used to quickly check if a program was compiled correctly.
For example, if the Tracker refuses to load an add-on, a
quick check with pedump/pefdump may reveal an improperly
exported process_refs(). The dependency information can help
identify the libraries an application requires; no longer
shall the "Not an executable" error message leave you
scratching your head.
The /bin directory contains a variety of additional tools
helpful for debugging while your application is running.
Among these are several programs that display information
about kernel constructs. A sampling of the available
programs and the information they reveal:
Jean-Louis Gassée is hard at working researching the
usefulness of BeOS cooking software in Alsace, France. A
wine cellar inventory application may also be required...
About This Space...
1997 Be Newsletters |
1995 & 1996 Be Newsletters
Copyright ©1998 Be, Inc.
Be is a registered trademark, and BeOS, BeBox, BeWare, GeekPort, the Be logo and the BeOS logo
are trademarks of Be, Inc. All other trademarks mentioned are the property of their respective owners.
Comments about this site? Please write us at webmaster@be.com.