Be Developers' Conference Technical Discussion of the New Media Kit Jon Watte

Jon Watte: Hi, everyone, welcome to the more technical overview of what the new Media Kit is and what it's going to do for you. I hope everybody is comfortable, I hope you had a good lunch, the house lights are dim, good seats, technical droning, if anyone needs a pillow I'm sure the hotel can provide it.

My name again is Jon Watte and I'm in charge of the Media Kit stuff. Today we are going to talk about how we envision the Media Kit and how we are currently busily implementing the Media Kit. The Media Kit is of course the part of the BeOS that's going to provide handling of time sensitive media for you guys in your applications.

So while yesterday I talked about the Translation Kit, which focused on importing and exporting data, the Media Kit involves a time axis, it handles realtime presentation, or realtime capture, or realtime processing of data of really any kind of format.

We are going to go over the technical architecture of how it's all supposed to fit together, and we are going to go over when and how you can get your hands on this stuff. And we are going to solicit all of your feedback. That is very important to us. Obviously we are working hard to make the Media Kit as good as possible but we know that developers run into all kinds of crazy special cases where with a little help from the system they might create something truly excellent.

If you have a specific requirement in your app for something the Media Kit needs to be able to handle, we are very interested in hearing that at this stage because we still have time to change for the final release.

When we started out doing the new Media Kit we had a set of goals, you know, obviously there are things we would like it to do. We would like it to not suck. We would like it to be high performance, it shouldn't get in the way of getting done what you need to have done. As an application developer you should never feel compelled to reimplement the Cinepak codec just because you can do it faster than the Media Kit, then the Media Kit has no reason for existence. It has to be capable. It has to handle input and output of realtime media. It has to handle the processing of that media and handle recording and playback, which is different from input and output. Input is the actual capture of the data process without ever recording or playing back. Whereas recording data exists in storage. So the VCR demo that you saw a while ago, that is something that does input and recording in the same application.

We want to make it very easy for you developers to create a simple application like that without much thought on your part. Thinking hurts.

And the last goal is of course to make it all extendable. Users should be able to get a floppy with an add-on for his camera or compression card or whatever he went to Fry's and bought and stick that in the machine and your application will transparently take advantage of whatever the capabilities are in the system.

Also it should work across applications. You shouldn't have one big monolithic do-all application, everybody lives within plug-ins for this application. Some models on some images operating systems do it that way, I'm not going to name names. We don't like it.

Some restrictions. We started working on this a while ago. Last year. But we still, you know, it would be nice to go off for three years and spend all three years working on the ultimate media everything. Right? Some big companies do it that way. They never seem to actually ship anything, though. So we figure that that's not a good thing to do. We have to ship soon. Of course the BeOS, the media OS needs to have a rich set of media capabilities soon as well. And soon in this case is defined as R4.

We have to implement all of this using the standard C++ interface used all over the Be kits, we can't go off and invent our own language or write it all in a C API that's different than what you are doing in other applications, that would alienate developers, to deal with disparate things.

And also it has to be user level implementation, as our kernel guys, as good as they are, do not have more than 24 hours a day, seven days a week to work. They are busy doing all this low level stuff that we take advantage of in applications and kits. We couldn't keep pushing more media stuff on them. I know that Windows is currently on a heavy drive to push, push media stuff into the kernel and for some reason they do that. We don't like it because things that live in the kernel can crash the entire system. Things that live in their own address space can only crash that address space. We feel that's much fairer.

So those already are a little comparison to some other technologies that are out there. Compared to like AVI, AVI is just a file format designed for delivering movies. The Media Kit does much more than just play movies, it's a generic framework into which you can plug movie functionality, you can also plug media processing functionality or realtime say and speech data processing functionality or any kind of data that has realtime importance can be plugged into and processed through the Media Kit.

It's also not at all tied to files. I know that some early versions of QuickTime could do nothing unless it came from a file. We are more of a streaming framework and files are just one class of things that can produce streams of data.

Of course to do all that we utilize the OS foundations that Steve talked about before lunch. We use shared memory, threading, interapplication communication using ports, all these low overhead high throughput things that the BeOS kernel provides, we gladly take advantage of.

So this is what the Media Kit is shaping up to look like. Your application sits in the upper right-hand corner, always in the upper right-hand corner and talks to the Media Kit library, which allows your application to seamlessly penetrate the address space barriers without your application needing to do that itself. So if you have a bug, it's not supposed to crash your other applications. The Media Kit library also talks to media add-ons that talks to cameras, compression cards, all the output hardware, what have you. Talks to the media server.

And at the bottom of this all is the kernel and the kernel level device drivers. We kind of like kernel level device drivers to be very shallow. The basic goal of a device driver is to service interrupts and have access to physical memory and anything that doesn't need one of these two capabilities should sit in the media add-on and not in the device driver. Because that way we can protect, we can protect our applications from what you are doing. Not that you would ever be doing anything wrong, right?

The new Media Kit is the actual shared library that you talk to. It's just like lib.be or any shared libraries they sit in system/lib. It's a C++ API that allows you to take advantage of all the various media that is provided through the BeOS and through user installed add-ons to the Media Kit.

This shared library does most of its work by messaging or talking to the media server, which in turn talks to add-ons. But it also, the shared library also exports participant base classes that you have to derive from to be good citizens in the media world. And a participant is both your application, if your application wants to receive audio, do something to it and spit it out again, then it's a participant, just like the add-on is also a participant. There is not a whole lot of difference except in your application you are controlling your UI whereas an add-on gets called upon by other applications and the media server to perform things.

The media server is a regular BeOS server, it subsumes the audio server which we shall retire and it plays the startup sounds, the first thing it does. Amazing. It handles routing of data, this stream graph of where data goes. It handles cataloging what are the current participants, what are the add-ons that are available to become participants but not yet loaded to become participants. And it handles some synchronization and timing issues. Obviously since the Media Kit handles anything that has real time constraints on it, the timing paradigm implemented in the Media Kit is kind of at the core of

The add-ons, which are user installable in a subdirectory of the add-ons directory, and also Be ships a certain set of add-ons already with the system in R4. They implement the actual functionality of talking to devices of implementing, say, audio effects or video transitions, all that stuff typically goes into add-ons, unless you build that into your application.

To do anything you need to derive from these system bases that are exported from the Media Kit library and those base classes are the buffer producer, which is anything that produces a stream of data, a buffer consumer, which is anything that sits on the other end of the stream of data, and obviously it could derive from both buffer producer and consumer anything that eats data, processes it, spits it out.

There is a file capable API for anything that records to or plays back from a file. There is the time source class, which anything which feels that it's in need of supplying time to other parts of the system should derive from the BTimeSource base class.

And last there is the controllable class which is the least fleshed out and I'm going to tell you why in a little while, but it's for anything that either wants to control external devices or has control from external devices.

And lastly, as I said, these add-ons are what you use to talk to device drivers. The media server does not talk to device drivers, the applications should not talk to any of the device drivers, the device drivers should be the shell layer at the bottom and the add-ons are what should talk to the device drivers and provide the uniform API for the media server and applications to use.

So, how does data stream through the Media Kit? Well, data is typed, first of all. And part of the type is not only... this is video, right, this is video, it's 320 pixels wide by 240 pixel high, interlaced so there are two fields, it's played at 60 fields a second, it has uniform square aspect ratio or some other aspect ratio and also the row bytes is part of the type. The row byte is this stride between each scan line of pixels. And this is very important. Suppose you have a Media Kit plug in, which is a BeDirectWindow which displays video. Obviously even though the scan line is 32 pixels wide, you want it to skip one K or one K ahead until the next scan line for the next line of video data. So that's also part of the type of the data that is being streamed.

Data sits in buffers, which live in shared memory and ultimately that's a shared memory area that gets created through the kernel area. However, if you are streaming small buffers of audio or video, you don't want to have one area for each little chunk of audio, that drags down on performance. So the BBuffer isolates this concept so you can have more than one buffer living in the same area, referencing the exact same memory, which can be useful. Suppose you have a BDirectWindow, you can have even and odd scan lines, one buffer for even scan lines, one buffer for odd scan lines. And if you have this BDirectWindow as a client, as a buffer consumer, it can of course require that only my buffers can be used to send data to. It wouldn't be useful to have a BDirectWindow that receives data through some other buffer and then has to copy the data into the window. That would be an unnecessary copy step, so the BDirectWindow, should you have a subclass that implements BBuffer consumer, it can specify that only use these buffers that write directly to my window.

You can also have a producer, if you have requirements such as you have an on card frame buffer visible through the PCI bus, you can say I can only produce buffers, using my own buffers, I cannot accept anyone else's buffers. Obviously if you want to connect one of the cards to one of those windows, something has to go in the middle. Typically either one or the other has a requirement, it is good to have one conform to whatever requirements, the more stringent requirement the other has.

Of course we have made every conceivable... we have taken every conceivable precaution to be sure that passing on these buffers should be really fast. Messaging overhead should not kill a Media Kit.

Timing is very important. I think half of you are here just to hear about the timing, right? A little show of hands. How many care deeply and intensely on a personal level about timing of media? That's only one-third. I guess one-third is actually here just to rest after lunch.

The user, which is usually the application but it could be any participant, can specify who is the master of time. And in a typical system that would be the audio player, the DAC that will actually play audio samples, you want the audio to sync to the video. So the audio has the most stringent requirements to just keep playing samples. It would be the source of timing information for the rest of the nodes in the system. But suppose you are playing an Mpeg data stream from satellite, you can't flow control the satellite because your neighbors would be very upset with you. So instead the satellite has to be the timing master and all the other nodes would just have to cope. You would have to drop a frame now and then to make sure that audio and video stays in sync.

If you have a simple participant audio echo plug in it doesn't really care, it just gets data, and spits it out, it doesn't need to worry about timing, but there are a class of sophisticated add-ons that are participants that need to know this. It's not just participants that actually do realtime presentation or capture of data, there are some Codecs like Intel Indeo actually ask the system how much more time do I have to decode this video? It can do a better job if there is more time available. It has to have the timing information available to it to decide how good a job of decode to do.

The back side is that any add-on or participant that you write that actually cares has to be prepared to deal with what happens on underrun or overrun of data. There is a concept of system level policy, what to do when data overrun or underrun happens. And your add-on, each individual add-on has to implement the system set policy as it sees fit.

Device drivers are of course important to the Media Kit because without devices there wouldn't be much media to do kit things with.

There are some device classes that have a well defined API already, /dev/audio is new in R3 and there is only one driver that implements API and that's for a card no longer made. We are hoping to implement future audio drivers to the same API. There is /dev/MIDI and the port in R3 now knows how to iterate over all the devices available in /dev/midi and choose whatever one of those the user wants to.

And the protocol for these device drivers is publicly available from Be. If you are writing one of these kind of drivers, send us an e-mail and we will send you a sample driver. But of course it's still an add-on, you can write /dev/misc/my_midi and nobody would know how to talk to it except for your add-on, the media server and application only talks to add-ons, not to drivers, you can fix it up in the add-on. Still you have to think about accuracy, which is the ability to promise performance at some specific point in time or with some specific parameters. And you have to think about latency, which is how long does it take from wanting to do something to something actually happening. Both of these are important to media. Accuracy is more important when you are doing off line rendering. When you are playing a QuickTime movie it doesn't matter that much if there is a 200 millisecond delay before it starts playing as long as it stays in sync very accurately.

For latency that's more important when you do on line rendering, if you have a MIDI keyboard and you are generating audio that goes out the audio port, the latency has to be as short as possible for getting from the MIDI to spitting out the front. So sometimes there is a trade-off between these two and you have to figure out what the right thing is to do for your specific device.

Here is some typical usage of how to write a subclass from the Media Kit. Showing that these slides are at least a week old because this changed. We will go to the next slide.

Suppose you have a buffer producer, there is a well-known interface for how to derive from the BBuffer producer which allows you to spit out buffers that the rest of the Media Kit will understand. There is some negotiation about what kind of formats you can handle, what kind of formats can the other guy handle, and that's implemented through the suggest format method. The Media Kit will call suggest format and ask you to suggest a format and it will call, you know, do you like this format, and the other guy can say yes, I do, or modify it and then the Media Kit goes ahead and says please try and hook up together using this format that both guys got to hook up. It will say yes, I can; or no, I can't do that and the user will probably get an alert.

Oh, oh. This should say IterateSource and DisposeSourceCookie. You are a producer, you are a source. As a producer you can have more than one source output, we used to call them plugs. So a client that sits somewhere else might want to ask you so how many outputs do you really have? If you are an audio capture card with 16 channels of input, you may want 16 channels of output as individual sources. So to iterate over all available sources, that's how you implement.

Set buffer group is the call that gets called on your add-on when the client has specific requirements for buffers, so you get past the buffer group and are told please use these buffers and only these buffers when spitting out data.

And last there is two functions that you would never call yourself but gets called within your add-on, which is hook up and break, which basically tells you to please establish a connection to start streaming data to this guy and please break this connection.

So, as an application developer, you would use this, something like this. You would have your own buffer producer or buffer consumer that you would instantiate, you would ask for the default output mixer, which is... I don't have a slide but I can quickly explain this. There is a default video input, default audio input. There is a default video output, default audio output, but there is also a default audio mixer, because if you have more than one application playing, you want them in parallel and you need to have a mixer that mixes them so the applications don't have to do their own mixing. The conversion can actually play samples at different sample rates, then spit it out.

So typically you as an application would never talk to the sound output, you would talk to the default sound mixer.

You get your own format, all your formats that you want to play, and you hook up to the default audio mixer using this format.

So this is how you would hook your own subgenerated class to the rest of the system such that you can play sounds.

So when are you going to get your grubby little hands on this? There is going to be a first comment release, which is a release that we would like you to not send us your first comments about, but your second and well-considered opinions about, it's going to be late April, as we said. And it will have very few add-ons actually supported. You might want to have... you might have to go out and find a specific brand of sound card just to work with this first comment release. Because we don't have the time to do all the Codecs we want to do. It's going to have most of the functionality there, but almost none of the convenience classes, so to play on you have to have your own buffer producer that knows how to produce audio whereas in some future release there will be a Be sound player that says here, play this sound.

There is going to be a second developer release, developer beta release sometime in June, I think they said June 1 on the slide, huh, good date. It could get lost in the mail for two weeks.

It will incorporate all the good feedback we get from the first comment release.

So basically at the developer beta release the API had better be frozen whereas the first comments release may see smaller or bigger change up on to the beta. And we are hoping to have at least first versions, first iterations of the chord, the capture playback, compression, audio, support add-ons at the developer beta release time frame.

So what's going to happen with the old Media Kit or which only does audio so it should be named the audio kit. We are going to stay compatible as much as we can. We are going to reimplement the audio kit classes on top of the current Media Kit API. Exactly how good a job of that we are going to do we don't know yet because we haven't started but we think that we can provide compatibility at a level where all the current applications will keep making noises even though all the little special details may not still be there. Of course we hope to do this all by R4, otherwise it's kind of pointless. Future releases, R5 and forward, we are going to add the convenience classes, of course.

As I said, the first version of the Media Kit may not be very convenient to use. Just because we can't spend time writing those convenience classes while we still need to get the core functionality to the level we expect. And we will add the DV coding, the Mpeg 2 codec, more hardware that our OEMs may request from us or that developers may provide.

And so if you have specific requirements, as I said, if you care deeply on a personal level about this stuff, if you feel that you have good feedback to provide, or if you have an application that you need to ship with R4, you know who you are and why you would want to do that. You should get the comment release but you should also realize that the comment release is for your comments and we will listen to your comments, which means it will change between the comment release and beta release.

If you are just sort of interested in doing media stuff but you don't have the current application or you don't feel like you want to give us free engineering help, you should probably wait for the beta release. It will be announced in the Be newsletter. And probably in sync with the newsletter article starting to talk about how media will work on the BeOS.

So thanks all for coming. This was fairly quick because I want to leave a lot of time for you guys to kind of grill me, you know, did you think about how would it work? That kind of thing. So thanks.

I will open the floor to questions.

A Speaker: Hi, Jon. Three questions. One is I'm in the middle of writing an audio device driver right now using current device API. How will that translate into the new dev audio stuff if I just write it as is right now, will it be easy to...

Jon Watte: The question is he is writing an audio device driver right now and he is using the current audio device API, how easily will that translate to the new device API? Unfortunately not very. A question for you, where did you get the documentation for the current API?

A Speaker: Bob Herold's talk last year. It's a standard device driver, it's like open, read/write.

Jon Watte: Right. Every device driver will of course have open, read/write and ioctl. If you want it to work with the current audio server there is a well-defined, well, not well-defined, but set of ioctls that you need to implement with complicated structs in them. Are you doing that right now?

A Speaker: No, I am not.

Jon Watte: Good. You might be closer to the /dev/audio API. It has one simple well-defined ioctl to set the sample format, just write to the device to play data and to read from the device to record data. So you may be close to doing the right thing. You had a second question.

A Speaker: Actually two more. Speaking of that, is there any kind of configuration for each device? Let's say I want to set my audio device to 44 kilohertz or 48 kilohertz, 96 kilohertz, is there a defined framework for telling the device go to this mode.

Jon Watte: A defined framework for configuring devices for going to specific modes? Yes, there is. At two levels, for a device driver, that's an audio device driver, part of the configuration struct has float sample rate and there is like format of samples, byte order of samples, the size of your buffers, things like that. And on the higher level Media Kit, part, part of the type information of the stream data is what's the sample rate? So when you write an add-on that produces audio, it gets requested I want data in this specific format. Which includes a sample rate. And then it can go to that mode, if you can, or you can say no, I can't do this but I can do that instead. You have a third question.

A Speaker: Yes, third question. You mentioned that you had a default mixer. Can you configure actions on, let's say you have a lot of channels coming into a node. Is it possible to associate like actions of each channel so when this channel goes active it can recall the other channels or stuff like that? Can you basically tell when a channel goes active?

Jon Watte: Okay. He is asking if you can tell when a channel goes active. Yes, there are two levels of activity, there is one you get hooked up, you get the this guy got hooked up message, it may or may not be streaming at that point. There is also a second system callback which tells you that the system is now in play mode, so data starts streaming.

So, you know, combining those two, yes you can know when a channel actually starts getting data. If you want to do something like ramp up the volume when that happens, that's easy for you to do.

A Speaker: I have a quick question about multiple devices with one like I say one I/O board with eight sound recording capabilities or something. Is that all under one /dev/audio virtual device? How would you iterate and interact with each individual one?

Jon Watte: So the question is suppose you have an eight channel input PCI board, would that live as one device /dev/audio or several devices? How would you iterate over the different imports that you have? Good question. You might want, as a developer you can choose to treat each pair as a single audio input stream. If your board is called the X board, you might have X board 1, X board 2, X board 3, X board 4. Or you might choose to have X board or X board 1 if you have more than one card.

And then the API for an audio device driver allows you to specify the number of channels. So you might want to have, you know, specify I want to have all eight channels in one buffer. However, again, it's your add-on that actually talks to the device. You might implement your own private API and just provide those outputs, those plugs from your audio capture, those inputs, the sockets for your buffer consumer.

A Speaker: You build your own device.

Jon Watte: No, you build your own entirely creative plug in and have as much fun as you want as long as you stay within the API and you provide as many or as little input and output as you feel.

A Speaker: Are there any restrictions on full duplex, reading writing at the same time, or is it all virtualized?

Jon Watte: Any restrictions on full duplex? No, no restrictions. There is also no requirements, again that's up to your plug in. If you have one of those cheesey, old style ISA sound cards that can only do one or the other, typically you have a... when someone connects to input you would make output disappear and the other way around. If you can do full duplex, sure you should publish that within the API.

Is this it? No hard questions? Hard question down there.

A Speaker: Probably not a hard question. Are there any plans about spatial audio? Spatial?

Jon Watte: Any plans for spatialized audio, playing Dolby surround sound, Dolby Pro Logic demo, we have no current plans to have a plug in that does that for... we might do that in R5. We might just invent a data type for streaming spatial information that could go parallel to audio information and then have plug-ins worry about it. If you want to develop that, please stay in contact with us and, you know, we can work something out.

A Speaker: What is the realtime performance of the system?

Jon Watte: The realtime performance of the system is fairly good. Typically since we use sending a buffer of data assuming the data is already in the buffer, it is one kernel port right off a 64-byte data structure, which takes a low number of microseconds. So you can do that 10,000 times a second if you wanted to. Of course if you actually do 10,000 times a second, you wouldn't have much time to process the data you were doing. But typically it's more dependent on the actual hardware you have. Some hardware actually needs 256 samples buffers to play audio without glitches. And then the realtime performance, the latency that you get is determined from the buffer size that you play. And that's kind of variable between the different hardwares. Did that answer your question?

A Speaker: No. What's the interim latency, what's the task switch time?

Jon Watte: Task and thread switch time are basically the same. Task switch time, what you care about is the time it takes from write port to read port on the other end. That's on the order of 100 microseconds. You know, with the full messaging overhead that's involved.

The interrupt latency depends again on if your hardware is the PC platform. I have when I have made my experiments typically on the order of a couple hundreds microseconds, again it's like 600 microseconds but occasionally for some reason my system which is a dual Pentium II motherboard picks up and goes to 2.7 milliseconds in interrupt latency. That's probably just something to do with the specific hardware you are running on. If you are targeting a consumer type PC, I think you cannot expect the interrupt latency to never go above 600 microseconds. I think with current hardware you have to be prepared to deal with latencies up to 2 or 3 milliseconds.

But again that's more dependent on hardware than software.

A Speaker: You talked about this new facility, are you expecting an application, would it be responsible for mixing itself and its own output gets mixed with everything else or does it dish out as much as it wants (inaudible)?

Jon Watte: The question is would an application that has many sounds going, suppose it's a game, explosions and background, would it mix its own sounds and then spit it out to the default mixer or would it switch to the mixer at eight different points and spit out eight different channels? And the answer to that is yes to both. You can do both. The default system mixer will in its first incarnation not be all that great. It will do drop sample conversion frame rate conversion, it will do some kinds of scaling for leveling, there may be a discrete number of levels. But that's what we have time to produce for R4. And as it turns out if you run at 48 kilohertz and the sound output hardware runs at 48 kilohertz, there is not going to be a sample rate conversion. If you care you run at the system rate and there will be no conversion.

Also, there will be a control panel for specifying what the default input and output and mixing devices are. So if you write a better mixer, you can ship that, install that, and just have the user go into the control panel and choose your mixer instead of the Be mixer.

But to go back to your question, it would be kind of neat to have a game, you know, that had different sounds outputs like explosions, music, gun shots and you would have those labeled and the user could mix them individually. That would be neat and you could do that.

A Speaker: I think I missed it. You listed... among the items you listed you said you were going to talk about later, controller.

Jon Watte: I didn't get back to the controller interface, that's because that's the least fleshed out. We are probably not going to have a lot of functionality in that area for R4 but we have thought through how to abstract both a device that can take a number of controls, you might have some pan and volume and effects and whatnot, axis of control, and an add-on can definitely export, you know, these are my axes of control and these are the various values they can take, they can be discrete like the input source would take line or AUX or CD or mike. A volume control might be a more floating point value, and also there is some labeling and naming issues we should label the volume and it is should have the units DB.

So that kind of level of functionality might be there for R4 when we go to automation where you have a mixing desk and you want the mixing desk to hook up to an internal part in your system or you want to MIDI device control, we have been looking at that. We are fairly confident that the system can transparently handle that but there is not going to be any functionality for it in R4.

There was something else I forgot. Right. If you as an application don't want to deal with whatever axis of control some add-ons may publish for you to find you can also add anything that's controllable, you can just export a Replicant, that's a view that the add-on implements that's supposed to show whatever the add-on is doing and allow the user to manipulate whatever can be manipulated. If you are a convenience type of application developer, you would say hey, add-on your view and you can add the view to the window and that view would give whatever the user needs to control that add-on. Did that answer your question? Good.

A Speaker: So is it true then that, am I understanding correctly that with the new Media Kit the way things work is your application sort of doesn't have to know what the Codecs are and what the devices are and so forth? If you have a video stream coming through you see through the sequence of bits, presumably if you have both they are somehow hooked together for timing purposes and you sort of don't need to know whether things are going out as JPEG or QuickTime.

Jon Watte: So the question was am I correct in understanding that your application doesn't need to know... doesn't necessarily need to know what format video is, whether it's JPEG or whatever and what form audio is in, it will just get buffers of frames and buffers of samples of audio with enough timing information in there to enable to present them in sync in the end, and the answer is that is correct. Of course, good applications want to know because they want to do cool stuff, but you don't need to know.

Well, we still have some time to go but I see no more hands.

A Speaker: In the diagram that you showed on... you are showing the Media Kit, and then you had a media add-on that the media server talks to and you had another add-on and a media library that penetrates the other space. What is that, the second add-on? What does it do?

Jon Watte: There are two different add-on blocks on this frame. It's the same add-on that gets loaded in both address spaces and that's because of this controller that we talked about. An add-on might choose to export a view that provides the user interface for the add-on itself. And for a view to be added to a window that lives in the application the add-on has to be loaded into the application address space.

So your add-on, if it exports this view, has to be prepared to live in more than one address space at the same time. So that's what this slide, why the add-on is on both sides.

A Speaker: Just the same add-on that happens to sit in both address spaces.

Jon Watte: Yes. In the media server it does media streaming stuff. And in the application space it does the view thing. You might, since it's a Replicant, you might have the add-on as two separate executables, one that does streaming and the other that does user interface and that would actually be two different executables that are loaded.

And this will be explained in more depth once we get around to finalizing the API and then documenting it in the Be Book.

A Speaker: So is there anything special about the buffer (inaudible)?

Jon Watte: Is there anything special about?

A Speaker: You were talking about the buffer consumers and producers and the producers get their buffers from some sort of bank. What are those buffers?

Jon Watte: What are buffers? Right now a buffer can only live in an area, in a kernel area but you can have more than one buffer in the same kernel area.

A Speaker: You can introduce...

Jon Watte: You can introduce your own areas, yes. That's why you can create a BBuffer object and then tell someone else to use it. If you want data to go in this specific part of physical memory, like BDirectWindow, there can be a buffer that references that specific area and just tell the producer please use this area or this buffer. We foresee data living somewhere else. Suppose you have hardware capture and Mpeg. Sorry?

A Speaker: Position I/O.

Jon Watte: It's not a Be position I/O, no. You get into performance problems there.

A Speaker: Right.

Jon Watte: So yes, no. No. It's all buffered.

There is also the separate file capable add-on, if you want to play a QuickTime movie in, you are going to need a file capable add-on, and the purpose is to take a file and turn it into buffers or the other way around.

A Speaker: What are the plans for network-centric data sources or data transport?

Jon Watte: What are the plans for network-centric data sources? We designed into our kit such that it will work when data comes from a satellite or network or some other such system. However, we don't have the manpower to currently implement any add-ons for receiving any of the proprietary network streaming data formats. You as a developer, it's an excellent third party opportunity. Of course like anything else we don't have time to do right now, but it's something that we are aware of and if there is a good solution out there, that might be available to us, we would probably talk to those people.

A Speaker: I was looking at the HTML version of the Be Book last night and I looked at the Media Kit and it was talking about Be subscribers and such. I didn't see any of that.

Jon Watte: Right. This is called the new Media Kit as opposed to old Media Kit. The question is where is the Be subscriber that you can read about in the Be Book? As I briefly, briefly mentioned, the old Media Kit API including BSubscriber and BStream and whatever they are called are going to be reimplemented in terms of the new Media Kit. Unfortunately we didn't find enough flexibility in the old API to do all the things we wanted to do, so we had to create a totally new kit. The old API will be there, it will be deprecated but hopefully old applications will be running.

Right now if you want to play a sound all you can do is play with the old API. We want to get early versions in the hands of developers as soon as possible so you can take advantage of trying out the new stuff. We want people to develop to the new API so we will try to make that available as soon as possible.

A Speaker: Is this the first little bit of legacy code in the OS?

Jon Watte: Is this the first legacy code into the OS? Actually, no it's not.

A Speaker: Data types.

Jon Watte: Data types, right. In R3 we already load and work with data type handlers for the old data types library, even though it's a new translation kit, API we prefer, there is stuff before that. In PR2 we replaced some virtual functions in some system classes and we have some legacy code to deal with what applications expect if they were compiled under PR1. So it uses run time type information; what am I doing with BListView, so already PR2 that snuck in there. We are trying to keep it as a manageable level, we don't want it to overpower us.

A Speaker: Steve said that in the session this morning that the main thrust in development for drivers for sound cards was toward the AC 97 standard but there are still quite a few, half a dozen that had the fancy ISA cards that are popular with professional musicians. I understand you guys aren't going to be writing drivers for that. Is the new Media Kit available to people writing add-ons to support ISA cards of that type?

Jon Watte: The question is our thrust is definitely going to be AC 97 PCI style Codecs, that's what our OEM partners want to put on their motherboard. But there are already like Turtle Beach sound cards and others out there that might use ISA, but not AC 97 compliant. How do we get support for those boards? Yes, developers can write their own drivers and write their own plug ins and it will work, but also as you said, developers can write that.

So no, we don't foresee supporting ISA sound card except for the SoundBlasters, and the SoundBlasters we might never support fully because they are quirky and don't always do full duplex and those things.

Hey, we are out of time. Thanks a lot for coming. (Applause.)