Christopher Tate: We've already talked about how your applications can do all of these wondrous and magnificent things with real-time-based media. We haven't yet talked about how your users will tell you what they want to do. That's the topic of this next section, the user interface for media nodes.
Some nodes have the ability to be configured at run time. An example that I'll keep coming back to is the audio mixer. The gain for each channel that the audio mixer handling can be independently adjusted. In fact, left and right can be independently adjusted. Those channels can be muted at run time. These are all things that can be affected by the user, and theoretically can also be affected in software at run time.
Nodes that can be adjusted, configured, controlled at run time we call -- sensibly enough -- controllable nodes. They derive from BControllable, which is a kind of media node, and these configurable nodes make their parameters visible to the outside world encapsulated with BParameter objects.
Each BParameter object represents one conceptual thing that be can be adjusted at run time. For example, the master gain slider which you all see on your Genki 4 CD, corresponds to a parameter. That is the master gain parameter for the system.
The set of BParameters that a given node publishes to the world is called its parameter web. I'll come back to that class BParameterWeb, and we'll get to that in more detail a little bit later.
Change of behavior is affected by your applications setting the value of a parameter. Setting the value of a parameter on a node is an event just like starting the node or stopping the node. It is given a performance time at which to apply the change. So the application can anticipate the need to adjust the gain sometime in the future, queue up that event, and let the node take care of it when it gets there.
As far as generating a user interface for a node, there are these parameters, and you affect change by setting their values, but from whence do you derive these values? Well, there are a couple of ways that you can present an interface to the user. The first way that you can start a control panel that does everything, and your application doesn't do anything at all. Or you can build a UI at some level of detail within your application. First we'll talk about the simplest case.
StartControlPanel takes as its parameter the node that it wishes to parameterize. That node is responsible for creating the window and running the event loop and creating all of the controls and putting them in a window and setting the values of all the parameters in response to the user's manipulation of the control. Your application has nothing whatsoever to do with the process. You have a new window. You may actually get a new team. This might involve the operating system's running the node as an application. Or running an application that co-resides with the node. I don't remember the details on that.
For you the app writer's purposes, you just need to know that it's fire-and-forget. You start the control panel and then the user is suddenly working with a live interface.
Building the user interface can be almost as simple, or it can be an arbitrarily detailed process. You can go through the options in order. In dealing with building an interface for nodes, these are the key concepts that you will keep coming back to, classes that you will be using.
Skipping around a little bit, I already talked about parameters and setting their values. There's also a class called BMediaTheme. That is the primary class involved with the creation of user interfaces for nodes. There are these tantalizingly named methods, ViewFor and MakeControlFor. ViewFor takes as its parameter a parameter web. MakeControlFor takes a single parameter. And I'm sure you can anticipate where this is going -- but not yet.
The media theme, as I said, is responsible for providing user interface elements at some level of detail. There is one system theme installed by default at least. It's available to any application by requesting the preferred theme from the Media Roster.
Once you have a pointer to a BMediaTheme object, you just talk directly to it. You're not talking to the Media Roster to do user interface. You will talk to the Roster to affect nodes, whereas the media theme is your control point for getting interface elements.
A media theme. This is a kind of high level architectural view of it. It implements two policies. It implements a policy of control instantiation, where "control" in this case means BControl derivatives always. So it controls the mapping of parameters to the controls that affect those parameters.
And it implements a policy of visual layout, how it arranges those views in a window or those controls, rather, in views. These two policies are accessible somewhat independently, as we'll see in a moment.
The parameter web, which I said we'd get back to, is not just a list of parameters with no relationships expressed between them. It's a hierarchical structure where parameter groups exist to illustrate to your node's logical organization and the logical association of smaller sets of parameters.
I will actually show you some examples and explain where the groups are so that this is a little clearer.
The parameter web is essentially a top level group. And the parameter web's special property -- unlike the BParameterGroups -- is that given a parameter web, you can just iterate over all of the parameters that exist anywhere within that web, in some repeatable order. It's not necessarily a particularly useful order, but that's how you can scan for a particular parameter if you know that it exists, or if you're looking for one of a particular description.
There's a lot of information in the parameter that helps you identify which parameter you're interested in. Here I have the parameter web for the audio mixer, and I want to adjust the gain associated with my connection to it. How do I do that? How do I find that particular parameter? Well, as you see here there's the name, the kind, it's string data, but the possible values are operating system defined constants and they have some other names like B_MASTER_GAIN.
Units are "decibels" or "frames per second," or some other user-presentable label that could be applied on the control to make it clear what's going on.
Type, ID, type code, we'll get into these a little later. They seem kind of redundant: like, why is there a type, and a kind and a type code and a value type. Yes, it's a little confusing sometimes.
So we'll go for an example. This is the mixer's master gain parameter information. It has a name. Look, it's the system gain. It's kind is the B_MASTER_GAIN. The fact that it's a char* and not an int should be almost irrelevant to your application.
If you use decibels, you use B_CONTINUOUS_PARAMETER. There are currently three different types of parameter that we think pretty much cover all of the different kinds of parameters that applications will need to deal with. If you can come up with a good case where you will need a different kind of parameter, please let us know, because due to some odd technical reasons, we have to build support for different parameter types directly into the Media Kit.
The ID just happens to be where it is in the mixer's parameter web. That's not usually very useful. And the parameter has two channels of data and the value type of those channels of data is floating point. That's very useful. That tells you that when you set the master gain, you have to set in stereo floating point.
There are three types of parameter are discrete, continuous and null.
Discrete parameters can take one of a limited number of possible values. Good examples are the list of available input jacks on an audio card. It can take line in and it can take microphone in, say, or if it's a high-end multi-channel card, it might have 8 or 10 or 12 channels of input. Selecting which one you wanted to use might be done with a pop-up menu, or if it's an on or off type parameter that's discrete it might be implemented as a check box to enable or disable, et cetera.
Continuous parameters are for the case when there is a range of possible values and you want to accept anything in that range. These are always in floating point in order to give representation within the range. Usually you'll see this as some kind of a slider, like the gain controls for audio.
Again, I reiterate that any parameter can be multi-channel. Even a discrete parameter can be multi-channel. In that case it might be rendered as a control with more than one check box or a radio group or something like that.
Null parameters are kind of a special case. Setting their value doesn't do anything. They don't exist in order to effect change in the node. They exist as a user interface "hint." They provide a label for some other element in the parameter web to be used for user interface purposes. I'll illustrate what that is in just a second.
This is a screen shot. I hope you can see this at least reasonably well. This is a screen shot that I've snipped out of the audio mixer portion of the Media control panel that is new on the Genki 4 CD. So you'll be able to see this when you mount the CD. It says "Audio Mixer" at the top and then it has a number of gain sliders. This slider here is the master gain. And then the sliders off to that direction are the system beep, old programs, which is a particularly dedicated channel for all software that uses the old sound API, and then a couple of free channels. Here we have some labels. Here is the Master gain, 44.1 kilohertz. Here is a mute check box and then the gain control and then it says output. These are groups. Each one of these vertical stacks represents the parameters that are within a single parameter group. Each of those groups is then within a parameter group that is the entire parameter web.
The association, how you know which parameters go in which order, is that there is a concept of parameters being inputs and outputs to one another. So there is a null parameter with that label that has as an output a null parameter with this other label that has as an output a mute button that has as an output a gain parameter, that has as a parameter another null parameter. This isn't usually going to be information that you need to deal with, unless you choose to build your own interface at a very low level. We'll get to that in a minute.
The simplest way to build an interface for a node that runs within your app, rather than in this thing off over here that we call a control panel, is to ask the media theme to make a view for the whole parameter web. And it then returns a BView object that you place in a window, wherever it is that you want, and you leave it alone. The view is not accessible to your application. Even though it's a BView and it has BControls in it, your application will not see any messages from those controls. The view takes steps to intercept and properly process the control manipulations.
This is a BMediaTheme method, you'll notice, and there's a system theme, but that might not be the only theme available. If you have, say, the KPT Bryce media theme available to your application, then you could instantiate one of those and tell it to create the interface, rather than asking the preferred theme to create an interface that looks like the rest of the operating system.
Now, if you need more control than simply allowing the media theme to exercise its full user interface layout policy, then things get interesting. You can ask for individual controls for individual parameters. This is particularly useful if you know that all you really want is a gain slider for your output. You don't care about the rest of the parameter web. You just want to have that one in your application and you want it to look like it would everywhere else in the system. Then you would just get the system media control for that parameter, and then put that control in your own view somewhere.
With power comes responsibility. So now that you have power over that control, you have the responsibility to handle its messages and to set that parameter's values accordingly. There is some sample code available that does this.
Before I go on, I need to tell you that you're not working completely in the dark. A few weeks ago there was a developer list newsletter article about parameters and controllables and how you use MakeViewFor and how you use MakeControlFor and how you handle those messages. So go to the Be web site. Look up the sample code. It's there. It works. It worked even then if you had whatever current beta was available then. It definitely works in Genki 4 so it's there.
You're going to get these messages from the control, but because you didn't instantiate the control, you don't really know what identifier the system theme is going to be using to identify messages that came from that control, as opposed to messages that came from anywhere else, and in your event loop you're going to get this message and you're going to need to affect some parameter, so you need to make sure that you can keep track of which messages refer to which parameter.
I suggest that what you do so that you can recognize the messages is to use the BControl method SetMessage. This is where you fill out a BMessage with a known "what" code, set the control to use that message and then you'll know what to look for in your window's event loop.
Similarly, you can add to that message any other fields that you want to be included with every message that the control sends. We strongly recommend that one of those fields be the BParameter pointer for the parameter. That way, cleverly, your window can get the message, say "aha, it's a message from one of my controls," extract the BParameter pointer from it directly, and set the value right there without having to maintain any other mappings.
Here is some code that does just that. We declare some arbitrary message identifier and name for the field. We get a control from the theme, and then fill out the message and set the control to use that message.
You also have to know how to recognize the value that is coming out in that message. BControls by default and throughout the system use the "be:value" name to identify the value field and the messages that they sent. That's how they report what the new setting of the control is.
Because we have introduced multi-channel controls in Genki, you need to know how to recognize multi-channel data. We've introduced a new name that's "be:channel_data" to identify an array of values that need to be all set on the corresponding parameter. Before I go on, the BMultiChannel control class is actually a new Application Kit class. It's not restricted to the Media Kit. It's available in any context. Sort of go forth and use it. It's great.
For ultimate control, if you don't want to use the theme's layout policies or the theme's control allocation policies, you can simply instantiate your own controls and handle them however you wish. At that point everything is on your own head and you didn't need to sit through the rest of this in the first place.
However, it might be really useful -- especially to other people -- if you implemented a BMediaTheme to provide your layout policy and your control allocation. That way you could sell it or ship it to other people, and we too could have really nifty user interfaces.
So let's talk about implementing your own BMediaTheme.
There are really only two functions in BMediaTheme that you need to implement. They are both pure virtual to make sure that you implement them.
One of them is MakeViewFor and one of them is MakeControlFor. As you might guess, MakeViewFor implements the layout policy, and MakeControlFor implements the make-control implementation policy.
I'm going to take these in order. It's pretty self-explanatory, I hope, that when you parse the parameter web looking at the relationships between the parameters, allocate the controls that you want to use for those parameters. We strongly suggest that your theme allocate the controls by using its own MakeControlFor method. Otherwise, users are liable to get confused.
As I indicated, there is this input and output relationship between parameters that exists in order to help your application identify which parameter controls should be visually connected in some way.
Again, this is a chain of parameters with their inputs and out puts set to indicate a relationship. MakeViewFor in this media theme has chosen to interpret that as meaning vertical stacking. It could just as easily have implemented horizontal stacking or some other association.
When you get the control, those controls are going to be producing messages. The view that is produced is intended to be opaque to the application. The messages from that view and from those controls are not permitted to be delivered to the application. It won't know what to do with them. So you need to set up a BView that has an appropriate message loop for handling those controls, and then you have to arrange a mechanism whereby the controls' messages go to your view, rather than to the window.
The way you do that is by installing a message filter on the window. There is a behavior that says that you can set the target of any BControl's messages to be any appropriate BHandler. Unfortunately, there's also a behavior that when you add a view to a window, via the AddChild method, all of the controls in that view are retargeted to the window. It would be really nice to be able to create the control, set the control to the view, put the control in the view and do this for all of the controls and then just hand the view out without doing any more work. But you can't do that because the controls' messages will go to the window instead and you can't do anything about that.
Well, you can't do anything about that at first. What you can do is in your view's AllAttached method. This is a great little method. When your view has been attached to a window, and all of the processing to retarget controls and steps to be taken in order to ensure that the view functions properly within the window, once that's finished, the view's AllAttached method is called. So that's your hook to install a message filter at that time that reroutes messages from all of your controls to your view.
What your message filter should do is it should use your own private control "what" code which is set up just like as if you were intercepting them in your app to something recognizable and install a filter that looks for that "what" code and informs them to a view instead. It's a fairly straightforward piece of code, and we'll have sample code for that pretty darned soon now.
AllDetached is a method that's called when things are being removed from the window. That's your opportunity to remove the message filter, similarly to AllAttached.
MakeControlFor in comparison is quite simple. And although the MakeViewFor didn't do it at first, it is going to need it last. You look at the parameters and you look at its characteristics, and then you allocate an appropriate control. If it's a continuous parameter, you want to allocate some sort of slider and you look at the parameter's range to decide how to label the slider and what parameters to give the control what sort of scale it should use.
There is a MakeFallbackControl method, if you want to use the system controls, but in some kind of different layout, you can fall back to the built in system theme. This is an easier way of getting the controls than having to derive and instantiate your own BControl classes.
There is, as I mentioned a new BControl subclass in the Genki release that is multi-channel data. That's what's used for the gain sliders. That's what you should use for any kind of multi-channel slider. It's very nice to be able to couple channel input or allow the channels to be manipulated independently as alternative behaviors in the same control.
I know I kind of blazed through this, but in a sense there's only so many words I can say about the subject. Questions?
Audience Member: When setting targets, where you're saying you have a BMessageFilter and then use that to set the target to the view, could you instead use in AllAttached loop through the controls and then set the target to this instead?
Christopher Tate: Absolutely. It's just --
Audience Member: That way we don't have to manage --
Christopher Tate: There is a complexity trade off. If you use the message filter, then you don't have to retarget anything and you don't have to know what controls are in the view.
Audience Member: Well, there is a way to iterate through the controls; right?
Christopher Tate: Yes.
Audience Member: And you can do a dynamic_cast on the base class?
Christopher Tate: Right. That's actually in some ways more work than just using a message filter whose only implementation is to look at the "what" code and if it matches what you've used for your controls, send it to the view. Otherwise, send it to the window either way works. It's a question of personal preference.
Audience Member: I have a question. Say I have a node and say the user sets up some values for it. Who is supposed to take care of the persistence?
Christopher Tate: What persistence?
Audience Member: I would say the values, such as the volume for that particular control. I would say I want to save that set up for the data used.
Christopher Tate: There is no mechanism for the persistence of parameter settings. It's a little bit tricky because there might be more than one active control for a given parameter at a time. So just because you have a control for it and you know that your control didn't change, that nobody moved your control for that parameter, that doesn't mean that somebody else didn't set the value of that parameter. So really the only way to do it is to pull the set of values from the parameters in some way and persist that. And when you choose to do that, it is sort of an application policy decision.
Audience Member: So yesterday when we saw two copies of the media player, you still have two BControls for volume and you set one, isn't that going to the same theme?
Jon Wätte: The volume is per connection to the mixer.
Christopher Tate: Right. In the case of the media player, the gain slider for each window affects the audio just for that window. It's not at all hard to set up a situation where you have more than one control for the same parameter. Like if you run the ParameterSample application that I referred to earlier that was in the newsletter article, then you will wind up with multiple active controls for the system master gain, and if you move one, it adjusts the output and if you move the other one, it adjusts the same output.
This was not previously the case. I believe that it is now the case. Jon, help me out. Did SetParameterValue broadcast messages get implemented yet?
Jon Wätte: No.
Christopher Tate: Okay. This is actually a more sophisticated question than you think it is, and there are some reasons why we might not want to support that. So I say might, but, Jon?
Jon Wätte: SetParameterValue has time stamp so it's actually a queued event, and the message should not get broadcast until the event is actually implemented. Therefore, the node needs to broadcast it.
We have an added hooks for a node to broadcast the value change. The problem there is if somebody pulls on the slider there is going to be a storm of messages, and what we're looking at doing is just collating these messages and filtering the screen so we don't get a flood of messages, but that filtering is not there yet. So if you get Genki and you see broadcast parameter value change, just be gentle.
Audience Member: Just to clarify on the persistence question, if you're changing a parameter many times over the span of time you're playing back volume or playing back, say, your sound, for instance, say you're slowly decreasing the volume of sound over time, you are going to be calling SetParameterValue several times with the time stamp, and when you want to save this all to a file somewhere, you've got to handle all that yourself because you can't just query the web because it's got the last value set; is that right?
Christopher Tate: Right. I was illustrating the case where you have some sort of static value that you want to persist. In fact, what you're describing sounds like recording the changing of parameters as an effect, basically. I don't think that there is currently a way to do that, except by catching and recording the set parameter value messages that are broadcast by the node. And that's when you would just accumulate those to some sort of script, which could then be replayed.
Jon Wätte: [to Christopher Tate] Did you talk about recordable controllables?
Christopher Tate: No.
Jon Wätte: Okay.
Christopher Tate: Because this was all in flux at the time. If you would care to talk about recordable controllables, you're welcome to... [audience laughter]
Jon Wätte: [to the audience] There are these two utility functions BControllable, one that will take the values of some number of parameters that you specify, and save them in some flattened state in the buffer, and there is another convenience function that will take such a buffer and apply those values to your parameters.
Supposing you also worked via a producer and a consumer who accepted connections of the type not raw audio or encoded video but parameter data, which is actually a defined media type, then you could send buffers of parameter values when they get changed with time stamps and you could receive as a consumer those same buffers, and replay them so that way you can actually record parameter value changes just like you record all your video, but that's kind of an advanced feature that we haven't gotten around to doing in our nodes.
So there are no nodes that do it right now, but as an application if you wanted to do automation, that is the intended way of doing it. If you want to get involved in this, just feel free to send me e-mail and we'll talk more about it.
Christopher Tate: This approach has the advantage of not sending out storms of BMessages at the application level, and getting the application kit involved and doing all this really non-realtime stuff in the midst of realtime data processing. It's much lower overhead to catch it as metadata in the media stream.
I invite questions on the entire afternoon session, because we're done.
Audience Member: I just have a question about the theme. Say someone makes a new theme. Is there a way to tell the rest of the system that that theme is available or set it as the default theme or anything like that?
Christopher Tate: You can set it as the default theme. There is a roster function to set the preferred theme. So if you had an add-on and you wanted to install yourself as the preferred media theme, then you would just invoke that when you reloaded.
Audience Member: And can applications browse the list of available themes or anything like that?
Christopher Tate: I don't think there is currently an API for that, although the subject has come up in the past.
Jon Wätte: Do we have the headers here?
Christopher Tate: I don't think so.
Owen Smith: Not for Genki.
Audience Member: I have a question about the media theme because it looked and it tastes like a layout library to me a little bit.
Christopher Tate: Well, no, BParameters are actually part of the Media Kit and the media theme mechanism relies on the Media Kit, therefore, it relies on BParameters. I mean, the interface is that you pass it a parameter web or you pass an individual parameter. That is to say, a BParameter web object or a BParameter object. One could quite easily use it as a template for a more general layout engine, hint, hint. But I don't think that it would be directly applicable, no. The answer to the previous question is it's not public, so you can't do it. Any other questions?
Audience Member: A general question. This release besides the Media Kit, the BMultiChannelControl is new, but what else is new that we should be digging into Be book?
Christopher Tate: The question is what else is new in Genki in the media kit that means that you need to read the Be book again?
Jon Wätte: Not just of the media kit; right?
Audience Member: No.
Christopher Tate: The whole thing? The short answer to that is "lots." There have been quite a number of tweaks here and there. Actually, one thing I didn't mention is that if you get a view from a theme for a parameter web, you put that in your application in a window somewhere, that window has to use asynchronous controls, which are new in Genki, I believe.
Jon Wätte: The problem is that the Be book is not updated for everything that's new. Headers are your friend.
Christopher Tate: Yes. And unfortunately header files are the ultimate reference for what's new and exciting in any given release. The Be book has a lot longer latency for the changes [audience laughter]. So yeah, we try to get the Media Kit documentation in particular much more up to date for this release. Unfortunately, our technical writers only have so many fingers. So we can't update everything all the time.
Audience Member: When you hook up to the mixing example. Let's say I have two audio data files that one is the higher sampling rate, and one is the low sampling rate. My question is whose responsibility to adjust these sample rates? I don't want to get -
Owen Smith: It is the mixer's responsibility in this case. What the mixer will do is take various data streams and it will get a buffer. Well, actually, it's in the connection. When you make the connection to the mixer, you're going to tell it what format of audio you're going to be sending. So in one case you have 44.1 K and its full floating point and it's just gorgeous. In the other case you have this crappy 8-bit audio that you're sending. What the mixer has to do then is to convert these to whatever the output format is going to do. So it's going to do whatever resampling and stuff, and it already does that. I mean, if you try and play a nice sound and a bad sound at the same time, it will do the resampling and mixing for you. Does that answer your question?
Audience Member: Yes.
Christopher Tate: Yes. In general the system mixer already handles this. You can plug 44.1 and it 22.05 and it will do the right thing, and you'll get 44.1 out the other side. If you want to handle these streams in your own private way, not involving the system mixer class, then you are a mixer and you are responsible for resampling them to match your desired output rate.
Owen Smith: That's exactly why I'm so pleased that the audio mixer is something that I can instantiate myself because I don't want to do this stuff myself.
Christopher Tate: The guy who wrote the system mixer did a nice job.
Audience Member: So if I may, I am going to do some fancy way of treating the data between the two samples, I can do it by myself?
Owen Smith: Yes.
Christopher Tate: Yes. You can always do this yourself.
Audience Member: Thanks.
Audience Member: Is it only upsample though, the system mixer or the add-on? For example, can I get a float out?
Jon Wätte: No.
Christopher Tate: The question was, does the system mixer only up-sample to 44.1 floating point audio output and the answer was no, you can also get 16 bit integer if you want.
The general everybody-from-Be-stands-up-on-the-podium question and answer session is in 20 minutes; that's it for this track.