|
Be Newsletter
Issue 23, May 15, 1996
Table of Contents
BE ENGINEERING INSIGHTS: Heap
Holes and How to Cure Them
By Cyil Meurillon
Every application on the BeBox has access to
malloc(), the standard C library memory
allocator function. Programmers that have used C or C++
should be familiar with malloc's protocol: The function
takes an integer argument that declares the number of bytes
of memory that you want to allocate, and returns a pointer
to this memory. The memory is taken from your application's
heap; when you're done using the memory, you pass the
pointer to free(), thus freeing the memory for
re-use.
(Macintosh programmers have a similar mechanism called
NewHandle(). The only difference between the
two mechanisms is that a block of memory that's allocated
through malloc() can't be moved.)
The primary advantage of malloc() is that it's
very easy to use. But it does have some drawbacks. First of
all, it imposes a strict discipline upon the programmer:
Memory blocks that are allocated with malloc()
must be explicitly returned to the system with
free(). The BeBox doesn't provide a "garbage
collector" (a process that looks for and frees unreferenced
memory blocks), so programmers have to make sure that their
malloc() and free() calls are balanced. Failure
to do so results in a memory leak: Your application keeps
using more and more memory.
A more significant problem -- and one that can't be
corrected simply by adopting healthy programming practices
-- is that malloc() and free() can
create "holes" in your heap. When you call
free() the heap isn't "compacted": The
remaining allocated blocks aren't shifted to "squish" the
freed block. This can results in "heap fragmentation," a
situation in which free space is scattered throughout the
heap. Fragmentation is bad because the "holes" in the heap
actually consume physical memory, but they don't contain
useful data -- they're just wasted space. (A freed block can
be re-used in a subsequent malloc(), but only
if that malloc() requests a block of memory
that's no greater than the freed block.)
As the kernel swaps out untouched pages, the hole-ridden
sections of your heap will eventually move out of RAM, but
even this is a waste, since the VM system will write these
empty pages to a swap file on the disk.
In general, large blocks that are malloc'd and freed during
the life of an application are problematic because they can
leave large holes in the heap. malloc() isn't
the appropriate allocator for such objects. Fortunately, the
Be OS provides another level of memory allocation. With the
pair of system calls create_area() and
delete_area(), you can manage separate memory
segments -- or "areas" -- in a program's address space. A
large object can efficiently be allocated in an area because
all the RAM that's used by the object is returned to the
system when the object is deallocated and the area is
deleted. In other words, areas don't leave holes in the
heap.
Areas aren't as light-weight as blocks created by
malloc(). For one thing, the minimum size for
an area is 4K; also, create_area() isn't as
fast as malloc() for small blocks, so some
judgement is needed. What's the critical size beyond which
create_area() is more efficient than
malloc()? I would say that a block of memory
that's 100K or greater should certainly be allocated in an
area.
Let's consider a concrete example. An application programmer
wants to allocate a 100K memory block for each open
document. As these are large blocks that are dynamically
allocated, the programmer may want to use
create_area() instead of malloc()
to limit heap fragmentation. The header file OS.h specifies
the following prototype for create_area():
area_id create_area(const char *name,
void **start_addr,
ulong addr_spec,
long size,
ulong lock,
ulong protection);
The function and its arguments are thoroughly documented
in the Kernel Kit chapter of The Be Book; here, we'll
summarize the arguments:
- "
name" is a handle of your own
choosing.
- "
start_addr" and
"addr_spec" work together to let you specify
the location of the memory that you want to allocate.
Typically, you let the kernel choose the address for you
by setting "addr_spec" to B_ANY_ADDRESS. The
address that the kernel chooses will be passed back by
reference through start_addr.
- "
size" has to be a multiple of
B_PAGE_SIZE (4 K, or 4096 bytes).
- "
lock" specifies the allocation scheme
to use for the area's pages. B_NO_LOCK fits most cases.
It means that the pages are allocated on demand (when
accessed for the first time), and that pages may be
swapped out by the kernel if necessary. Other lock values
let you allocate pages and lock them into RAM
immediately, or lock them as they're paged in. Typically,
only real-time applications need to make such
demands.
- Finally, the typical "
protection" is
B_READ_AREA + B_WRITE_AREA; this declares that the memory
in the area is both readable and writable.
Here's the sample code:
/* our structure that is 100K large */
struct s {
...
};
area_id aid;
void *b;
struct s *p;
/* create the area */
aid = create_area("fred", &b,
B_ANY_ADDRESS,
25*B_PAGE_SIZE,
B_NO_LOCK,
B_READ_AREA | B_WRITE_AREA);
/* check the error, and deal with it */
if (aid < B_NO_ERROR) {
...
}
/* create_area() returned the address in b
* cast the address as a pointer to our structure
*/
p = (struct s *) b;
/* play with the structure */
...
/* dispose of the area
* when you're done with the structure
*/
delete_area(aid);
In a future article, I will address interesting features
that areas and VM offer, such as memory sharing between
applications, lazy copying of segments, and file mapping.
For now, feel free to contact me by e-mail (at
cyril@be.com) if you have
questions or comments regarding what has been discussed
here.
BE DEVELOPER PROFILE: Jim B. Moy
Jim B. Moy, a Be developer in Fort Collins, Colorado,
describes himself as "a jaded Macintosh programmer who
enjoys programming at the OS level." He just started
developing for the BeBox, and tells us, "The Be OS got me
excited about programming in ways I haven't felt in quite
awhile -- thanks!"
Jim has been developing software professionally on Macintosh
and UNIX platforms for more than 10 years. He's worked on a
number of scanning applications (DeskScan, Picture Place,
Picture Scan, and others) that Hewlett-Packard bundles with
its desktop scanner products, and developed the Macintosh
version of the TWAIN Adobe Photoshop plug-in that ships with
Photoshop.
Jim does his BeBox development mostly in the evenings after
his family is asleep. "I'm always thinking about what I'm
going to do next on my BeBox. Even when it looks like I'm
watching TV with my family, I'm really not; I'm thinking
about how I'm going to get something to compile."
Jim is particularly attracted by the BeBox's
price/performance ratio. "It's an incredibly robust box for
the price." That, combined with "a cool OS, a cool API, real
multiprocessing and multitasking," drew Jim to the BeBox. "I
hope the BeBox carves out a niche with a strong following,
then expands into other markets. I hope it kicks butt on
current PC operating systems!"
"What I'd really like is for the Be OS to become a real,
viable alternative to current OS choices," Jim says. "The Be
OS is way more fun to program than other operating systems.
The API is much simpler than the current Macintosh API, and
with the proper tools, I'll be able to turn my ideas into
reality more cleanly, and more quickly with the BeBox."
So what exactly are Jim's BeBox product ideas? He thinks
he'll start off developing a Kagi shareware registration
application. (Kagi is a shareware payment processing
service.) "A neat feature of the BeBox version of the
application is that it can connect directly to Kagi over the
Internet, making it easy for users to make shareware
payments." In addition, Jim has plans for a GUI resources
editor, and after that, a desktop scanner software
application (including drivers) with a TWAIN interface.
Speaking of TWAIN, Jim adds, "I'd like to see TWAIN adopted
as a standard interface for image acquisition devices on the
BeBox. This would allow developers to leverage their
programming investments so that software written to the
TWAIN standard on Macintosh and Windows could be easily
moved to the BeBox, thereby reducing the amount of effort
required to support more than one device." Toward that end,
Jim hopes to provide some layers of the TWAIN standard that
BeBox applications can easily integrate.
After he's delivered all that? "I'll develop where the
spirit moves me, where stuff is needed on the platform that
other people aren't working on yet." We can't wait to see
what he comes up with.
For more information, visit Jim's web site at
http://www.verinet.com/~jbm or send e-mail to
jbm@verinet.com.
Born-Again CHRP
By Jean-Louis Gassée
At the inauguration of the building bearing his name on
the Stanford campus last February, Bill Gates was asked to
comment on Apple's situation and strategy. He replied, in
substance, that Apple's problems were rooted in
implementation, not strategy -- they lacked determination,
not intellect.
Gil Amelio has the reputation of enforcing accountability
while giving clear direction and offering fairly wide
autonomy in meeting agreed-upon goals. This gives us hope
that under Gil's firm guidance, Apple will implement its
chosen strategies and bounce back to financial and spiritual
health.
Beyond the welfare of our many friends at Apple, we have a
vested interest in one of Apple's efforts, one in which Dr.
Amelio's thoughtful and disciplined approach is guaranteed
to bring results. I'm referring to the PPCP, nee and, we're
told this week, reborn CHRP. Never mind the name, the goal
is to give momentum to the PowerPC world by agreeing upon a
Common Hardware Reference Platform. Call it our PC/AT if you
will: A standard for a platform that will flourish in its
variations, extensions, perversions, for the benefit the
entire PPC commonwealth.
Standards are strange creatures. There's no blueprint for
creating a new one -- the rise to standardom is governed by
rules that can only be stated a posteriori. But if you
follow the trajectory back to the source, very often you'll
find a common element: A strong-willed individual who's
willing to steamroll the obstacles, and share the loot with
others.
Look at Java. Follow Java back to its roots and you'll find
OAK, a language developed by First Person, a Sun skunkworks,
for "home information terminals" (a/k/a set-top boxes). The
First Person project was given up for dead in early 1995 --
and all would have been forgotten, if not for the
determination of Jim Clark and Scott McNealy. They took a
considered, well-engineered but aimless language, pointed it
in a single direction, cleared the path, and then
pushed...hard.
I'm aware this is hindsight, and that while all steamrollers
flatten equally, they don't always head in the right
direction. Still, Gil Amelio's leadership, as analyzed in
his book, "Profit From Experience," will be much needed in
order to remove the obstacles in the way of a CHRP standard
for the PowerPC world.
The first obstacle, of course, is setting the terms so that
when we speak of a PowerPC motherboard, we know we will be
understood. The motherboard needs an assumed set of core
functions that will give it the sort of easy certainty that
we associate, today, with an Intel motherboard. Surely, the
Intel world isn't perfect -- but go to Fry's and pick out
any Intel motherboard. You stand a good chance of running
DOS, PC/DOS, OS/2, Windows 3.1, 95, NT, various flavors of
Unix and Linux, Geos and Microware's OS, perhaps NEXTSTEP
and GNUSTEP, and a few others I can't quite remember at the
moment. Such a bounty may be out of the PowerPC family's
reach, but we need to give wide currency to the PPCP or CHRP
motherboard token.
This week we show the BeBox at Apple's World Wide
Developer's Conference in San Jose. We're grateful for
Apple's gracious hospitality and, for our part, we're
restating our support for a PPC motherboard standard. We're
encouraged by CHRP demos at the conference and by various
announcements tying Apple, IBM, the PowerPC, and the MacOS
together. We hope their implementation will clear up
lingering questions, such as the full documentation of the
standard, the supply of MacOS-specific building blocks, the
availability of CHRP-compliant MacOS versions, the
unbundling of MacOS sales from hardware sales, and the
migration to 100% CHRP-based Apple hardware. This is a long
list and this is where Dr. Amelio's new regime of firm
direction and accountability can make a difference for all
of us in the PowerPC camp.
Already there have been more than enough comments on Dr.
Amelio's 100 days speech that he delivered to the Conference
on Monday. The usual suspects, from Peter Hartsook to Tim
Bajarin, have let oracular pronouncements fall from their
Delphic lips, and dutiful reporters have carefully spliced
the quotes into their guarded accounts of the event. If
we're not completely amnesic, we know the limits of such
speeches. They're a tradition that must be honored, but
unless you really kill -- or die -- the impact doesn't last
long.
Be Demo Tour: Washington DC Meeting
Report
(On May 11, Be's Dominic Giampaolo and Mark Gonzalez
demonstrated the BeBox to the May general meeting of the Mac
Users' Group at the Northern Virginia Community College in
Washington DC. The following is an impression of the demo,
excerpted from a larger report, as recorded by the meeting's
chairman, Don Essick.)
Mark and Dominic of Be were fantastic! Mark gave most of the
presentation while Dominic was the "designated mouser" and
fountain of technical knowledge. They both knew their
product inside and out.
It's the Be operating system that supplies the magic to the
BeBox. The Be OS is a real-time, preemptive multitasking,
multithreaded operating system with a built-in database.
Quite frankly, it blew my socks off! Fast, agile, and
powerful are words that come to mind. Some of this is
because Be doesn't have 10 years of "legacy compatibility"
code, (you know, the code that lets today's Mac run the
original MacWrite), but there are a lot of in-box
innovations that make this system unique. My personal
favorite was the "active window dragging" feature. When you
click on a window and drag it, the program in the window
just keeps on running: The 3-D rendering of the Be logo just
kept twirling away as Dominic moved the window around the
screen. There is none of that "clear and redraw" stuff that
happens on the Mac.
Mark and Dominic continued to show innovation after
innovation in the system. Not only can you use the system
database to store application data, it also manages the file
system (automatically). You can create "live" database
queries that monitor changes to the records in the database:
As values change, your application is automatically and
instantly informed. This type of message passing capability
is the wave of the future -- and available now from Be.
Following the demo, the presenters invited everyone down for
a look at the BeBox. The back of the machine is packed with
connectors of just about every flavor: A parallel port, four
serial ports, two MIDI in/out ports, four joystick ports,
stereo RCA line in/out jacks, microphone and headphone
jacks, three infrared controller ports -- there's even a
"GeekPort," Be's general-purpose,
connect-just-about-anything I/O port. It also ships with a
complete application development environment including an
the Metrowerks CodeWarrior IDE and a full set of code
libraries and documentation.
Everyone seemed quite impressed with the BeBox; many people
had glowing praise for the demo. I hope Be comes back in a
few months, when applications are shipping, to show us what
they and their development partners have been up to.
|
