These are comments on the page: http://www.keithl.com/fixfirefox.html. It isn't clear (at least in Opera) how to append to or modify the page.


The issue of memory usage in Firefox is a complex one. To grasp it effectively one has to have knowledge about the memory management hierarchy in the system – from the user program (e.g. Firefox) to the C library (.../malloc/malloc.c), through the Linux kernel (.../linux/mm/*.c). The comments below are related to Firefox running under Linux but they may apply to other operating systems to a greater or lesser extent.

Firefox uses (heap) memory to store a variety of data structures, incoming data from the internet, parsed HTML text, image files, graphics data structures (the Gtk/Gdk/glib interface to X windows), the bookmarks and history data, etc. etc. All of this sits on top of the C library malloc/free ANSI standard memory allocation interface (even though this is largely hidden in the Firefox source because there are "Netscape Portable Runtime (NSPR) functions as well as implicit C++ functions (Firefox is mostly written in C++) which sit between the Firefox source code and the underlying C library functions. Though there appear to be means within the Firefox source for managing different memory allocation "heaps" (handling images different from Javascript data structures for example) it does not appear at this time any of these are implemented. As far as I've been able to tell all Firefox memory management sits on top of ANSI standard malloc() and free() 1. The standard glibc malloc() and free() code is complex. It can be "tuned" to alter memory allocation and deallocation policies but it does not appear that Firefox does that. It can also be compiled with various compile options to make it behave differently. Firefox builds do not appear to do that either (The Firefox source would have to be distributed with or grab the glibc malloc files). The "default" situation which I suspect is in place for common Firefox usage, is that Firefox calls malloc(), malloc uses sbrk() and/or brk() and sometimes mmap() to get memory from the Linux kernel.

Here is one primary problem. The heap is considered to be one large chunk of data which is subdivided by malloc(). So long as the "last" element in the heap is allocated (say to the last page visited "history" record), the allocated heap memory size (and therefore the maximum virtual memory size one sees on a system-monitor, ps, etc.) will never decrease! So if you used 500 MB of heap space opening dozens of windows/tabs over a few days and you have bookmarked recently visited pages, are keeping a visited page history, or happen to have a Javascript running in the last page, etc. and close every tab/window page up to the last page your heap size remains the same. Only if you manage to deallocate all of the most recently allocated memory data structures in the heap and the heap is merged into one big (490+MB) chunk of free memory, will that virtual memory be returned by malloc.c to Linux and thus shrink the process virtual memory size.

So, while this looks bad, it really really isn't if the memory is really one big "free" chunk. It gets paged out to swap space (which can be a problem if you have limited swap space) but it doesn't impact performance all that much. Where it becomes a problem is when over time, lots of pages have been opened and closed, pages redrawn, scripts run, downloaded ads, various plug-ins doing what is anyones guess, etc., and the heap becomes heavily "fragmented" – ones resident set is small but ones virtual memory use is large. Most of the Firefox heap memory is paged out to the swap disk. So when you want to allocate, 100KB, 500KB, 5 MB, etc. as one big chunk (X bytes) for some jpeg file, some flash stream (???), etc. Firefox turns to malloc and says give me X. Firefox has to go through all of the small chunks in the heap to discover that it doesn't have X and needs to request more from Linux. The problem is that paging in all of the linked pages in the heap to discover it doesn't have X doesn't work well under Linux. What should happen is that ones CPU use should increase significantly and I/O to the swap device should go through the roof. That doesn't happen. I believe this is a fundamental bug in the Linux virtual memory management system that it isn't tuned for "workstation" performance where it is ok to max out the disk I/O bandwidth and CPU utilization to provide better paging performance. It could be compensated for by better malloc() code, and I get the impression that it tries to do this (using "bins" of various chunk sizes to minimize list length) but it doesn't seem to work well with Firefox.

So one really has 3 problems. 1) Firefox doesn't try to manage different types of memory differently (at least not much) and doesn't try to tune C-library memory functions for its usage. 2) Glibc malloc() doesn't seem to handle fragmented heaps (like those produced by Firefox) really well. 3) Linux VM paging code isn't tuned to allow maximal usage of system resources.

1. The NSPR source includes a "private" memory management system which seems to include some interesting tracing and statistics keeping functions but to the best of my knowledge it isn't in any 'public' version of Firefox.