While there are some extensions to do this (okay, I found one but it was broken), in Joomla 1.5 (decent product, a good balance between ease of use and uber flexibility), it seems that each template (in /templates/) have a favicon.ico file. Simply replace this file (keeping the same name obviously), with your own version. Google favicons and how to create them if you don’t know what they are :P. Note that favicons are usually cached by the browser, you’d either need to do several reloads, or clear your browser’s cache before the icon shows up/refreshes.

— Kamal