Yes, mixed version clusters are supported in SQL Server.
Installing SQL 2008 is different from SQL 2005. Read up on the new
procedures. They are much easier. You will like them a lot better than the
SQL 2005 installer.
Follow the rules for installing a second instance and you will be fine. You
will need a new disk, IP address, and network name for the new instance.
Geoff N. Hiten
Principal SQL Infrastructure Consultant
Microsoft SQL Server MVP