I loaded both 64 bit and 32 bit versions of 1.8-RC3 onto separate machines respectively running Windows 7 64bit Ultimate and Windows XP-SP3 today and both run considerably more slowly than version 1.7.
Thank you for your report. We have detected a performance regression in version 1.8.0 RC3, which affects irregularly to some tools. I have already identified and fixed this problem today, and right now I'm preparing a new 1.8.0 RC4, which hopefully will be the last one before the final 1.8.0 release.
As for the performance regression in RC3, some tools are more affected than others, and StarMask is indeed one of the worst cases. However, the numbers you're showing are much more severe (more than twice) than what we have measured on Intel-based machines. One possibility is that the performance loss be hardware-dependent, being worse on AMD processors. Not too surprising because the problem is related to atomic operations (reference/dereference operations in multithreaded code).
As for ATrousWaveletTransform, take into account that the previews (including real-time previews) of this process in version 1.7 were so inaccurate that they were basically useless. They are accurate in version 1.8, but this requires computing the transform for the whole image, not just for the preview or the real-time preview. Once the whole transform has been cached, if you avoid modifying parameters that invalidate the cache (as resetting the tool, changing the number of layers, or the scaling function), previews are (especially once we have fixed this performance regression in RC4) considerably faster than in version 1.7. The same applies to MultiscaleMedianTransform, where I have implemented the same cache system.