Richard,
I see the same behavior on my XP system. Although, on my system, it generally only takes a few seconds for all the sub-directories to appear. From a programming perspective, this may be the expected behavior, at least , when there's a delay of only a few seconds. It seems to be populating the view with the most recent directories you've viewed in the past. This gives fast access to those you most commonly use; however, it can become annoying if you switch directories a lot. Perhaps, there should be an option to turn this behavior off if it is unwanted, assuming this is the expected behavior.
I'm not sure why it's taking your system so long to refresh.
Wade