Please do not set default thread priority to 'time critical'

[Nocturnal]

Hi Juan,

we talked about this before but I need to ask you again as 1.7 set the thread priority to time critical again. This is not cool. PI needs to be a good neighbor and share resources. If you can demonstrate that there is a benefit to having time critical threads let's see some numbers and have a discussion. Just now I saw that PI made my mouse cursor stutter when it performed an ACDNR operation. I set the thread priority to 'high' and repeated the operation. No perceivable delay and the mouse kept working as it should.

I think the current default behavior is not what new users should see. If you feel strongly (with data to back it up, preferably) that it should be time-critical you could add a note to the documentation or print a warning on the console for all I care.

ACDNR: Processing view: NGC6946
Writing swap files...
129.65 MB/s
In-place color space conversion: 100%
Building lightness mask: 100%
Processing CIE a* component: 100%
Processing CIE b* component: 100%
Processing CIE L* component: 100%
In-place color space conversion: 100%
11.16 s
Calculating view histograms...

Preferences: Global context
0.004 s

ACDNR: Processing view: NGC6946
In-place color space conversion: 100%
Building lightness mask: 100%
Processing CIE a* component: 100%
Processing CIE b* component: 100%
Processing CIE L* component: 100%
In-place color space conversion: 100%
11.18 s



In any case, please do not reset preferences upon upgrade. I'm not sure how my setting got changed but I hope (strongly) it'll remain untouched from now on.

[Juan Conejero]

1.7 set the thread priority to time critical again.

The following code is excerpted from pcl/Thread.h, lines 67-104:

Code: [Select]

/*!
   \namespace ThreadPriority
   \brief     Thread scheduling priorities.

   <table border="1" cellpadding="4" cellspacing="0">
   <tr><td>ThreadPriority::Inherit</td>      <td>Inherit the caller's thread priority. This is the default value.</td></tr>
   <tr><td>ThreadPriority::Idle</td>         <td>Schedule when no other threads are busy.</td></tr>
   <tr><td>ThreadPriority::Lowest</td>       <td>Schedule more often than \e idle priority, but less than \e low priority.</td></tr>
   <tr><td>ThreadPriority::Low</td>          <td>Schedule more often than \e lowest priority, but less than \e normal priority.</td></tr>
   <tr><td>ThreadPriority::Normal</td>       <td>Standard thread priority.</td></tr>
   <tr><td>ThreadPriority::High</td>         <td>Schedule more often than \e normal priority, but less than \e highest priority.</td></tr>
   <tr><td>ThreadPriority::Highest</td>      <td>Schedule more often than \e high priority, but less than \e time-critical priority.</td></tr>
   <tr><td>ThreadPriority::TimeCritical</td> <td>Schedule as often as possible, taking precedence over any other threads.</td></tr>
   <tr><td>ThreadPriority::DefaultMax</td>   <td>Default maximum priority. This is a platform-dependent, maximum thread priority recommended for processing modules.</td></tr>
   </table>

   Note that not all platforms and operating systems handle thread scheduling
   priorities in the same way. In general, modules should use the default
   maximum thread priority (ThreadPriority::DefaultMax) instead of hard-coded
   specific priority values. Time-critical priority should be reserved for
   performance-critical, very brief tasks exclusively. This is particularly
   important on Windows platforms.
*/
namespace ThreadPriority
{
   enum value_type
   {
      Inherit,       // Inherit caller's thread priority.
      Idle,          // Schedule when no other threads are busy.
      Lowest,
      Low,
      Normal,        // Standard thread priority
      High,
      Highest,
      TimeCritical,  // Schedule as often as possible, taking precedence over any other threads.
      DefaultMax = Highest
   };
}

As you see, the default priority is not time-critical, but highest. This applies to all platforms, not just Windows, and is being observed by all standard modules (I have verified this). In fact, time-critical priority is not being used by any module; just by the core application in some screen rendering routines that don't run unless you scroll or zoom an image interactively (e.g. with the mouse). The funny part is that this code has not changed since the last time we talked about this (two years ago?).

Fortunately I have a physical Windows machine at hand now (a Core i5 laptop) so I've been able to make some tests. Decreasing thread priority to Highest and High makes no difference at all. In order to keep the rest of the system responsive, thread priority has to be set to Normal. It seems something has changed in Windows recently (perhaps in Windows 7) because PI remains the same. For some reason thread priorities are now distributed in a rougher way: everything above Normal seems to be interpreted as TimeCritical, or nearly.

I'll change DefaultMax=Normal in the next version of PCL (Windows only) and will release updated versions of all standard modules for Windows. This will have to wait until September though.

[Nocturnal]

Hi Juan,

I did not examine actual OS thread priorities although that would have been a good idea. My 1.7 install said 'time critical' in the settings dialog. I'm quite certain I did not configure that on purpose by hand but I suppose it's possible. I recall a recent thread started by Steve I think where he ran into the same issue.

On this laptop I installed 1.7 a few months ago and its default setting is 'time critical'.

[Juan Conejero]

Note that the Preferences' maximum module thread priority setting does not force a thread priority; it is just the maximum allowed priority for modules, but no standard module uses anything above Highest by design.

[Nocturnal]

Well that's great but PI does not behave properly on W7 with that preference set to 'time critical'. Looks like some of these magic internal threads that will still use time critical by default are affecting the system in unpleasant ways. I therefore still request that the max priority gets lowered. Users should not have to fiddle with that unless there's demonstrable benefit to doing it this way. In my mind general system responsiveness (ability to move the mouse, bring windows back/forth, play music) should never be compromised by user applications.

[Juan Conejero]

So are you suggesting we implement file browsers and music players in PI?

Just kidding ... 

[Nocturnal]

Ha!

[georg.viehoever]

This problem still seems to exist in 1.74. No way to watch a DVD while integrating images 

[RobF2]

I know I'm a dinosaur still running XP32 SP3, but I wouldn't have ever known how to unwind this annoying behaviour from PI without help from this forum.  XP32 is paralysed unless you wind the thread priority back to normal.  Surely most modern OS's could organise themselves pretty effiiciently these days? 

+1 for Sander's suggestion.
(this was the very first thing I did after installing 1.7 recently)

[Juan Conejero]

The default thread priority used by all modules is always 'High', *never* 'Time-Critical', on the Windows version of PixInsight. This is true unless you select a maximum priority lower than 'High' (e.g. 'Normal') in Preferences.

Some processes use high-level parallelism that tend to sink a lot of processor resources, even with normal thread priority. Two examples are ImageCalibration and ImageIntegration, and also the next (still unpublished) major revision of StarAlignment. CPU affinity control also tends to aggravate this because it 'captures' specific processor cores, which become unavailable to other processes.

Lowering thread priority and disabling CPU affinity, and/or decreasing the number of processors used by PixInsight, will free more resources for other applications. PixInsight allows you to fine tune parallel processing to adapt it to your specific requirements, but the default configuration tries to maximize performance. PixInsight is a computing intensive application, so this default behavior seems reasonable IMO. I'll consider setting the default thread priority to 'Normal' or even 'Low' in the Windows version, but first I must be completely sure that this won't damage us in terms of competing performance.

[georg.viehoever]

... I'll consider setting the default thread priority to 'Normal' or even 'Low' in the Windows version, but first I must be completely sure that this won't damage us in terms of competing performance.

I believe there have been a couple of posts on this in the forum that indicate that there is no major performance penalty for running in normal or low priority. From my own experience with other programs, this is true for both Linux and Windows. I consider running compute intense processes in high priority harmful, because it will prevent vital other functionalities (such as GUI updates) and other less vital functions (such as recording/viewing) from working properly.

Georg

[RobF2]

I don't think users will care too much if a complex processing operation takes an extra few minutes versus another application.  I DO think a lot of people notice how much longer pre-processing operations take in PI, particularly as there is so much repetitive user interaction to get things done.

2 cents said - will pull head in now   

[twade]

To all,

I'm beginning to wonder if the "Maximum module thread priority" value is sticking (i.e. saved and used the next time you open PixInsight).  The reason I ask is that after closing my PixInsight session with it set to Normal I started to get the STF window disappearing again (see the topice here http://pixinsight.com/forum/index.php?topic=3729.0).

I also continue to see strange mouse behaviour in this version of PixInsight.  At times, it gets very jumpy where it pauses for a moment.  Perhaps, the user chosen value is being overwritten somewhere else?

Wade

[Juan Conejero]

Hi Wade,

As all preferences settings, the maximum module thread priority value is saved and restored in subsequent sessions. Otherwise we have a regression, but I have checked this with 1.7.4 and AFAIK it works correctly on all platforms.

Instead of the Preferences tool you can use the command line. Enter this command:

parallel -q

to learn the current parallel processing settings. You can also define the maximum thread priority:

parallel -t=4

4 is for normal thread priority.

Anyway I suspect this strange mouse behavior is due to other causes. Are you using a wireless device that emulates a mouse, such as a tablet? We have seen some problems with these devices.

[twade]

Juan,

Are you using a wireless device that emulates a mouse, such as a tablet?

No.

I initial had my thread count equal to the number of cores, instead of threads.  I set it to maximum and now the STF window stays.  For how long I'm not sure. 

I'm still getting a "Not responding" briefly in the area above the menu bar.  This is why I'm thinking PixInsight and Windows aren't "talking" properly. 

It is very strange to say the least.

If you need any screen shots or any help please let me know.  As stated in the other thread, PixInsight completely turns white during the unresponsive state. 

Wade