Author Topic: PixInsight 1.6.0 - New Multithreading Capabilities  (Read 7327 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
PixInsight 1.6.0 - New Multithreading Capabilities
« on: 2010 March 30 13:52:34 »
Hi there!

The new version 1.6 of PixInsight introduces several new features that improve parallel processing on Windows and Linux/X11 platforms. Here is a brief description of these new features.



Thread CPU Affinity Control

PixInsight 1.6 allows you to enable/disable thread CPU affinity control. This is a global setting that affects threads created by the core application as well as all external modules.

The affinity of a thread defines the set of logical processors on which the thread is eligible to run. Thread affinity allows to improve execution speed by restricting each thread to run on a separate processor. This prevents the performance cost caused by the cache invalidation that occurs when a process ceases to execute on a processor and restarts execution on a different one.

When thread CPU affinity control is enabled, PixInsight automatically assigns each running thread to a logical processor. Logical processors include both physical processors and processor cores, as well as virtual processors that the operating system sees on systems and platforms with hyper-threading technology.

CPU affinity control is available on Linux and Windows exclusively (it will also be available in the upcoming port of PixInsight to FreeBSD). Unfortunately, Mac OS X does not provide the necessary system resources to control CPU affinity at the level necessary to implement this features.

Real-Time Thread Scheduling (Linux only)

On Linux, PixInsight 1.6 automatically selects the SCHED_FIFO real-time scheduling policy when it detects that the user (actually, the process that has created the PixInsight Core process) has sufficient privileges. This greatly improves the efficiency of parallel processes and threads for all standard modules and the core application, yielding true workstation performance on multiprocessor and multicore machines. For detailed information on Linux thread scheduling policies, see the following online documents:

http://linux.die.net/man/2/sched_setscheduler
http://linux.die.net/man/2/getrlimit

To enable this feature, the user must have the necessary resource limits assigned. In particular, the user's RLIMIT_RTPRIO limit (see the second document above) must be set through the /etc/security/limits.conf configuration file. The standard procedure is as follows:

- As root, edit /etc/security/limits.conf

- Add a line specifying the desired RLIMIT_RTPRIO limit. For example:

juan hard rtprio 90

is the corresponding setting on my workstation (I am user juan on my Linux box). The above line says that I have the right to increase my real-time thread priority up to 90. Valid values are from 0 to 99.

When PixInsight detects a rtprio value above zero at startup, it automatically scales all thread priorities accordingly, such that real-time priority corresponds to the rtprio value (for example, rtprio=90 in my case).

If you set your rtprio above 80 and configure PixInsight to use all processor cores available on your Linux box, be sure that your machine has very good refrigeration :)
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: PixInsight 1.6.0 - New Multithreading Capabilities
« Reply #1 on: 2010 March 30 16:47:28 »
Juan,

do you have examples for the benefits of those 2 features? How much additional performance to I get? In my applications, it is usually well below 10%.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Re: PixInsight 1.6.0 - New Multithreading Capabilities
« Reply #2 on: 2010 March 30 16:49:21 »
Juan,

do you have examples for the benefits of those 2 features? How much additional performance to I get? In my applications, it is usually well below 10%.

Georg


I think the second benchmark went from 14 to 10 sec (Juan must confirm this).


V.

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: PixInsight 1.6.0 - New Multithreading Capabilities
« Reply #3 on: 2010 March 31 08:13:21 »
Time to fork over the goods Juan. You are just toying with us. :(

Btw, did figure a way to assign a reference image to so you do a sequence such as registration and then integration in process/image containers?  A nice script will do too.  ;)

Max