[1.8RC1 Python]: Output from Python Threads throws exception

[georg.viehoever]

In the Python module I am currently developing,  I have redirected output from Python  (e.g. via print statement) to the PixInsight console by mapping Python's sys.stdout to the Python Console object, which is a wrapper for the pcl::Console object. There, the Python::Console::write() method is mapped to pcl::Console::Write(). This works perfectly fine.

However, if I create threads in Python (http://docs.python.org/2.7/library/threading.html), and those threads try to print something, I get an exception (see stack trace in attached screenshot). The problem apparently is that pcl::Console::Write() notices that the Write method is called from a foreign thread, and thus throws an API error with "WriteConsole(): Low-level API function error".

Question: Which method can I use from any thread to output something to the PI console. I dont really care if characters are mixed with other threads (that's something that python threads need to consider anyway in other environments as well).

Georg

[Juan Conejero]

You cannot invoke GUI output operations, such as writing to pcl::Console, from a running thread. However, in PCL 2.0 a pcl::Thread has off-line console output capabilities, which you can use to implement your Python console output quite easily.

The key member function is:

String pcl::Thread::ConsoleOutputText() const

The basic idea is: Instead of redirecting Python's sys.stdout to pcl::Console, define a pcl::Thread descendant and perform all of your console text output from a running instance of this class. For example (just an outline):

Code: [Select]

#include <pcl/Atomic.h>
#include <pcl/AutoLock.h>
#include <pcl/Console.h>
#include <pcl/Thread.h>

namespace pcl
{

class ThreadSafeConsoleOutput : public Thread
{
public:

   ThreadSafeConsoleOutput() : Thread(), console(), mutex(), scheduled(), finished( 0 )
   {
   }

   virtual void Run()
   {
      for ( (void)finished.FetchAndStore( 0 ); !finished; )
      {
         // Keep the UI responsive
         Module->ProcessEvents();

         // If there is some text scheduled for output, flush it.
         // Use Mutex::TryLock() to reduce thread contention.
         if ( mutex.TryLock() )
         {
            if ( !scheduled.IsEmpty() )
            {
               console.Write( scheduled );
               scheduled.Clear();
            }

            mutex.Unlock();
         }
      }
   }

   void Write( const String& s )
   {
      volatile AutoLock lock( mutex );
      scheduled.Append( s );
   }

   void WriteLn( const String& s )
   {
      volatile AutoLock lock( mutex );
      scheduled.Append( s );
      scheduled.Append( '\n' );
   }

   void WriteLn()
   {
      volatile AutoLock lock( mutex );
      scheduled.Append( '\n' );
   }

   friend ThreadSafeConsoleOutput& operator <<( ThreadSafeConsoleOutput& o, const String& s )
   {
      o.Write( s );
      return o;
   }

   void Finish()
   {
      (void)finished.FetchAndStore( 1 );
   }

   String Text() const
   {
      volatile AutoLock lock( mutex );
      return ConsoleOutputText();
   }

   void Flush()
   {
      volatile AutoLock lock( mutex );
      FlushConsoleOutputText();
   }

private:

   Console   console;
   Mutex     mutex;
   String    scheduled;
   AtomicInt finished;
};

} // pcl
As you see, the ThreadSafeConsoleOutput class above mimics the relevant functionality of pcl::Console, so it can be used to redirect the output of any text-based asynchronous system, such as Python's sys.stdout. Basically, ThreadSafeConsoleOutput allows you to gather console output text in a thread-safe way. At any moment, i.e. while the thread is running, you can call its Text() or Flush() member functions to retrieve its accumulated text or send it to the console, respectively. For example, you can use an external object driven by a pcl::Timer to flush console output at regular intervals.

Let me know if this helps you.

(P.S.: Do you see now why PCL threads cannot be implemented as OpenMP threads? With the current PCL 2.0 Thread class, we have a lot of rich functionality and control that we can only have with our own implementation).

[georg.viehoever]

Juan,
thanks for the advice. I think I can implement something along these lines.

Regarding your PS: I have spent half a day to drill down to this unexpected behaviour-the original error message that I saw within Python pointed into a fairly different direction. So forgive me if I say that sometimes (imperfect) standards are preferable to (perfect) homegrown solutions such as large parts of PCL-especially if the philosophy+details behind it are less documented/known than for instance QThreads, ThreadBuildingBlock or OpenMP. I know that we have different points of view on this, and I respect your opinion. Mine sometimes is just different.

Thanks for helping with this.

Georg