PI on Linux and color management

[Andrés Vattuone]

Hi dear PI friends,

I am looking forward to buying PI and I am also planning the acquisition of new hard and soft. In this regard  I am trying to decide about the best system configuration for PI, since I am planning to make PI one of my top applications. On the other hand I am also very keen on color management, having dealt with this matter in the case of traditional potography.
I've had no experience whatsoever with Linux distributions but from what I have learned in this forum it shows the best performance. Since I am not willing to split my workflow accross different OS, I am wondering about the possibilities of color management in Linux. I've come across Argyll Color Management System.
Are there any opinions or experiences that you could share with this forum?
Thanks and regards.

Andrés

[Nocturnal]

If you do not have any experience with Linux are you sure this is how you want to gain it? I mean you'll be new to PI and to the OS. Why not start on Windows or Mac and then try the linux version later? Your only reason to use linux was performance and I don't think that's really so clear cut. On W7-64 with a fast CPU PI runs very well. If it's not fast enough, get a faster CPU and faster drives. If you love Linux for other reasons, go for it.

[astropixel]

Andres'. I run PI on 64bit ubuntu, and it runs very well. I suspect that one of the developers may be able to advise you on color management. I would suggest, after using Linux for almost 10 years that a system such as ubuntu, where there is plently of support and automatic updates and system configuration, is probably the path of least resistance. You may also find that someone is already doing what you are proposing with color management and find support in the ubuntu forums. Having said that, there are numerous Linux systems, each with its own attributes and drawbacks. A desktop that has received praise on these forums is FreeBSD - though I believe a little short on applications to choose from. PI runs from the home/user directory in Linux and requires no special installation or library dependencies - no brainer. You just need to link the executable to your desktop or application bar for convenience.

[Juan Conejero]

Hola Andrés,

Welcome to PixInsight Forum.

You've addressed several important topics that deserve separate discussions. As for color management in PixInsight, it works exactly the same on all supported platforms: FreeBSD, Linux, Mac OS X and Windows. This is because PixInsight has its own, OS-independent color management system. PI's color management system is based on the Little CMS engine and its quality and performance is at least as good as (and in some aspects better) than CMSs provided by well-known commercial applications. In addition, please note that (to my knowledge) PixInsight is the only image processing application specialized in astrophotography with color management capabilities.

Regarding operating systems, the choice depends on many factors that are always different for each user. As the author of a multiplatform application, I can have a certain OS-agnostic perspective (yes, I can have it if I want ). Something I can say without hesitation is that performance isn't a good reason for selecting an OS --it is in fact one of the worst reasons. Actually, performance in terms of execution speed is basically the same on all platforms for a given hardware, once everything has been well configured. For me, a few important questions one has to answer in order to make an OS decision are the following:

- (a) I want an easy-to-use OS / I don't want to have problems. In ascending order of difficulty: Mac OS X, Windows, 'soft' Linux distributions (Ubuntu), 'hardcore' Linux distributions (Fedora), FreeBSD.

- (b) I want to have freedom of choice. For example, one can select among many different desktop environments on X11 based systems (FreeBSD, Linux) but this is not possible on Windows and Mac OS X.

- (c) I want to have full control over my system / I don't want features designed so you don't have to think about ..., aka designed to help you. In ascending order of control possibilities: Mac OS X, Windows, Linux, FreeBSD.

- (d) I love open source. In ascending order of openness: Windows, Mac OS X, Linux, FreeBSD.

My personal choices are Linux and FreeBSD, mainly due to reasons (b) and (c), among others. While I am writing this on my Linux workstation, my Linux laptop has hung due to a kernel crash when it attempted to suspend the machine after a period of inactivity (happens rarely, though). This is where (a) bites you...

[Nocturnal]

Juan's list is good but conveniently he left out the one factor where Windows wins easily and the one that's most important to 95% of users. Windows has the highest compatibility with software and hardware, by far. This is true for 'normal' software and astronomy software both. What this means is that a Windows PC is the most universal compute tool you can buy. There are so many software and hardware choices that competition keeps innovation going and prices low. Macs are more expensive than windows systems and you can do less with them. Sounds like a terrible choice to me. Linux systems are as expensive as windows systems ($100 Windows license is negligible difference, if this truly matters to you get out of astro photography now) but offer less hardware compatibility and far less software compatibility.

I buy computers because they are tools. Not because they are jewelry.

[Sean]

"Macs are more expensive than windows systems and you can do less with them."

Oh boy, here we go, religious OS war

Macs are the cheapest systems I've ever used (I was a PC user and developer for over 20 years) because I don't waste all my time with bugs, crashes, viruses, head-pounding, swearing, etc etc. My dog likes Macs because there's a lot less shouting in my office these days.

I guess if you don't value your time very highly, PC's are OK.

Thank you Juan for a Mac version of PI.

Sean

Glad I switched and never ever going back.

[Andrés Vattuone]

Thank you all for your feedback. Thanks Juan for your warm welcame greeting and for the neat OS summary. When considering performance I saw the oportunity to try Linux, though I realise now that it might not be the most important consideration. On the course of my investigation I discovered Argyll CMS and will implement it on Windows. I can grasp the feeling of control and freedom you get with this kind of sofware, and I like it indeed. It is also one of the reasons I chose PI, the confidence in its algorithms being another one. Now I am a proud owner of a PI license. If control and freedom weren't enough, well, then let's say that guys who implement Linux seem on average very smart to me. So why not try it?
But the question of my post that concerns me the most is color management.
I believe that the requirement of a color managed workflow is central to any serious imaging application. Even if you haven't got a camera profile, even if you don't care about the G2V method or if you are doing narrow banding, you would want to share your work consistently.
I already checked that PI fully implements color management in Windows. The Little CMS engine seems to do a great job. But what about the rest of the system in a Linux implementation? Has anyone calibrated and profiled a display in Linux and confirmed that the video card effectively loads the profile?
Perhaps the only question I would pose to PI ( Little CMS) would be this: can it handle and convey 10 bit images to a capable video card? I am not sure about the state of the art on this matter or whether I should ask this question to the SO's color management module instead.
Thanks and regards,

Andrés

[astropixel]

I'm not sure if I am answering your question. I copied the inbuilt PI color profile to /usr/share/color and made it the default profile. There should be plenty of help and tutorials on the forums.

[Juan Conejero]

I believe that the requirement of a color managed workflow is central to any serious imaging application. Even if you haven't got a camera profile, even if you don't care about the G2V method or if you are doing narrow banding, you would want to share your work consistently.

Please note that color management has nothing to do with color calibration. Color management is just a set of standardized procedures and tools to allow a device to correctly interpret the pixel values in an image. Color management and color calibration are completely unrelated processes.

I already checked that PI fully implements color management in Windows. The Little CMS engine seems to do a great job. But what about the rest of the system in a Linux implementation? Has anyone calibrated and profiled a display in Linux and confirmed that the video card effectively loads the profile?

As I said before, PI provides the same color management system on all platforms. A monitor color profile can be easily created on Linux with the Argyll system. Once you have generated an ICC profile characterizing your monitor, you just have to select it with the ColorManagementSetup tool in PixInsight (Edit > Color Management Setup). As there is no truly standardized color management system on UNIX/Linux, this tool allows you to select a new monitor profile manually (this option is not available on Mac OS X and Windows, where the monitor profile is a standard system-wide setting). Use the System Settings (X11 Only) section of the tool's interface to select a new monitor profile (and read the tool tip).

As for the rest of applications, color management support on Linux varies greatly. Most leader open-source applications provide fully color managed environments (and most of them use the LittleCMS engine as PI does). This includes GIMP, Inkscape and Scribus. Probably more applications provide color management support, but I have not checked them. In general, you should have no problems, unless you need a specific application that has no CM support.

[Juan Conejero]

Has anyone calibrated and profiled a display in Linux and confirmed that the video card effectively loads the profile?/quote]

I detect a possible misunderstanding here. Note that ICC profiles are managed and used through software exclusively. The graphics card knows nothing about the meaning that a specific numeric value has for your monitor. It is in your operating system and/or in your software applications where the appropriate translations take place.

[Andrés Vattuone]

Juan, you are right in that, strictly speaking, color management refers to the technology of color conversion and gamut mapping. But in our context, a conditio sine qua non to achieve its goal is characterization of input and output devices, or at least monitor characterization, i.e profiling. Astropixel, let me be categorical:  in order to take advantage of color management as defined by ICC and implemented through LittleCMS you MUST have your monitor profiled. For this you need a colorimeter/spectrometer and a piece of software. For the sake of better conveying our work to others I humbly  encourage every member of this forum to implement it.
It seems that neither Argyll nor LPROF, the only profiling tools in Linux that I know of,  are V4 compliant. The newest version of LittleCMS is.To see the importance of this you may read:

http://www.color.org/ICC_white_paper_6_v2_and_v4_display_profile_differences.pdf

Please note that the use of a monitor profile presupposes that the monitor is in the same state of calibration (white point, brightness, etc.) as it was when characterised. My painful experience with laptops running under Windows told me that this can be tricky. I am not sure were this calibration settings are stored in this case.  See however "vcgt" tag below.
With respect to the video card digression I am very much confused, since I don't know which element in the pipeline does what and to what extent. Your comments have been helpful Juan, though.

I quote from Wikipedia

>>Monitor calibration on a computer is done preferably with a colorimeter and software
>>that supports it. In most cases, a monitor profile created by the calibration software
>>(whether it relies on a hardware device or on user input) includes both an accurate
>>description of monitor output and instructions to the video card to change the colors
>>it displays. Video card lookup table (LUT) adjustments are usually required to help
>>the display reach the white point and gamma desired by the user.
>>Monitor calibration is, in that respect, different from calibration of other devices. A
>>monitor profile created by calibration software (as opposed to the profiles supplied by
>>monitor manufacturers) usually includes instructions to the video card that are stored
>>in a special "vcgt" tag of the profile.
>>Video card LUT adjustments need to be loaded for a monitor to match the
>>description in the profile. Mac OS X and Microsoft Windows 7 load LUT adjustments
>>automatically, while Linux and older versions of Microsoft Windows require
>>standalone LUT loaders. Although such loaders perform just as good a job when
>>adjusting video card output as the operating system would, their existence tends to
>>confuse many people with regards to the function these loaders perform.

>>Use an LUT loader to actually load the LUT adjustments contained within the profile
>>prepared during calibration. According to the documentation, these loaders do not
>>modify the video card LUT by itself, but achieve the same type of adjustment by
>>modifying the X server gamma ramp. Loaders are available for Linux distributions that
>>use X.org or XFree86—the two most popular X servers on Linux. Other X servers are
>>not guaranteed to work with the currently available loaders. There are two LUT
>>loaders available for Linux:
>>Xcalib is one such loader, and although it is a command-line utility, it is quite easy
>>to use.
>>dispwin is a part of Argyll CMS.

The sentence "these loaders do not modify the video card LUT by itself, but achieve the same type of adjustment by modifying the X server gamma ramp" is unintelligible to me.
If I understood you well Juan, Little CMS in Pi does "load" the monitor LUT by itself without any other intervention by loaders or SO in Linux. How do Linux and PI (Little CMS) treat the "vcgt" tag of the profile?
In the same vein, I guess that LittleCMS  implements 16bit transformation at least. So if a monitor has an effective 10bit depth per channel (e.g. Eizo monitors), the only bottle neck in Linux would be the video card or its drivers. Right?

Thanks and regards,

Andrés

[Andrés Vattuone]

An update as a result of my ongoing research:

http://ninedegreesbelow.com/2011/imaging/color-management-tutorial/monitor-profile-calibrate-confuse.html

I think it clarifies most of the discussion, at least theoretically.
There remains the question of 10bit support. Any suggestion will be much appreciated.
Thanks.

[Juan Conejero]

Hi Andrés,

PixInsight Core uses 32-bit floating point for all color management operations, such as ICC profile transformations. In the ICCProfileTransformation tool there is an option to disable floating point (and hence to use 16-bit integer arithmetics), but it is enabled by default.

PixInsight ignores the vcgt profile tag. The reason is that loading hardware LUTs and gamma curves is not the responsibility of an image processing application. These operations must be implemented by the operating system or a specialized software system, such as a video card driver or a calibration system. PixInsight just loads the monitor profile and uses it to perform the necessary transformations to render images appropriately on the screen. The transformations are actually implemented in LittleCMS, which PixInsight uses as its color management back-end.

[Andrés Vattuone]

PixInsight ignores the vcgt profile tag. The reason is that loading hardware LUTs and gamma curves is not the responsibility of an image processing application. These operations must be implemented by the operating system or a specialized software system, such as a video card driver or a calibration system.

Understood, seems fair and customary. People who want to implement a color managed workflow must ensure that the profiled monitor and video card are in the same state of calibration, eventually loading settings with approriate loaders. Argyll seems to be able to do that in a Linux environment. I've already checked this functionality of Argyll under Windows.

PixInsight Core uses 32-bit floating point for all color management operations, such as ICC profile transformations. In the ICCProfileTransformation tool there is an option to disable floating point (and hence to use 16-bit integer arithmetics), but it is enabled by default.

No doubt at this point about the precision of operations in PI. However, how does PI communicate data to the video interface? Is it capable of choosing 10 bit pixel format in the sense of the following article? Is it a 10-bit aware OpenGL application or something of the like?

http://www.amd.com/us/Documents/10-Bit.pdf


Regards

Andres

[Andrés Vattuone]

Hi Juan/PI team,

One question was shaded from my eyes in our previous posts:

Does PI implement the latest version of LittleCMS, i.e. the v4 compliant version?

And one question remains to be answered unless I didn't grasp it.

Appart from its internal precision, is PI capable of transmitting 10bit depth to the monitor?

Thanks and regards,

Andrés