PixInsight Forum (historical)

PixInsight => Bug Reports => Topic started by: Marzio on 2019 November 23 12:32:29

Title: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 23 12:32:29
hi,

I installed the update about the batch preprocessing and new script weighted preprocessing script ...version 1.8.8-1.
I tried the new script to compare the image obtained with the one developed about a month ago with the classic script, but unfortunately the PixInsight task crash.
Out of curiosity I tried and rework the image with the old scrip ... but pixinsight crash.
Processing involves bias, dark and flat as master.
The images to be processed are 119.
I tried to reduce the number of images to be processed and discovered that with 10 it works.
So I made several attempts, enlarging the number of photos to be processed and I found out that PI crashes past the 100 photos ...100 frame = ok ...101 frame = crash !
PI crashes immediately after starting the debayer section.
System is win10, 64bit, 16gb ram, intel i7 4790 @ 3.60 ghz, 8 logical processor

this is the system (http://www.marziobambini.it/images/photoscratch/pi_system.jpg)
pi at start (http://www.marziobambini.it/images/photoscratch/pi_start.jpg)
pi at script diagnostic (http://www.marziobambini.it/images/photoscratch/pi_script.jpg)
pi at last map before crash with greather then 100 frame (http://www.marziobambini.it/images/photoscratch/pi_100.jpg)

Can you help me?

Thank you so much, CS.

Marzio

ps: and this is the final photos ...my small contribution to cassiopea ... https://www.astrobin.com/w07bwx/B/?nc=user


Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 24 00:48:59
hi,

with new script wbpp and 110 - 119 frame this is what you see:

(http://www.marziobambini.it/images/photoscratch/wbpp-err-110.jpg)

(http://www.marziobambini.it/images/photoscratch/wbpp-err-119-a.jpg)

I tried with 50 photos and it works ... now I try around 100 to see where the dead line is

tks

CS
Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 November 24 12:45:37
Hi Marzio,

Bug confirmed, and fixed. I have already answered to a similar bug report (https://pixinsight.com/forum/index.php?topic=14265.msg85732#new) to describe this problem:

During the intensive tests we have carried out today, we have discovered an obscure bug in version 1.8.8-1 of PixInsight that has been causing sporadic segmentation faults during execution of several scripts, including BPP and WBPP. The bug happens during garbage collection (GC) of closed image window objects performed by the JavaScript engine. Since JavaScript's GC is non-deterministic, the bug is difficult to reproduce. Furthermore, it only happens during relatively aggressive GC tasks, which are more likely during very long script executions and/or working with large amounts of data (keep in mind that the garbage collector has a fixed threshold of 4 GiB to start collecting unused objects automatically, so this is independent on the amount of available RAM).

This bug is now fixed in version 1.8.8-2 of the PixInsight core application. We'll release it as a regular update tomorrow. Hopefully this will fix all of these problems during script execution.


I apologize for the inconvenience. Sorry for the trouble.
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 25 00:25:54
Hi Juan,

absolutely not a disturbance, developing complex systems is normal to run into situations like these.
Indeed, it is I who thank you for your timeliness, even from here you can understand how alive this project is.
Today I can't, but tonight when I return from work I will download the update and try both BPP and WBPP with the complete project database of 119 frames and I will report the result.

Thanks again.

Ciao, Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 25 11:57:02
Hi Juan,

I downloaded the 1.8.8-2 update and tried the classic BPP script with all 119 frames. Sorry to say but PixInsight Crash again.

Do you want me to send you the material to perform tests in your development environment?

Thanks, Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 25 12:09:16
Hi Juan,

I also tried WBPP, the system crashes at the usual point, I attach an image of the screen before the crash.

Tell me how I can help you.

Hello Marzio

(http://www.marziobambini.it/images/photoscratch/bug_wbpp_1882_a.jpg)
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 November 26 02:28:40
Hi Marzio,

This is not a crash in PixInsight. An 'unknown exception' denotes an error generated outside of PixInsight (otherwise it wouldn't be unknown), which has interrupted an ongoing operation. In this case, the problem has occurred during execution of the Debayer process.

Unknown exceptions can be caused by a wide variety of issues. Looking at the Activity Monitor in your screenshot, the most probable causes can be:

- Your machine only has an Intel HD graphics adapter. There are known problems with these GPUs on Windows in recent versions of PixInsight; for example, see the Known Problems section in the official announcement for version 1.8.8-2 (https://pixinsight.com/forum/index.php?topic=14301.0).

- At the moment of the unknown exception, both your processor and your hard disk were subject to heavy pressure. This might be a disk I/O error or hardware failure. It could be also a file system error.

Anyway, if you can upload the whole data set (dropbox, etc), I'd like to repeat the WBPP process on our test machines. This would be of great help. Thank you for your support.
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 26 23:24:32
Hi Juan,

wow ...I saw !!!

I took the whole database and moved it to an old laptop that I use to control shooting sessions, a PC with only 8GB of RAM and a 4 core logical unit.
Both script, bpp and wbpp, work correctly.
Now I have to try to understand what could have happened to the main pc, I have been using it for 5 years without problems ... bho !!!
Thank you for everything and sorry for the inconvenience.

CS
Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: dchamberlain on 2019 November 27 08:11:12
Hi Marzio,

This is not a crash in PixInsight. An 'unknown exception' denotes an error generated outside of PixInsight (otherwise it wouldn't be unknown), which has interrupted an ongoing operation. In this case, the problem has occurred during execution of the Debayer process.

Unknown exceptions can be caused by a wide variety of issues. Looking at the Activity Monitor in your screenshot, the most probable causes can be:

- Your machine only has an Intel HD graphics adapter. There are known problems with these GPUs on Windows in recent versions of PixInsight; for example, see the Known Problems section in the official announcement for version 1.8.8-2 (https://pixinsight.com/forum/index.php?topic=14301.0).


- At the moment of the unknown exception, both your processor and your hard disk were subject to heavy pressure. This might be a disk I/O error or hardware failure. It could be also a file system error.

Anyway, if you can upload the whole data set (dropbox, etc), I'd like to repeat the WBPP process on our test machines. This would be of great help. Thank you for your support.

Hi Juan,

I am getting consistent crashes using WBPP version 1.4.2 with PixInsight 1.8.8-2. Always crashing during “Computing descriptors” so the weighing feature of this script is doing me no good. The error is attached. Now my system has a single graphics adapter Intel HD Graphics 4600. Are you saying I will need to put in a non-Intel HD graphics card to avoid this error?

Thanks

Dale
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 November 27 10:20:17
Hi Dale,

Quote
my system has a single graphics adapter Intel HD Graphics 4600. Are you saying I will need to put in a non-Intel HD graphics card to avoid this error?

If you want to continue using PixInsight on Windows, then probably yes. Assuming you are using Windows 10, try updating your graphics drivers to the latest version available. If this does not solve these issues, then there is nothing more I can do.

In all of our tests, AMD graphics cards have caused zero problems on all platforms. We have seen some issues with NVIDIA graphics cards on Windows 10, although they have been fixed completely by installing the latest driver versions.

There are no problems at all on Linux and FreeBSD. There have been minimal usability issues on macOS with integrated Intel HD graphics (MacBook laptops), which are very easy to work around, although we have not seen them since we have upgraded to macOS 10.15.
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 27 13:51:11
Hello everybody,

my card is also an Intel 4600 HD.

I checked the drivers and they are up to date.
But are we sure it's a graphics card problem? and if I change it and the problem remains?
I apologize for my incompetence, but how can it be that Intel distributes a card with defects? ... or bad drivers? ... is it not in their interest to ensure that the card has no problems?
I don't understand ... I confess I am dazed.

Tks, Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 27 15:43:49
Hi Juan,

I apologize, this is the reworking done this evening, how can I think that it is the fault of the graphics card?

(http://www.marziobambini.it/images/photoscratch/BPP_a.jpg)

and then immediatly

(http://www.marziobambini.it/images/photoscratch/BPP_b.jpg)

Tks again, Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 November 28 00:01:08
Hello everybody,

in the weekend I will proceed to uninstall PI, I will test on the ram and on the hd, then I will proceed to the installation of PI 1.8.8. I will try to stress the BPP script and if everything is OK I will proceed with the 1.8.8-1/2 update. Then I do the test again and I make the point.

Thank you very much.

CS
Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 November 28 02:24:11
Marzio,

Your graphics card is not the problem. It surely is working fine. The problem is with Intel HD graphics drivers on Windows, which cause problems with Qt-based applications. This has been happening for a long time (since 2016 at least, if I can remember well). For example (found just after a quick Google search):

https://bugreports.qt.io/browse/QTBUG-72531

https://forums.intel.com/s/question/0D50P0000490HYmSAM/qtbased-applications-working-with-opengl-were-broken-after-driver-update?language=en_US

https://invent.kde.org/kde/kdenlive/issues/277

Some links selected from the above page:

https://discourse.vtk.org/t/problem-with-view-update-since-intel-uhd-graphics-620-driver-update/1224

https://discourse.vtk.org/t/intel-uhd-620-vtk-qt-bug-with-new-drivers-on-windows/1694

https://stackoverflow.com/questions/45541690/all-qt-applications-on-my-windows-machine-have-garbled-gui

Intel HD graphics drivers have a long history of bugs and incompatibilities, especially in their OpenGL implementation on Windows. The access violation errors happen because PixInsight is trying to draw something on the screen using OpenGL acceleration. It expects everything to work as described by OpenGL standards, but the graphics driver is probably doing something in a different and unexpected way.

You can try to disable OpenGL acceleration by running PixInsight from a command line (press Windows+R) as follows:

C:\Program Files\PixInsight\bin\PixInsight.exe --opengl=raster

In theory this disables OpenGL GPU acceleration on the whole PixInsight platform. However, with recent versions of Qt this does not always work reliably on Windows, and OpenGL is still being used. Anyway, try it and perhaps this solves the problem in your particular case.

Please realize that there is nothing more I can do because this problem is completely beyond my control.
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: dchamberlain on 2019 November 28 07:48:25
Hi Juan,

I apologize, this is the reworking done this evening, how can I think that it is the fault of the graphics card?

(http://www.marziobambini.it/images/photoscratch/BPP_a.jpg)

and then immediatly

(http://www.marziobambini.it/images/photoscratch/BPP_b.jpg)

Tks again, Marzio

Hi Marzio,

I was also having problems with WBPP using the latest versions of PI, WBPP and the graphics adapter driver. My graphics adapter is an Intel HD 4600. Yesterday I installed a new GTX 1650 graphics card, disabled the integrated adapter via the BIOS settings and reran WBPP twice with no problems. I am convinced the Intel HD graphics drivers are to blame for this situation.

Hope this helps!

Dale
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 November 29 02:56:44
Hi Dale,

Quote
I was also having problems with WBPP using the latest versions of PI, WBPP and the graphics adapter driver. My graphics adapter is an Intel HD 4600. Yesterday I installed a new GTX 1650 graphics card, disabled the integrated adapter via the BIOS settings and reran WBPP twice with no problems. I am convinced the Intel HD graphics drivers are to blame for this situation.

Thank you for posting this information. It is very helpful and reiterates what we have already been experiencing during years.

We have seen also a few problems on Windows with NVIDIA cards, but all of them have been fixed by simply updating drivers to their latest versions.
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: alphascorpi on 2019 November 29 10:03:26
Hi Juan
I have the same problems with a GEFORCE RTX 2080. I have inbstalled today the latest driver.
In the last 12 month PI runs always without problems with version 1.8.6. Now with the latest version 1.8.8.-2 I have the same errors with BPP and WBPP.
In my posting 3 days ago I have described the rig and the procedures.
Thanks for any help
Ed (alphascorpi)
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Marzio on 2019 December 15 01:19:01
Hi Dale,
thank you for your contribution, as well as I thank Juan for patience and Alphascorpi which I hope with my contribution to help.
I apologize for not having responded immediately, but it has been a rather busy period of work and only yesterday I managed to devote myself to the problem.
Reading your post Dale I went to a lab yesterday and installed the GTX 1650 graphics card. Unfortunately the result has not changed. PI crashes. Then we tried with 2 other graphics cards ... but nothing to do, PI crashes.
The lab technician tells me that he thinks it's not a graphics card problem, but it's more likely a problem related to hard disk performance.
He proposes to change the disk to go on ssd technology, unfortunately not now, I have no time, I agree with him but I refer to the operation in January.
I come home very sad, I haven't solved the problem and I still don't know if the proposed solution will solve it.
thinks he thinks about it ... in xp I remember you were going to touch the paging file to help the system.
I look for and find the parameter set to automatic management with the size set exactly equal to the size of the ram. 16 GB. 32gb septum manual management just to try and

..... magic .... problem solved !!!

ED (alphascorpi) I hope my contribution can help you.

A greeting to all and the wish for peaceful holidays
Ciao
Marzio
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: Juan Conejero on 2019 December 17 11:44:27
Hi Marzio,

I am very happy to know that. Yes, computers can be magical sometimes. The change to SDD is a very good recommendation. Happy Holidays!
Title: Re: After installation of release 1.8.8-1 script batch prepocessing crash
Post by: dchamberlain on 2019 December 17 12:39:17
Hi Dale,
thank you for your contribution, as well as I thank Juan for patience and Alphascorpi which I hope with my contribution to help.
I apologize for not having responded immediately, but it has been a rather busy period of work and only yesterday I managed to devote myself to the problem.
Reading your post Dale I went to a lab yesterday and installed the GTX 1650 graphics card. Unfortunately the result has not changed. PI crashes. Then we tried with 2 other graphics cards ... but nothing to do, PI crashes.
The lab technician tells me that he thinks it's not a graphics card problem, but it's more likely a problem related to hard disk performance.
He proposes to change the disk to go on ssd technology, unfortunately not now, I have no time, I agree with him but I refer to the operation in January.
I come home very sad, I haven't solved the problem and I still don't know if the proposed solution will solve it.
thinks he thinks about it ... in xp I remember you were going to touch the paging file to help the system.
I look for and find the parameter set to automatic management with the size set exactly equal to the size of the ram. 16 GB. 32gb septum manual management just to try and

..... magic .... problem solved !!!

ED (alphascorpi) I hope my contribution can help you.

A greeting to all and the wish for peaceful holidays
Ciao
Marzio

One thing to underscore here: I'm not sure if your motherboard supports this, but I had to turn off the integrated graphics. Even after installing the GTX1650, PI crashed until I changed this setting. Now it works like a champ. Also, my primary drive (Windows) is SSD, but my working drives are not.

Hope this helps!

Dale