Author Topic: Read Write speeds  (Read 554 times)

Offline blueridgeDSIA

  • Newcomer
  • Posts: 26
    • View Profile
Read Write speeds
« on: 2018 July 19 10:39:01 »
Im not sure if there is a bottleneck somewhere in my system but I would really like Pi to push my system at least to 75% when it is performing a process.  It does a good job of this during image registration and integration but when I do a batch file conversion for instance or an image container/process container batch process on lets say 200 dslr images, I find that Pixinisght is not pushing my system very hard and it seems to take longer than necessary to complete an operation. 

For example today I converted a big stack of RGB images to Lum and my 960evo m.2 ssd has very high read wrote speeds, but Pi was utlizing less than 25% of the write speed, the cpu and ram were not maxed out.  I dont see a bottle neck anywhere, is there something in the software that limits read/writespeeds?  Could this be user adjusted in Windows?


Online pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4549
    • View Profile
Re: Read Write speeds
« Reply #1 on: 2018 July 19 14:11:24 »
can a single x86 thread saturate an m.2 ssd? meaning, are disk benchmarks which test maximum disk performance multi-threaded? there's a certain amount of handshaking and queuing going on all up and down thru the memory hierarchy that could limit the performance of a single thread.

rob

Offline Xplode

  • Newcomer
  • Posts: 5
    • View Profile
Re: Read Write speeds
« Reply #2 on: 2018 December 17 09:00:59 »
The problem is that Batch file conversion only uses a single thread, i would love to see it upgaded to handle multi thereads since i convert all my files to XISF with compression, i save around 40-50% space and that's a lot with a 250GB+ with XISF files.
I believe it shouldn't be that much work to make it able to run parallel threads on different images

Online pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4549
    • View Profile
Re: Read Write speeds
« Reply #3 on: 2018 December 17 09:32:06 »
i think the issue is that the javascript engine is single-threaded, thus all PI scripts are single-threaded. that's what i was alluding to in my reply.

rob

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 6934
    • View Profile
    • http://pixinsight.com/
Re: Read Write speeds
« Reply #4 on: 2018 December 17 12:32:02 »
Yes, this is a bottleneck that has to be removed ASAP. I think I can write a C++ version of the format conversion script quite easily. It's in the to-do list with high priority. Thanks for the suggestion.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/