Author Topic: PixInsight 1.8 — Renewed Script Editor  (Read 16438 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
PixInsight 1.8 — Renewed Script Editor
« on: 2012 August 14 03:56:52 »
Hi again everybody,

As I said in my previous post, the Script Editor window has also been revamped in PixInsight 1.8 Ripley. In fact, I have rewritten about an 80% of it from scratch, so we could say that it is actually a 'new feature'.


For a development platform like PixInsight, a powerful and versatile code editor is a very important piece of the puzzle. While most non-developer users won't find the usefulness of Script Editor very evident in an image processing application, we have many developers working on the PJSR (PixInsight JavaScript Runtime) and PCL (PixInsight Class Library) frameworks who really need and will love the new editor. So far our Script Editor has been useful, but not serious enough to become a code editor of choice. This is going to change in the new version. In fact, I am now using PI's Script Editor routinely to write source code of the PI core application.

The first change that most of you will notice in the screenshot above is the lack of the old Object Browser window, which was integrated with Script Editor in previous versions. This functionality—and a lot more—has been moved to a new window in 1.8 Ripley, namely the Object Explorer window, which I'll describe in a separate post.

Instead of the old Object Browser, the Script Editor window has a new document selection panel at its left side, where all open documents can be accessed quickly and efficiently on a tree structure. The documents are automatically organized by folders as they are being opened and saved. You can open new files at a specific folder by simply double-clicking the corresponding item, or you can close or save all files in a folder in a single operation. All of these options are available through context menus. The document selection panel has replaced the old tabbed control used to organize documents in previous versions.

One of the most powerful new features of Script Editor is its ability to handle two types of text selections: line and block selections. Line selections are the type of selections that can be found in normal text editors, and were the only ones available in previous versions of SE. This is an example:


Unlike line selections, which always include whole lines except for the first and last lines selected, block selections define rectangular regions of text, also known as column selections. This is a block selection defined with the same starting and ending points as in the example above:


Block selections provide the same basic operations as line selections: copy, cut, paste and delete. However, block selections are very important for efficient edition of source code because they allow to apply complex changes to large portions of text at once, in contrast to line selections, where the same operations usually require a lot of repetitive manual actions. Block selections can be defined with the mouse or movement keys in the usual way.

Another nice feature of the new Script Editor is automatic highlighting, as shown in the following example:


In this screenshot, the user has selected (by double-clicking for example) the word 'm_image', and the editor has highlighted all occurrences in the whole document. All coders know how important this feature is to quickly analyze and understand source code fragments.

All text search operations have been redesigned and reimplemented completely. By pressing Ctrl+F (Cmd+F on the Mac) , the Text Find widget is shown at the bottom side of the editor window:


The Text Find widget implements a live search operation, also called "search as you type". As you type or delete characters in the "Find" box, the text that you are typing is immediately searched for in the document, from the current cursor position, and selected if one match is found. The Highlight All button is very useful to preview all matches in the whole document.

As in previous versions, you can search for plain text with and without case sensitivity, you can search for whole words only, or search for regular expressions.

The Text Replace widget is shown as an extension of Text Find at the bottom of the editor window by pressing Ctrl+R (Cmd+R):


To complete the set of interactive search and replacement tools, the Go To Line widget is activated by pressing Ctrl+L (Cmd+L):


You'll notice also a new menu item in Script Editor: Tools. Under this menu you can perform a number of useful operations on selected text (or on the whole document if there is no text selection): Sort text lines (with and without case sensitivity, both locale-aware and UFT code point based sorting), convert text to uppercase and lowercase characters, join selected text lines into a single line, apply automatic word wrap to text lines, and compute MD5 and SHA1 checksums.

The Script Editor Preferences dialog is similar to previous versions, but it has been extended to include new features, such as automatic word wrapping, and much more syntax highlighting items that are necessary to cover the new languages that Script Editor can highlight now.


Script Editor supports now automatic syntax highlighting for the following programming and scripting languages: JavaScript, C++, XML, PIDoc and PixelMath. For JavaScript, Script Editor implements special highlighting items for PJSR keywords and macros (for example, all installed processes are highlighted as 'external objects'). For C++, PCL special keywords and macros are also highlighted (such as all macros starting with "__PCL_" and keywords such as size_type, uint32, etc.), as well as Qt keywords and macros (such as signal, slot, qobject_cast<>, etc., mainly because we use Script Editor to write PI Core code).

As noted above, PixelMath source code can also be highlighted with specific items, such as all operators and functions available in the PixelMath tool. And this leads us to the final point in this brief description: Starting from version 1.8.0, Script Editor can be instantiated as a PCL control (the same as PushButton or TreeBox for example), which means that any module can embed its own instances of the Script Editor component as part of its graphical interface. PixelMath is an example of this new integration functionality. This is still not 100% functional but I am working on it right now; it will be available when we release 1.8 in September, along with a new version of the PCL framework.

That's it for now. I'll be posting more information on PixInsight 1.8 Ripley soon on this forum.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #1 on: 2012 August 14 07:45:12 »
Could you add a replace feature, where it replaces a string in all the opened documents? This would be great to change the name of variables, or the name of the process, for example.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #2 on: 2012 August 14 09:08:25 »
"In other news tonight, Pleiades Astrophoto has declared Emacs and Vi being ancient toys not suitable for serious code work and intends to replace them with an astro photo editor which happens to have a text editor built in. When asked about a built in javascript debugger to facilitate code development beyond the typing 'n staring phase, the answer was ....."

I'll admit to using the Visual Studio text editor when I use that IDE but I much prefer vim. Then again I'm pretty old school.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #3 on: 2012 August 14 09:16:18 »
Talking about old school, I use Kwrite, or gedit :)
(yeah, I could be using vim or nano, but sometimes you have to take advantages of a more modern UI :D )
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #4 on: 2012 August 14 09:24:08 »
Some of my first programming was done on a DEC TOPS 20 machine if I recall correctly. It used a line editor with a paper terminal. When we were allowed to use a VAX 750 with CRT terminals and a full screen editor we were very very happy.

Of course I'm lucky I skipped the paper index card and tape phase. My dad brought plenty of those home for crafts though.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #5 on: 2012 August 14 11:33:21 »
How about integrating xyz (insert your favorite IDE) with PI (instead of implementing it in PI)?
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #6 on: 2012 August 14 11:42:30 »
Why any integration at all? All we need is the ability to run arbitrary scripts (already works of course), the ability to debug (not yet but on the roadmap I believe) and proper documentation for the library (pretty good last time I checked).

PCL has no integration with PI and that works fine too. I understand the desire to promote PJSR (PSJR? I forget) programming but I'm not sure this is the way to do it.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #7 on: 2012 August 15 02:49:12 »
I think that you don't fully understand what the PixInsight project is all about. PixInsight is not a photo editing application. There are enough photo editors out there; adding one more to the bag would be too boring.

PixInsight is an image processing and analysis platform specialized in astronomy and other technical imaging fields. This is not a toy and we are not here to do anything but pursuing the excellence in everything we do. Our goals are very ambitious and challenging for us, but unfortunately our resources are scarce and we have to work in a less-than-ideal environment (to put it smoothly, we don't live in a place where 'development' is a very valued concept), so we cannot carry out our development tasks as quickly and exhaustively as we would like many times, but we keep trying.

A code editor is a necessary component in the PixInsight platform, mainly because PixInsight is a development platform. And it has to be a very good and useful editor, not just "something to cover the basics", or "something to do the least possible effort" because both concepts are incompatible with PixInsight. The new script editor in PI 1.8 is just a step in this direction.

Regarding a debugger, I am waiting while Mozilla's new debugger API reaches a reasonable state of maturity. PixInsight 1.8 uses version 1.8.7 of the SpiderMonkey engine, where the debugger API should be included (PI 1.7 used SM version 1.8.5). When this happens we'll have a very good JavaScript debugger in PixInsight. And then you'll love the idea of having a good debugger integrated with a good and powerful editor, not just a 'so-so' thing.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #8 on: 2012 August 15 04:26:38 »
...Our goals are very ambitious and challenging for us, but unfortunately our resources are scarce and we have to work in a less-than-ideal environment...
Hi Juan,

please don't get me wrong: I really value the effort you put into PI, and the overall quality and functionality of PI is very very high. And setting the development priorities  is entirely up to you.

But: I believe there have been not too many requests for a new editor within PI, and there have not been too many requests for an improved file selection dialog. On the other hand, there have been quite a lot of requests for improved documentation, or improved ease of use. At the same time, there are quite a lot of development environments out there (for instance Eclipse and Visual Studio) that have amazing capabilities, support C++, JavaScript, Python and you-name-it, including documentation/object browsers and debuggers. Integrating these with PI is not easy, but possible thanks to plugin-concepts and documented APIs. And for the file dialog: At least for Windows there are quite a number of tools that extend the Windows Explorer (which is also used within file dialogs) to understand a large number of file formats, such as the FastPictureViewer Codec pack http://www.fastpictureviewer.com/codecs/ (with the notable exception of FITS).

I feel that it is not wise to invest scarce resources into file dialogs and editors. I fully agree with you that PI is also a development platform (one of the reasons why I love it). But at the core of it is image processing, and IMHO that is what PI should focus on. I would love to be able to use Eclipse to write PJSR scripts or PCL modules with direct access to API documentation, class hierarchies, syntax coloring, debugging, automated module installation, ... . PI should not try to do all of this by itself.

Again: I really appreciate the effort you put into PI, and I cant wait to get my hands on the new release. And I am sure I will love the new editor. I just think it would be possible to get more for less effort if PI focused on its special abilities and used existing tools for the other things.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #9 on: 2012 August 15 10:50:53 »
I'm with Georg. I wrote a much larger reply but that's pretty pointless so I've deleted it. I doubt anyone will be persuaded to buy PI because it has a text editor but maybe I'm misjudging people's priorities.


Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #10 on: 2012 August 15 16:19:58 »
i'm going to go out on a limb and say that such projects are Juan's equivalent of the google "20% time" project. meaning, when you are stuck on your main work, the 20% project gives you something useful to procrastinate on. while not a trivial undertaking, writing a text editor is probably a good deal more straightforward than some of the astro stuff, and it's fun because it's completely different.

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #11 on: 2012 August 16 03:56:36 »
Sander & Georg,

I am sure that your intention is helping us, but once again I think that you don't fully understand what PixInsight is and what our goals are regarding PixInsight as a development platform.

Since you are questioning the way we are investing our resources—and I have no problem at all with that—, here are the main reasons to invest significant development resources in a code editor:

1. The JavaScript engine is one of the most important components of PixInsight. Have you seen the kind of development projects that have been implemented by PI users during the last months? For example, take a look at the ImageSolver and AnnotateImage scripts by Andrés del Pozo. These are really serious tools, not "little utilities". These scripts have about 6000 lines of source code. Or consider the BatchPreprocessing script, which has more than 5000 lines and is currently one of our most famous and useful tools. Note that each of these scripts provides more functionality than some standalone applications entirely dedicated to the same tasks.

2. The PJSR in PI 1.8 is much more powerful than previous versions, with a new engine (SpiderMonkey 1.8.7) and more predefined objects that provide more image processing capabilities and full access to virtually any aspect of the PixInsight platform. We'll have also an integrated documentation system for PJSR: the new Object Explorer window, similar to Process Explorer, which I'll describe in a separate post. This makes PJSR a very attractive and powerful framework for many development and research tasks.

3. We have plans to improve PixInsight's JavaScript engine considerably. We'll have, among others:

- An integrated JavaScript debugger.

- A mixed JavaScript/C++ model based on an integrated C++ compiler (see the next point). Basically, the PJSR will allow you to write sections of your script as special C++ modules (including multithreaded code), which will be compiled and linked by PI transparently on demand.

4. PixInsight will have an integrated C++ compiler, based on clang. This means that, along with the JavaScript/C++ mixed model that I have described above, PCL-based C++ development will also be integrated, to some extent, with the core application.

5. Some PixInsight tools will benefit from an advanced editor component accessible through PCL. The best example is PixelMath. As I said in the introductory post, the whole functionality of Script Editor will be available as a Control subclass in PCL.

In my opinion, these reasons more than justify that a powerful code editor is absolutely necessary in an application like PixInsight. I am not trying to compete with Eclipse or Visual Studio (although I have to say that I like PI's editor much more than both of them, especially with its new automatic code hinting feature, which will also be available during the 1.8 cycle). It is just that we need a really good and powerful editor because we cannot implement most of the above features without it. That has been my only intention when I have rewritten most of the Script Editor component in PI 1.8.


Quote
I believe there have been not too many requests for a new editor within PI

If I'd guide my development work based only on what most users request, then we'd still be in the PixInsight LE stage. For example, many (lots of?) PI users limit themselves to DBE and a few more things, then continue working on their images using inferior image retouching applications (and no, the lack of documentation is not a reason for that). Based on that fact, we'd have no special reasons to work on new multiscale processing tools, to improve our image registration tools, to research new noise reduction algorithms, or to implement new deconvolution tools based on total generalized variation algorithms, just to name a few examples.

PixInsight is a long-term project. My responsibility is to create the necessary building blocks to make it possible. Most of the time I have to look at my feet to know the ground I'm walking on, but sometimes I also have to look at the horizon.


Quote
...FastPictureViewer Codec pack...

Very nice, but besides *all* the information that that utility provides, the new File Explorer component of PixInsight provides:

* Exhaustive FITS format support.
* Support for 8, 16 and 32 bit integer images, plus 32 and 64 bit floating point images.
* Automatic stretch functions applied to raw images.
* Real-time histogram previews.
* Real-time image statistics.
* FITS header keywords.
* Multithreaded, non-blocking thumbnail generation and recursive directory analysis.
* Multithreaded, non-blocking thumbnail and image information cache.
* Multiplatform implementation: available on FreeBSD, Linux, Mac OS X and Windows.
* Fully integrated with PixInsight.
* Does not alter the images (thumbnails are maintained in a dedicated cache, *not* inserted in image files).

My intention with File Explorer is to create a *useful* tool for all PixInsight users. Again, I am not trying to compete with Windows Explorer, Finder or KDE's Dolphin; I am just trying to provide some functionality that will increase PixInsight's usability and performance.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #12 on: 2012 August 16 04:47:56 »
Hello Juan,

thanks for this carefully written answer. We probably have to agree that we disagree on some of your priorities. I do understand and honor your point of view, but my idea of the priorities is different  :-[ .

I am looking forward to 1.8 Ripley.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline bitli

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 513
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #13 on: 2012 August 16 13:24:01 »
For my own usage, and since version 1.0, the priority is to get documentation of the key functions easily accessible.  I bought PI 1.0 based on the high quality of documentation if PI LE, and on the promise that the documentation will follow soon.
If the new editor helped doing a cut and paste between the release notes and the PI doc format, so that we have at least some basic information available on most processes, then it will have been useful.
I am glad that PI is a better development platform for the people developing new tools (they are key contributors), but I really though that the goal of PI was also to be a astro image processing tool for the rest of us. May be reasonably complete documentation will be part of 1.8? In this case I welcome the new editor as an additional capability.
-- bitli

Offline Javio

  • PTeam Member
  • PixInsight Enthusiast
  • **
  • Posts: 79
    • PixInsight
Re: PixInsight 1.8 — Renewed Script Editor
« Reply #14 on: 2012 August 21 08:47:36 »
Hi guys,

just to say there is life beyond the forum. We also have direct requests of features. By the way there is a lot of planning work behind the new features and sometimes we also need to add some feature to help us work too. I can´t imagine how we can go ahead with a project of some hundreds of thousands code lines with vim or emacs (check Stallman's mental health). Personally, I´m fed up of Kwrite, Gedit and Anjuta bugs (and even Kate until the last release...), so I really thing this new editor will improve (at least) my/our day by day job confort. Javio dixit.

An image platform without a file explorer? Come on, this feature was ultra necessary. We were hammering Juan´s head for months asking him to implement it, and finally he did it during his holidays. You guys you never catch some holidays?

Dealing with the documentation, there is something like 100 modules and tons of scripts and features. All of you know that the documentation explorer was a late addition to PI. So we have to combine docs, issues, improvements, concept design and coding, as all of you know. We will come back to documentation as soon as some major concerns will be solved and, personally, when I´ll come back to work from my paternity leave (coders we also have life, wife, childs,  and labor rights to keep everything going on). So keep on enjoying dark skies  :o

Javio
Javier Sanchis Muñoz
PixInsight Development Team
http://pixinsight.com/