Learn IDL Programming to Visualize Data with

IDL, or Interactive Data Language, is a computer programming language primarily used for data analysis and data visualization. While it first appeared in 1977 (with roots going back even further), and is a bit specialist in its scope, it remains an important language forty years on.

IDL is still under active development and is used extensively in space science — including in major data analysis work done by NASA and The European Space Agency.

History of IDL

IDL’s current usefulness in space-oriented data analysis is attributable largely to its conception and history. In the 1970s, David Stern was working at the University of Colorado at Boulder in the Laboratory for Atmospheric and Space Physics (LASP).

While there he began working to design ways to allow scientists and other researchers to test hypotheses and perform data analysis on their own, without bringing in computer programmers to develop purpose-specific applications.

The first attempt at this was a program for the PDP-12, called Rufus. PDP, or Programmed Data Processor, was a sort of proto-minicomputer used for various data analysis tasks.

Rufus wasn’t a language yet, just a calculating program written in Fortran. Rufus evolved along with various needs at LASP, including a version used to analyze data from the Mariner missions to Mars.

The work on Rufus eventually led to a full-fledged programming language called SOL. This was designed to run on the PDP-8 (the same PDP version used for the Mariner work) and was very similar to Fortran in syntax and style.

David Stern founded a company called Research Systems Inc (RSI), in order to commercialize the work begun on Rufus. RSI’s first product was the Interactive Data Language, designed to run on the PDP-11.

RSI’s very first customer was NASA. Subsequent versions written for the VAX/VMS platform were adopted by other major players in the aerospace and astronomy industry including the National Center for Atmospheric Research (NCAR), the University of Michigan, the University of Colorado, and the Naval Research Laboratory.

With the rise of general-purpose computers, RSI shifted the focus of IDL development from VAX/VMS to Unix. This required a rewrite in C, and this rewritten version is the “genetic ancestor” of the language currently in use. However, the language has hardly remained static.

As technology has changed, many new features have been adopted into IDL. For example, in the early 1990s, event-driven graphical user interfaces (GUIs) were introduced.

In the late 1990s, both web services and object-oriented programming were introduced. The language is currently (as of this writing) in version 8.5, and is available on all major platforms and operating systems.

RSI changed owners a few times, including a short period when it was owned by Eastman Kodak (the film company).

The company changed names to Exelis Visual Information Solutions, which was a wholly owned subsidiary of Exelis ITT. Exelis was bought by Harris Corporation in 2015. David Stern is no longer involved with development.

IDL has been used in a number of important astronomy-related projects. It was used to process the images of Halley’s Comet taken by the European Space Agency’s Giotto spacecraft, and it was also used to diagnose problems with the Hubble telescope.

About the Language

IDL is a dynamically-typed, vector-based language with a syntax similar to Fortran. It is particularly adept at dealing with arrays and matrices. The language provides tools for mathematical analysis and — most crucially — for data visualization.

Other features of the language include:

• Separate namespaces for variables, functions and procedures
• Multi-threaded functions and procedures
• Function arguments are passed by reference
• Named parameters called keywords which are passed by reference
• Named parameter inheritance in nested routine c

  alls;

• Common block declarations and system variables that share variables between separate routines
• A basic form of object-oriented programming
• A persistent, global heap for pointer and object variables
• Composite data types such as character strings, homogeneous-type arrays, lists, hash tables, and simple (non-hierarchical) record structures of mixed data types.

Resources

Online Resources

• The JHU/APL/S1R IDL Library: a library of over 600 general-purpose astronomy related routines for IDL.
• Craig Markwardt’s IDL Library: routines and tutorials.
• Learn IDL: collection of routines and learning resources maintained by a professor at Texas A&M University who has written several books on the language.
• Coyote’s Guide to IDL Programming: one of the best online sources of information about IDL; almost entirely the work of a single IDL expert and enthusiast.
• A Slug’s Guide to IDL: simplified explanations of IDL concepts, for students.
• Robbie’s IDL programming: a somewhat disorganized collection of personal IDL projects and tools.
• MichaelGalloy.com: blog about IDL development from the author of Modern IDL.
• Rob Dimeo’s IDL Programs: another personal collection of tools from an IDL developer.

Tools

• Idlwave: IDL Emacs editing and shell mode
• The Motley IDL Library: a collection of routines several Object Graphics classes, a base class for widget applications and several applications built on it, functions that make it easy to represent scaled data on axes, classes for accessing netCDF files and several utility routines.
• TeXtoIDL: a collections of routines that make it simple to use Greek letters, subscripts and superscripts in making labels for plots in IDL.

Implementations

The canonical implementation of IDL is now managed by Harris Corporation and can be found at their website.

There are two open source alternatives:

• GDU Data Language
• Fawlty Language.

Books on IDL

• An Introduction to Programming with IDL: Interactive Data Language by Kenneth Bowman: a great introductory text suitable even for people with little programming experience of any kind.
• Practical IDL Programming by Liam Gumley: a very clear and straightforward look at building real IDL programs for solving actual problems.
• Object Oriented Programming with IDL by Ronn Kling: OOP capability was added to IDL in the late 1990s. At this point, that was a long time ago, but the culture of language use had already had 20 years to develop. Even today, many IDL programmers are not using OOP techniques. This book, which does not require knowing OPP concepts to get started, shows how to apply object oriented programming methodology in IDL.
• Application Development with IDL by Ronn Kling: this book moves the IDL developer from ad-hoc programming for specific analysis tasks to building full-scale applications in the language.

Not to Be Confused With…

IDL also stands for Interface Definition Language. This is not a specific language but is rather a description of a certain type of language functionality.

IDLs are specification languages used to design the interaction between two software systems in a language-independent way. There are a number of Interface Definition Languages.