There has been a tradition of operators, physicists, and other experts outside of the Controls Group writing software tools for beam-line monitoring, processing diagnostics and other complicated but automatable tasks. There were a few issues with some of these tools. Some were web pages, run on personal servers, some were simple scripts that had to be deployed onto the controls consoles. The one thing that was consistent was the inconsistency, especially between user interfaces. Some programs relied on paid software licenses that broke occasionally. Many people had interesting ideas for applications but did not have the expertise to write something from scratch. Further, there was no obvious group or community to ask for advice.
TRIUMF's first attempt at creating the HLA development environment was to use the open source application framework library OpenXAL originally developed at SNS. Some HLAs using this framework were created by students. However, this approach does not meet all of the requirements that follow.
We wanted a way to cultivate a productive development environment, where developers could easily share code and expertise. We envisioned a system where:
The solution we came up with was a Web Application framework written in Python. Python is the obvious choice because it is the prevailing language for writing scientific software. Most physics students will have learned some simple scripting in Python, and for everyone else it is a very accessible programming language to learn.
We decided to use web applications instead of native/local applications for many reasons:
For a more technical history and a summary of the early HLA architecture one should read the Beam Physics Design Note: TRI-BN-19-01 "A Brief History of High Level Applications at TRIUMF" by Carla Barquest. Since then, a lot has changed, but the vestiges of our history cling to us, because well, we're still using a lot of the old code.
In the essay "The Cathedral and the Bazaar" open source developer Eric S. Raymond reflects on his experiences as a open source software project manager.
The following is a summary of the 6 first and most important lessons, retrieved on the 30th of January 2024, they are:
- Every good work of software starts by scratching a developer's personal itch.
- Good programmers know what to write. Great ones know what to rewrite (and reuse).
- "Plan to throw one away; you will, anyhow." (Fred Brooks, The Mythical Man-Month, Chapter 11)
- If you have the right attitude, interesting problems will find you.
- When you lose interest in a program, your last duty to it is to hand it off to a competent successor.
- Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
The full essay is well worth a read, especially for aspiring software developers, and anyone interested in complex projects or open source. This wisdom is very important for how we structure the HLA enterprise. To highlight how this relates to the HLA project explicitly:
"The Rise of Worse is Better" is a section of a 1991 essay on Lisp, written by software developer and computer scientist Richard P. Gabriel. In it, he describes the cultural divide between the East and West coast of the United States regarding the quality of software.
Dr. Gabriel concludes his thoughts on "Worse is Better" with the following, retrieved on the 15th of February 2024:
The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.
This is also worth a read. Individual developers have varying tastes and will inevitably fall somewhere between the styles of the East Coast and the West Coast. Dr. Gabriel himself spent many subsequent articles arguing back and forth for either side. Wherever you fall, this discussion can help inform both day-to-day HLA development decisions, and more broadly, the software architecture of the HLA framework.
Some wisdom we can directly apply to HLA development from the "Worse is Better" style is:
TRIUMF as a part of the global scientific enterprise values collaboration, inclusion and transparency. We rely heavily on free open-source software to get work done. The open-source software community thrives only when developers share their projects, experiences and feedback. Thus, we contribute by making the source code of many of our projects freely available, licensing permitting.
More specifically to our framework, the Python code that powers the backend of the Web framework is divided up into a set of individual software packages that functions independently with scientific workflows and scripts. This means that those students and staff who do not need to write web applications can still utilize parts of our framework to do productive work.
One of TRIUMF's core missions is cultivating Highly Qualified Personnel. Not all staff and students come to the HLA team with a comprehensive programming background. We aim to train developers though the HLA ecosystem in such a way that they learn skills and technologies that are broadly useful in both their academics and industry.