Prev: P4.1 Next: P4.3

P4.2: Allen, Christopher
Christopher Allen (Smithsonian Astrophysical Observatory)
Francis A. Primini (Smithsonian Astrophysical Observatory)
Joseph B. Miller (Smithsonian Astrophysical Observatory)





Theme: Data Science: Workflows Hardware Software Humanware
Title: Optimization of Aperture Photometry in the Chandra Source Catalog

As part of the creation of the Chandra Source Catalog, the Aperture Photometry systems characterize source photon counts, count rates, energy flux, and photon flux for over 300,000 sources. These sources are characterized per observation and across blockings of similar observations, leading to approximately three million unique characterizations. As development progressed and our test sets grew both in both complexity and size, runtime issues were identified - scenarios where the system was too resource-intensive or had projected processing times far outside of the project timeframe. These issues lead to a continual reevaluation of our implementation of the tool, all the while keeping the underlying scientific algorithms intact. Herein we will discuss the algorithm, the challenges we encountered scaling it up for production, and the optimizations we added as a result. These design decisions cover a broad scope, from the mundane of restructuring nested loops to minimize file I/O, to the intricate of modifying our statistical sampling methods and replacing our hypermesh-based fitting with a Markov chain Monte Carlo sampling that was then converted into a probability distribution. Analysis tools used to identify code hotspots will be reviewed, as well as the coding tools we used to refactor some computationally expensive sections of the tool. In exploring these decisions in detail, the pros and cons of both the chosen solution and alternative options considered but not chosen will be discussed. The tool’s high-level design will also be analyzed, elaborating on how we were able in a single tool to perform analysis on the broad variety of sources the catalog encounters - from point sources to broad extended emissions, and from the faintest detections to brightest.

Link to PDF (may not be available yet): P4-2.pdf