Containerizing the telemetry data pipeline for MMTO subsystems

J. Duane Gibson; Carlos Burguillo-Rodriguez; Timothy Pickering; Dallan Porter; Scott Swindell; Will Goble

doi:10.1117/12.2626505

29 August 2022 Containerizing the telemetry data pipeline for MMTO subsystems

J. Duane Gibson, Carlos Burguillo-Rodriguez, Timothy Pickering, Dallan Porter, Scott Swindell, Will Goble

Proceedings Volume 12189, Software and Cyberinfrastructure for Astronomy VII; 121891B (2022) https://doi.org/10.1117/12.2626505
Event: SPIE Astronomical Telescopes + Instrumentation, 2022, Montréal, Québec, Canada

Conference Poster

Abstract

The telemetry data pipeline for the MMT Observatory (MMTO) describes the flow of data sampled from diverse hardware devices within MMTO subsystems, through logging into various databases, to user interfaces and monitoring services. Subsystems within the pipeline include the telescope mount, primary and secondary mirrors, instruments, and environmental sensors. Data acquisition services within the pipeline post new data with a uniform data structure to a master Redis server. These incoming data are transported in real-time to replicated Redis servers where they are logged into local MariaDB relational databases. Database tables for logged data from the subsystems are highly optimized for data storage, allowing the archival of billions of data points for thousands of parameters over the past 10-15 years. Because of ever increasing difficulty in supporting legacy servers and software, a large-scale containerization effort is underway of the various components of the telemetry pipeline and underlying cyberinfrastructure. These critical servers and services are single points of failure that could result in up to weeks of operational downtime. Containerization helps to reduce the risk of potential hardware failure, operating system upgrades, and software incompatibilities. Containerizing a service defines all the software requirements for that service, including the code, runtime, system tools, system libraries, and settings. It allows rapid and reliable redeployment of new and legacy services with minimal concern for the underlying hardware. Finally, a summary of the ongoing and planned future work is presented.

Citation Download Citation

J. Duane Gibson, Carlos Burguillo-Rodriguez, Timothy Pickering, Dallan Porter, Scott Swindell, and Will Goble "Containerizing the telemetry data pipeline for MMTO subsystems", Proc. SPIE 12189, Software and Cyberinfrastructure for Astronomy VII, 121891B (29 August 2022); https://doi.org/10.1117/12.2626505

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available