Analysts at Sandia National Laboratories who have been tasked with answering strategic questions posed by executives, program managers, and sponsors typically perform data analysis studies. However, the challenges of data provisioning for these studies—data collection, integration, quality, and other considerations—are complex and time-consuming, limiting the number of studies performed (and therefore questions asked) at any given time, as well as making the studies long, drawn-out affairs. When the analysts provide answers, the data they used is no longer current, and the results represent a look back at a point in time, resulting in significant additional effort to “refresh” the results.
Sandia’s response to this problem was to architect and build a platform that we called “Analytics for Sandia Knowledge.” This platform utilizes a set of tools, techniques, and custom web applications to create a centralized, integrated, and virtual data repository designed for data analysis and visualization. This platform has significantly reduced Sandia’s “time-to-answer” for data studies, allows more data studies to occur, supports security and privacy needs, and allows us to create sustainable and up-to-date visualizations of data study results. The architecture can support both traditional and Big Data approaches and lays a foundation that is scalable, extendable, and adaptable.
This experience report will provide an overview of the architecture goals, quality attributes, final design, and some lessons learned along the way in creating the virtual data repository and data visualization platform.
