Prev: O10.2 Next: F3

O10.3: Buchschacher, Nicolas
Nicolas Buchschacher (University of Geneva)
Fabien Alesina (University of Geneva)
Julien Burnier (University of Geneva)




Time: Tue 10.00 - 10.15
Theme: Databases and Archives: Challenges and Solutions in the Big Data Era
Title: No-SQL databases: An efficient way to store and query heterogeneous astronomical data in DACE.

Data production is growing every day in all domains. Astronomy is particularly concerned with the recent instruments. While SQL databases have proven their performances for decades and still performs in many cases, it is sometimes difficult to store, analyse and combine data produced by different instruments which do not necessarily use the same data model. This is where No-SQL databases can help to solve our requirements: how to efficiently store heterogenous data in a common infrastructure ? SQL database management systems can do a lot of powerful operations like filtering, relation between tables, sub-queries etc. The storage is vertically scalable by adding more rows in the tables but the schema has to be very well defined. In the opposite, No-SQL databases are not restrictive. The scalability is horizontal by adding more shards (nodes) and the different storage engines have been designed to easily modify the structure. This is why it is well suited in the big data era. DACE (Data and Analysis Center for Exoplanets) is a web platform which facilitates data analysis and visualisation for the exoplanet research domain. We are collecting a lot of data from different instruments and we regularly need to adapt our database to accept new data sets with different models. We recently decided to do a major change in our infrastructure after using PostgreSQL to use CASSANDRA for the storage and Apache Solr as an indexer to do sophisticated queries among a huge number of parameters. This recent change accelerated our queries and we are now ready to accept new data sets from futur instruments and combine them with older data to do better science. DACE is funded by the Swiss National Centre of Competence in Research (NCCR) PlanetS, federating the Swiss expertise in exoplanet research.

Link to PDF (may not be available yet): O10-3.pdf