publications

2023

1. TiQuE: Improving the Transactional Performance of Analytical Systems for True Hybrid Workloads

   Faria, Nuno, Pereira, José, Nunes Alonso, Ana, Vilaça, Ricardo, Koning, Yunus, and Nes, Niels

   Proceedings of the VLDB Endowment 2023

   PDF

2022

1. Scalable transcriptomics analysis with Dask: applications in data science and machine learning

   Moreno, Marta, Vilaça, Ricardo, and Ferreira, Pedro G.

   BMC Bioinformatics Nov 2022

   Abs PDF

   Gene expression studies are an important tool in biological and biomedical research. The signal carried in expression profiles helps derive signatures for the prediction, diagnosis and prognosis of different diseases. Data science and specifically machine learning have many applications in gene expression analysis. However, as the dimensionality of genomics datasets grows, scalable solutions become necessary.

2. Adaptive Database Synchronization for an Online Analytical Cloud-to-Edge Continuum

   Costa, Daniel, Pereira, Jose, Vilaça, Ricardo, and Faria, Nuno

   In Proceedings of the 37th Annual ACM Symposium on Applied Computing 2022

   Abs PDF

   Wide availability of edge computing platforms, as expected in emerging 5G networks, enables a computing continuum between centralized cloud services and the edge of the network, close to end-user devices. This is particularly appealing for online analytics as data collected by devices is made available for decisionmaking. However, cloud-based parallel-distributed data processing platforms are not able to directly access data on the edge. This can be circumvented, at the expense of freshness, with data synchronization that periodically uploads data to the cloud for processing. In this work, we propose an adaptive database synchronization system that makes distributed data in edge nodes available dynamically to the cloud by balancing between reducing the amount of data that needs to be transmitted and the computational effort needed to do so at the edge. This adapts to the availability of CPU and network resources as well as to the application workload.

3. AIDA-DB: A Data Management Architecture for the Edge and Cloud Continuum

   Faria, Nuno, Costa, Daniel, Pereira, Jose, Vilaça, Ricardo, Ferreira, Luis, and Coelho, Fabio

   In The 1st International Workshop on Secure FunctiON ChAining and FederaTed AI - SONATAI 2022 co-located with IEEE 19th Annual Consumer Communications Networking Conference (CCNC) 2022

   Abs PDF

   There is an increasing demand for stateful edge computing for both complex Virtual Network Functions (VNFs) and application services in emerging 5G networks. Managing a mutable persistent state in the edge does however bring new architectural, performance, and dependability challenges. Not only it has to be integrated with existing cloud-based systems, but also cope with both operational and analytical workloads and be compatible with a variety of SQL and NoSQL database management systems. We address these challenges with AIDA-DB, a polyglot data management architecture for the edge and cloud continuum. It leverages recent development in distributed transaction processing for a reliable mutable state in operational workloads, with a flexible synchronization mechanism for efficient data collection in cloud-based analytical workloads.

2021

1. Geolocate: A geolocation-aware scheduling system for Edge Computing

   Vilaça, Joao, Paulo, Joao, and Vilaça, Ricardo

   In Proceedings of the 1st Workshop on High-Performance and Reliable Big Data (HPBD 2021) co-located with 40th International Symposium on Reliable Distributed Systems (SRDS 2021), Chicago, USA, September 20, 2021 2021

   Abs PDF

   Edge computing has emerged as an important paradigm that moves the computation and data storage of distributed services closer to users. While virtualization technologies, such as containers, have eased the task of running distributed services in heterogeneous edge hardware, these are still lacking adequate scheduling and orchestration algorithms that can indeed place computation near data producers. We present Geolocate, a generic scheduler for workload orchestration and distribution in containerized edge deployments. As the main novelty, the proposed solution takes into account both the available computational resources and the geographical location of nodes when deploying service components in these. A preliminary experimental evaluation of our prototype shows that Geolocate scheduling time is on par with other schedulers such as the KubeEdge default scheduler. Moreover, Geolocate shows average gains, even on single isolated requests, of about 62% in the response times of scheduled services.

2. Towards Generic Fine-Grained Transaction Isolation in Polystores

   Faria, Nuno, Pereira, José, Alonso, Ana, and Vilaça, Ricardo

   In Proceedings of the polystore systems for heterogeneous data in multiple databases with privacy and security assurances (Poly 2021) co-located with 47th International Conference on Very Large Data Bases (VLDB 2021), Copenhagen, Denmark, August 20, 2021 2021

   Abs PDF

   Transactional isolation is a challenge for polystores, as along with the limited capabilities of each datastore, we have to contend with their sheer diversity. However, transactional isolation is increasingly desirable as a variety of datastores are being sought after for roles that go beyond simple data lakes, where information is mostly static. Transactional guarantees are also relevant for reliability at scale. Finally, it would also close the gap to what is available in multi-model database systems. In this paper, we propose that transactional isolation in polystores can be achieved by leveraging the query engine, i.e., we implement some of the responsibilities of a transactional storage manager (TSM) in the query language itself. This has the key advantage of greatly simplifying design and implementation, as it doesn’t need to be re-invented for each datastore, and should increase performance, by taking advantage of dynamic query optimization where available. We demonstrate the feasibility of the proposal with a simple proof-of-concept and experiment.

3. Detailed Black-Box Monitoring of Distributed Systems

   Neves, Francisco, Vilaça, Ricardo, and Pereira, José

   SIGAPP Appl. Comput. Rev. Jul 2021

   Abs PDF

   Modern containerized distributed systems, such as big data storage and processing stacks or micro-service based applications, are inherently hard to monitor and optimize, as resource usage does not directly match hardware resources due to multiple virtualization layers. For instance, interapplication traffic is an important factor in as it directly indicates how components interact, it has not been possible to accurately monitor it in an application independent way and without severe overhead, thus putting it out of reach of cloud platforms.In this paper we present an efficient black-box monitoring approach for gathering detailed structural information of collaborating processes in a distributed system that can be queried for various purposes, as it includes both information about processes, containers, and hosts, as well as resource usage and amount of data exchanged. The key to achieving high detail and low overhead without custom application instrumentation is to use a kernel-aided event driven strategy. We validate a prototype implementation by applying it to multi-platform microservice deployments, evaluate its performance with micro-benchmarks, and demonstrate its usefulness for container placement in a distributed data storage and processing stack (i.e., Cassandra and Spark).

4. Horus: Non-Intrusive Causal Analysis of Distributed Systems Logs

   Neves, Francisco, Machado, Nuno, Vilaça, Ricardo, and Pereira, José

   In 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) 2021

   Abs PDF

   Logs are still the primary resource for debugging distributed systems executions. Complexity and heterogeneity of modern distributed systems, however, make log analysis extremely challenging. First, due to the sheer amount of messages, in which the execution paths of distinct system components appear interleaved. Second, due to unsynchronized physical clocks, simply ordering the log messages by timestamp does not suffice to obtain a causal trace of the execution. To address these issues, we present Horus, a system that enables the refinement of distributed system logs in a causallyconsistent and scalable fashion. Horus leverages kernel-level probing to capture events for tracking causality between application-level logs from multiple sources. The events are then encoded as a directed acyclic graph and stored in a graph database, thus allowing the use of rich query languages to reason about runtime behavior. Our case study with TrainTicket, a ticket booking application with 40+ microservices, shows that Horus surpasses current widely-adopted log analysis systems in pinpointing the root cause of anomalies in distributed executions. Also, we show that Horus builds a causally-consistent log of a distributed execution with much higher performance (up to 3 orders of magnitude) and scalability than prior state-of-the-art solutions. Finally, we show that Horus’ approach to query causality is up to 30 times faster than graph database built-in traversal algorithms.

5. CODBS: A cascading oblivious search protocol optimized for real-world relational database indexes

   Pontes, Rogério, Portela, Bernado, Barbosa, Manuel, and Vilaça, Ricardo

   In 40th Symposium on Reliable Distributed Systems, SRDS 2021, online event, [Chicago, USA], September 20-23, 2021 2021

   Abs PDF

   Encrypted databases systems and searchable encryption schemes still leak critical information (e.g.: access patterns) and require a choice between privacy and efficiency.We show that using ORAM schemes as a black-box is not a panacea and that optimizations are still possible by improving the data structures. We design an ORAM-based secure database that is built from the ground up: we replicate the typical data structure of a database system using different optimized ORAM constructions and derive a new solution for oblivious searches on databases. Our construction has a lower bandwidth overhead than state-ofthe- art ORAM constructions by moving client-side computations to a proxy with an intermediate (rigorously defined) level of trust, instantiated as a server-side isolated execution environment. We formally prove the security of our construction and show that its access patterns depend only on public information. We also provide an implementation compatible with SQL databases (PostgresSQL). Our system is 1.2 times to 4 times faster than state-of-the-art ORAM-based solutions.

2020

1. Black-box inter-application traffic monitoring for adaptive container placement

   Neves, Francisco, Vilaça, Ricardo, and Pereira, José

   In SAC ’20: The 35th ACM/SIGAPP Symposium on Applied Computing, online event, [Brno, Czech Republic], March 30 - April 3, 2020 2020

   Abs PDF

   A key issue in the performance of modern containerized distributed systems, such as big data storage and processing stacks or microservice based applications, is the placement of each container, or container pod, in virtual and physical servers. Although it has been shown that inter-application traffic is an important factor in placement decisions, as it directly indicates how components interact, it has not been possible to accurately monitor it in an application independent way, thus putting it out of reach of cloud platforms. In this paper we present an efficient black-box monitoring approach for detecting and building a weighted communication graph of collaborating processes in a distributed system that can be queried for various purposes, including adaptive placement. The key to achieving high detail and low overhead without custom application instrumentation is to use a kernel-aided event driven strategy. We evaluate a prototype implementation with micro-benchmarks and demonstrate its usefulness for container placement in a distributed data storage and processing stack (i.e., Cassandra and Spark).

2019

1. d’Artagnan: A Trusted NoSQL Database on Untrusted Clouds

   Pontes, Rogério, Maia, Francisco, Vilaça, Ricardo, and Machado, Nuno

   In 38th Symposium on Reliable Distributed Systems, SRDS 2019, Lyon, France, October 1-4, 2019 2019

   Abs PDF

   Privacy sensitive applications that store confidential information such as personal identifiable data or medical records have strict security concerns. These concerns hinder the adoption of the cloud. With cloud providers under the constant threat of malicious attacks, a single successful breach is sufficient to exploit any valuable information and disclose sensitive data. Existing privacy-aware databases mitigate some of these concerns, but sill leak critical information that can potently compromise the entire system’s security. This paper proposes d’Artagnan, the first privacy-aware multicloud NoSQL database framework that renders database leaks worthless. The framework stores data as encrypted secrets in multiple clouds such that i) a single data breach cannot break the database’s confidentiality and ii) queries are processed on the server-side without leaking any sensitive information. d’Artagnan is evaluated with industry-standard benchmark on marketleading cloud providers.

2018

1. Parallel Polyglot Query Processing on Heterogeneous Cloud Data Stores with LeanXcale

   Kolev, Boyan, Levchenko, Oleksandra, Pacitti, Esther, Valduriez, Patrick, Vilaça, Ricardo, Gonçalves, Rui C., Jiménez-Peris, Ricardo, and Kranas, Pavlos

   In IEEE International Conference on Big Data, Big Data 2018, Seattle, WA, USA, December 10-13, 2018 2018

   Abs PDF

   The blooming of different cloud data stores has turned polystore systems to a major topic in the nowadays cloud landscape. Especially, as the amount of processed data grows rapidly each year, much attention is being paid on taking advantage of the parallel processing capabilities of the underlying data stores. To provide data federation, a typical polystore solution defines a common data model and query language with translations to API calls or queries to each data store. However, this may lead to losing important querying capabilities. The polyglot approach of the CloudMdsQL query language allows data store native queries to be expressed as inline scripts and combined with regular SQL statements in ad-hoc integration queries. Moreover, efficient optimization techniques, such as bind join, can still take place to improve the performance of selective joins. In this paper, we introduce the distributed architecture of the LeanXcale query engine that processes polyglot queries in the CloudMdsQL query language, yet allowing native scripts to be handled in parallel at data store shards, so that efficient and scalable parallel joins take place at the query engine level. The experimental evaluation of the LeanXcale parallel query engine on various join queries illustrates well the performance benefits of exploiting the parallelism of the underlying data management technologies in combination with the high expressivity provided by their scripting/querying frameworks.

2017

1. HTAPBench: Hybrid Transactional and Analytical Processing Benchmark

   Coelho, Fábio, Paulo, João, Vilaça, Ricardo, Pereira, José Orlando, and Oliveira, Rui

   In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering, ICPE 2017, L’Aquila, Italy, April 22-26, 2017 2017

   Abs PDF

   The increasing demand for real-time analytics requires the fusion of Transactional (OLTP) and Analytical (OLAP) systems, eschewing ETL processes and introducing a plethora of proposals for the so-called Hybrid Analytical and Transactional Processing (HTAP) systems. Unfortunately, current benchmarking approaches are not able to comprehensively produce a unified metric from the assessment of an HTAP system. The evaluation of both engine types is done separately, leading to the use of disjoint sets of benchmarks such as TPC-C or TPC-H. In this paper we propose a new benchmark, HTAPBench, providing a unified metric for HTAP systems geared toward the execution of constantly increasing OLAP requests limited by an admissible impact on OLTP performance. To achieve this, a load balancer within HTAPBench regulates the coexistence of OLTP and OLAP workloads, proposing a method for the generation of both new data and requests, so that OLAP requests over freshly modified data are comparable across runs. We demonstrate the merit of our approach by validating it with different types of systems: OLTP, OLAP and HTAP; showing that the benchmark is able to highlight the differences between them, while producing queries with comparable complexity across experiments with negligible variability.

2. Prepared scan: efficient retrieval of structured data from HBase

   Neves, Francisco, Vilaça, Ricardo, Pereira, José Orlando, and Oliveira, Rui

   In Proceedings of the Symposium on Applied Computing, SAC 2017, Marrakech, Morocco, April 3-7, 2017 2017

   Abs PDF

   The ability of NoSQL systems to scale better than traditional relational databases motivates a large set of applications to migrate their data to NoSQL systems, even without aiming to exploit the provided schema flexibility. However, accessing structured data is costly due to such flexibility, incurring in a lot of bandwidth and processing unit usage. In this paper, we analyse this cost in Apache HBase and propose a new scan operation, named Prepared Scan, that optimizes the access to data structured in a regular manner by taking advantage of a well-known schema by application. Using an industry standard benchmark, we show that Prepared Scan improves throughput up to 29+ and decreases network bandwidth consumption up to 20+.

3. Performance Trade-Offs on a Secure Multi-Party Relational Database

   Pontes, Rogério, Pinto, Mário, Barbosa, Manuel, Vilaça, Ricardo, Matos, Miguel, and Oliveira, Rui

   In Proceedings of the Symposium on Applied Computing 2017

   Abs PDF

   The privacy of information is an increasing concern of software applications users. This concern was caused by attacks to cloud services over the last few years, that have leaked confidential information such as passwords, emails and even private pictures. Once the information is leaked, the users and software applications are powerless to contain the spread of information and its misuse.With databases as a central component of applications that store almost all of their data, they are one of the most common targets of attacks. However, typical deployments of databases do not leverage security mechanisms to stop attacks and do not apply cryptographic schemes to protect data. This issue has been tackled by multiple secure databases that provide trade-offs between security, query capabilities and performance. Despite providing stronger security guarantees, the proposed solutions still entrust their data to a single entity that can be corrupted or hacked. Secret sharing can solve this problem by dividing data in multiple secrets and storing each secret at a different location. The division is done in such a way that if one location is hacked, no information can be leaked. Depending on the protocols used to divide data, functions can be computed over this data through secure protocols that do not disclose information or actually know which values are being calculated.We propose a SQL database prototype capable of offering a trade-off between security and query latency by using a different secure protocol. An evaluation of the protocols is also performed, showing that our most relaxed protocol has an improvement of 5+ on the query latency time over the original protocol.

2016

1. SafeRegions: Performance Evaluation of Multi-party Protocols on HBase

   Pontes, R., Maia, F., Paulo, J., and Vilaca, R.

   In 2016 IEEE 35th Symposium on Reliable Distributed Systems Workshops (SRDSW) Sep 2016

   Abs PDF

   On-line applications and services are now a critical part of our everyday life. Using these services typically requires us to trust our personal or company’s information to a large number of third-party entities. These entities enforce several security measures to avoid unauthorized accesses but data is still stored on common database systems that are designed without data privacy concerns in mind. As a result, data is vulnerable against anyone with direct access to the database, which may be external attackers, malicious insiders, spies or even subpoenas. Building strong data privacy mechanisms on top of common database systems is possible but has a significant impact on the system’s resources, computational capabilities and performance. Notably, the amount of useful computation that may be done over strongly encrypted data is close to none, which defeats the purpose of offloading computation to third-party services. In this paper, we propose to shift the need to trust in the honesty and security of service providers to simply trust that they will not collude. This is reasonable as cloud providers, being competitors, do not share data among themselves. We focus on NoSQL databases and present SafeRegions, a novel prototype of a distributed and secure NoSQL database that is built on top of HBase and that guarantees strong data privacy while still providing most of HBase’s query capabilities. Safe Regions relies on secret sharing and multi-party computation techniques to provide a NoSQL database built on top of multiple, non-colluding service providers that appear as a single one to the user. Strikingly, service providers, individually, cannot disclose any of the user’s data but, together, are able to offer data storage and processing capabilities. Additionally, we evaluate SafeRegions exposing performance trade-offs imposed by security mechanisms and provide useful insights for future research on performance optimization.

2. Reducing Data Transfer in Parallel Processing of SQL Window Functions

   Coelho, Fábio, Pereira, José, Vilaça, Ricardo, and Oliveira, Rui

   In Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1 and 2 2016

   Abs PDF

   Window functions are a sub-class of analytical operators that allow data to be handled in a derived view of a given relation, while taking into account their neighboring tuples. We propose a technique that can be used in the parallel execution of this operator when data is naturally partitioned. The proposed method benefits the cases where the required partitioning is not the natural partitioning employed. Preliminary evaluation shows that we are able to limit data transfer among parallel workers to 14% of the registered transfer when using a naive approach.

3. Resource usage prediction in distributed key-value datastores

   Cruz, Francisco, Maia, Francisco, Matos, Miguel, Oliveira, Rui, Paulo, João, and Vilaça, Ricardo

   In Proceedings of the international conference on Distributed applications and interoperable systems 2016

   Abs PDF

   In order to attain the promises of the Cloud Computing paradigm, systems need to be able to transparently adapt to environment changes. Such behavior benefits from the ability to predict those changes in order to handle them seamlessly. In this paper, we present a mechanism to accurately predict the resource usage of distributed key-value datastores. Our mechanism requires offline training but, in contrast with other approaches, it is sufficient to run it only once per hardware configuration and subsequently use it for online prediction of database performance under any circumstance. The mechanism accurately estimates the database resource usage for any request distribution with an average accuracy of 94 %, only by knowing two parameters: (i) cache hit ratio; and (ii) incoming throughput. Both input values can be observed in real time or synthesized for request allocation decisions. This novel approach is sufficiently simple and generic, while simultaneously being suitable for other practical applications.

4. Holistic Shuffler for the Parallel Processing of SQL Window Functions

   Coelho, Fábio, Vilaça, Ricardo, Pereira, José, and Oliveira, Rui

   In Proceedings of the international conference on Distributed applications and interoperable systems 2016

   Abs PDF

   Window functions are a sub-class of analytical operators that allow data to be handled in a derived view of a given relation, while taking into account their neighboring tuples. Currently, systems bypass parallelization opportunities which become especially relevant when considering Big Data as data is naturally partitioned. We present a shuffling technique to improve the parallel execution of window functions when data is naturally partitioned when the query holds a partitioning clause that does not match the natural partitioning of the relation. We evaluated this technique with a non-cumulative ranking function and we were able to reduce data transfer among parallel workers in 85 % when compared to a naive approach.

2015

1. CumuloNimbo: A Cloud Scalable Multi-tier SQL Database

   Jimenez-Peris, Ricardo, Patiño-Martinez, Marta, Kemme, Bettina, Brondino, Ivan, Pereira, José, Vilaça, Ricardo, Cruz, Francisco, Oliveira, Rui, and Ahmad, Yousuf

   Bulletin of the Technical Committee on Data Engineering Mar 2015

   Abs PDF

   This article presents an overview of the CumuloNimbo platform. CumuloNimbo is a framework for multi-tier applications that provides scalable and fault-tolerant processing of OLTP workloads. The main novelty of CumuloNimbo is that it provides a standard SQL interface and full transactional support without resorting to sharding and no need to know the workload in advance. Scalability is achieved by distributing request execution and transaction control across many compute nodes while data is persisted in a distributed data store. In this paper we present an overview of the platform.

2014

1. DATAFLASKS: Epidemic Store for Massive Scale Systems

   Maia, Francisco, Matos, Miguel, Vilaça, Ricardo, Pereira, José, Oliveira, Rui, and Rivière, Etienne

   In 2014 IEEE 33rd International Symposium on Reliable Distributed Systems 2014

   Abs PDF

   Very large scale distributed systems provide some of the most interesting research challenges while at the same time being increasingly required by nowadays applications. The escalation in the amount of connected devices and data being produced and exchanged, demands new data management systems. Although new data stores are continuously being proposed, they are not suitable for very large scale environments. The high levels of churn and constant dynamics found in very large scale systems demand robust, proactive and unstructured approaches to data management. In this paper we propose a novel data store solely based on epidemic (or gossip-based) protocols. It leverages the capacity of these protocols to provide data persistence guarantees even in highly dynamic, massive scale systems. We provide an open source prototype of the data store and correspondent evaluation.

2. pH1: A Transactional Middleware for NoSQL

   Coelho, Fábio, Cruz, Francisco, Vilaça, Ricardo, Pereira, José, and Oliveira, Rui

   In Proccedings of SRDS’14: The 33rd IEEE International Symposium on Reliable Distributed Systems 2014

   Abs PDF

   NoSQL databases opt not to offer important abstractions traditionally found in relational databases in order to achieve high levels of scalability and availability: transactional guarantees and strong data consistency. In this work we propose pH1, a generic middleware layer over NoSQL databases that offers transactional guarantees with Snapshot Isolation. This is achieved in a non-intrusive manner, requiring no modifications to servers and no native support for multiple versions. Instead, the transactional context is achieved by means of a multiversion distributed cache and an external transaction certifier, exposed by extending the client’s interface with transaction bracketing primitives. We validate and evaluate pH1 with Apache Cassandra and Hyperdex. First, using the YCSB benchmark, we show that the cost of providing ACID guarantees to these NoSQL databases amounts to 11% decrease in throughput. Moreover, using the transaction intensive TPC-C workload, pH1 presented an impact of 22% decrease in throughput. This contrasts with OMID, a previous proposal that takes advantage of HBase’s support for multiple versions, with a throughput penalty of 76% in the same conditions.

3. On the Support of Versioning in Distributed Key-Value Stores

   Felber, Pascal, Pasin, Marcelo, Rivière, Etienne, Schiavoni, Valerio, Sutra, Pierre, Coelho, Fábio, Oliveira, Rui, Matos, Miguel, and Vilaça, Ricardo

   In 2014 IEEE 33rd International Symposium on Reliable Distributed Systems Oct 2014

   Abs PDF

   The ability to access and query data stored in multiple versions is an important asset for many applications, such as Web graph analysis, collaborative editing platforms, data forensics, or correlation mining. The storage and retrieval of versioned data requires a specific API and support from the storage layer. The choice of the data structures used to maintain versioned data has a fundamental impact on the performance of insertions and queries. The appropriate data structure also depends on the nature of the versioned data and the nature of the access patterns. In this paper we study the design and implementation space for providing versioning support on top of a distributed key-value store (KVS). We define an API for versioned data access supporting multiple writers and show that a plain KVS does not offer the necessary synchronization power for implementing this API. We leverage the support for listeners at the KVS level and propose a general construction for implementing arbitrary types of data structures for storing and querying versioned data. We explore the design space of versioned data storage ranging from a flat data structure to a distributed sharded index. The resulting system, ALEPH, is implemented on top of an industrial-grade open-source KVS, Infinispan. Our evaluation, based on real-world Wikipedia access logs, studies the performance of each versioning mechanisms in terms of load balancing, latency and storage overhead in the context of different access scenarios.

4. Workload-Aware Table Splitting for NoSQL

   Cruz, Francisco, Maia, Francisco, Oliveira, Rui, and Vilaça, Ricardo

   In Proceedings of the 29th Annual ACM Symposium on Applied Computing 2014

   Abs PDF

   Massive scale data stores, which exhibit highly desirable scalability and availability properties are becoming pivotal systems in nowadays infrastructures. Scalability achieved by these data stores is anchored on data independence; there is no clear relationship between data, and atomic inter-node operations are not a concern. Such assumption over data allows aggressive data partitioning. In particular, data tables are horizontally partitioned and spread across nodes for load balancing. However, in current versions of these data stores, partitioning is either a manual process or automated but simply based on table size. We argue that size based partitioning does not lead to acceptable load balancing as it ignores data access patterns, namely data hotspots. Moreover, manual data partitioning is cumbersome and typically infeasible in large scale scenarios. In this paper we propose an automated table splitting mechanism that takes into account the system workload. We evaluate such mechanism showing that it simple, non-intrusive and effective.

2013

1. Evaluating Cassandra as a Manager of Large File Sets

   Beernaert, Leander, Gomes, Pedro, Matos, Miguel, Vilaça, Ricardo, and Oliveira, Rui

   In Proceedings of the 3rd International Workshop on Cloud Data and Platforms 2013

   Abs PDF

   All companies developing their business on the Web, not only giants like Google or Facebook but also small companies focused on niche markets, face scalability issues in data management. The case study of this paper is the content management systems for classified or commercial advertisements on the Web. The data involved has a very significant growth rate and a read-intensive access pattern with a reduced update rate.Typically, data is stored in traditional file systems hosted on dedicated servers or Storage Area Network devices due to the generalization and ease of use of file systems. However, this ease in implementation and usage has a disadvantage: the centralized nature of these systems leads to availability, elasticity and scalability problems.The scenario under study, undemanding in terms of the system’s consistency and with a simple interaction model, is suitable to a distributed database, such as Cassandra, conceived precisely to dynamically handle large volumes of data.In this paper, we analyze the suitability of Cassandra as a substitute for file systems in content management systems. The evaluation, conducted using real data from a production system, shows that when using Cassandra, one can easily get horizontal scalability of storage, redundancy across multiple independent nodes and load distribution imposed by the periodic activities of safeguarding data, while ensuring a comparable performance to that of a file system.

2. DATAFLASKS: an epidemic dependable key-value substrate

   Maia, Francisco, Matos, Miguel, Vilaça, Ricardo, Pereira, José, Oliveira, Rui, and Riviere, Etienne

   In International Workshop on Dependability of Clouds, Data Centers and Virtual Computing Environments (DCDV 2013) Jun 2013

   Abs PDF

   Recently, tuple-stores have become pivotal structures in many information systems. Their ability to handle large datasets makes them important in an era with unprecedented amounts of data being produced and exchanged. However, these tuple-stores typically rely on structured peer-to-peer protocols which assume moderately stable environments. Such assumption does not always hold for very large scale systems sized in the scale of thousands of machines. In this paper we present a novel approach to the design of a tuple-store. Our approach follows a stratified design based on an unstructured substrate. We focus on this substrate and how the use of epidemic protocols allow reaching high dependability and scalability.

3. An effective scalable SQL engine for NoSQL databases

   Vilaça, Ricardo, Cruz, Franciso, Pereira, José, and Oliveira, Rui

   In Proceedings of the 13th IFIP WG 6.1 international conference on Distributed applications and interoperable systems 2013
4. pH1: middleware transacional para NoSQL

   Coelho, Fábio, Cruz, Francisco, Pereira, José, Vilaça, Ricardo, and Oliveira, Rui

   In Inforum 2013

   Abs PDF

   As bases de dados NoSQL optam por n ao oferecer importantes abstracões tradicionalmente encontradas nas bases de dados relacionais, de modo a atingir elevada escalabilidade e disponibilidade: garantias transacionais e critérios de coerência de dados fortes. Estas limitações resultam em maior complexidade no desenvolvimento de aplicações sobre bases de dados NoSQL e, logo, são um obstáculo à adopção do paradigma. Neste trabalho, propomos uma camada de middleware sobre bases de dados NoSQL que oferece garantias transacionais com Snapshot Isolation. A abordagem  e não-intrusiva, apresentando aos clientes a mesma interface NoSQL acrescida do contexto transacional. Este contexto transacional  e o cerne da nossa contribuição e assenta modularmente num repositório distribuído, não persistente, que mantém várias versões dos dados manipulados e num certificador de transacções concorrentes. Apresentamos uma implementação do nosso sistema pH1 sobre Cassandra e, recorrendo a um benchmark (YCSB) extensamente utilizado na avaliação de desempenho de bases de dados NoSQL, medimos o custo do suporte do paradigma transacional com garantias transacionais ACID, que n ao resulta numa diminuição significativa da latência das operações quando comparado com o Cassandra.

5. MeT: Workload aware elasticity for NoSQL

   Cruz, Francisco, Maia, Francisco, Matos, Miguel, Oliveira, Rui, Paulo, Joao, Pereira, José, and Vilaça, Ricardo

   In EuroSys 2013

   Abs PDF

   NoSQL databases manage the bulk of data produced by modern Web applications such as social networks. This stems from their ability to partition and spread data to all available nodes, allowing NoSQL systems to scale. Unfortunately, current solutions’ scale out is oblivious to the underlying data access patterns, resulting in both highly skewed load across nodes and suboptimal node configurations. In this paper, we first show that judicious placement of HBase partitions taking into account data access patterns can improve overall throughput by 35%. Next, we go beyond current state of the art elastic systems limited to uninformed replica addition and removal by: i) reconfiguring existing replicas according to access patterns and ii) adding replicas specifically configured to the expected access pattern. MeT is a prototype for a Cloud-enabled framework that can be used alone or in conjunction with OpenStack for the automatic and heterogeneous reconfiguration of a HBase deployment. Our evaluation, conducted using the YCSB workload generator and a TPC-C workload, shows that MeT is able to i) autonomously achieve the performance of a manual configured cluster and ii) quickly reconfigure the cluster according to unpredicted workload changes.