QUEST FOR A BETTER DATABASE
As a Big Data Engineer who’s worked for a few years alongside Google Developers & Microsoft Professionals, before that, I’ve been on a quest to find better tools and technologies to improve application and database performance at scale. Like many developers in my space, I was only familiar with the biggest brand name DWHs such as Snowflake, Big Query, Synapse, Redshift, etc. But I recently stumbled across an advert to try out and test a new database technology (SingleStore) for free, and I was surprised at the results.
In a traditional DWH, we need to look over the provisioning of hardware resources. It takes huge time to manage and monitor it. Even if we look at cloud solutions, there is still a lot of managing to do. But in SingleStore’s Case, everything is managed by the Organization, it acts as a Database as a Service. This is very handy for organizations who want to drive straight into development without stressing over the physical resources. It is easily scalable as well. It is a great alternative to Snowflake with great pricing.
SingleStoreDB was good at being able to handle complex queries against large datasets out of the box. In the past, we’ve had to do quite a bit of manual configuration and database performance-tuning in most of the DWHs, but SingleStore (so far) has seemed to require minimal configuration in this aspect. Both data imports/exports, as well as queries against the data, run very fast.
I had the opportunity to load 100 billion (Yes you heard that right, 100 billion) rows and it loaded them without breaking a sweat. It took approx 10 minutes for all of the rows to be loaded and I was awed at the glance of it.
The performance and simplicity of the platform are too good to be true. It is similar to Snowflake but it needs time to get there due to the improvements needed in the UI. But as far as the performance is concerned, it is truly amazing.
Not only that, It also provided a great experience for anyone who is looking forward to testing the platform via its Free trial and a big chunk of support/documentation.
Some suggestions that I have regarding SingleStore remind me of when we had to choose between AWS and Azure. Aws is definitely robust but not User friendly like Azure. I believe a similar is the case with SingleStore. I think some inspiration from snowflake in terms of the User Experience would definitely help. So I believe we would focus on a better User Experience, Integration with Other DWH & ETL Tools. we would also provide more regions.
The conclusion is that the Product is great. It simplifies the entire headache of managing and maintaining the hardware resources for a DWH… But it’s bigger and better than that, because SingleStore can uniquely handle both operations and analytics in one location (unlike data warehouses). This has huge potential. I would definitely place my bets on it.
It seems good at being able to handle complex queries against large datasets out of the box. In the past, we’ve had to do quite a bit of manual configuration and database performance-tuning, but SingleStore (so far) has seemed to require minimal configuration in this aspect.
