BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

How To Test Drive Big Data Analytics

Following
This article is more than 9 years old.

Cloud-centric ‘autonomic’ application management company Qubell has announced Qubell for Hadoop Developers. The software is intended to enable Hadoop programmers and data engineers to get access to on-demand test environments for big data analytics projects. The assertion here (made by IDC analyst Melinda Ballou) is that too many companies are still at what has been called “immature levels” of quality and configuration management for production environments that can support big data analytics -- and so, logically, these firms are not really using or taking advantage of big data analytics yet.

Testing big data, why it’s tough

If we can accept the contention that big data analytics testing is not easy, then we can drill down to ask why exactly this job and procedure is so hard to execute in real world workplaces.

  1. It is still comparatively early days for both Hadoop and for big data analytics, so it’s not unfair to suggest that firms will often exhibit a lack of the required levels of in-house testing infrastructure required to execute big data projects.
  2. As a direct corollary and consequence of point 1) firms also often suffer from a lack of skills related to configuring, testing and managing analytics tools and environments.

So what happens?

Qubell says that because of the associated costs and time involved, testing big data applications is most commonly done in production by fencing off part of the production system for the developers, but this approach presents two problems for organizations.

  1. Firstly, changes made by developers can oftentimes wreak havoc to production systems.
  2. Secondly, we could argue that innovation is stifled when developers are severely restricted in terms of what they can actually do in a fenced-off environment.

This latter statement is obviously the bait for Qubell to tell us that its Hadoop Developer offering answers these points -- and, in fairness, the product does combine an on-demand cloud infrastructure with fully provisioned applications stacks configured with required datasets, libraries and tools.

“The result is a self-service solution that enables analytics groups to do something they were never able to do before: provide a push-button experience for data scientists to get test clusters on-demand, and tear them down when no longer needed,” said the company, in a press statement.

The software is built on an open platform and enables users to add new components of the analytics stack and mix and match traditional and big data analytics technologies, so they can be managed as one unified system. The solution combines the pre-loaded and pre-configured Hadoop-related tools optimized to run on AWS, with the option to expand the system by adding more application components, customizing the workflows or moving to a multi-cloud environment.

“We are lowering the barriers to leveraging Hadoop and related tools and bringing the best practices of continuous integration, continuous delivery and continuous upgrades to the world of big data,” said Victoria Livschitz, CEO of Qubell. “On the other hand, we know that big data projects don’t exist in isolation and our customers need to mix, match and integrate big data infrastructure with all the other enterprise and web technology. Having a unified autonomic application management platform that can handle all types of applications, including Hadoop, is a big deal.”

Qubell for Hadoop Developers is a pay-per-use solution that uses the firm’s own autonomic application delivery and management platform, which provides self-service test environments, continuous delivery, continuous live updates and visibility of configuration changes. There are developer sandboxes here to spin up and dispose of small private Hadoop environments that are pre-loaded with code from a personal branch, test datasets and all the necessary tools.

 

Follow me on Twitter or LinkedIn