Big Data Technology Evaluation Checklist
Anyone who’s been following the
rapid-fire technology developments in the world that is becoming known as “big
data” sees a new capability, product, or company founded literally every week.
The ambition of all of these players, established and newcomer, is tremendous,
because the potential value to business is enormous. Each new arrival is aimed
at addressing the pain that enterprises are experiencing around unrelenting
growth in the velocity, volume, and variety of the data their operations
generate.
What’s being lost, however, in some
of this frothy marketing activity, is that it’s still early for big data
technologies. There are vexing problems slowing the growth and the practical
implementation of big data technologies. For the technologies to succeed at
scale, there are several fundamental capabilities they should contain,
including stream processing, parallelization, indexing, data evaluation
environments and visualization.
When evaluating big data technology, it
can be valuable to ask companies about their ability to deliver some of these
fundamental capabilities. If you get an unsophisticated answer, you may find
that the company is not as serious or capable as you might have expected. (For
my research on this topic please see: Designing
a Scalable and Agile Big Data Platform.)
In this article, we examine some of
the big data requirements that are partially defined or in early stages of
maturity. Any big data vendor worth considering should be able to address these
requirements now or in the near future, or confidently explain their position.
We sat down with Mike Driscoll, CTO of Metamarkets, a big data company that
delivers predictive analytics solutions for digital media, to develop a
checklist for evaluating new solutions and their fit criteria against the
challenges of big data:
Some general questions to begin the
evaluation process:
- Does the solution allow for stream processing,
and incremental calculation of statistics?
- Does the solution parallelize processing and
take advantage of distributed computing?
- Does the solution perform summary indexing to
accelerate queries of huge datasets?
- What are the solution’s data exploration and
evaluation environments that enable a quick understanding of the value of
new datasets?
- How does a solution directly provide or easily
integrate with visualization tools?
- What is the strategy for verticalization of
the technology?
- What is the ecosystem strategy? How does the
solution provider fill the gaps in its capabilities through partnerships?
Getting these questions answered will
put most vendors to the test and help improve your understanding of the
technology you are evaluating.
Stream processing
As the pace of business has
increased, and the number of instrumented business processes has expanded,
increasingly our attention is focused not on “data sets,” but on “data
streams.”
“Decision-makers are interested
putting their finger on the pulse of their organization, but to get answers in
real time; they require architectures that can process streams of data as they
happen,” Driscoll says. “Current database technologies are not well suited to
do this kind of stream processing.”
For example, calculating an average
over a group of data can be done in a traditional batch process, but far more
efficient algorithms exist for calculating a moving average of data as it
arrives, incrementally, unit by unit. If you want to take a repository of data
and perform almost any statistical analysis, that can be accomplished with open
source products like R or commercial products like SAS. But if you want to
create a set of streaming statistics, to which you incrementally add or remove
a chunk of data as a moving average, the libraries either don’t exist or are
immature.
“The entire ecosystem around
streaming data is underdeveloped,” says Driscoll.
In other words,
if you’re talking to a vendor about a big data project, you have to determine
whether this kind of stream processing is important to your project, and if it
is, whether they have a capability to provide it. This axiom extends all the
way down, to not just the analytical algorithms that run over the streams, but
also to the way in which those streams are queued, ingested, managed and
ultimately processed Many architectures exist for ingesting and queuing data
streams, some of which are proprietary. TIBCO, Esper and ZeroMQ all offer
solutions. But those solutions are only about moving packets of data around.
Actually doing analysis of the streams requires practitioners to build on a
lower level, for which there is another rapidly evolving toolset.
Parallelization
There are many
definitions of big data. Here’s a useful one: “Small data” is data that fits
in-memory on a single desktop or machine with a capacity of between 1 GB and 10
GB of disk space. “Medium data” fits on a single hard drive of 100 GB to 1 TB
in size. “Large data” is distributed over many machines, comprising 1 TB to
multiple petabytes.
“If you want to
work with distributed data, and you expect to have any hope of processing that
data in a reasonable amount of time, that requires distributed processing,”
Driscoll says.
Parallel
processing comes to the fore in distributed data. Hadoop is one of the
better-known examples of distributed or parallelized processing. Hadoop can do
more than distributed processing. It can also conduct distributed queries,
which have been a subject of interest recently for designers of massively
parallel processing (MPP) databases, whose object is to take a query and
parallelize it across a set of nodes. Each of those nodes does partial work for
the query in parallel, and then combines those partial answers into a single
unified answer, Driscoll explains. Parallelizing queries is not a simple
affair. When something is being analyzed in a parallel stream, each new unit of
data must be combined with an existing unit of data in order to produce an
answer.
Therefore, if a
vendor is trying to sell you a solution for addressing big data at scale, their
salespeople should be able to articulate their special secret sauce and
strategy for parallelization.
“One of the most
important features that current data warehouse vendors must offer is the
ability to do parallel copy from Hadoop into their warehouse,” Driscoll says.
“So, whether it’s EMC Greenplum offering the ability to do distributed parallel
copy from Hadoop to Greenplum, Netezza or Oracle, the ability to parallelize
data transfer is a critical feature.”
Hadoop has a
massively distributed file system and can support distributed queries on top of
it. But it does not inherently support parallelization—running a parallel
process on Hadoop without an algorithm for optimizing queries can significantly
slow down the process, taking minutes to return an answer. This is acceptable
for some queries, but it won’t support real-time analytics in a big data world.
The power and speed of that algorithm will be a determining factor of the
robustness and cost of the solution, and that should be appropriately scaled to
your needs, says Driscoll.
Summary
indexing
Summary indexing
is the process of creating a pre-calculated summary of data to speed up running
queries. The problem with summary indexing is that it requires you to plan in advance
what kind of queries you are going to run, so it is limiting.
The most common
form of summary indexing is the star schema used to support speedy searches in
data warehouses. A star schema prioritizes one master dimension (such as
location or product) in advance of running a multi-dimensional data cube,
organizing subordinate dimensions in relation to the master. The technique
works well, but has one huge problem. If you want to ask a new question, it
takes a long time to reconfigure the schema and associated data cubes and
recompute them. When we begin to ask questions of all our data, not just the
structured data, it is practically impossible to create a star schema to answer
every possible question. The problem is not pre-processing; it’s the difficulty
of reconfiguring the pre-processing as needed to ask new questions and get a
speedy response.
The ideal
solution would easily adjust the summaries being created as new questions
arose. With a quickly created, summarized form of the data, it would then be
possible to use data-analysis tools such as QlikView, Tableau, or TIBCO
Spotfire for exploration and analysis. But there is currently a gap in
available tools to make this summary creation easier, as many available tools
don’t reach down to machine-level data, says Driscoll. The result is that the
IT department becomes involved in building a custom query.
Some help is on the way for this
problem. Vendors such as Splunk have emerged with a solution based on their
search language that makes creating summary indexes far faster than other
approaches, like star schemas. The designers of technology like SAP HANA, 1010
Data, and Metamarkets recommend an in-memory approach that completely abandons
summarizing, by keeping vast amounts of data in in-memory systems.
But data volumes are growing fast and
the need for summarizing will never go away completely. For the short and
medium term, vendors must have a strategy for agile creation of summary
indexes.
Data evaluation environments
How does your vendor’s solution understand, or allow you to understand the
meaning of new datasets and incorporate them into your analysis?
For example, if your business has a
retail store, and you are studying transactions, you could pick up the
movements of people around the web site, or even around brick-and-mortar stores
through opt-in GPS signals and cell-phone tracking. Once you acquire the data:
- How do you incorporate and understand what
that data can tell you?
- How do you develop a model of the store that
will help you analyze customer behavior?
- How do you understand when those movements
become events?
- How fast can you figure that out—before the
customer leaves the store?
To answer these questions, your
solution needs to be able to join disparate datasets. Very few vendors have a distinguished
capability to join datasets. And, just as the most critical areas of a building
are at the joints, the same is true of data architecture. Where data
interfaces, tremendous value can be unlocked, Driscoll says.
“The holy grail for anyone in the
online retail space is to understand the connection between online impression
events that lead to actions, such as clicks, or some level of engagement that
eventually lead to purchasing behaviors, which eventually lead to long-term
customer adoption,” Driscoll says. “Right now, all of these datasets live in
different places. American Express knows where you’ve purchased your Starbucks
coffee and Foursquare knows where you checked in at Starbucks, and Yahoo! knows
when you clicked on an ad discount for a Starbucks latte on a hot summer day.
And yet, people are struggling to thread these various data streams together.”
As enterprises begin to draw in
disparate data feeds—particularly mobile data feeds—it’s critical that a vendor
has a solution for rapidly joining disparate datasets, because the information
they contain is critical for enterprises.
Visualization
There are two broad categories of
visualization tools, according to Driscoll.
Exploratory visualization describes
tools that allow a decision-maker and an analyst to explore different axes of
the data for relationships, which usually involves some kind of visual “mining
for insights.” Tools such as Tableau and TIBCO Spotfire, and to a lesser extent
QlikView, fit into this category, Driscoll says.
Narrative visualizations are designed
to examine a particular axis of the data in a particular way. For instance, if
you say want to look at a time series visualization of sales broken up by
geography for an enterprise, a format for that visualization can be pre-created.
The data can be played back month-by-month for every geography, and is sorted
into a pre-cast formula. Vendors such as Perceptive Pixel fit into this
category.
In narrative visualizations, “Certain
knobs are free to explore the data, but it’s not completely open to ask any
question,” Driscoll says. “These visualizations are designed to tell a certain
story about the data, just as certain pre-computations or reports are designed
to tell a certain story. And some tools are better for the first, ad-hoc
exploratory model, and others are better for the second, constrained
narrative.”
Mind the Verticals
There are nearly as many types of
decision-making needs in different verticals as there are ways to collect,
process and analyze data. Vendors should be wary that different decisions
makers within an organization and between verticals have different kinds of
visualizations that they are accustomed to seeing.
“Any vendor that
wants to serve the needs of those decisions-makers ought to be well aware of
what those expected narratives are, because that will speed the adoption of
that visualization tool,” Driscoll says, citing the preference for candlestick
plots in the financial services industry.
Ecosystem
Strategy
The largest most
successful companies all spend tens of millions creating ecosystems around
their products. The ecosystems are supported by product features and business
models that allow the product to do its job but also work with other
technologies or partners who extend the product or craft it to special uses. If
a product doesn’t have an ecosystem strategy, you may find that it is hard to
adapt to your needs and that finding expertise to help with implementation and
configuration may be hard to come by.
This list of
requirements for big data technology is not exhaustive, but it is a good start.
Using these topics when evaluating big data technology will only lead to deeper
understanding.
Source: Wikipedia.
