| By Adrian Bridgwater | Article Rating: |
|
| December 23, 2015 12:00 PM EST | Reads: |
554 |
We can reasonably break down the core imperatives of data analytics into three essential areas. Firms that wish to gain the full potential out of engaging in data analytics today are obliged to at least recognize these rudimentary cornerstones as the central building blocks of data intelligence:
- Prepare
- Model
- Automate
By way of clarification, there are probably a whole lot more than three steps to consider here - but calling out this threesome is a prudent step in what (for many IT shops) are still early days with technologies such as Hadoop and its wider world of Big Data.
Data preparation
It's a simple statement to make, but before we can start to work with data it has to be prepared. Before we get to the insightful insights involving dashboards, we need to subject data to combined stages of both machine and human processing.
If we have to define this action, we can say that data preparation is a procedure for gathering, cleaning, amalgamating and consolidating our target information sources into one location (a file, database or table). This process also covers the error correction, merging of data sources, deduplication of extraneous data and accommodating for areas where nulls and zeros exist (or where the data is incomplete in some other way) so that we can progress to building our data model.
Data preparation may also incorporate ‘data enrichment,' i.e., the augmentation of one core set of data with additional intelligence sourced from some other disconnected data stream.
For want of an example here, preparing a movie database would include names of movies with actors' names and movie names and years. But it could also include the interrelationship between actors who have been in the same movie together, or a group of movies that have all been made in the same year and so on. Enrichment could come from identifying background information on actors, directors and more.

Data modeling
Given the myriad and often unanticipated ways data can be consumed, it can be risky to normalize (or integrate) data too far in advance as part of the preparation process... so as we come out of stage one, we enter our data modeling phase.
Data modeling gives us the ability to decide how various elements of data relate to each other and how the heart of the data will behave inside our chosen database. Put simply, data modeling is the classification, documentation and formalization of procedures and events involved within the software in hand. Potentially hugely complex as an overall task, data modeling tools help capture and translate multifaceted system designs into representations of data that are more easily comprehended.
For a retail firm approaching its first major implementation of Big Data analytics, the ability to model the data form allows the business function to specify and describe all the myriad components of the business into data. The model here would include price, product, size, shape, product type, product variants and perhaps also special offers, i.e., the elements of the retail data universe are large.
As complex as data modeling is, the industry now provides drag-and-drop functionality in a variety of products and services to make the entire process easier to manage.
Automate
If you ask a technology evangelist to name the top ten trends for the next half-decade of computing, then ‘automation' will be one of the terms highlighted. Today we understand automation to be the elements within a software data system that we ‘could' build from scratch, but (and here's the crucial point) these are elements that we can more productively find in tools provided by specialists in this layer of computing.
Take the construction of a marketing software system (for example) as a productive and beneficial place to bring in automation software. Automation controls here could include process-driven campaign workflows, built in Business Intelligence (BI), campaign sharing and reuse options, integrated checklists and elements relating to design and navigation. When subject to the positive force of automation, these tasks become more efficient from front to back.
What this brief tour of Big Data shows us, if anything, is that data analytics is no plug-and-play process. There are multifarious multifaceted stages and steps to be aware of. But if we approach this task studiously and in a Zen-like one step at a time fashion, then we can make analytics work for us and our business.
It's as simple as one, two, three - well, kind of.
This post is brought to you by the Big Data Forum, sponsored by SAS.
SAS is a leader in business analytics software and services and the largest independent vendor in the business intelligence market.
Published December 23, 2015 Reads 554
Copyright © 2015 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Adrian Bridgwater
Adrian Bridgwater is a freelance journalist and corporate content creation specialist focusing on cross platform software application development as well as all related aspects software engineering, project management and technology as a whole.
Feb. 22, 2016 01:00 PM EST Reads: 570 |
By Pat Romanski SYS-CON Events announced today that VAI, a leading ERP software provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
VAI (Vormittag Associates, Inc.) is a leading independent mid-market ERP software developer renowned for its flexible solutions and ability to automate critical business functions for the distribution, manufacturing, specialty retail and service sectors. An IBM Premier Business Part...Feb. 22, 2016 01:00 PM EST Reads: 756 |
By Liz McMillan WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was.
You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.Feb. 22, 2016 12:45 PM EST Reads: 1,061 |
By Carmen Gonzalez Internet of @ThingsExpo, taking place June 7-9, 2016 at Javits Center, New York City and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo New York Call for Papers is now open.
Feb. 22, 2016 12:45 PM EST Reads: 1,415 |
By Elizabeth White SYS-CON Events announced today that Kintone has been named "Bronze Sponsor" of SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
kintone promotes cloud-based workgroup productivity, transparency and profitability with a seamless collaboration space, build your own business application (BYOA) platform, and workflow automation system. Feb. 22, 2016 12:30 PM EST Reads: 1,081 |
By Elizabeth White There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...Feb. 22, 2016 12:15 PM EST Reads: 234 |
By Elizabeth White As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense.
In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...Feb. 22, 2016 12:00 PM EST Reads: 311 |
By Liz McMillan The best-practices for building IoT applications with Go Code thatattendees can use to build their own IoT applications.
In his session at @ThingsExpo, Indraneel Mitra, Senior Solutions Architect & Technology Evangelist at Cognizant, will provide valuable information and resources for both novice and experienced developers on how to get started with IoT and Golang in a day.
He will provide information on how to use Intel Arduino Kit, Go Robotics API and AWS IoT stack to build an application t...Feb. 22, 2016 12:00 PM EST Reads: 126 |
By Pat Romanski Feb. 22, 2016 12:00 PM EST Reads: 124 |
By Elizabeth White SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2016 New York.
The 18th Cloud Expo and 5th @ThingsExpo will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
"The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Internet to enable us all to im...Feb. 22, 2016 12:00 PM EST Reads: 151 |
By Liz McMillan WebRTC is about the data channel as much as about video and audio conferencing. However, basically all commercial WebRTC applications have been built with a focus on audio and video. The handling of “data” has been limited to text chat and file download – all other data sharing seems to end with screensharing. What is holding back a more intensive use of peer-to-peer data?
In her session at @ThingsExpo, Dr Silvia Pfeiffer, WebRTC Applications Team Lead at National ICT Australia, looked at diffe...Feb. 22, 2016 11:30 AM EST Reads: 230 |
By Liz McMillan Feb. 22, 2016 11:00 AM EST Reads: 526 |
By Elizabeth White The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas.
In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound...Feb. 22, 2016 10:00 AM EST Reads: 1,013 |
By Elizabeth White Feb. 22, 2016 09:45 AM EST Reads: 588 |
By Elizabeth White SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including clou...Feb. 22, 2016 09:30 AM EST Reads: 885 |
By Carmen Gonzalez There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content. Join @ThingsExpo conference chair Roger Strukhoff (@IoT2040), June 7-9, 2016 in New York City, for three days of intense 'Internet of Things' discussion and focus, including Big Data's indespensable role in IoT, Smart Grids and Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) IoT's use in Vertical Markets.
Feb. 22, 2016 09:15 AM EST Reads: 1,379 |
By Elizabeth White SYS-CON Events announced today that SpeedyCloud will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
SpeedyCloud is the IaaS brand owned by Beijing SpeedyCloud Technology Co., Ltd., a leader in the cloud industry. As a general solution provider, SpeedyCloud provides IaaS for Internet and enterprises companies. Feb. 22, 2016 09:00 AM EST Reads: 169 |
By Maria C. Horton Two weeks ago (November 3-5), I attended the Cloud Expo Silicon Valley as a speaker, where I presented on the security and privacy due diligence requirements for cloud solutions.
Cloud security is a topical issue for every CIO, CISO, and technology buyer. Decision-makers are always looking for insights on how to mitigate the security risks of implementing and using cloud solutions. Based on the presentation topics covered at the conference, as well as the general discussions heard between sessi...Feb. 22, 2016 08:00 AM EST Reads: 745 |
By Pat Romanski Feb. 22, 2016 06:30 AM EST Reads: 598 |
By Liz McMillan We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will w...Feb. 22, 2016 04:15 AM EST Reads: 694 |

SYS-CON Events announced today that VAI, a leading ERP software provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
VAI (Vormittag Associates, Inc.) is a leading independent mid-market ERP software developer renowned for its flexible solutions and ability to automate critical business functions for the distribution, manufacturing, specialty retail and service sectors. An IBM Premier Business Part...
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was.
You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Internet of @ThingsExpo, taking place June 7-9, 2016 at Javits Center, New York City and Nov 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 18th International @CloudExpo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo New York Call for Papers is now open.
SYS-CON Events announced today that Kintone has been named "Bronze Sponsor" of SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
kintone promotes cloud-based workgroup productivity, transparency and profitability with a seamless collaboration space, build your own business application (BYOA) platform, and workflow automation system.
There will be new vendors providing applications, middleware, and connected devices to support the thriving IoT ecosystem. This essentially means that electronic device manufacturers will also be in the software business. Many will be new to building embedded software or robust software. This creates an increased importance on software quality, particularly within the Industrial Internet of Things where business-critical applications are becoming dependent on products controlled by software. Qua...
As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense.
In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...
The best-practices for building IoT applications with Go Code thatattendees can use to build their own IoT applications.
In his session at @ThingsExpo, Indraneel Mitra, Senior Solutions Architect & Technology Evangelist at Cognizant, will provide valuable information and resources for both novice and experienced developers on how to get started with IoT and Golang in a day.
He will provide information on how to use Intel Arduino Kit, Go Robotics API and AWS IoT stack to build an application t...
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2016 New York.
The 18th Cloud Expo and 5th @ThingsExpo will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
"The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Internet to enable us all to im...
WebRTC is about the data channel as much as about video and audio conferencing. However, basically all commercial WebRTC applications have been built with a focus on audio and video. The handling of “data” has been limited to text chat and file download – all other data sharing seems to end with screensharing. What is holding back a more intensive use of peer-to-peer data?
In her session at @ThingsExpo, Dr Silvia Pfeiffer, WebRTC Applications Team Lead at National ICT Australia, looked at diffe...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas.
In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including clou...
There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content. Join @ThingsExpo conference chair Roger Strukhoff (@IoT2040), June 7-9, 2016 in New York City, for three days of intense 'Internet of Things' discussion and focus, including Big Data's indespensable role in IoT, Smart Grids and Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) IoT's use in Vertical Markets.
SYS-CON Events announced today that SpeedyCloud will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY.
SpeedyCloud is the IaaS brand owned by Beijing SpeedyCloud Technology Co., Ltd., a leader in the cloud industry. As a general solution provider, SpeedyCloud provides IaaS for Internet and enterprises companies.
Two weeks ago (November 3-5), I attended the Cloud Expo Silicon Valley as a speaker, where I presented on the security and privacy due diligence requirements for cloud solutions.
Cloud security is a topical issue for every CIO, CISO, and technology buyer. Decision-makers are always looking for insights on how to mitigate the security risks of implementing and using cloud solutions. Based on the presentation topics covered at the conference, as well as the general discussions heard between sessi...
We are reaching the end of the beginning with WebRTC, and real systems using this technology have begun to appear. One challenge that faces every WebRTC deployment (in some form or another) is identity management. For example, if you have an existing service – possibly built on a variety of different PaaS/SaaS offerings – and you want to add real-time communications you are faced with a challenge relating to user management, authentication, authorization, and validation. Service providers will w...
DevOps bridges the gap between Development and Operations to accelerate software delivery and increase business agility and time-to-market. With its roots in the Agile movement, DevOps fosters collaboration between teams and streamlines processes, with the goal of breaking silos in order to “go fast.”
Information Security (InfoSec) and compliance are critical to businesses across the globe, especially given past examples of data breaches and looming cybersecurity threats. InfoSec has long been ...
This week’s Top 10 is a little bit different...The latest episode of our Continuous Discussions (#c9d9) podcast focused on Microservices and Continuous Delivery. Panelists Usman Ismail, Daniel Rolnick, Darko Fabijan and our own Anders Wallgren discussed some of the best practices and tips to approaching a transition to microservices.
It’s easy to understand the confusion that is caused when the terms Agile and DevOps start getting tossed around. Buzzword bingo, here we come. But these two terms are more than just marketing speak – they’re technical movements born out of the desire to deliver better software, collaborate more effectively and bring a more robust end product to users.
Father business cycles and digital consumers are forcing enterprises to respond faster to customer needs and competitive demands. Successful integration of DevOps and Agile development will be key for business success in today’s digital economy.
In his session at DevOps Summit, Pradeep Prabhu, Co-Founder & CEO of Cloudmunch, covered the critical practices that enterprises should consider to seamlessly integrate Agile and DevOps processes, barriers to implementing this in the enterprise, and pr...
Imagine that you are designing the 2017 Ford Mustang. Like all gas-powered vehicles, each one needs an exhaust muffler. Ford has already vetted and narrowed in on a preferred provider of mufflers. But imagine what would happen if the designers and factory line workers could pick from any one of 20+ past versions of that exhaust muffler from any provider for the new model year—even if that part is outdated, has lower performance, does not meet current emission standards or has a known defect.
When major companies start getting into the nitty gritty of updating their software delivery processes, governance always pokes its head into the mix. The ability to control and secure a software release has traditionally been a hands-on experience due to its many complexities. But at what costs?
As more software delivery processes become automated, the traditional manual processes used for corporate governance are creating bottlenecks that slow down software releases and promote inefficiency. ...
If we look at slow, traditional IT and jump to the conclusion that just because we found its issues intractable before, that necessarily means we will again, then it’s time for a rethink.
As a matter of fact, the world of IT has changed over the last ten years or so. We’ve been experiencing unprecedented innovation across the board – innovation in technology as well as in how people organize and accomplish tasks.
Let’s take a look at three differences between today’s modern, digital context...
If you pay any attention to modern software delivery conversations, it sometimes feels like you are being beaten over the head with a Continuous magic wand. Continuous Integration, Continuous Delivery, Continuous Deployment, Continuous Documentation, etc. The idea is so easy that it’s frustrating: Go fast. But the benefits of going fast are well beyond more builds to your user base. Perhaps the greatest value is long term, and hidden in how you can break the application continuously without fear...
Enterprises are finally realizing that digital transformation must be end-to-end if they want to remain competitive in today’s software-driven, customer-focused world – but virtually every large organization has a massive boat anchor dragging them down – slow IT.
Legacy technology woes are bad enough – but what really makes slow IT slow are the people and the processes they follow. Nobody likes to change, yet change is what digital transformation is all about.
For microservices to work in an organization, there must be a business initiative attached to it. Questions arise among IT professionals on whether microservices are suited only for giant Web applications like Google and Facebook. However, scale is only one of the business benefits of microservices.
In today’s computing environment, innovation and speed are critical. The movement toward microservices is generated by the need to create new software that can enhance and improve a monolithic syste...
In a previous article, I demonstrated how to effectively and efficiently install the Dynatrace Application Monitoring solution using Ansible. In this post, I am going to explain how to achieve the same results using Chef with our official dynatrace cookbook available on GitHub and on the Chef Supermarket. In the following hands-on tutorial, we’ll also apply what we see as good practice on working with and extending our deployment automation blueprints to suit your needs.
I’m not going to start with a reminder of how important app performance is. Let’s just all agree we already know this as the first app economy axiom and get on with the post payload. Actually, I know this is true because I’ve watched a staggering increase in the past year in use of web acceleration services including techniques like compression. app optimizations jan 15 to jan 16 ihealth
That’s why today’s ops briefing focuses on app performance and brings to your attention two emerging effor...
We've said it on this blog before, and we'll repeat the same message: Solving a software engineering problem is simple physics. It takes a set amount of energy to move a project from point A to point B, and changing the process or path doesn't change the unavoidable fact that difficult software problems are difficult to solve. DevOps integration is no different.
If you have a complex, highly-orchestrated release no amount of automation can change the fact that the large, successful systems foun...
Agile software development is slowly but surely becoming the norm for a significant contingency of software developers. What used to be rigid, step-by-step processes that were siloed between specific teams have transformed into a collaborative, adaptive methodology that ultimately leads to the creation of superior solutions.
As an integral component of software development, agile test management has also been evolving to be more accommodating to agile processes. Testing automation in particular...
If you work for a large company, you'll often look around and wonder if moving to a daily release cycle or automating deployments is even possible given the number of meetings and the amount of process your releases are subject to. Between the CAB meetings, the QA schedules, and the coordinated conference calls it's tough to imagine a monthly release process being condensed down to a process that could run in a single day. For most organizations dealing with real risk - it's true - you will not...
After more than five years of DevOps, definitions are evolving, boundaries are expanding, ‘unicorns’ are no longer rare, enterprises are on board, and pundits are moving on. Can we now look at an evolution of DevOps? Should we? Is the foundation of DevOps ‘done’, or is there still too much left to do? What is mature, and what is still missing? What does the next 5 years of DevOps look like?
In this Power Panel at DevOps Summit, moderated by DevOps Summit Conference Chair Andi Mann, panelists l...
With live TV, there is no room for error. There’s no reset button. Any mistake can have a catastrophic effect on their brand. If during the World Series or the Super Bowl or a Presidential debate a system fails to handle the capacity or load – it would affect millions of people around the country. Simply put, it’s a front page news story waiting to happen.
Continuous innovation is a concept that has taken hold with internet companies such as Amazon, Netflix, eBay, Google and countless other con...
Microservices are all the rage right now — and the industry is still learning, experimenting, and developing patterns, for successfully designing, deploying and managing Microservices in the real world. Are you considering jumping on the Microservices-wagon? Do Microservices make sense for your particular use case? What are some of the “gotchas” you should be aware of?
This morning on #c9d9 we had experts from popular chat app Kik, SMB SaaS platform Yodle and hosted CI solution Semaphore sha...
























