| By Trevor Parsons | Article Rating: |
|
| November 22, 2014 10:00 PM EST | Reads: |
2,285 |
Unlimited Logging: A New Chapter in Log Management
It's no secret that log data is quickly becoming one of the most valuable sources of information within organizations. There are open source, on-premise, and cloud-based solutions to help you glean value from your logs in many different ways.
Largely, organizations use logs for debugging during development, for monitoring and troubleshooting production systems, for security audit trails and forensics, and (more and more) for different business use cases that transcend product management and marketing teams.

I love seeing logs used in untraditional ways, for example:
- How Elon Musk used logs to call out a New York Times journalist after an unfavorable review of the performance of the (at the time) new Model S Telsa. Musk went back to the logs to outline ‘exactly' what happened during the test drive vs. what was claimed, highlighting the value of maintaining the log level evidence of your systems just in case you ever need it.
- Monitoring your users in real time. Using javascript logging you can log directly from the client's browser as they navigate your app - giving you insight into your customers' behavior. In the past this was achieved by observing activity on the store room floor, where you could see how customers were congregating, and determine what items were popular. Today, where your customers are always online, you can gather the same customer insights by logging such activity and viewing this in a ‘live streaming' mode. Because logs allow you to record this information, you can obtain a much more analytical understanding of customer trends and even individual customer behavior; enabling you to better position your offering and drive more value for your business. For a product manager or a founder of a SaaS company like myself, it can be addictive to sit and watch your users in real time, as they use new features and interact with your technology.
- Logs used as simple data structures to build powerful distributed systems. Jay Kreps' article is a must read for every developer interested in understanding the power of the humble log as a simple data structure for solving complex problems.
Logs continue to be one of the fastest growing data sources at organizations today. For example the largest DB hosted at AWS contains machine-generated statistics on AWS itself.
Log Management's Ugly Secret
Organizations manage logs similar to how we managed email in the 1990's.
Organizations are constantly worried about data volumes, exceeding data limits, and incurring unpredictable (and costly) fees. You end up always looking over your shoulder, concerned about your next log management bill, which is almost always based on GB/TB of data you produce.
The constant murmur we hear from organizations is something along the lines of - ‘look don't get me wrong, we love our logs and would find it very difficult to operate our business without them.... BUT it's bloody expensive!'
This cost largely comes in two flavors:
- Costs associated with the traditional vendors' per GB, pay for everything pricing model can become prohibitive as log volumes increase.
- Organizations frustrated with this model who look at open source/roll your own solutions often end up in an even more expensive situation. They are left footing the bill for the infrastructure required to run their internal logging cluster, as well as the developer(s) salary required to build and consistently maintain the solution.
Enter Unlimited Logging - Logentries is to logs as Gmail was to Email

At Logentries we're moving away from charging organizations per GB for everything they log, and instead, want you to send us ALL your log data and not worry about the cost.
Think about how you felt when Gmail came along and you never had to worry about running out of inbox space - they opened a new chapter in how email as a service was delivered; most certainly for the better. At Logentries we are doing the same for our users with our new Unlimited Logging - send us all your data and don't worry about it.
You do not necessarily get 2X the value from your logs when your log volumes double.
Value is more aligned with the type of analysis you can perform and the valuable trends you can extract from your data.
How Unlimited Logging Works
At Logentries we have a fundamentally different perspective:
Log Management and analysis should be simple to use and real time:
- You should not need to be a data scientist to work with and understand your logs.
- You should not have to learn a complex search query language to navigate and get value from your logs.
- Analysis should be performed in real time and you shouldn't have to wait 10 mins to get an alert on an important event that occurred in your system.
At Logentries we have built a technology from more than a decade of research in distributed systems, with a unique pre-processing engine that analyses your data up front, in real time with built-in intelligence, so that you do not need to construct complex search queries. We do the hard work so you don't have to, and we aim to make your log data analysis quick, painless but still super powerful.
Send as much data as you like:
Our unique pre-processing engine can be used to dynamically route your logs for real time analysis, or alternatively, into cloud storage for on-demand analytics. Generally, an organization will have log data that needs to be analyzed immediately, in real time. But organizations also tend to have a lot of data that MAY need to be analyzed at some point in the future - this is where on-demand analytics comes into play.
Traditionally, logging providers have tried to apply a one-size fits all approach.
All your data gets indexed up front- so that they can charge you more per GB for all the log data indexed. At Logentries we let YOU decide what data you want to analyze right now, and what data you want to analyze at some point in the future - on demand.
We allow you to send as much data as you like to cloud storage, and we only charge you for what you actually ingest into the Logentries service for analysis. This provides a very flexible way for organization to significantly reduce and cap their logging costs without having to worry about log ‘inflation' as their systems and business grow - as they invariable do.
Want to check out our unlimited logging? You can get more details on how it works here and how you can start to better manage and cut your logging costs.
Published November 22, 2014 Reads 2,285
Copyright © 2014 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Trevor Parsons
Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.
Jan. 4, 2015 03:00 AM EST Reads: 1,002 |
By Liz McMillan We’re no longer looking to the future for the IoT wave. It’s no longer a distant dream but a reality that has arrived. It’s now time to make sure the industry is in alignment to meet the IoT growing pains – cooperate and collaborate as well as innovate.
In his session at @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, will examine the key ingredients to IoT success and identify solutions to challenges the industry is facing. The deep industry expertise behind this presentation will provide attendees with a leading edge view of rapidly emerging IoT oppor...Jan. 4, 2015 02:00 AM EST Reads: 1,139 |
By Carmen Gonzalez Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
Jan. 4, 2015 02:00 AM EST Reads: 2,759 |
By Liz McMillan The explosion of connected devices / sensors is creating an ever-expanding set of new and valuable data. In parallel the emerging capability of Big Data technologies to store, access, analyze, and react to this data is producing changes in business models under the umbrella of the Internet of Things (IoT). In particular within the Insurance industry, IoT appears positioned to enable deep changes by altering relationships between insurers, distributors, and the insured.
In his session at @ThingsExpo, Michael Sick, a Senior Manager and Big Data Architect within Ernst and Young's Financial Servi...Jan. 4, 2015 12:00 AM EST Reads: 1,204 |
By Elizabeth White The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information.
In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focused on understanding how industrial data can create intelligence for industrial operations. Imagine ...Jan. 3, 2015 11:30 PM EST Reads: 923 |
By Elizabeth White Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...Jan. 3, 2015 10:30 PM EST Reads: 2,538 |
By Carmen Gonzalez In the consumer IoT, everything is new, and the IT world of bits and bytes holds sway. But industrial and commercial realms encompass operational technology (OT) that has been around for 25 or 50 years. This grittier, pre-IP, more hands-on world has much to gain from Industrial IoT (IIoT) applications and principles. But adding sensors and wireless connectivity won’t work in environments that demand unwavering reliability and performance.
In his session at @ThingsExpo, Ron Sege, CEO of Echelon, will discuss how as enterprise IT embraces other IoT-related technology trends, enterprises with i...Jan. 3, 2015 08:45 PM EST Reads: 828 |
By Liz McMillan For years, we’ve relied too heavily on individual network functions or simplistic cloud controllers. However, they are no longer enough for today’s modern cloud data center. Businesses need a comprehensive platform architecture in order to deliver a complete networking suite for IoT environment based on OpenStack.
In his session at @ThingsExpo, Dhiraj Sehgal from PLUMgrid will discuss what a holistic networking solution should really entail, and how to build a complete platform that is scalable, secure, agile and automated.Jan. 3, 2015 07:00 PM EST Reads: 1,167 |
By Liz McMillan Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...Jan. 3, 2015 12:00 PM EST Reads: 2,350 |
By Liz McMillan Sensor-enabled things are becoming more commonplace, precursors to a larger and more complex framework that most consider the ultimate promise of the IoT: things connecting, interacting, sharing, storing, and over time perhaps learning and predicting based on habits, behaviors, location, preferences, purchases and more.
In his session at @ThingsExpo, Tom Wesselman, Director of Communications Ecosystem Architecture at Plantronics, will examine the still nascent IoT as it is coalescing, including what it is today, what it might ultimately be, the role of wearable tech, and technology gaps stil...Jan. 3, 2015 12:00 PM EST Reads: 1,101 |
By Liz McMillan There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immediate and actionable interpretation of events as they happen. Another aspect concerns how to deliver ...Jan. 3, 2015 12:00 PM EST Reads: 2,735 |
By Liz McMillan The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets.
According to a recent IDG Research Services Survey this rate of traffic will only grow. What's driving t...Jan. 3, 2015 11:00 AM EST Reads: 2,576 |
By Pat Romanski One of the biggest impacts of the Internet of Things is and will continue to be on data; specifically data volume, management and usage. Companies are scrambling to adapt to this new and unpredictable data reality with legacy infrastructure that cannot handle the speed and volume of data.
In his session at @ThingsExpo, Don DeLoach, CEO and president of Infobright, will discuss how companies need to rethink their data infrastructure to participate in the IoT, including:
Data storage: Understanding the kinds of data: structured, unstructured, big/small?
Analytics: What kinds and how responsiv...Jan. 3, 2015 10:45 AM EST Reads: 1,081 |
By Carmen Gonzalez Wearable devices have come of age. The primary applications of wearables so far have been "the Quantified Self" or the tracking of one's fitness and health status. We propose the evolution of wearables into social and emotional communication devices. Our BE(tm) sensor uses light to visualize the skin conductance response. Our sensors are very inexpensive and can be massively distributed to audiences or groups of any size, in order to gauge reactions to performances, video, or any kind of presentation.
In her session at @ThingsExpo, Jocelyn Scheirer, CEO & Founder of Bionolux, will discuss ho...Jan. 3, 2015 09:15 AM EST Reads: 1,136 |
By Pat Romanski Since 2008 and for the first time in history, more than half of humans live in urban areas, urging cities to become “smart.” Today, cities can leverage the wide availability of smartphones combined with new technologies such as Beacons or NFC to connect their urban furniture and environment to create citizen-first services that improve transportation, way-finding and information delivery.
In her session at @ThingsExpo, Laetitia Gazel-Anthoine, CEO of Connecthings, will focus on successful use cases.Jan. 3, 2015 07:45 AM EST Reads: 1,223 |
By Pat Romanski An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and assessments, including a decade of leading incident response and digital forensics. He is co-author of t...Dec. 31, 2014 06:00 PM EST Reads: 3,009 |
By Pat Romanski How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends:
Exposing the device to a management framework
Exposing that management framework to a business centric logic
Exposing that business layer and data to end users.
This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles ...Dec. 31, 2014 02:00 PM EST Reads: 2,591 |
By Pat Romanski The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand.
In his session at @ThingsExpo, Sean Lorenz, Technical Product Manager for Xively at LogMeIn, demonstrated how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.Dec. 31, 2014 12:00 PM EST Reads: 2,392 |
By Yeshim Deniz The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from hardware to software, or as we like to say, it’s an Internet of many different things. The difference ...Dec. 31, 2014 12:00 PM EST Reads: 2,944 |
By Elizabeth White "There is a natural synchronization between the business models, the IoT is there to support ,” explained Brendan O'Brien, Co-founder and Chief Architect of Aria Systems, in this SYS-CON.tv interview at the 15th International Cloud Expo®, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.Dec. 31, 2014 11:30 AM EST Reads: 3,173 |

We’re no longer looking to the future for the IoT wave. It’s no longer a distant dream but a reality that has arrived. It’s now time to make sure the industry is in alignment to meet the IoT growing pains – cooperate and collaborate as well as innovate.
In his session at @ThingsExpo, Jim Hunter, Chief Scientist & Technology Evangelist at Greenwave Systems, will examine the key ingredients to IoT success and identify solutions to challenges the industry is facing. The deep industry expertise behind this presentation will provide attendees with a leading edge view of rapidly emerging IoT oppor...
Cloud Expo 2014 TV commercials will feature @ThingsExpo, which was launched in June, 2014 at New York City's Javits Center as the largest 'Internet of Things' event in the world.
The explosion of connected devices / sensors is creating an ever-expanding set of new and valuable data. In parallel the emerging capability of Big Data technologies to store, access, analyze, and react to this data is producing changes in business models under the umbrella of the Internet of Things (IoT). In particular within the Insurance industry, IoT appears positioned to enable deep changes by altering relationships between insurers, distributors, and the insured.
In his session at @ThingsExpo, Michael Sick, a Senior Manager and Big Data Architect within Ernst and Young's Financial Servi...
The industrial software market has treated data with the mentality of “collect everything now, worry about how to use it later.” We now find ourselves buried in data, with the pervasive connectivity of the (Industrial) Internet of Things only piling on more numbers. There’s too much data and not enough information.
In his session at @ThingsExpo, Bob Gates, Global Marketing Director, GE’s Intelligent Platforms business, to discuss how realizing the power of IoT, software developers are now focused on understanding how industrial data can create intelligence for industrial operations. Imagine ...
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
In the consumer IoT, everything is new, and the IT world of bits and bytes holds sway. But industrial and commercial realms encompass operational technology (OT) that has been around for 25 or 50 years. This grittier, pre-IP, more hands-on world has much to gain from Industrial IoT (IIoT) applications and principles. But adding sensors and wireless connectivity won’t work in environments that demand unwavering reliability and performance.
In his session at @ThingsExpo, Ron Sege, CEO of Echelon, will discuss how as enterprise IT embraces other IoT-related technology trends, enterprises with i...
For years, we’ve relied too heavily on individual network functions or simplistic cloud controllers. However, they are no longer enough for today’s modern cloud data center. Businesses need a comprehensive platform architecture in order to deliver a complete networking suite for IoT environment based on OpenStack.
In his session at @ThingsExpo, Dhiraj Sehgal from PLUMgrid will discuss what a holistic networking solution should really entail, and how to build a complete platform that is scalable, secure, agile and automated.
Cultural, regulatory, environmental, political and economic (CREPE) conditions over the past decade are creating cross-industry solution spaces that require processes and technologies from both the Internet of Things (IoT), and Data Management and Analytics (DMA). These solution spaces are evolving into Sensor Analytics Ecosystems (SAE) that represent significant new opportunities for organizations of all types. Public Utilities throughout the world, providing electricity, natural gas and water, are pursuing SmartGrid initiatives that represent one of the more mature examples of SAE. We have s...
Sensor-enabled things are becoming more commonplace, precursors to a larger and more complex framework that most consider the ultimate promise of the IoT: things connecting, interacting, sharing, storing, and over time perhaps learning and predicting based on habits, behaviors, location, preferences, purchases and more.
In his session at @ThingsExpo, Tom Wesselman, Director of Communications Ecosystem Architecture at Plantronics, will examine the still nascent IoT as it is coalescing, including what it is today, what it might ultimately be, the role of wearable tech, and technology gaps stil...
There is no doubt that Big Data is here and getting bigger every day. Building a Big Data infrastructure today is no easy task. There are an enormous number of choices for database engines and technologies. To make things even more challenging, requirements are getting more sophisticated, and the standard paradigm of supporting historical analytics queries is often just one facet of what is needed. As Big Data growth continues, organizations are demanding real-time access to data, allowing immediate and actionable interpretation of events as they happen. Another aspect concerns how to deliver ...
The Domain Name Service (DNS) is one of the most important components in networking infrastructure, enabling users and services to access applications by translating URLs (names) into IP addresses (numbers). Because every icon and URL and all embedded content on a website requires a DNS lookup loading complex sites necessitates hundreds of DNS queries. In addition, as more internet-enabled ‘Things' get connected, people will rely on DNS to name and find their fridges, toasters and toilets.
According to a recent IDG Research Services Survey this rate of traffic will only grow. What's driving t...
One of the biggest impacts of the Internet of Things is and will continue to be on data; specifically data volume, management and usage. Companies are scrambling to adapt to this new and unpredictable data reality with legacy infrastructure that cannot handle the speed and volume of data.
In his session at @ThingsExpo, Don DeLoach, CEO and president of Infobright, will discuss how companies need to rethink their data infrastructure to participate in the IoT, including:
Data storage: Understanding the kinds of data: structured, unstructured, big/small?
Analytics: What kinds and how responsiv...
Wearable devices have come of age. The primary applications of wearables so far have been "the Quantified Self" or the tracking of one's fitness and health status. We propose the evolution of wearables into social and emotional communication devices. Our BE(tm) sensor uses light to visualize the skin conductance response. Our sensors are very inexpensive and can be massively distributed to audiences or groups of any size, in order to gauge reactions to performances, video, or any kind of presentation.
In her session at @ThingsExpo, Jocelyn Scheirer, CEO & Founder of Bionolux, will discuss ho...
Since 2008 and for the first time in history, more than half of humans live in urban areas, urging cities to become “smart.” Today, cities can leverage the wide availability of smartphones combined with new technologies such as Beacons or NFC to connect their urban furniture and environment to create citizen-first services that improve transportation, way-finding and information delivery.
In her session at @ThingsExpo, Laetitia Gazel-Anthoine, CEO of Connecthings, will focus on successful use cases.
An entirely new security model is needed for the Internet of Things, or is it? Can we save some old and tested controls for this new and different environment? In his session at @ThingsExpo, New York's at the Javits Center, Davi Ottenheimer, EMC Senior Director of Trust, reviewed hands-on lessons with IoT devices and reveal a new risk balance you might not expect. Davi Ottenheimer, EMC Senior Director of Trust, has more than nineteen years' experience managing global security operations and assessments, including a decade of leading incident response and digital forensics. He is co-author of t...
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends:
Exposing the device to a management framework
Exposing that management framework to a business centric logic
Exposing that business layer and data to end users.
This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles ...
The Internet of Things promises to transform businesses (and lives), but navigating the business and technical path to success can be difficult to understand.
In his session at @ThingsExpo, Sean Lorenz, Technical Product Manager for Xively at LogMeIn, demonstrated how to approach creating broadly successful connected customer solutions using real world business transformation studies including New England BioLabs and more.
The definition of IoT is not new, in fact it’s been around for over a decade. What has changed is the public's awareness that the technology we use on a daily basis has caught up on the vision of an always on, always connected world. If you look into the details of what comprises the IoT, you’ll see that it includes everything from cloud computing, Big Data analytics, “Things,” Web communication, applications, network, storage, etc. It is essentially including everything connected online from hardware to software, or as we like to say, it’s an Internet of many different things. The difference ...
"There is a natural synchronization between the business models, the IoT is there to support ,” explained Brendan O'Brien, Co-founder and Chief Architect of Aria Systems, in this SYS-CON.tv interview at the 15th International Cloud Expo®, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
The potential adoption of Internet of Things in industries like manufacturing, healthcare will definitely bring much needed automation and agility into the operations, but if they are not properly planned out these initiatives may lead into an explosion of data whose magnitude will simply derail the whole initiative or result in a huge cost of operations that will out smart the benefits they bring in.
As I mentioned in my last article, the concept of Fog Computing (which is term coined by CISCO) is more of an EDGE computing where by much of data analytics and decision making are happeni...
Big Data is a loose term for the collection, storage, processing, and sophisticated analysis of massive amounts of data, far larger and from many more kinds of sources than ever before. The definition of Big Data can be traced back to the 3Vs model defined by Doug Laney in 2001: Volume, Velocity, and Variety. The fourth V was later added in different fashions, such as “Value” or “Veracity”.
Interestingly the conceptualization of Big Data in the beginning of this century seems to gain wider use now after nearly 14 years. This sounds a little strange as the present dynamic world has evolved s...
As the holiday rush is winding down, I sit here reflecting on all the companies that lost business/revenue during the busiest time of the year. Loss of business not because of technology failure, although this is always a manifestation of a problem, but because of process failure in order to remedy the failures of technology. I’ve offered some tips on preparing for the holiday traffic from the system architecture perspective, but perhaps I should have concentrated on preparing for the rush from the organizational perspective.
Another year comes to a close. What did we see as significant technology events? In the disruption category, we saw Uber getting valued at $41B even with all its issues in the news. When you disrupt an entrenched business such as taxi service, it is only natural that resistance will happen. But consumers like me love the value-added service from Uber. This is unstoppable as evident from the investor’s confidence in providing $1.2B funding. In the disruption category, companies like Snapchat, Instagram, Airbnb, Instacart, and others made good progress. Re-imagination is the catchword here. See...
DevOps Summit at Cloud Expo 2014 Silicon Valley was a terrific event for us. The Qubell booth was crowded on all three days. We ran demos every 30 minutes with folks lining up to get a seat and usually standing around. It was great to meet and talk to over 500 people! My keynote was well received and so was Stan's joint presentation with RingCentral on Devops for BigData. I also participated in two Power Panels – ‘Women in Technology’ and ‘Why DevOps Is Even More Important than You Think,’ both featuring brilliant colleagues and moderators and it was a blast to be a part of.
Fraud is definitely top of mind for all banks. Steve Rosenbush at the Wall Street Journal recently wrote about Visa’s new Big Data analytic engine which has changed the way the company combats fraud. Visa estimates that its new Big Data fraud platform has identified $2 billion in potential annual incremental fraud savings. With Big Data, their new analytic engine can study as many as 500 aspects of a transaction at once. That’s a sharp improvement from the company’s previous analytic engine, which could study only 40 aspects at once. And instead of using just one analytic model, Visa now opera...
Cloud has become an extension of today’s enterprise and the traditional perimeter has long disappeared. Increasing business requirements for agility and flexibility make the cloud-extended enterprise ideal for a workforce that works anywhere, anytime and any place. This is especially true as organizations are increasingly made up of third-party resources, partners and suppliers compared to just employees.
However, traditional security models are not equipped to deal with the fluid nature of data and network flows that extend from the enterprise into a mix of multiple clouds. This not only cr...
In the last year, conversations about In-Memory Computing (IMC) have become more and more prevalent in enterprise IT circles, especially with organizations feeling the pressure to process massive quantities of data at the speed that is now being demanded by the Internet. The hype around IMC is justified: tasks that once took hours to execute are streamlined down to seconds by moving the computation and data from disk, directly to RAM. Through this simple adjustment, analytics are happening in real-time, and applications (as well as the development of applications) are working at-pace with this...
Gordon E. Moore's famously predicted tech explosion was prophetic, but it may have hit a snag. While the number of transistors on integrated circuits has doubled approximately every two years since his 1965 paper, the ability to process and transact on data hasn't. We're now ingesting data faster than we can make sense of it, leaving computing at an impasse. Without a new approach, the innovation promised by the combination of big data and internet scale may be like the flying cars we thought we'd see by 2014. Fortunately, this is is not the case, as in-memory computing offers a way to bridge...
"The year ahead brings accelerated disruption, as those technologies which we spoke about last year as being emergent -- have now begun to evolve to practical application," stated Puneet Gupta, Brillio's Chief Technology Officer. "We are seeing the industry's rapid adoption of next-generation platforms and services centered on cloud, mobility, big data and the Internet of Things. As our customers continue to leverage technology for digital transformation, Brillio looks towards 2015 as the year we push the limits of where technology can go -- as a way to bridge the gap between existing and new ...
I try to keep on top of the news, particularly as it relates to the nature and severity of cyber attacks taking place. Sadly, there’s been no shortage of reading material lately.
Last month, there were reports on breaches at Kmart and Dairy Queen (my family loves Blizzards). Updates then came out about a massive breach at Home Depot. Then more recently, there’s been the spate of nation-state attacks on the USPS, National Oceanic and Atmospheric Administration (NOAA), Sony Entertainment and the White House. Between the time I finish writing this post and the time you read this, who knows what ...
The Internet of Things or IoT is the next big trend promising to connect literally every device on the planet to the internet. IoT will fuel a data explosion that will provide the data needed to improve services, offerings and life in general by analysis and use of the information generated. Much of this data will be sensitive, personal and protected information – driving a critical requirement to safeguard this information wherever it resides – on devices, in transit, in storage and when analyzed.
What do you get when you combine Big Data technologies….like Pig and Hive? A flying pig?
No, you get a “Logical Data Warehouse”.
My general prediction is that Cloudera and Hortonworks are both aggressively moving to fulfilling a vision which looks a lot like Gartner’s “Logical Data Warehouse”….namely, “the next-generation data warehouse that improves agility, enables innovation and responds more efficiently to changing business requirements.”
Now is the age of information analytics. We have (very arguably) reached a point where the insight arising from data analytics can be applied to almost every aspect of a company, in every business vertical.
But what shape should that analytics be? Increasingly we talk about embedded analytics, but what do we mean? Should we be embedding analytics inside a) applications themselves, or should we b) look to embed analytics as business rules inside complete corporate processes – or should it be both?

























