Monday, March 21, 2016

Big Data vs. SDN

During one Software Defined Networking (SDN) workshop I hosted in Jakarta early this year, my friend was presenting a session with thought provoking title: Big Data vs. SDN. He is the CEO of a Deep Packet Inspection (DPI) and Data Analytic company that relies on Big Data technologies, so I can understand why he brought up such topic. But just like the new movie Batman vs. Superman that will be released this week, should the two heroes are fighting each other? Should the two are competing between each other? Big Data and SDN obviously solve different problems. And the way I look at it, they are actually closer to work together to deliver platform to help business with CAPEX reduction, OPEX reduction and agility in delivering new services.


The most natural approach to define Big Data is with the bigness. However according to Gartner, Big Data is defined as “high volume, high velocity and/or high variety information assets” that can be used to improve decision making and provide better insights.The majority of raw data, particularly Big Data, does not offer a lot of value in its unprocessed state. Big Data Analytic is the process of examining Big Data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions.

SDN paradigm to separate control plane and data plane has objective to create network abstraction for faster innovation. We can create new network applications without any need to interact directly with the actual network devices, instead our applications just need to consume the APIs provided by SDN controllers. SDN can be coupled with NFV, Network Function Virtualization, to run network functions and software on any open standard-based hardware to reduce CAPEX, OPEX, power and space. The orchestration of the network based on SDN and NFV can automate, provision, and interconnect all physical and virtual network resources.


Last year Gartner showed in their Hype Cycle that SDN & NFV is moving from hype to reality. The hype reached the peak on mid 2013 when most people thought that central controller using OpenFlow would control basic network devices completely, all network functions would transition to run on x86 platforms, and this paradigm would apply across to all aspects of networking. However, by mid 2015 most people understood that SDN & NFV implementation will use hybrid control plane consisting of a mix of distributed and centralized control plane, the network will consist of a mix of physical and virtual network functions, and SDN solutions are based on specific requirements / features and use-cases driven approach.

How can SDN & NFV be connected to Big Data & Analytic?
SDN applications must be responsive to the world in which they exist. If we are going to use SDN technologies to auto-provision network services, and turn the network to be truly dynamic, we will need to have feedback loops to ensure the desired behavior is actually occurring after a change is made. The natural progression from manual to automated will pass first through network analytics. As Michael Bushong wrote for Infoworld: "Ultimately, the promise of SDN will be inherently tied to the information that surrounds the network and drives the decisions that make SDN applications interesting. With more and more endpoints driving increasing traffic to a growing number of users, that adds up to big data."


Two weeks ago I was in Brussels to try Cisco WAN Automation Engine (WAE), a sample implementation of network analytic for Wide Area Network. To many network operators and service providers, WAN are oceans of uncertainty. Resource-constrained and multivendor, WANs produce delays and outages in far-flung and sometimes remote areas, posing a special set of issues that are distinct from those we see in data centers and access networks. WAN bandwidth is the most expensive bandwidth in the network and failure impacts are large.

WAE allows service providers to respond to overall traffic growth in terms of keeping the whole network stable and not just provisioning to traffic peaks. WAE offers what-if scenarios and failure analysis, empower the customer to be able to examine the impact of network failures in more detail, and was able to adopt a new policy by only upgrading links that were in danger of dropping packets on single-circuit failures. Using predictive analytics capabilities of WAE, we can build a powerful SDN solution that allows us to deploy and optimize innovative new services, such as global load balancing, bandwidth calendaring, bandwidth on demand, and premium network routing.


So we have seen how Big Data (Batman) and SDN (Superman) can work together, who should be the Wonder Woman in this story? It is obvious, isn't it? Internet of Things.

IoT and Big Data relationship forms Fog Computing, that uses one or a collaborative multitude of end-user clients or near-user edge devices to carry out a substantial amount of storage, communication and control, configuration, measurement and management. Fog computing can be perceived both in large cloud systems and big data structures, making reference to the growing difficulties in accessing information objectively.

With IoT it means everything is now connected. With huge explosion of number of endpoints, IoT will require a network that is not only self-provisioned, but as well as adaptive, dynamic and responsive that can be achieved with SDN. The network must provide broad and deep visibility into network traffic flow patterns too, the network must become the sensors, and this information can be used in different implementation such as for network security where by leveraging rich threat intelligence information allows for more rapid identification of security threats.



When the three superheroes are together, what should we call them?
Introducing AI @ Computer Networking. JP Vasseur, a Cisco Fellow, during Cisco Live Milan last year presented this as Self Learning Networks. Ex-Cisco CEO John Chambers mentioned this terminology in his keynote for Cisco Live 2014. Packet Pushers call this Machine Learning for Networks.

Artificial Intelligence, a singular consciousness that spawned an entire race of machines, as Morpheus explained to Neo in The Matrix back in 1999, is coming to unexpected places like computer networking. David Meyer, CTO & Chief Scientist at Brocade and formerly at Cisco, wrote the following notes: "it doesn’t take a lot of imagination to see how sophisticated unsupervised machine learning will revolutionize many of the more complex network tasks that are solved today in (much) less dynamic ways. For example, problems such as data center orchestration could benefit greatly from this technology. Clearly functionality that we’re talking about in networking these days, including Network Function Virtualization, Service Function Chaining, Mobility and the like are great candidates for treatment by Deep Learning. More generally, orchestration and optimization of Compute, Storage, Networking, Security and Energy (CSNSE) are prime candidates for consideration by Deep Learning technology. And consider what DevOps-style automation might look like when combined with Deep Learning."

Even IBM believes Watson, not SDN, is the future of networking. IBM's cognitive system demonstrates the power of artificial intelligence which will run our networks and transform IT management. They even come up with another cool name: AI-defined networks. And it won’t be about automating routine tasks, programmatically managing networks or providing a platform where humans create evermore clever access policies. Instead, we will teach machines that watch our network traffic, monitor our applications and cognitively recognize novel behavior on our firewalls.

So SDN is not the future of networking. And I agree. It is simply part of network evolution that is happening right now. If your network has not embraced SDN & NFV technologies, it is not a question of If but When. SDN will happen eventually.

The future of networking lies on the combination between SDN & NFV with Big Data & Analytics, and IoT. The three superheroes are working together and we may call them Self Learning Network, or Self Managing Network, or AI @ Computer Networking, or AI-defined Network.

Now let's just enjoy Batman vs. Superman movie.

19 comments:

Alfred Scott said...

Amazing article, a lot of useful information. I've often read your blog, keep going. Want recommend you one great article about How to protect network from different kinds of malware, sure it will be useful for you to read it, and you find there a lot of good tips and tricks.

murali karthik said...

In near future, big data handling and processing is going to the future of IT industry. Thus taking Hadoop Training in Chennai | Big Data Training in Chennai will prove beneficial for talented professionals.

arulnath d said...

If anyone expects nolan touch in this movie ,then they will be dissapointed.

Seo training in Chennai

Anita Romanenko said...

Thank you so much for taking the time for you http://awriter.org/ultiuscom personally to share such a nice info. I truly favor to reading your post

Briley Celia said...

Valuable very touchable information. I would like to thank you for sharing your great facts.www.thebusinessgenerator.net |

Colvin Cord said...

Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great workwww.carscopy.net |

Danita Delman said...

I just read your blog and get the information which searching everywhere. Now it’s become on my top preference of reading for acquiring the updates without hassle.www.saleahome.net |

Alden Barclay said...

This topic is the one that I have been looking into from last 3 days and your insight is extremely useful. Thanks for sharing this information. www.tacosoftware.net |

Bram Well said...

I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing.http://www.oralhealtheurope.net |

Andrew Son said...

Hadoop is especially used for Big Data maintenance. It uses Hadoop distributed file system . Its operating system is in cross platform. Its framework mostly written in java programming language. The other languages which are used by hadoop framework are c, c++ (c with classes) and sometimes in shell scripting.Thank you..!!
Regards,
Hadoop Training in Chennai | Big Data Training in Chennai

byash said...

Valuable very touchable information. I would like to thank you for sharing your great facts.http://www.rackmounttechnology.org |

Ashton Belton said...

Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great workhttp://www.devonboathouse.org |

Bowie Callan said...

I just read your blog and get the information which searching everywhere. Now it’s become on my top preference of reading for acquiring the updates without hassle.http://www.business-pictures.org |

Cecilia Dean said...

This topic is the one that I have been looking into from last 3 days and your insight is extremely useful. Thanks for sharing this information. http://www.gotravel24.org |

Presely Preston said...

I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing.http://www.garealestategroup.org |

Xplorant Seo said...

Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

iOS Training in Chennai

Brave Technologies said...

Excellent post. Thanks for your great information!
ERP software in Chennai | ERP Providers in Chennai

Siva Seo said...

Nice Blog. Thank you for Sharing. Enterprise Resource Planning Software Solutions in Chennai. For more details contact us info@bravetechnologies.in
erp in Chennai | erp software providers chennai

Evelin Harriet said...

I have read your blog its very Interesting. Thanks for sharing.
best erp software providers in chennai | leading erp software development companychennai