Monday, March 21, 2016

Big Data vs. SDN

During one Software Defined Networking (SDN) workshop I hosted in Jakarta early this year, my friend was presenting a session with thought provoking title: Big Data vs. SDN. He is the CEO of a Deep Packet Inspection (DPI) and Data Analytic company that relies on Big Data technologies, so I can understand why he brought up such topic. But just like the new movie Batman vs. Superman that will be released this week, should the two heroes are fighting each other? Should the two are competing between each other? Big Data and SDN obviously solve different problems. And the way I look at it, they are actually closer to work together to deliver platform to help business with CAPEX reduction, OPEX reduction and agility in delivering new services.


The most natural approach to define Big Data is with the bigness. However according to Gartner, Big Data is defined as “high volume, high velocity and/or high variety information assets” that can be used to improve decision making and provide better insights.The majority of raw data, particularly Big Data, does not offer a lot of value in its unprocessed state. Big Data Analytic is the process of examining Big Data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions.

SDN paradigm to separate control plane and data plane has objective to create network abstraction for faster innovation. We can create new network applications without any need to interact directly with the actual network devices, instead our applications just need to consume the APIs provided by SDN controllers. SDN can be coupled with NFV, Network Function Virtualization, to run network functions and software on any open standard-based hardware to reduce CAPEX, OPEX, power and space. The orchestration of the network based on SDN and NFV can automate, provision, and interconnect all physical and virtual network resources.


Last year Gartner showed in their Hype Cycle that SDN & NFV is moving from hype to reality. The hype reached the peak on mid 2013 when most people thought that central controller using OpenFlow would control basic network devices completely, all network functions would transition to run on x86 platforms, and this paradigm would apply across to all aspects of networking. However, by mid 2015 most people understood that SDN & NFV implementation will use hybrid control plane consisting of a mix of distributed and centralized control plane, the network will consist of a mix of physical and virtual network functions, and SDN solutions are based on specific requirements / features and use-cases driven approach.

How can SDN & NFV be connected to Big Data & Analytic?
SDN applications must be responsive to the world in which they exist. If we are going to use SDN technologies to auto-provision network services, and turn the network to be truly dynamic, we will need to have feedback loops to ensure the desired behavior is actually occurring after a change is made. The natural progression from manual to automated will pass first through network analytics. As Michael Bushong wrote for Infoworld: "Ultimately, the promise of SDN will be inherently tied to the information that surrounds the network and drives the decisions that make SDN applications interesting. With more and more endpoints driving increasing traffic to a growing number of users, that adds up to big data."


Two weeks ago I was in Brussels to try Cisco WAN Automation Engine (WAE), a sample implementation of network analytic for Wide Area Network. To many network operators and service providers, WAN are oceans of uncertainty. Resource-constrained and multivendor, WANs produce delays and outages in far-flung and sometimes remote areas, posing a special set of issues that are distinct from those we see in data centers and access networks. WAN bandwidth is the most expensive bandwidth in the network and failure impacts are large.

WAE allows service providers to respond to overall traffic growth in terms of keeping the whole network stable and not just provisioning to traffic peaks. WAE offers what-if scenarios and failure analysis, empower the customer to be able to examine the impact of network failures in more detail, and was able to adopt a new policy by only upgrading links that were in danger of dropping packets on single-circuit failures. Using predictive analytics capabilities of WAE, we can build a powerful SDN solution that allows us to deploy and optimize innovative new services, such as global load balancing, bandwidth calendaring, bandwidth on demand, and premium network routing.


So we have seen how Big Data (Batman) and SDN (Superman) can work together, who should be the Wonder Woman in this story? It is obvious, isn't it? Internet of Things.

IoT and Big Data relationship forms Fog Computing, that uses one or a collaborative multitude of end-user clients or near-user edge devices to carry out a substantial amount of storage, communication and control, configuration, measurement and management. Fog computing can be perceived both in large cloud systems and big data structures, making reference to the growing difficulties in accessing information objectively.

With IoT it means everything is now connected. With huge explosion of number of endpoints, IoT will require a network that is not only self-provisioned, but as well as adaptive, dynamic and responsive that can be achieved with SDN. The network must provide broad and deep visibility into network traffic flow patterns too, the network must become the sensors, and this information can be used in different implementation such as for network security where by leveraging rich threat intelligence information allows for more rapid identification of security threats.



When the three superheroes are together, what should we call them?
Introducing AI @ Computer Networking. JP Vasseur, a Cisco Fellow, during Cisco Live Milan last year presented this as Self Learning Networks. Ex-Cisco CEO John Chambers mentioned this terminology in his keynote for Cisco Live 2014. Packet Pushers call this Machine Learning for Networks.

Artificial Intelligence, a singular consciousness that spawned an entire race of machines, as Morpheus explained to Neo in The Matrix back in 1999, is coming to unexpected places like computer networking. David Meyer, CTO & Chief Scientist at Brocade and formerly at Cisco, wrote the following notes: "it doesn’t take a lot of imagination to see how sophisticated unsupervised machine learning will revolutionize many of the more complex network tasks that are solved today in (much) less dynamic ways. For example, problems such as data center orchestration could benefit greatly from this technology. Clearly functionality that we’re talking about in networking these days, including Network Function Virtualization, Service Function Chaining, Mobility and the like are great candidates for treatment by Deep Learning. More generally, orchestration and optimization of Compute, Storage, Networking, Security and Energy (CSNSE) are prime candidates for consideration by Deep Learning technology. And consider what DevOps-style automation might look like when combined with Deep Learning."

Even IBM believes Watson, not SDN, is the future of networking. IBM's cognitive system demonstrates the power of artificial intelligence which will run our networks and transform IT management. They even come up with another cool name: AI-defined networks. And it won’t be about automating routine tasks, programmatically managing networks or providing a platform where humans create evermore clever access policies. Instead, we will teach machines that watch our network traffic, monitor our applications and cognitively recognize novel behavior on our firewalls.

So SDN is not the future of networking. And I agree. It is simply part of network evolution that is happening right now. If your network has not embraced SDN & NFV technologies, it is not a question of If but When. SDN will happen eventually.

The future of networking lies on the combination between SDN & NFV with Big Data & Analytics, and IoT. The three superheroes are working together and we may call them Self Learning Network, or Self Managing Network, or AI @ Computer Networking, or AI-defined Network.

Now let's just enjoy Batman vs. Superman movie.

61 comments:

Alfred Scott said...

Amazing article, a lot of useful information. I've often read your blog, keep going. Want recommend you one great article about How to protect network from different kinds of malware, sure it will be useful for you to read it, and you find there a lot of good tips and tricks.

murali karthik said...

In near future, big data handling and processing is going to the future of IT industry. Thus taking Hadoop Training in Chennai | Big Data Training in Chennai will prove beneficial for talented professionals.

arulnath d said...

If anyone expects nolan touch in this movie ,then they will be dissapointed.

Seo training in Chennai

Anita Romanenko said...

Thank you so much for taking the time for you http://awriter.org/ultiuscom personally to share such a nice info. I truly favor to reading your post

Briley Celia said...

Valuable very touchable information. I would like to thank you for sharing your great facts.www.thebusinessgenerator.net |

Colvin Cord said...

Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great workwww.carscopy.net |

Danita Delman said...

I just read your blog and get the information which searching everywhere. Now it’s become on my top preference of reading for acquiring the updates without hassle.www.saleahome.net |

Alden Barclay said...

This topic is the one that I have been looking into from last 3 days and your insight is extremely useful. Thanks for sharing this information. www.tacosoftware.net |

Bram Well said...

I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing.http://www.oralhealtheurope.net |

Andrew Son said...

Hadoop is especially used for Big Data maintenance. It uses Hadoop distributed file system . Its operating system is in cross platform. Its framework mostly written in java programming language. The other languages which are used by hadoop framework are c, c++ (c with classes) and sometimes in shell scripting.Thank you..!!
Regards,
Hadoop Training in Chennai | Big Data Training in Chennai

byash said...

Valuable very touchable information. I would like to thank you for sharing your great facts.http://www.rackmounttechnology.org |

Ashton Belton said...

Its a great pleasure reading your post.Its full of information I am looking for and I love to post a comment that "The content of your post is awesome" Great workhttp://www.devonboathouse.org |

Bowie Callan said...

I just read your blog and get the information which searching everywhere. Now it’s become on my top preference of reading for acquiring the updates without hassle.http://www.business-pictures.org |

Cecilia Dean said...

This topic is the one that I have been looking into from last 3 days and your insight is extremely useful. Thanks for sharing this information. http://www.gotravel24.org |

Presely Preston said...

I recently found many useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing.http://www.garealestategroup.org |

Xplorant Seo said...

Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

iOS Training in Chennai

Brave Technologies said...

Excellent post. Thanks for your great information!
ERP software in Chennai | ERP Providers in Chennai

Siva Seo said...

Nice Blog. Thank you for Sharing. Enterprise Resource Planning Software Solutions in Chennai. For more details contact us info@bravetechnologies.in
erp in Chennai | erp software providers chennai

Evelin Harriet said...

I have read your blog its very Interesting. Thanks for sharing.
best erp software providers in chennai | leading erp software development companychennai

anitha a said...

thank you for such a great article with us. hope it will be much useful for us. please keep on updating..
Web design institute chennai

Anonymous said...

Thanks for the attractive, i am serching for aws jobs in hyderabad can u please share information about amazon web servers

devo said...

Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas. Dot Net Training Institute in Chennai | Dot Net Training Institute in Velachery.

Vamshi Krishna said...


Useful information and Please keep updating us..... Thanks

deepti said...

Great article, thanks for sharing your informative Blog..

CCNP Training Institute in Taramani | CCNP Training Institute in Velachery | Online Training in Chennai

sukere said...

Really an amazing post..! By reading your blog post i gained more information.
Bulk SMS Chennai
Internet Marketing Company Chennai

john stany said...


I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.

Android App Development Company

john stany said...


Nice it seems to be good post... It will get readers engagement on the article since readers engagement plays an vital role in every blog.i am expecting more updated posts from your hands.
iOS App Development Company
iOS App Development Company

vmware jobs in hyderabad said...

I get a lot of great information here and this is what I am searching for. Thank you for your sharing. I have bookmark this page for my future reference.view vmware openings in hyderabad.

louis philip said...

Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving..
Fitness SMS
Fitness Text
Salon SMS
Salon Text
Investor Relation SMS
Investor Relation Text

sunitha vishnu said...

great and nice blog thanks sharing..I just want to say that all the information you have given here is awesome...Thank you very much for this one.
web design Company
web development Company
web design Company in chennai
web development Company in chennai
web design Company in India
web development Company in India

isabella jacob said...

it is really amazing...thanks for sharing....provide more useful information...
Mobile app development company

Joe Fung said...

A software-configurable network is a data network in which the level of network management is separated from data transfer devices and implemented programmatically, one of the forms of virtualization of computing resources. Activewizards.com describes this definition. If we consider a modern network device (a router or a switch, not fundamentally), then it, like a pie, logically consists of three components.
1. The management layer is the CLI, the embedded web server or the API and management protocols. The task of this level is to ensure the device's controllability.
2. The level of traffic management - a variety of algorithms and functional whose task is to automatically respond to changes in traffic, ie, the intelligence device.
3. Traffic transmission - functional providing physical data transmission, level of microcircuits and network packets. (SDN from English Software-defined Networking, also a program-defined network) is a data network in which the level of network management is separated from data transfer devices and implemented programmatically, one of the forms of virtualization of computing resources.

ALINAAMEL said...


I would like to say thank you for the amazing details and concepts you are sharing in this.
Webseiten Gestaltung Lüdenscheid

Seo Maya said...


It’s quite particularly open-handed of people like Jasa Seo you to convey easily exactly what numerous people would have distributed for an ebook to generate some dough for their own end, particularly now that you could have done it if you ever considered necessary.
These concepts also acted to become a easy way to be Jasa Seo aware that many people have the same dream really like my very own to know much more in respect of this matter.
I’m certain there are many more fun times ahead for Jasa Seo individuals who read carefully your jasa seo blog post.
Certainly there Pakar Seo Jakarta Indonesia are numerous more pleasurable times ahead Jasa Seo for those who look into your master seo site.

PLC SCADA Training in Chennai said...

Thanks for sharing such a great article....


PLC Training in Chennai | PLC Training Institute in Chennai | PLC Training Center in Chennai | PLC SCADA Training in Chennai | Inplant Training in Chennai | PLC Training in Kerala

Embedded System Training in Chennai said...

nice information to sharing the blog. I really enjoyed the post.




Best Embedded System Training in Chennai | Embedded Course in Chennai | Embedded Training in Chennai | Best Embedded Training in Chennai | Embedded Systems Course in Chennai | Matlab Training in Chennai | LabVIEW Training in Chennai | VLSI Training in Chennai

PLC Training Institute in Chennai said...

Congrats for your work. Thanks a lot for providing this with us...


PLC Training in Chennai | PLC Training in Chennai | Embedded System Training in Chennai | VLSI Training in Chennai | Inplant Training in Chennai

PLC Training in Chennai said...

Thanks for your valuable information..........


Inplant Training for in Chennai | Inplant Training for in Chennai For ECE | Inplant Training in Chennai For EEE | Inplant Training in Chennai For EIE | Inplant Training in Chennai For IT Students | Inplant Training in Chennai For CSE Students | Inplant Training in Chennai For Mechanical Engineering Students | Inplant Training in Chennai For CIVIL Enginnering Students | Inplant Training For MECHATRONICS


PLC Training in Aluva said...

This post was quite awesome and interesting to read!! Thanks for sharing..

PLC Training in Kerala | PLC Training Institute in Kerala | AutoCAD Training Institute in Kerala | Inplant Training in Kerala

kri pav said...

Nice blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it.
No.1 IOS Training Institute in Chennai | Best Android Training Institute in Chennai | Java Training Institute in Chennai

deepti said...

Thank you for sharing such a nice post.
PCB Training Institute in Chennai | PCB Training Center in Chennai | PCB Training in Velachery

Adam lee said...

It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
I’ve bookmarked your site, and I’m adding your RSS feeds to my Google account.
AWS training in bangalore

rose said...

Great blog.you put Good stuff.All the topics were explained briefly.so quickly understand for me.I am waiting for your next fantastic blog.Thanks for sharing.Any coures related details learn...
Java Training in Bangalore|

annika romy said...

Thanks For Your valuable posting, it was very informative
microsoft office 2010 professional download

sunitha vishnu said...

Really it was an awesome article...very interesting to read..You have provided an nice article....Thanks for sharing..
Android Training in Chennai
Ios Training in Chennai

Fuzzy International said...

The best thing is that your blog really informative thanks for your great information!
cashew nuts suppliers and exporters in dubai
A4 paper suppliers and exporters in dubai
onion suppliers and exporters in dubai
potato suppliers and exporters in dubai
spices&grains suppliers and exporters in dubai

Ali Khan said...

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post. upload to instagram from pc

Ali Khan said...

I like this post,And I guess that they having fun to read this post,they shall take a good site to make a information,thanks for sharing it to me. instagram online web viewer

Ali Khan said...

I like this post,And I guess that they having fun to read this post,they shall take a good site to make a information,thanks for sharing it to me. instagram web viewer

annika romy said...

Informative and impressive. Keep Updating
office 2013 professional plus key deutsch

annika romy said...

Informative and impressive. Keep Updating
office 2013 professional plus key deutsch

ALINAAMEL said...

Interesting post! This is really helpful for me. I like it! Thanks for sharing!

Webdesign Lüdenscheid

ALINAAMEL said...

Interesting post! This is really helpful for me. I like it! Thanks for sharing!

Webdesign Lüdenscheid

gopalpadhan said...

If you are search any training institute in noida than Webtrackker is the best option for you.
Salesforce training institute in noida
Java training institute in noida
Software Testing Training institute in Noida
Qtp training institute in noida
Python Training Institute in Noida
SEO Training institute in noida
Amazon Web Services training Institute in Noida

gopalpadhan said...

Sap Training Institute in Noida-Webtrackker is the best SAP training institute in noida. SAP is drastically a completely state-of-the-art element ever to analyze and to excel in it. However with the ever-growing price, and being the most accessible organization solution key, agencies are greater involved to have their employees specialized in it. The education courses effectively deliver you with the fundamental recognition of SAP, in a grade by grade way.
Sas Training Institute in Noida
PHP Training Institute in Noida
Hadoop Training Institute in Noida
Oracle Training Institute in Noida
Linux Training Institute in Noida
Dot net Training Institute in Noida

mary Brown said...

Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Java developer learn from Java Training in Chennai. or learn thru Java Online Training in India . Nowadays Java has tons of job opportunities on various vertical industry.


or Javascript Training in Chennai. Nowadays JavaScript has tons of job opportunities on various vertical industry.

mary Brown said...

Java Training Institutes Java Training Institutes Java EE Training in Chennai Java EE Training in Chennai Java Spring Hibernate Training Institutes in Chennai J2EE Training Institutes in Chennai J2EE Training Institutes in Chennai Core Java Training Institutes in Chennai Core Java Training Institutes in Chennai

Java Online Training Java Online Training Java Online Training Java Online Training Java Online Training Java Online Training

Anonymous said...

Ok first I want to say that your Agen Judi Bola Online Terpercaya baby is so cute at the laptop and with the Hire Me sign! I can see other posts Agen Judi Bola Online Terpercaya you have written in the preview area and it is so adorable. Agen Judi Bola Online Terpercaya:) I've never heard of these tips, but I have been out of the working world for over 2 years now and I had my first job out of college for almost 4 years before that so you can probably see why this stuff is new to me!
Thank you for sharing with us at Countdown in Style! Agen Judi Bola Online Terpercaya Don't forget to stop by to see if you are featured on Friday! xo

Joel Wong said...

Really great website you have! Are you using wordpress? By the way, if you ever want to accounting company, I know the best singapore company incorporation assistant! register a business today!

ALINAAMEL said...

Your website content nice nice and interesting to observe.
seo lüdenscheid

devo said...

Great article. I like it, Thanks for sharing...Android Projects Center in Chennai | Android Projects Center in Velachery