Got lots of data that you want to make use of? Not so easy to set up an environment to do so and maintain it, eh? Symantecâ€™s data lake is a large scale example of marrying OpenStack platform technologies with big data enabling technologies such as Hadoop, Hive, Storm, Kafka, Spark, etc. This talk will cover what Symantec has done to allow our various teams to easily leverage our many petabytes of security data to increase the protection of our customers against threats such as APTs, identity thieves, and malicious web sites.
Symantec leverages our OpenStack cloud to create multiple analytics clusters, ranging in size from multi-PB to just a few VMs. We use various OpenStack services through a CloudBreak plug-in. Some other technologies we use in setting up and operating these clusters include Ambari, Puppet, a home-grown synthetic transaction system, Zabbix, and Dasher.