Hadoop： Apache Hadoop 软件库是一个框架，允许使用简单的编程模型跨计算机集群分布式处理大型数据集。简单来说，Hadoop 是一个处理“大数据”的框架。它旨在从单个服务器扩展到数千台机器，每台机器都提供本地计算和存储。 Hadoop 是开源软件。 Apache Hadoop 的核心由称为 Hadoop 分布式文件系统 (HDFS) 的存储部分和是 Map-Reduce 编程模型的处理部分组成。 Hadoop 将文件拆分成大块并将它们分布在集群中的节点之间。然后它将打包的代码传输到节点以并行处理数据。 Hadoop 是由 Doug Cutting 和 Mike Cafarella 于 2005 年创建的。
Splunk： Splunk 是一款主要用于通过 Web 风格的界面搜索、监控和检查机器生成的大数据的软件。 Splunk 在可搜索的容器中执行捕获、索引和关联实时数据，它可以从中生成图形、报告、警报、仪表板和可视化。 Splunk 是一个监控工具。它旨在构建可用于组织的机器生成数据，并能够识别数据模式、生成指标、诊断问题并为业务运营目的授予情报。 Splunk 是一种用于应用程序管理、安全性和合规性以及业务和 Web 分析的技术。 Michael Baum、Rob Das 和 Erik Swan 于 2003 年共同创立了 Splunk。
下表列出了 Hadoop 和 Splunk 之间的差异：
| Feature | Hadoop | Splunk |
|---|---|---|
| Definition | Hadoop is an open source product. It’s a framework that allows storing and processing Big data using HDFs and MapR | Splunk is Real-time monitoring tool. It could br for application, security, performance and management |
| Components | HDFS-Hadoop distributed file system. Map Reduce algorithm. Reducer |
Splunk Indexer Splunk Forwarder Deployment server |
| Architecture | Hadoop architecture follows distributed fashion and it’s a master worker architecture for transforming and analyzing large datasets | Splunk architecture includes components that are in charge for data ingestion, indexing and analytics. Splunk deployment can be of two type’s standalone and distributed |
| Relation | Hadoop passes the result sets to Splunk | Collection of data and processing will be done by hadoop, visualization of those results and reporting will be done by Splunk |
| Benefits | Hadoop identifies the insights in the raw data and helps business to make good choices. | Splunk gives operational intelligence to optimize the IT operations cost |
| Features | Flexibility Cost-effective Scalability Data replication Very fast in data processing |
Splunk collects and indexes the data from many sources Real time monitoring Splunk has very powerful search, analysis capabilities Splunk supports reporting and alerting Splunk supports software installation and cloud service |
| Products | Hortonworks Hadoop Spark R server Interactive Query |
Splunk Enterprise Splunk Cloud Splunk Light Splunk Enterprise Security |
| Designed for | Financial Domain Fraud Detection and Prevention |
Create Dashboard to analyze result Monitor Business metrics |