📜  wordcount pyspark - Python 代码示例

📅  最后修改于: 2022-03-11 14:45:53.637000             🧑  作者: Mango

代码示例1
text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")