Python NLTK | nltk.tokenize.StanfordTokenizer()

在nltk.tokenize.StanfordTokenizer()方法的帮助下，我们可以使用tokenize.StanfordTokenizer()方法从字符串或数字字符提取标记。它遵循生成令牌的斯坦福标准。

Syntax : tokenize.StanfordTokenizer()
Return : Return the tokens from a string of characters or numbers.

示例 #1：
在这个例子中，我们可以看到通过使用tokenize.SExprTokenizer()方法，我们能够使用斯坦福标准从字符或数字流中提取标记。

# import StanfordTokenizer() method from nltk
from nltk.tokenize.stanford import StanfordTokenizer
     
# Create a reference variable for Class StanfordTokenizer
tk = StanfordTokenizer()
     
# Create a string input
gfg = "Geeks f o r Geeks"
     
# Use tokenize method
geek = tk.tokenize(gfg)
     
print(geek)

输出 ：

[‘Geeks’, ‘f’, ‘o’, ‘r’, ‘Geeks’]

示例 #2：

# import StanfordTokenizer() method from nltk
from nltk.tokenize.stanford import StanfordTokenizer
     
# Create a reference variable for Class StanfordTokenizer
tk = StanfordTokenizer()
     
# Create a string input
gfg = "This is your great author."
     
# Use tokenize method
geek = tk.tokenize(gfg)
     
print(geek)

输出 ：

[‘This’, ‘is’, ‘your’, ‘great’, ‘author’, ‘.’]