📜  后缀树应用程序2 –搜索所有模式

📅  最后修改于: 2021-04-17 12:45:53             🧑  作者: Mango

给定一个文本字符串和一个模式字符串,在字符串查找模式的所有出现。

已经讨论了很少的模式搜索算法(KMP,Rabin-Karp,朴素算法,有限自动机),可用于此检查。
在这里,我们将讨论基于后缀树的算法。

在1后缀树应用程序(子串检查),我们看到了如何检查一个给定的模式是否串文字与否。这是应经过检查的子串1
在本文中,我们将进一步探讨相同的问题。如果模式是文本的子字符串,那么我们将在文本中找到模式上的所有位置。

作为前提,我们必须知道如何以一种或另一种方式构建后缀树。
在这里,我们将使用Ukkonen算法构建后缀树,该算法已在下面进行了讨论:
Ukkonen的后缀树构造–第1部分
Ukkonen的后缀树构造–第2部分
Ukkonen的后缀树构造–第3部分
Ukkonen的后缀树构造–第4部分
Ukkonen的后缀树构造–第5部分
Ukkonen的后缀树构造–第6部分

让我们看下图:
后缀树应用

这是字符串“ abcabxabcd $”的后缀树,显示后缀索引和边缘标签索引(开始,结束)。边缘上的(子)字符串值仅出于说明目的而显示。我们从不将路径标签字符串存储在树中。
路径的后缀索引告诉该路径上子字符串(从根开始)的索引。
考虑上面树中带有后缀索引7的路径“ bcd $”。它告诉子字符串b,bc,bcd,bcd $在字符串中的索引7处。
同样,带有后缀索引4的路径“ bxabcd $”表明子字符串b,bx,bxa,bxab,bxabc,bxabcd,bxabcd $位于索引4。
类似地,后缀索引为1的路径“ bcabxabcd $”表明子字符串b,bc,bca,bcab,bcabx,bcabxa,bcabxab,bcabxabc,bcabxabcd,bcabxabcd $位于索引1。

如果我们同时看到以上所有三个路径,则可以看到:

  • 子字符串“ b”位于索引1、4和7
  • 子字符串“ bc”位于索引1和7

通过上面的解释,我们应该能够看到以下内容:

  • 子字符串“ ab”位于索引0、3和6
  • 子字符串“ abc”位于索引0和6
  • 子字符串“ c”位于索引2和8
  • 子字符串“ xab”位于索引5
  • 子串“ d”位于索引9
  • 子字符串“ cd”位于索引8

    …..
    …..

您能看到如何查找字符串所有出现的模式吗?

  1. 所有的1,检查如果给定模式字符串确实存在与否(正如我们在子串检查所做的那样)。为此,请针对该模式遍历后缀树。
  2. 如果在后缀树中找到模式(不要从树上掉下来),则遍历该点下方的子树,并在叶节点上找到所有后缀索引。所有这些后缀索引将是字符串的模式索引

后缀树应用

// A C program to implement Ukkonen's Suffix Tree Construction
// And find all locations of a pattern in string
#include 
#include 
#include 
#define MAX_CHAR 256
   
struct SuffixTreeNode {
    struct SuffixTreeNode *children[MAX_CHAR];
   
    //pointer to other node via suffix link
    struct SuffixTreeNode *suffixLink;
   
    /*(start, end) interval specifies the edge, by which the
     node is connected to its parent node. Each edge will
     connect two nodes,  one parent and one child, and
     (start, end) interval of a given edge  will be stored
     in the child node. Let's say there are two nods A and B
     connected by an edge with indices (5, 8) then this
     indices (5, 8) will be stored in node B. */
    int start;
    int *end;
   
    /*for leaf nodes, it stores the index of suffix for
      the path  from root to leaf*/
    int suffixIndex;
};
   
typedef struct SuffixTreeNode Node;
   
char text[100]; //Input string
Node *root = NULL; //Pointer to root node
   
/*lastNewNode will point to the newly created internal node,
  waiting for it's suffix link to be set, which might get
  a new suffix link (other than root) in next extension of
  same phase. lastNewNode will be set to NULL when last
  newly created internal node (if there is any) got it's
  suffix link reset to new internal node created in next
  extension of same phase. */
Node *lastNewNode = NULL;
Node *activeNode = NULL;
   
/*activeEdge is represented as an input string character
  index (not the character itself)*/
int activeEdge = -1;
int activeLength = 0;
   
// remainingSuffixCount tells how many suffixes yet to
// be added in tree
int remainingSuffixCount = 0;
int leafEnd = -1;
int *rootEnd = NULL;
int *splitEnd = NULL;
int size = -1; //Length of input string
   
Node *newNode(int start, int *end)
{
    Node *node =(Node*) malloc(sizeof(Node));
    int i;
    for (i = 0; i < MAX_CHAR; i++)
          node->children[i] = NULL;
   
    /*For root node, suffixLink will be set to NULL
    For internal nodes, suffixLink will be set to root
    by default in  current extension and may change in
    next extension*/
    node->suffixLink = root;
    node->start = start;
    node->end = end;
   
    /*suffixIndex will be set to -1 by default and
      actual suffix index will be set later for leaves
      at the end of all phases*/
    node->suffixIndex = -1;
    return node;
}
   
int edgeLength(Node *n) {
    if(n == root)
        return 0;
    return *(n->end) - (n->start) + 1;
}
   
int walkDown(Node *currNode)
{
    /*activePoint change for walk down (APCFWD) using
     Skip/Count Trick  (Trick 1). If activeLength is greater
     than current edge length, set next  internal node as
     activeNode and adjust activeEdge and activeLength
     accordingly to represent same activePoint*/
    if (activeLength >= edgeLength(currNode))
    {
        activeEdge += edgeLength(currNode);
        activeLength -= edgeLength(currNode);
        activeNode = currNode;
        return 1;
    }
    return 0;
}
   
void extendSuffixTree(int pos)
{
    /*Extension Rule 1, this takes care of extending all
    leaves created so far in tree*/
    leafEnd = pos;
   
    /*Increment remainingSuffixCount indicating that a
    new suffix added to the list of suffixes yet to be
    added in tree*/
    remainingSuffixCount++;
   
    /*set lastNewNode to NULL while starting a new phase,
     indicating there is no internal node waiting for
     it's suffix link reset in current phase*/
    lastNewNode = NULL;
   
    //Add all suffixes (yet to be added) one by one in tree
    while(remainingSuffixCount > 0) {
   
        if (activeLength == 0)
            activeEdge = pos; //APCFALZ
   
        // There is no outgoing edge starting with
        // activeEdge from activeNode
        if (activeNode->children] == NULL)
        {
            //Extension Rule 2 (A new leaf edge gets created)
            activeNode->children] =
                                          newNode(pos, &leafEnd);
   
            /*A new leaf edge is created in above line starting
             from  an existing node (the current activeNode), and
             if there is any internal node waiting for its suffix
             link get reset, point the suffix link from that last
             internal node to current activeNode. Then set lastNewNode
             to NULL indicating no more node waiting for suffix link
             reset.*/
            if (lastNewNode != NULL)
            {
                lastNewNode->suffixLink = activeNode;
                lastNewNode = NULL;
            }
        }
        // There is an outgoing edge starting with activeEdge
        // from activeNode
        else
        {
            // Get the next node at the end of edge starting
            // with activeEdge
            Node *next = activeNode->children];
            if (walkDown(next))//Do walkdown
            {
                //Start from next node (the new activeNode)
                continue;
            }
            /*Extension Rule 3 (current character being processed
              is already on the edge)*/
            if (text[next->start + activeLength] == text[pos])
            {
                //If a newly created node waiting for it's 
                //suffix link to be set, then set suffix link 
                //of that waiting node to current active node
                if(lastNewNode != NULL && activeNode != root)
                {
                    lastNewNode->suffixLink = activeNode;
                    lastNewNode = NULL;
                }
  
                //APCFER3
                activeLength++;
                /*STOP all further processing in this phase
                and move on to next phase*/
                break;
            }
   
            /*We will be here when activePoint is in middle of
              the edge being traversed and current character
              being processed is not  on the edge (we fall off
              the tree). In this case, we add a new internal node
              and a new leaf edge going out of that new node. This
              is Extension Rule 2, where a new leaf edge and a new
            internal node get created*/
            splitEnd = (int*) malloc(sizeof(int));
            *splitEnd = next->start + activeLength - 1;
   
            //New internal node
            Node *split = newNode(next->start, splitEnd);
            activeNode->children] = split;
   
            //New leaf coming out of new internal node
            split->children] = newNode(pos, &leafEnd);
            next->start += activeLength;
            split->children] = next;
   
            /*We got a new internal node here. If there is any
              internal node created in last extensions of same
              phase which is still waiting for it's suffix link
              reset, do it now.*/
            if (lastNewNode != NULL)
            {
            /*suffixLink of lastNewNode points to current newly
              created internal node*/
                lastNewNode->suffixLink = split;
            }
   
            /*Make the current newly created internal node waiting
              for it's suffix link reset (which is pointing to root
              at present). If we come across any other internal node
              (existing or newly created) in next extension of same
              phase, when a new leaf edge gets added (i.e. when
              Extension Rule 2 applies is any of the next extension
              of same phase) at that point, suffixLink of this node
              will point to that internal node.*/
            lastNewNode = split;
        }
   
        /* One suffix got added in tree, decrement the count of
          suffixes yet to be added.*/
        remainingSuffixCount--;
        if (activeNode == root && activeLength > 0) //APCFER2C1
        {
            activeLength--;
            activeEdge = pos - remainingSuffixCount + 1;
        }
        else if (activeNode != root) //APCFER2C2
        {
            activeNode = activeNode->suffixLink;
        }
    }
}
   
void print(int i, int j)
{
    int k;
    for (k=i; k<=j; k++)
        printf("%c", text[k]);
}
   
//Print the suffix tree as well along with setting suffix index
//So tree will be printed in DFS manner
//Each edge along with it's suffix index will be printed
void setSuffixIndexByDFS(Node *n, int labelHeight)
{
    if (n == NULL)  return;
   
    if (n->start != -1) //A non-root node
    {
        //Print the label on edge from parent to current node
        //Uncomment below line to print suffix tree
       // print(n->start, *(n->end));
    }
    int leaf = 1;
    int i;
    for (i = 0; i < MAX_CHAR; i++)
    {
        if (n->children[i] != NULL)
        {
            //Uncomment below two lines to print suffix index
           // if (leaf == 1 && n->start != -1)
             //   printf(" [%d]\n", n->suffixIndex);
   
            //Current node is not a leaf as it has outgoing
            //edges from it.
            leaf = 0;
            setSuffixIndexByDFS(n->children[i], labelHeight +
                                  edgeLength(n->children[i]));
        }
    }
    if (leaf == 1)
    {
        n->suffixIndex = size - labelHeight;
        //Uncomment below line to print suffix index
        //printf(" [%d]\n", n->suffixIndex);
    }
}
   
void freeSuffixTreeByPostOrder(Node *n)
{
    if (n == NULL)
        return;
    int i;
    for (i = 0; i < MAX_CHAR; i++)
    {
        if (n->children[i] != NULL)
        {
            freeSuffixTreeByPostOrder(n->children[i]);
        }
    }
    if (n->suffixIndex == -1)
        free(n->end);
    free(n);
}
   
/*Build the suffix tree and print the edge labels along with
suffixIndex. suffixIndex for leaf edges will be >= 0 and
for non-leaf edges will be -1*/
void buildSuffixTree()
{
    size = strlen(text);
    int i;
    rootEnd = (int*) malloc(sizeof(int));
    *rootEnd = - 1;
   
    /*Root is a special node with start and end indices as -1,
    as it has no parent from where an edge comes to root*/
    root = newNode(-1, rootEnd);
   
    activeNode = root; //First activeNode will be root
    for (i=0; isuffixIndex > -1)
    {
        printf("\nFound at position: %d", n->suffixIndex);
        return 1;
    }
    int count = 0;
    int i = 0;
    for (i = 0; i < MAX_CHAR; i++)
    {
        if(n->children[i] != NULL)
        {
            count += doTraversalToCountLeaf(n->children[i]);
        }
    }
    return count;
}
  
int countLeaf(Node *n)
{
    if(n == NULL)
        return 0;
    return doTraversalToCountLeaf(n);
}
  
int doTraversal(Node *n, char* str, int idx)
{
    if(n == NULL)
    {
        return -1; // no match
    }
    int res = -1;
    //If node n is not root node, then traverse edge
    //from node n's parent to node n.
    if(n->start != -1)
    {
        res = traverseEdge(str, idx, n->start, *(n->end));
        if(res == -1)  //no match
            return -1;
        if(res == 1) //match
        {
            if(n->suffixIndex > -1)
                printf("\nsubstring count: 1 and position: %d",
                               n->suffixIndex);
            else
                printf("\nsubstring count: %d", countLeaf(n));
            return 1;
        }
    }
    //Get the character index to search
    idx = idx + edgeLength(n);
    //If there is an edge from node n going out
    //with current character str[idx], traverse that edge
    if(n->children[str[idx]] != NULL)
        return doTraversal(n->children[str[idx]], str, idx);
    else
        return -1;  // no match
}
  
void checkForSubString(char* str)
{
    int res = doTraversal(root, str, 0);
    if(res == 1)
        printf("\nPattern <%s> is a Substring\n", str);
    else
        printf("\nPattern <%s> is NOT a Substring\n", str);
}
   
// driver program to test above functions
int main(int argc, char *argv[])
{
    strcpy(text, "GEEKSFORGEEKS$"); 
    buildSuffixTree();    
    printf("Text: GEEKSFORGEEKS, Pattern to search: GEEKS");
    checkForSubString("GEEKS");
    printf("\n\nText: GEEKSFORGEEKS, Pattern to search: GEEK1");
    checkForSubString("GEEK1");
    printf("\n\nText: GEEKSFORGEEKS, Pattern to search: FOR");
    checkForSubString("FOR");
    //Free the dynamically allocated memory
    freeSuffixTreeByPostOrder(root);
  
    strcpy(text, "AABAACAADAABAAABAA$");
    buildSuffixTree();    
    printf("\n\nText: AABAACAADAABAAABAA, Pattern to search: AABA");
    checkForSubString("AABA");
    printf("\n\nText: AABAACAADAABAAABAA, Pattern to search: AA");
    checkForSubString("AA");
    printf("\n\nText: AABAACAADAABAAABAA, Pattern to search: AAE");
    checkForSubString("AAE");
    //Free the dynamically allocated memory
    freeSuffixTreeByPostOrder(root);
  
    strcpy(text, "AAAAAAAAA$");
    buildSuffixTree();    
    printf("\n\nText: AAAAAAAAA, Pattern to search: AAAA");
    checkForSubString("AAAA");
    printf("\n\nText: AAAAAAAAA, Pattern to search: AA");
    checkForSubString("AA");
    printf("\n\nText: AAAAAAAAA, Pattern to search: A");
    checkForSubString("A");
    printf("\n\nText: AAAAAAAAA, Pattern to search: AB");
    checkForSubString("AB");
    //Free the dynamically allocated memory
    freeSuffixTreeByPostOrder(root);
  
    return 0;
}

输出:

Text: GEEKSFORGEEKS, Pattern to search: GEEKS
Found at position: 8
Found at position: 0
substring count: 2
Pattern  is a Substring


Text: GEEKSFORGEEKS, Pattern to search: GEEK1
Pattern  is NOT a Substring


Text: GEEKSFORGEEKS, Pattern to search: FOR
substring count: 1 and position: 5
Pattern  is a Substring


Text: AABAACAADAABAAABAA, Pattern to search: AABA
Found at position: 13
Found at position: 9
Found at position: 0
substring count: 3
Pattern  is a Substring


Text: AABAACAADAABAAABAA, Pattern to search: AA
Found at position: 16
Found at position: 12
Found at position: 13
Found at position: 9
Found at position: 0
Found at position: 3
Found at position: 6
substring count: 7
Pattern  is a Substring


Text: AABAACAADAABAAABAA, Pattern to search: AAE
Pattern  is NOT a Substring


Text: AAAAAAAAA, Pattern to search: AAAA
Found at position: 5
Found at position: 4
Found at position: 3
Found at position: 2
Found at position: 1
Found at position: 0
substring count: 6
Pattern  is a Substring


Text: AAAAAAAAA, Pattern to search: AA
Found at position: 7
Found at position: 6
Found at position: 5
Found at position: 4
Found at position: 3
Found at position: 2
Found at position: 1
Found at position: 0
substring count: 8
Pattern  is a Substring


Text: AAAAAAAAA, Pattern to search: A
Found at position: 8
Found at position: 7
Found at position: 6
Found at position: 5
Found at position: 4
Found at position: 3
Found at position: 2
Found at position: 1
Found at position: 0
substring count: 9
Pattern  is a Substring


Text: AAAAAAAAA, Pattern to search: AB
Pattern  is NOT a Substring

Ukkonen的后缀树构造需要O(N)的时间和空间来构建长度为N的字符串的后缀树,此后,遍历子字符串检查将使用O(M)来确定长度为M的模式,然后如果Z出现模式,将需要O(Z)查找所有这些Z出现的索引。
总体图案复杂度是线性的:O(M + Z)。

更详细的分析
长度为N的字符串后缀树中将有多少个内部节点?
答案:N-1(为什么?)
长度为N的字符串中将有N个后缀。
每个后缀都有一个叶子。
因此,长度为N的字符串的后缀树将具有N个叶子。
由于每个内部节点至少有2个子节点,因此N叶后缀树最多具有N-1个内部节点。
如果某个模式在字符串发生Z次,则意味着它将成为Z后缀的一部分,因此在模式匹配在树中结束的点(内部节点和边缘之间)中将存在Z个叶子,因此具有Z个叶子的子树在该点以下将具有Z-1个内部节点。带有Z个叶子的树可以在O(Z)时间中遍历。
总体图案复杂度是线性的:O(M + Z)。
对于给定的模式,Z(出现的次数)最多可以为N。
因此,最坏情况下的复杂度可能是:如果Z接近/等于N,则O(M + N)(具有N个节点的树遍历需要O(N)时间)。

后续问题:

  1. 检查模式是否为文本的前缀?
  2. 检查模式是否为文本的后缀?

我们已经发布了更多有关后缀树应用程序的文章:

  • 后缀树应用程序1 –子字符串检查
  • 后缀树应用程序3 –最长重复子串
  • 后缀树应用程序4 –构建线性时间后缀数组
  • 广义后缀树1
  • 后缀树应用程序5 –最长公共子串
  • 后缀树应用6 –最长回文子串