- (…) parentheses indicate a subshell. 
- { … } braces are like parentheses in that they group commands, but they only influence parsing, not grouping.
- (( … )) double parentheses surround an arithmetic instruction, that is, a computation on integers, with a syntax resembling other programming languages. This syntax is mostly used for assignments and in conditionals. This only exists in ksh/bash/zsh, not in plain sh.
- [ … ] single brackets surround conditional expressions. Conditional expressions are mostly built on operators such as -n "$variable" to test if a variable is empty and -e "$file" to test if a file exists. Note that you need a space around each operator (e.g. [ "$x" = "$y" ], not [ "$x"="$y" ]), and a space or a character like ; both inside and outside the brackets (e.g. [ -n "$foo" ], not [-n "$foo"]).
- [[ … ]] double brackets are an alternate form of conditional expressions in ksh/bash/zsh with a few additional features, for example you can write [[ -L $file && -f $file ]] to test if a file is a symbolic link to a regular file whereas single brackets require [ -L "$file" ] && [ -f "$file" ]. See Why does parameter expansion with spaces without quotes works inside double brackets [[ but not single brackets [? for more on this topic.