df.stack().reset_index()

   level_0   level_1  0
0        0  Column 1  A
1        0  Column 2  E
2        1  Column 1  B
3        1  Column 2  F
4        2  Column 1  C
5        2  Column 2  G
6        3  Column 1  D
7        3  Column 2  H