If you look at the way entire civilizations have nearly disappeared in our known time line, combine that with the way planets get pummeled by inter stellar activity; I feel it's entirely possible.

With the number of genetic bottlenecks in our history, I also think it's possible that previous civilizations may have not even been entirely "human" by our definition.

4.5 billion years worth of possibilities, kind of puts the classic Darwin vs Creationism argument in its place.