You are right, proving the Big Bang happened is no piece of cake. But, as a theory, it explains why the galaxies keep getting away from each other. By putting their courses in reverse, it is logical to assume that they started off from the same point.
What is puzzling is that galaxies seem to be moving faster and faster from that center. Obviously something we are missing there. I personally am not satisfied with the dark-matter explanation.
I am not sure what it means that time may not exist, to be honest. But time runs faster in a higher gravity field, so, while galaxies drift apart time should be moving slower and slower.
This is what I gather from the existing theories at least.