The Roman Arena in Nimes is the best preserved of its type in the world. It's in better shape even than the Colosseum in Rome. It was built around 100 AD and is still used for live bullfights and other events.
The Maison Carree ("square house") rivals Rome's Pantheon as the most complete and beautiful building that survives from the Roman Empire. It has been dated to AD 4 during the reign of Augustus Caesar. Rick Steves says that the video they show inside the building is not worth the money and you should just enjoy this building from the outside for free. The building is quite impressive either way. An interesting piece of trivia is that Thomas Jefferson used the Maison Carree as his model when designing the Virginia State Capitol building.
There are several other minor Roman sights in Nimes and a few museums, but they seem worthy of skipping in favor of visiting other places in Provence.