This is the perfect area for the history enthusiast. Check out underground caves with guided tours from Plain Road in the Golden Bay.
It made things much easier if you speak their languages, thus a private assisted Taiwan free-and-easy tour is typically advised.