Looping is weird and the game will weirdly hang open when it's closed, so you'll have to either use Task Manager to forcefully close the application or simply click Stop on Steam to close the game.
OR
It could just be an issue on my end, not entirely sure yet.