Ok, I found out what I did wrong:
I didn't ingest the audio file first, but tried to drag it straight from the finder.
Once I ingest the audio file, I can drag it to the rough cut as additional track. Only then does the right-click menu give me the option to add/delete a track.