Hemingway?
Heh, you're off by about only 700 miles. It's another Deborah Kerr film (that's William Holden looking all Ernest), this shot on St Thomas. My father watched it being filmed, so when he told me about it, I had to track it down. Lots of familiar places. The producers/studio wanted to capitalize on the success of From Here to Eternity, also with Kerr. The Proud and Profane.
