Going Crazy-Please help name this book!

Hey guys! I've been trying to find this book for forever and it's been at least 6-7 years since I've read it. Please bare with me since it's been awhile. It's a historical romance where the girl lives with her family with her son. The guy just comes back to town and is possibly a sailor?? I know they had something going on beforehand and her father's land is in danger. The biggest thing I remember is that they had sex in a field. I'm so sorry if this is confusing.