Most of the time, it is difficult to get these two words, confused or misused. That is because they are used interchangeably. Most of the time, people use the word abroad to mean in another country that is not close by. The same can be said for overseas. However, most people use the word abroad more often.
If you want to use the word elsewhere, it is usually used to mean to travel or refer to another country that is not located close by. However, it does not need to have a body of water between that country and the original country. If overseas is used, then it does mean to go to another country, and you would cross over a body of water.