How does your family feel about this? Are they accepting or would they disown you? And what do you mean by "family"? Parents and siblings or wife and children?
In the end, it's your choice to make and you have to live with the consequences. Think long and hard about this.
