Does this imply that my friend is working tomorrow? "Tomorrow, I'm going to tell my uncle, when you're off work. "

I had an argument with a friend regarding the meaning of this sentence. He says that they way I said this implies that he works "tomorrow". I argue, that I never said he worked tomorrow nor is it implied and that he is not working, this off work.