Well, FWIW, that may in fact be the problem.
You don't focus on the dot, you focus on the target.
Something else that may help at distance is a trick I got from Kyle Lamb, which is to line up the dot with the front sight post when shooting at distance.