Stall "influence" range

Hi folks and paesants! I am playing since release and the game is sooo good BUT...I can't get the logic behind the houses upgrade. I mean...I have like 20 stalls at literally 5 cm from a house but it doesn't let me upgrade it. What could be the reason?