Constructing optimal policies for agents with constrained architectures