Digi 003 console low latency issues

So suddenly low latency is not working properly in Pro tools 11, outputs are set right the plugins blue out yet there is still a delay and the plugins that are bypassed are still working. What is up with that? If I open a new session with new tracks it works fine, and then stops working. Pro tools 11, mac pro 3 ghz quad cpu, digi 003 console, plugins blue out but keep working as if there not.