I agree that in small companies the project manager should be responsible. Even when it comes to large firms delegation of some responsibilities are necessary, but the manager needs to stay on top of everything.
I view the manager as the final stop and believe all the decisions are on him not those he delegates the tasks too. They are responsible to him. He is responsible to the stakeholders in the end. It is important for him to know all he relies on and have good communication and review with them.