First, the graph:
Notice that administrative costs generally range from minimal to non-existent in all of these very large government programs. More interestingly, in the ones with higher admin costs, those costs are higher because of things conservatives insist on:
Programs like SNAP and Section 8 housing have fairly stringent means testing rules in order to root out folks trying to game the system, and the result of that is higher admin costs. It's pretty unavoidable. We could probably cut the overhead costs of housing vouchers by simply giving money to anyone under a certain income line and then calling it a day, but we don't. We make sure you really truly qualify, we make sure the vouchers are really spent on housing, and we make sure that landlords aren't scamming either tenants or the taxpayers. This is exactly the kind of thing conservatives are always urging us to do, and it costs money. There's no way around it.
The moralistic, heavy-handed nonsense that conservatives like to force upon agencies ends up increasing their administrative costs, and then conservatives turn around and accuse them of inefficiency. Now, it is possible that the money saved from scammers or whatever offsets the higher admin costs, and it is possible that some regulations are "worth it" even if they cost more than they save, but that doesn't change the fact that accusations of inefficiency are both inaccurate and deeply unfair.