I'll bet that after "dubya dubya two" they didn't have disposal facilities for SAFELY burning up WMDs like Russia and the US have now to destroy their Cold War stocks, so they dumped the stuff hoping that it sank deep enough not to be a problem. It might make sense to attempt salvage the containers are properly destroy the stuff, but salvage efforts may just break the containers more and cause more damage. I think the better choice is to sink concrete casings over the barrels.
As a little FYI: I read somewhere that a Russian disposal plant can destroy something like 4000 tons of chemical weapons a month and they've been working for a couple years and still have not destroyed the old Cold War stuff.
Funny thing is Hitler never ever aloud for Gas Shells to be used in Combat. He was injured in WW1 by the stuff and Was a bit sensitive about the whole issue.