Large-scale data management is essential for modern climate science. The CASCADE project has generated upwards of 3PB of data itself. LBNL data scientists have prepared a blog post describing the state of large scale data management across a variety of disciplines including climate science. See this link https://www.oreilly.com/ideas/the-big-data-ecosystem-for-science