| Инд. авторы: | Окладников И.Г., Гордов Е.П., Титов А.Г. |
| Заглавие: | Разработка модели хранения и обработки климатических данных |
| Библ. ссылка: | Окладников И.Г., Гордов Е.П., Титов А.Г. Разработка модели хранения и обработки климатических данных // Enviromis 2016: международная конференция и школа молодых ученых по измерению, моделированию и информационным системам для изучения окружающей среды. - 2016. - Томск: Томский центр научно-технической информации. - С.342-346. - ISBN: 978-5-89702-362-2. |
| Внешние системы: | РИНЦ: 27616014; |
| Реферат: | eng: Storage and processing model for climate datasets developed in the framework of a distributed research center for monitoring and predicting regional climate and environmental changes is presented. This research center is focused on processing and analysis of large volumes of climate data and able to help researchers (who are not experts in the field of information technologies) in the environmental and climate change study. The considered model is based on a «shared nothing» distributed computing architecture [1], where each com- puting node is independent and self-sufficient. On each compute node a dedicated software for processing and visu- alization of geospatialdata - processing core - is installed; it provides programming interfaces to communicate with other nodes. The nodes are interconnected by local network or Internet and exchange data and control instructions via a SSH connection and WPS. Geospatial data is stored in sets of NetCDF binary files organized in a special directory hierarchy in the framework of a file system used on each compute node. To speed up data reading and processing three approaches are proposed: preliminary calculation of intermediate products, distribution of data across multiple storage systems (with or without redundancy), and caching and reuse of previously obtained products. For a fast search and retrieval of a required data according to given parameters in the framework of the devel- oped data storage and processing model, a metadata database, containing descriptions of space-time features of da- tasets available for processing, locations of data files, as well as descriptions and run parameters of analysis and visualization software components is developed. In general, the developed model supports functioning of the distrib- uted research center for monitoring and predicting regional climate and environmental changes. |
| Издано: | 2016 |
| Физ. характеристика: | с.342-346 |
| Конференция: | Название: 9th International Multidisciplinary Conference and Early Career Scientists School on Environmental Observations, Modelling and Information Systems Аббревиатура: ENVIROMIS-2016 Город: Tomsk Страна: Russia Даты проведения: 2016-07-11 - 2016-07-16 Ссылка: http://www.scert.ru/ru/conference/enviromis2016/ |
| Цитирование: | 1. Лыкосов В.Н., Глазунов А.В., Кулямин Д.В., Мортиков Е.В., Степаненко В.М. Суперкомпьютерное моделирование в физике климатической системы // Издательство Московского университета, 2012. - 295 с. 2. Sandia sees data management challenges spiral. HPC Projects. 4 August 2009. (Электронный ресурс: http://www.s cientific-computing.com/news /news _s tory.php?news _id=922). 3. Reichman O.J., Jones M.B., Schildhauer M.P. Challenges and Opportunities of Open Data in Ecology // Science. - V. 331. - No. - 6018. - P. 703-705. DOI:10.1126/science.1197962. 4. Гордов Е.П., Кабанов М.В., Лыкосов В.Н. Информационно-вычислительные технологии для наук об окружающей среде: подготовка научной смены // Вычислительные технологи. - 2006. - Т. 11 (1). - Специальный выпуск. - С. 3-15. 5. Valduriez P. Shared-Nothing Architecture / Encyclopedia of Database Systems. ISBN: 978-0-387-35544-3. Eds.: Ling Liu, M. Tamer Özsu. - 2009. - P. 2638-2639. |