| Инд. авторы: | Kikhtenko V.A., Mamash E.A., Chubarov D.L. |
| Заглавие: | Computational infrastructure for remote sensing data processing at ICT SB RAS: design and evolution |
| Библ. ссылка: | Kikhtenko V.A., Mamash E.A., Chubarov D.L. Computational infrastructure for remote sensing data processing at ICT SB RAS: design and evolution // Современные информационные технологии для научных исследований в области наук о Земле: Материалы IV Международной конференции, Южно-Сахалинск,7-11 августа 2016 г. - 2016. - Владивосток: Дальнаука. - P.53-53. - ISBN: 978-5-8044-1605-9. |
| Реферат: | eng: We present a brief overview of the current state of remote sensing data processing infrastructure currently in use at the Institute of Computational Technologies of SB RAS. Over the years this infrastructure evolved into a sophisticated ensemble of facilities including a scalable storage, computational processing cluster and a number of services running on top of this hardware. This infrastructure supports heterogeneous workload consisting of multiple workload types. Each workload type has an associated subsystem for resource management and allocation that caters for its specific requirements. Access to common hardware by these workload specific subsystems is arbitrated by Apache Mesos ? a scalable fault-tolerant resource manager with pluggable architecture. Currently the following workload types are supported: an automated operative processing of direct readout satellite data implemented in the framework of data-driven architecture with the help of Taverna computational workflow management system; interactive data analysis is performed from within IPython notebooks running on top of a JupyterHub that isolates concurrent users in Docker containers. Access to satellite data products is enabled via hVault relational database technology that allows for SQL queries to be used to extract individual datapoints from a multiterabyte file archive. This technology also provides for easy implementation of web services to access the data using WMS and WCS standardized protocols. |
| Издано: | 2016 |
| Физ. характеристика: | с.53-53 |
| Конференция: | Название: IV Международная конференция "Современные информационные технологии для научных исследований в области наук о Земле" Аббревиатура: ITES-2016 Город: Южно-Сахалинск Страна: Россия Даты проведения: 2016-08-07 - 2016-08-11 Ссылка: http://sakhalin.fareastgeology.ru/ |