Back to Table of contents

Primeur weekly 2017-11-06

Quantum computing

Researchers develop data bus for quantum computer ...

Focus on Europe

Luxembourg 25th country to join PRACE ...

European Commission to invest 30 billion euro in new solutions for societal challenges and breakthrough innovation ...

Middleware

3i partners with DDN Storage to deliver demanding microscopy workflow solutions and unmatched performance ...

A third of the Internet is under attack ...

University of Nevada Las Vegas selects winning combination of Bright Cluster Manager and Advanced HPC ...

Hardware

Cray reports third quarter 2017 financial results ...

Asetek announces OEM partnership with E4 Computer Engineering and installation at CINECA in Italy ...

Asetek announces NEC Corporation as new data centre OEM partner ...

NVIDIA announces new AI partners, courses, initiatives to deliver deep learning training worldwide ...

Canadian Minister Qualtrough unveils new supercomputers to help forecast Canada's weather ...

KAUST IT receives iCMG Enterprise Architecture Award ...

Pawsey's Zeus expands to fulfil researchers' needs ...

NCI supercharges Raijin, Australia's fastest supercomputer ...

Supermicro showcases deep learning optimized systems ...

Applications

Sherlock deploys UC's first HIPAA-compliant Hadoop based data management system ...

Yu Liu appointed assistant professor of electrical & computer engineering at Clarkson University ...

Cutting datasets down to size ...

University of Delaware professor wins IEEE award ...

A bit of a 'quantum magic trick' ...

From quantum physicist to quantum CEO ...

High-performance computing methods focus of new text ...

Students explore immigration through a Big Data lens ...

Bright Computing announces reseller agreement with HPE for high performance computing and AI ...

The Cloud

IBM brings Cloud-native environment to private Clouds ...

IBM expands Watson Data Platform to help unleash AI for professionals ...

UberCloud brings parallel MPI to Univa Grid Engine's Docker support ...

Huawei acquires a seat in the Kubernetes Steering Committee ...

Red Hat improves IT flexibility and reduces complexity with Linux containers in latest version of production-ready OpenStack platform ...

Cutting datasets down to size


The KAUST supercomputer Shaheen II underpins the collaboration by providing high-performance computing applications and strategic advice and support. Credit: 2017 KAUST.
30 Oct 2017 Thuwal - A powerful statistical tool could significantly reduce the burden of analyzing very large datasets. By exploiting the power of high-performance computing, a new statistical tool has been developed by KAUST researchers that could reduce the cost and improve the accuracy of analyzing large environmental and climate datasets.

Datasets containing environmental and climate observations, such as temperature, wind speeds and soil moisture, are often very large because of the high spatial resolution of the data. The cost of analyzing such datasets increases steeply as the size of the dataset increases: for instance, increasing the size of a dataset by a factor of 10 drives up the cost of the computation by a factor of a 1000, and the memory requirements by a factor of 100, creating a computational strain on standard statistical software.

This spurred postdoctoral fellow Sameh Abdulah to develop a standalone software framework through a collaboration between KAUST's Extreme Computing Research Center (ECRC) and statisticians specializing in spatio-temporal dynamics and the environment.

The new framework, called Exascale GeoStatistics or ExaGeoStat, is able to process large geospatial environmental and climate data by employing high-performance computing architectures with a high degree of concurrency not available through universally used statistical software.

"Existing statistical software frameworks are not able to fully exploit large datasets", stated Sameh Abdulah. "For example, a computation that would normally require one minute to complete would take nearly 17h if the dataset were just 10 times larger. This leads to compromises due to the limitations in computing power, forcing researchers to turn to approximation methods that cloud their interpretation of results."

Leveraging linear algebra software developed by the ECRC, ExaGeoStat provides a framework for computing the maximum likelihood function for large geospatial environmental and climate datasets. It is able to predict unknown or missing data as well as reduce the effect of individual measurement errors, allowing the data to be easily analyzed and represented in a statistical model used for making predictions.

The researchers successfully applied ExaGeoStat to a large, real-world dataset of soil moisture measurements from the Mississippi basin in the United States. This could lead to the routine analysis of the larger datasets that are becoming available to geospatial statisticians, and could be used in a wide range of applications from weather forecasting, crop-yield prediction, and early-warning systems for flood and drought.

David Keyes, Director of the ECRC, which hosts the project, plans significant further improvements, tracking a rapidly developing technique in linear algebra: "We are now working on taking ExaGeoStat a step further on the algorithmic side by introducing a new type of approximation, called hierarchical tile low-rank approximation, which reduces memory requirements and operations by allowing for small errors that can easily be understood and controlled."

Abdulah S., Ltaief H., Sun Y., Genton M.G., and Keyes D.E. are the authors of the paper titled "ExaGeoStat: a high performance unified framework for geostatistics on manycore systems" - arXiv:1708.02835v2 (cs.DC) 2017.

Source: King Abdullah University of Science and Technology - KAUST

Back to Table of contents

Primeur weekly 2017-11-06

Quantum computing

Researchers develop data bus for quantum computer ...

Focus on Europe

Luxembourg 25th country to join PRACE ...

European Commission to invest 30 billion euro in new solutions for societal challenges and breakthrough innovation ...

Middleware

3i partners with DDN Storage to deliver demanding microscopy workflow solutions and unmatched performance ...

A third of the Internet is under attack ...

University of Nevada Las Vegas selects winning combination of Bright Cluster Manager and Advanced HPC ...

Hardware

Cray reports third quarter 2017 financial results ...

Asetek announces OEM partnership with E4 Computer Engineering and installation at CINECA in Italy ...

Asetek announces NEC Corporation as new data centre OEM partner ...

NVIDIA announces new AI partners, courses, initiatives to deliver deep learning training worldwide ...

Canadian Minister Qualtrough unveils new supercomputers to help forecast Canada's weather ...

KAUST IT receives iCMG Enterprise Architecture Award ...

Pawsey's Zeus expands to fulfil researchers' needs ...

NCI supercharges Raijin, Australia's fastest supercomputer ...

Supermicro showcases deep learning optimized systems ...

Applications

Sherlock deploys UC's first HIPAA-compliant Hadoop based data management system ...

Yu Liu appointed assistant professor of electrical & computer engineering at Clarkson University ...

Cutting datasets down to size ...

University of Delaware professor wins IEEE award ...

A bit of a 'quantum magic trick' ...

From quantum physicist to quantum CEO ...

High-performance computing methods focus of new text ...

Students explore immigration through a Big Data lens ...

Bright Computing announces reseller agreement with HPE for high performance computing and AI ...

The Cloud

IBM brings Cloud-native environment to private Clouds ...

IBM expands Watson Data Platform to help unleash AI for professionals ...

UberCloud brings parallel MPI to Univa Grid Engine's Docker support ...

Huawei acquires a seat in the Kubernetes Steering Committee ...

Red Hat improves IT flexibility and reduces complexity with Linux containers in latest version of production-ready OpenStack platform ...