Back to Table of contents

Primeur weekly 2019-06-11

Exascale supercomputing

Cray wins Gold and Silver Honours in 2019 American Business Awards ...

Quantum computing

New research unlocks properties for quantum information storage and computing ...

Focus on Europe

One of the most competitive supercomputers in the world to be placed in Kajaani, Finland ...

EuroHPC JU approved 840 million euro funding for eight supercomputers ...

First steps towards a made-in-Europe high-performance microprocessor ...

From 9,4 to 22 Petaflop/s in 2020: Inauguration of Joliot-Curie, the French supercomputer dedicated to French and European research ...

Atos to deliver most powerful supercomputer in Norway to national e-infrastructure provider Uninett Sigma2 ...

MareNostrum 5 will host an experimental platform to create European supercomputing technologies ...

EuroHPC selects Barcelona Supercomputing Center as entity to host one of the largest European supercomputers ...

Gauss Centre for Supercomputing resources support generation of first images of a black hole ...

Hardware

Supermicro authorized WekaIO OEM partner ...

WekaIO and Penguin Computing deliver record-breaking performance on STAC-M3 "Tick Analytics" Benchmark ...

Boston announces first European NVIDIA RAPIDS test drive programme ...

Check Point Software Technologies propels Mellanox past one million Ethernet switch ports ...

NEC provides CLAIX-2018 supercomputer to RWTH Aachen University in Germany ...

NEC enters into a technology partnership with ANEO for utilizing the NEC SX-Aurora Tsubasa-Vector Engine ...

Applications

Cray Shasta sparks new AI research programme at Indiana University ...

Michael Zentner joins SDSC to lead Sustainable Scientific Software Group ...

Researchers rely on Comet to showcase colour-changing materials ...

Supercomputing dynamic earthquake rupture models ...

German Aerospace and HPC join forces for global change research ...

ANSYS and BMW Group partner to jointly create the industry's first simulation tool chain for autonomous driving ...

ANSYS 2019 R2 strengthens digital thread between design, engineering and manufacturing ...

The Cloud

Microsoft and Oracle to interconnect Microsoft Azure and Oracle Cloud ...

Univa collaborates with Intel Corporation to offer an Intel Select Solution for HPC and AI Converged Clusters ...

WekaIO and Penguin Computing deliver record-breaking performance on STAC-M3 "Tick Analytics" Benchmark


Sample configuration of Penguin Computing FrostByte storage solution featuring the WekaIO File System.
6 Jun 2019 Campbell - WekaIO, an innovation expert in high-performance, scalable file storage for data-intensive applications, has achieved record-breaking performance on the STAC-M3 Benchmark from a new integrated solution with Penguin Computing - the FrostByte storage solution featuring the WekaIO File System. The STAC-M3 Antuco and Kanaga Benchmark Suites are the industry standard for testing solutions that enable high-speed analytics on time series data, such as tick database stacks. The WekaIO File System on Penguin Computing Relion servers and FrostByte storage system delivered record-breaking performance at scale for the STAC-M3 "Tick Analytics" Benchmark, affirming that the integrated solution is the ideal choice for algorithmic trading and quantitative analysis workloads common in financial services.

The STAC-M3 benchmark specifications are maintained by the STAC Benchmark Council, which consists of over 300 financial institutions and more than 50 vendor organisations whose purpose is to discuss technical challenges and solutions in financial services and to develop technology benchmark standards that are useful to financial organizations. User firms include the largest global banks, brokerage houses, exchanges, hedge funds, proprietary trading shops, and other market participants. The STAC-M3 results were audited by the Securities Technology Analysis Center (STAC), which facilitates the Council.

The system under test was a Kx kdb+ 3.6 database system distributed across 7 Penguin Computing Relion XO1132g servers with dual Intel Xeon Gold 6140 CPU @ 2.30 GHz and data stored on Penguin FrostByte with the WekaIO File System (v3.2.2) Software running on 8-1U RXE1112 server nodes with 9 NVMe SSDs per server, using Mellanox SB7780 36-port Non-blocking Managed EDR 100Gb/s InfiniBand Router and InfiniBand EDR/ Ethernet 100Gb 1-port 840QSFP28 Adapters. The tests were run on the baseline benchmark suite ("Antuco") and the scaling suite ("Kanaga"), a test suite with five years of simulated stock market data. The test system had a combined performance density of up to 87.5 GB/sec available bandwidth to the client nodes, 40.5 GB/sec of actual throughput, and 2.5M 4K IOPS in 8U, with the ability to fully distribute data, metadata and system services, as well as super low file system latency that leverages high-speed networks and NVMe-optimized storage.

The test system delivered record-breaking performance, breaking 8 STAC-M3 world records for mean query-response times and 4 world records for throughput.

Kx's time-series database kdb+ is commonly used for large scale complex analytics on streaming, real time, and historical datasets in industries ranging from financial services to marketing analytics, and for Industrial IoT. In the financial industry, typical kdb+ applications are for algorithmic trading, back testing (trading algorithm validation), surveillance, regulatory and compliance reporting and research environments. These workloads can demand a lot from the file system.

Better than block-based solutions for Financial Services, the WekaIO File System is the only NVMe-optimized, shared, coherent, POSIX-compliant file system with a fully distributed architecture for data resiliency. The WekaIO Software eliminates islands of storage, providing all applications access to a shared data pool with record-breaking performance. The WekaIO File System delivers 10x better performance than a local file system by bending the rules on data locality, distributing the data across the cluster over low latency InfiniBand or Ethernet networks.

WekaIO is a firm believer in the value of benchmark testing as a means to guide and inform development in the industry and frame results in a manner that delivers actionable intelligence to customers. The WekaIO File System currently holds one of the top positions on the independent SPEC SFS 2014 benchmark and the #1 position on VI4IO's 10 Node Challenge list.

The full STAC Report, which contains an explanation of the benchmark specifications and the results for this solution, is available online .

"Traditionally, IT organizations have used block-based storage solutions for their trading databases, rather than compromise performance due to network and file system latencies. The result has been large investments in specialized hardware and software to run different applications", stated David Hiatt, Director of Business Development and Alliances at WekaIO. "Now with the STAC-M3 Benchmark results, we have proven that with the WekaIO File System on Penguin Computing Relion servers and FrostByte delivers better application performance for Kx kdb+ by running in distributed mode over a network than with traditional block storage, with the additional benefit of simultaneously supporting other applications on the same platform. That's a huge leap in application acceleration and simplifying the storage infrastructure for financial services."

"The Penguin Computing FrostByte integrated solution featuring the WekaIO File System is designed, engineered and vetted to deliver a high-performance storage solution for Kx kdb+", added Dr. Kevin Tubbs, Sr. Director of Technology, Advanced Solutions Group at Penguin Computing. "To break eight STAC-M3 performance records is an incredible achievement and a testament to the innovative architecture of the WekaIO File System and leading HPC and AI solutions from Penguin Computing. For IT supporting different applications and Kx kdb+ time-series databases, this platform is an unbeatable combination for performance, scale, and ease of management."

Glenn Wright, Systems Architecture expert at Kx stated: "Kx customers demand both high performance and low latency for compute and storage. The ability to combine both a distributed kdb+ implementation running against a high-performance shared file system can be attractive to our customers. With WekaIO as this file system, our customers are able to choose how and when they distribute their historical data workload to be accessible by one or more of their analytics workloads."

"It's great to see another successful benchmark test outcome when the WekaIO File System is paired with Mellanox's network adapters. We have successfully collaborated with WekaIO on many benchmarks and together, we continue to set benchmark records", contributed Gilad Shainer, Senior Vice President Marketing at Mellanox Technologies. "The Mellanox InfiniBand EDR/Ethernet 100Gb 1-port 840QSFP28 Adapters provide the lowest latency available, and that coupled with the WekaIO distributed and parallel file system satisfies the performance of even the most demanding I/O intensive application requirements."

"There's now a proliferation of software and hardware configurations aiming to support I/O intensive workloads", commented David Floyer, Co-founder and CTO at Wikibon. "But to meet the performance demands of real-time and historical analytics in kdb+ environments, storage systems must offer both high performance and low latency. The results from the STAC-M3 Benchmark should help IT in making an informed decision when architecting storage systems to support algorithmic trading and quantitative analytics workloads."
Source: WekaIO

Back to Table of contents

Primeur weekly 2019-06-11

Exascale supercomputing

Cray wins Gold and Silver Honours in 2019 American Business Awards ...

Quantum computing

New research unlocks properties for quantum information storage and computing ...

Focus on Europe

One of the most competitive supercomputers in the world to be placed in Kajaani, Finland ...

EuroHPC JU approved 840 million euro funding for eight supercomputers ...

First steps towards a made-in-Europe high-performance microprocessor ...

From 9,4 to 22 Petaflop/s in 2020: Inauguration of Joliot-Curie, the French supercomputer dedicated to French and European research ...

Atos to deliver most powerful supercomputer in Norway to national e-infrastructure provider Uninett Sigma2 ...

MareNostrum 5 will host an experimental platform to create European supercomputing technologies ...

EuroHPC selects Barcelona Supercomputing Center as entity to host one of the largest European supercomputers ...

Gauss Centre for Supercomputing resources support generation of first images of a black hole ...

Hardware

Supermicro authorized WekaIO OEM partner ...

WekaIO and Penguin Computing deliver record-breaking performance on STAC-M3 "Tick Analytics" Benchmark ...

Boston announces first European NVIDIA RAPIDS test drive programme ...

Check Point Software Technologies propels Mellanox past one million Ethernet switch ports ...

NEC provides CLAIX-2018 supercomputer to RWTH Aachen University in Germany ...

NEC enters into a technology partnership with ANEO for utilizing the NEC SX-Aurora Tsubasa-Vector Engine ...

Applications

Cray Shasta sparks new AI research programme at Indiana University ...

Michael Zentner joins SDSC to lead Sustainable Scientific Software Group ...

Researchers rely on Comet to showcase colour-changing materials ...

Supercomputing dynamic earthquake rupture models ...

German Aerospace and HPC join forces for global change research ...

ANSYS and BMW Group partner to jointly create the industry's first simulation tool chain for autonomous driving ...

ANSYS 2019 R2 strengthens digital thread between design, engineering and manufacturing ...

The Cloud

Microsoft and Oracle to interconnect Microsoft Azure and Oracle Cloud ...

Univa collaborates with Intel Corporation to offer an Intel Select Solution for HPC and AI Converged Clusters ...