Anton Shterenlikht (Bristol)
Multi-scale cellular automata finite element (CAFE) framework for fracture in heterogeneous materials: to exascale via coarrays/MPI hybrid programming
All real materials are heterogeneous, e.g. polycrystalline metal alloys, reinforced concrete, carbon fibre reinforced polymer (CFRP), wood, nuclear graphite or bone. Modelling of such materials involves concurrent simulation of multiple interacting and competing physical processes, acting at different length and time scales, e.g. dislocation flow, ply debonding or separation of atomic layers. In this work we demonstrate a multi-scale fracture framework where cellular automata (CA) represents fracture at micro- or nano-scales and finite elements (FE) are used at structural scales. Fortran coarrays are native Fortran means for SPMD style of parallel programming, which offer simple and intuitive data structures for 3D CA. Design of a coarray CA microstructure evolution library CGPACK is described. We show how coarrays can be used together with an MPI FE library to create a two-way concurrent hierarchical multi-scale CAFE deformation and fracture framework. A highly portable MPI FE library ParaFEM was used in this work. The CAFE framework is based on mapping coarrays to MPI data structures. FE tensor fields are distributed over CA cells (localisation) based on existing damage and microstructure heterogeneity. After each fracture propagation increment at the CA scale, the microstructural damage is encoded in a scalar damage variable (homogenisation) which is passed back to the FE integration point. We show that independently CGPACK and ParaFEM programs can scale up well into tens of thousands of cores on Cray systems. An MPI/coarray CAFE miniapp of a fracture of a steel round bar under tension scales up to 7k cores. Coarrays is an example of a partitioned global address space (PGAS) programming model, with single sided communication, where a remote read or write operation can be performed by a single process/image, with no cooperation or even knowledge of the process whose data is being accessed. The single sided communication strategy is very different from two-way MPI send/receive communication. It poses specific challenges for synchronisation, data integrity and performance in coarray programs.