COLLECTIONS

Documents published in Scipedia

• Advances in FEFLO

  R. Lohner, F. Camelli, J. Baum, F. Togashi, O. Soto

  Aerospace Research Central (2013). Vol. 23 (1), pp. 1-20

  
  

  Abstract

  This paper summarizes the major improvements and developments that have taken place during the last year for FEFLO, a general-purpose CFD code based on adaptive, unstructured grids. All aspects of a comprehensive simulation pipeline: pre-processing, gridding, field solvers and post-processing saw important advances, and are treated. The advent of machines with millions of cores focused a lot of the developments on distributed memory aspects of field solvers and multiphysics modules. Parallel grid generation, particles and flow, parallel interpolation, parallel domain splitting and repartitioning for multiphysics were all considered. Furthermore, timings on Xeon and AMD chips led to the realization that memory transfers play a considerable role as compared to floating point operations, something previous chip generations did not exhibit. This led to a thorough analysis of numerical algorithms with subsequent re-writing. This in-depth analysis determined that at present achievable speeds are limited by the communication between processors (MPI). Even as the number of cores/domains can increase to several hundred thousand, due to communication overheads the wall clock time it takes to update a flowfield is bounded. We expect that as more users get access to hundreds of thousands of cores and the domain size per core starts shrinking, they will also encounter this 'minimum timestep barrier'.

  Abstract
  This paper summarizes the major improvements and developments that have taken place during the last year for FEFLO, a general-purpose CFD code based on adaptive, unstructured [...]

  • 4
  •  read

• Linear Sources for Mesh Generation

  R. Aubry, K. Karamete, E. Mestreau, S. Dey, R. Lohner

  SIAM J. on Scientific Computing (2013). Vol. 35 (2), pp. 886-907

  
  

  Abstract

  Sources offer a convenient way to prescribe a size distribution in space. For each newly created mesh point, the mesh generator queries the local size distribution, either to create a new point or element, depending on the underlying mesh generation method, to smooth the mesh, or to get a local relevant length scale. Sources may have different shapes such as points, edges, triangles, or boxes. They provide the size distribution given some user defined parameters and the distance of a point location to the source. Traditionally, the source strength is considered as constant. In this work, extensions to linear sources in space are proposed. It is shown that in the case of curvature refined mesh generation, substantial savings may occur due to the much better approximation of the curvature variation for a simple modification of traditional sources. Even though curvature refinement is the main application of this work, improvements through linear sources are relevant to other contexts such as user defined sources. The technique is very general as it deals with a fundamental aspect of mesh generation and can be easily incorporated into an existing mesh generator with traditional sources. Thorough details of source approximations and source filtering are provided. Relations with lower envelopes are highlighted. Practical examples illustrate the accuracy and efficiency of the method.

  Abstract
  Sources offer a convenient way to prescribe a size distribution in space. For each newly created mesh point, the mesh generator queries the local size distribution, either [...]

  • 4
  •  read

• Error and work estimates for high‐order elements

  R. Lohner

  Int. J. Numer. Meth. Fluids (2013). Vol. 67 (12), pp. 2184-2188

  
  

  Abstract

  Error and work estimates for high‐order elements are derived. The comparison of error and work estimates shows that for relative accuracy in the 1% range, which is typical of engineering interest, it may prove very difficult to improve on linear elements. As expected, the estimates also show that the optimal order of element in terms of work and storage demands depends on the desired relative accuracy.

  Abstract
  Error and work estimates for high‐order elements are derived. The comparison of error and work estimates shows that for relative accuracy in the 1% range, which is typical [...]

  • 4
  •  read

• Handling tens of thousands of cores with industrial/legacy codes: Approaches, implementation and timings

  R. Lohner, J. Baum

  Computers and Fluids (2013). Vol. 85, pp. 53-62

  
  

  Abstract

  The consequences that the recent stagnation in clockrates for CPUs has had on large-scale CFD runs are examined. At first sight, the conclusion is that only massive parallelism at the loop or domain decomposition level will lead to higher FLOP counts. However, the significant differences in advances for CPUs/GPUs versus RAM and interprocessor communication bandwidth lead to a so-called ‘limiting domain size’, below which communication dominates execution times and performance degrades drastically. The consequences of this ‘red-shift’ for the future of CFD are manifold: the time to advance the solution one timestep is limited, implying that even with unlimited number of processors/cores, LES, DES and DNS runs for realistic Reynolds-numbers will require days or weeks of execution.

  Abstract
  The consequences that the recent stagnation in clockrates for CPUs has had on large-scale CFD runs are examined. At first sight, the conclusion is that only massive parallelism [...]

  • 0
  •  read

• Scaling Up Multiphysics

  R. Lohner, J. Baum

  (2014). Computational Methods in Applied Sciences. Vol 33. Springer, Cham

  
  

  Abstract

  The present paper summarizes trends in supercomputing and the consequences they will have on coupled problems in computational mechanics. The appearance of parallel machines with more than 106 cores implies that the prevalent ‘scalar-pre, simple parallel solve, scalar-post’ environment will have to give way to a completely scalable simulation pipeline. The grid size alone will force distributed parallelization of meshing, domain splitting, load balancing and post-processing. Possible ways of addressing parallel meshing and dynamic load balancing for multiphysics are treated, and examples shown.

  Abstract
  The present paper summarizes trends in supercomputing and the consequences they will have on coupled problems in computational mechanics. The appearance of parallel machines [...]

  • 0
  •  read

• Butterfly-Effect for Massively Separated Flows

  R. Lohner, D. Britto, A. Michalski, E. Haug

  Engineering Computations (2014). Vol. 31 (4), pp. 742-757

  
  

  Abstract

  Purpose ‐ During a routine benchmarking and scalability study of CFD codes for typical large-scale wind engineering runs, it was observed that the resulting loads for buildings varied considerably with the number of parallel processors employed. The differences remained very small at the beginning of a typical run, and then grew progressively to a state of total dissimilitude. A "butterfly-effect" for such flows was suspected and later confirmed. The paper aims to discuss these issues. Design/methodology/approach ‐ A series of numerical experiments was conducted for massively separated flows. The same geometry ‐ a cube in front of an umbrella ‐ was used to obtain the flowfields using different grids, different numbers of domains/processors, slightly different inflow conditions and different codes. Findings ‐ In all of these cases the differences remained very small at the beginning of a typical run, they then grew progressively to a state of total dissimilitude. While the mean and maximum loads remained similar, the actual (deterministic) instantiations were completely different. The authors therefore suspect that for flows of this kind a "butterfly effect" is present, whereby even very small (roundoff) errors can have a pronounced effect on the actual deterministic instantiation of a flowfield. Research limitations/implications ‐ This implies that for flows of this kind the CFD runs have to be carried out to much larger times than formerly expected (and done) in order to obtain statistically relevant ensembles. Practical implications ‐ For practical calculations this implies running to much larger times in order to reach statistically relevant ensembles, with the associated much higher CPU time requirements. Originality/value ‐ This is the first time such a finding has been reported in the numerical wind engineering context.

  Abstract
  Purpose ‐ During a routine benchmarking and scalability study of CFD codes for typical large-scale wind engineering runs, it was observed that the resulting loads for buildings [...]

  • 1
  •  read

• On mesh-particle techniques

  R. Lohner, F. Camelli, J. Baum, F. Togashi, O. Soto

  Comp. Part. Mech (2014). Vol. 1, pp. 199-209

  
  

  Abstract

  The treatment of dilute solid (or liquid) phases via Lagrangian particles within mesh-based gas-dynamics (or hydrodynamic) codes is common in computational fluid dynamics. While these techniques work very well for a large spectrum of physical parameters, in some cases, notably for very light or very heavy particles, numerical instabilities appear. The present paper examines ways of mitigating these instabilities, and summarizes important implementational issues.

  Abstract
  The treatment of dilute solid (or liquid) phases via Lagrangian particles within mesh-based gas-dynamics (or hydrodynamic) codes is common in computational fluid dynamics. [...]

  • 1
  •  read

• Recent Advances in Parallel Advancing Front Grid Generation

  R. Lohner

  Archives of Comp. Meths. Engng. (2014). Vol. 21 (2), pp. 127-140

  
  

  Abstract

  The quest for scalable, parallel advancing front grid generation techniques now spans more than two decades. A recent innovation has been the use of a so-called domain-defining grid, which has led to a dramatic increase in robustness and speed. The domain-defining grid (DDG) has the same fine surface triangulation as the final mesh desired, but a much coarser interior mesh. The DDG renders the domain to be gridded uniquely defined and allows for a well balanced work distribution among the processors during all stages of grid generation and improvement. In this way, most of the shortcomings of previous techniques are overcome. Timings show that the approach is scalable and able to produce large grids of high quality in a modest amount of clocktime. These recent advances in parallel grid generation have enabled a completely scalable simulation pipeline (grid generation, solvers, post-processing), opening the way for truly large-scale computations using unstructured, body-fitted grids.

  Abstract
  The quest for scalable, parallel advancing front grid generation techniques now spans more than two decades. A recent innovation has been the use of a so-called domain-defining [...]

  • 1
  •  read

• Modeling subway air flow using CFD

  F. Camelli, G. Byrne, R. Lohner

  Tunnelling and Underground Space Technology (2014). Vol. 43, pp. 20-31

  
  

  Abstract

  A more realistic Computational Fluid Dynamics approach has been developed to model air flow in subway tunnels and stations. This modeling incorporates the effects of multi-car trains as they arrive and depart in a network of interconnected stations at realistic speeds. It also includes thermal sources within the stations and a coupling to the street level through inlets and outlets. This new approach is validated against experimental data, and numerical simulations are carried out in realistic subway station geometries. The air flow dynamics in these stations are quantified in terms of their spatial complexity and temporal stability. The air flow dynamics correlate to the dispersion and transport of pollutants though the stations.

  Abstract
  A more realistic Computational Fluid Dynamics approach has been developed to model air flow in subway tunnels and stations. This modeling incorporates the effects of multi-car [...]

  • 2
  •  read

• On maximum achievable speeds for field solvers

  R. Lohner, J. Baum

  Int. J. Numer. Meth. Heat and Fluid Flows (2014). Vol. 24 (7), pp. 1537-1544

  
  

  Abstract

  Purpose ‐ Prompted by the empirical evidence that achievable flow solver speeds for large problems are limited by what appears to be a time of the order of O(0.1) sec/timestep regardless of the number of cores used, the purpose of this paper is to identify why this phenomenon occurs. Design/methodology/approach ‐ A series of timing studies, as well as in-depth analysis of memory and inter-processors transfer requirements were carried out for a typical field solver. The results were analyzed and compared to the expected performance. Findings ‐ The analysis shows that at present flow speeds per core are already limited by the achievable transfer rate to RAM. For smaller domains/larger number of processors, the limiting speed of CFD solvers is given by the MPI communication network. Research limitations/implications ‐ This implies that at present, there is a "limiting useful size" for domains, and that there is a lower limit for the time it takes to update a flowfield. Practical implications ‐ For practical calculations this implies that the time required for running large-scale problems will not decrease markedly once these applications migrate to machines with hundreds of thousands of cores. Originality/value ‐ This is the first time such a finding has been reported in this context.

  Abstract
  Purpose ‐ Prompted by the empirical evidence that achievable flow solver speeds for large problems are limited by what appears to be a time of the order of O(0.1) sec/timestep [...]

  • 1
  •  read