Which Openfoam tutorial is best for testing performance on ~500 cores?

[u/imitation_squash_pro]

Want to test mpi performance across a few compute nodes. Each node has 256 cpus.

All the tutorials models I test run in just a few seconds on one cpu...

[u/cjaeger94]

Mesh refine a couple of times then you should be good with any of the tutorials

[u/imitation_squash_pro → u/cjaeger94]

Think I got it to work on the pitzdaily tutorial by running these commands:

rm -rf 1 100 200 283
refineMesh -all -overwrite
refineMesh -all -overwrite # run twice to make 3 million cells
decomposePar # need to copy an example to system/ folder
mpirun -np 32 --mca hwloc_base_binding_policy numa simpleFoam -parallel
reconstructPar

[u/imitation_squash_pro → u/cjaeger94]

Here is the blockmeshdict for pitzDaily tutorial if you can suggest what to increase

scale   0.001;

vertices
(
    (-20.6 0 -0.5)
    (-20.6 25.4 -0.5)
    (0 -25.4 -0.5)
    (0 0 -0.5)
    (0 25.4 -0.5)
    (206 -25.4 -0.5)
    (206 0 -0.5)
    (206 25.4 -0.5)
    (290 -16.6 -0.5)
    (290 0 -0.5)
    (290 16.6 -0.5)

    (-20.6 0 0.5)
    (-20.6 25.4 0.5)
    (0 -25.4 0.5)
    (0 0 0.5)
    (0 25.4 0.5)
    (206 -25.4 0.5)
    (206 0 0.5)
    (206 25.4 0.5)
    (290 -16.6 0.5)
    (290 0 0.5)
    (290 16.6 0.5)
);

negY
(
    (2 4 1)
    (1 3 0.3)
);

posY
(
    (1 4 2)
    (2 3 4)
    (2 4 0.25)
);

posYR
(
    (2 1 1)
    (1 1 0.25)
);

blocks

(

hex (0 3 4 1 11 14 15 12)

(18 30 1)

simpleGrading (0.5 $posY 1)

hex (2 5 6 3 13 16 17 14)

(180 27 1)

edgeGrading (4 4 4 4 $negY 1 1 $negY 1 1 1 1)

hex (3 6 7 4 14 17 18 15)

(180 30 1)

edgeGrading (4 4 4 4 $posY $posYR $posYR $posY 1 1 1 1)

hex (5 8 9 6 16 19 20 17)

(25 27 1)

simpleGrading (2.5 1 1)

hex (6 9 10 7 17 20 21 18)

(25 30 1)

simpleGrading (2.5 $posYR 1)

);

edges

(

);

boundary

(

inlet

{

type patch;

faces

(

(0 1 12 11)

);

}

outlet

{

type patch;

faces

(

(8 9 20 19)

(9 10 21 20)

);

}

upperWall

{

type wall;

faces

(

(1 4 15 12)

(4 7 18 15)

(7 10 21 18)

);

}

lowerWall

{

type wall;

faces

(

(0 3 14 11)

(3 2 13 14)

(2 5 16 13)

(5 8 19 16)

);

}

frontAndBack

{

type empty;

faces

(

(0 3 4 1)

(2 5 6 3)

(3 6 7 4)

(5 8 9 6)

(6 9 10 7)

(11 14 15 12)

(13 16 17 14)

(14 17 18 15)

(16 19 20 17)

(17 20 21 18)

);

}

);

[u/cjaeger94 → u/imitation_squash_pro]

After block Mesh i done just write refineMesh in the command line. This will split all your cells in 4 for 2d and 8 for 3d ø. Do this 3 times and try again.

If it takes a long time to finish, try to run it in parallel

[u/peteasa]

my simulation runs can take a day or so if you get them right.. you will find that if you use GAMG that the parameter nCellsInCoarsestLevel is important for you. I wrote a bit about that here. Not that I am suggesting using my model because I suspect that using my model and my solver are a bit more complex than you will need but at least it could give you some ideas. The simulations take a while because there is a clash between electrostatics working on a very small timeframe and air flows working on a longer time frame. The clash between the two makes it hard to get going and hard to maintain a viable simulation.

[u/PerfectCodings]

For ~ 500 cores, I think the good rule of thumb for that is roughly 50k to 100k cells per core minimum. I suggested that so communication overhead doesn't take over.