StaticCondensation can also be a DofMap

[lindsayad]

My goal is to be able to do field splits on the statically condensed system. This should sit in draft since it has meaningful impacts like converting DofMap methods into virtuals until I have some useful demonstration of capability

[moosebuild]

Job Coverage, step Generate coverage on f82ac57 wanted to post the following:

Coverage

	fb6cd8	#4140 f82ac5
	Total	Total	+/-	New
Rate	63.59%	63.62%	+0.03%	87.36%
Hits	75705	75840	+135	394
Misses	43341	43360	+19	57

Diff coverage report

Full coverage report

Warnings

• New new line coverage rate 87.36% is less than the suggested 90.0%

This comment will be updated on new commits.

[roystgnr]

I think my only pointed concern here is the backwards-compatibility (and forwards-compatibility, I suppose; feel free to libmesh_experimental() anything that you think is likely still in flux), but my biggest vague request is that we need to get much more documentation on this sometime. E.g. I can't follow vector_fe_ex9 - we're querying system.has_static_condensation(), but I don't see anywhere we might be enabling static condensation in that example.

[lindsayad]

E.g. I can't follow vector_fe_ex9 - we're querying system.has_static_condensation(), but I don't see anywhere we might be enabling static condensation in that example.

Static condensation is setup from the command line in run.sh. I'll add a comment in the source

[lindsayad]

I've restored the matrix_build_type

[lindsayad]

Cannot reproduce this single test failure on my ubuntu container so I'm going to say this is a bad test specification. Will relax the tolerancing comparisons in MOOSE

[lindsayad]

this should be ready for further review

[roystgnr]

If @jwpeterson isn't still on vacation or buried by more urgent things to catch up on, then I'd like to give him time to look at this too, but if he doesn't chime in by Monday I'd say let's merge this then.

[jwpeterson]

then I'd like to give him time to look at this too,

To be honest, I haven't followed the design discussion here closely, so I'm going to defer to you guys' expertise.

It's a little concerning to me that we are making DofMap::dof_indices() virtual... in the past we have put some effort into squeezing every last drop of performance out of that function, since it is called millions of times and often showed up in profiling. At the same time, we also won't be directly measuring any slowdowns, since I removed the logging from DofMap::dof_indices() in be55a07.

So I guess my main request would be if you guys have any performance regression testing on CIVET, to keep an eye on that before/after this is merged just to get an idea of whether there is any overall performance loss.

[lindsayad]

Will definitely monitor our stuff in MOOSE. Doing some libmesh benchmarking

After this PR

real    25m11.291s
user    24m9.356s
sys     0m49.729s

Before this PR

real    25m26.384s
user    24m7.176s
sys     1m9.175s

So that looks the same to me

[roystgnr]

Huh. That sys difference is kind of huge, but IIRC that measures time spent in the kernel (e.g. waiting for the filesystem to report a read or write as complete), not anything this PR would have affected. A user time 0.2% slower might be real, but it's definitely within the LIBMESH_BENCHMARK margin of error and also within my personal margin of "I don't care".

Will definitely monitor our stuff in MOOSE.

IIRC the one application to worry about here is Relap-7 - at least I think that was the one commonly doing solves on a zillion Edge2 elements where the actual calculations per element were thin enough that every little bit of overhead showed up. A bunch of the micro-optimizations in things like dof_indices date back to my attempt to optimize that kind of workload 8 or 9 years ago.

[lindsayad]

@joshuahansel you'll need to help us monitor performance in THM. Perhaps we can setup some performance logging stuff for THM case(s) with what @loganharbour has been working on recently?

[joshuahansel]

@joshuahansel you'll need to help us monitor performance in THM. Perhaps we can setup some performance logging stuff for THM case(s) with what @loganharbour has been working on recently?

Sounds good, lemme know whenever the capability is ready.

[loganharbour]

@joshuahansel you'll need to help us monitor performance in THM. Perhaps we can setup some performance logging stuff for THM case(s) with what @loganharbour has been working on recently?

Sounds good, lemme know whenever the capability is ready.

Metrics are already being collected, although not used.

See https://github.com/idaholab/moose/tree/next/modules/combined/performance for examples. Test spec is named "performance" instead of "tests" which are ran on master: https://civet.inl.gov/job/2938624/