Skip to content

Petsc operators fixes - #3522

Open
dschwoerer wants to merge 4 commits into
nextfrom
petsc-operators-fixes
Open

dschwoerer wants to merge 4 commits into
nextfrom
petsc-operators-fixes

Conversation

@dschwoerer

Copy link
Copy Markdown
Contributor

Fixes: #3517
With this, at least for 1D meshes, the x parallelization works.

@totork Can you verify for fully 3D?

@totork

totork commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

For some reason when I run with this it spams the output with

7.500e+03          1       5.88e-02    12.1    0.0    0.0   80.4    7.4
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
8.000e+03          7       9.81e-02    41.4    0.0    0.1   54.0    4.6
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
8.500e+03          1       6.45e-02     4.9    0.0    0.0   94.5    0.6
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
9.000e+03          1       6.49e-02    11.1    0.0    0.0   88.2    0.6
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
9.500e+03          1       6.78e-02    10.6    0.0    0.0   88.8    0.6
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)
1.000e+04          1       6.10e-02    11.8    0.0    0.0   87.5    0.6

Run finished at  : Wed Oct 28 01:04:00 2026
Run time : 3 s
	Option datadir = data (default)
	Option settingsfile = BOUT.settings (default)
Writing options to file data/BOUT.settings
	Option time_report:show = 0 (default)
Finalising PETSc

And it does not seem to run aswell.

@dschwoerer

Copy link
Copy Markdown
Contributor Author
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)

Is coming from YBoundary being re-created all over again. Is that maybe from Div_par_K_Grad_par_mod with petsc operator?

What is the error? Same as before?

@totork

totork commented Oct 28, 2026

Copy link
Copy Markdown
Contributor
	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)

Is coming from YBoundary being re-created all over again. Is that maybe from Div_par_K_Grad_par_mod with petsc operator?

When i run with the current BOUT++ commit in Hermes-3, it does not spam this.

What is the error? Same as before?

Same error as before

@stirkas

stirkas commented Oct 29, 2026

Copy link
Copy Markdown

I happened to see this issue/PR and wanted to mention there is also a related issue with using the new petsc operator for thermal conduction with hermes3 (with use_petsc_operator=true).

On top of your change in petsc_operators.hxx which changed PETSC_DECIDE there I also had to change PETSC_DECIDE to n_local on line 453 in petsc_operators.cxx:

from

BOUT_DO_PETSC(MatSetSizes(*temp, n_local, PETSC_DECIDE, n_global, n_global));

to

BOUT_DO_PETSC(MatSetSizes(*temp, n_local, n_local, n_global, n_global));

@dschwoerer

dschwoerer commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks a lot, that looks indeed like it needs to be fixed.

@totork does it work now?

	Option mesh:outer_x = true (default)
	Option mesh:inner_x = false (default)

Is coming from YBoundary being re-created all over again. Is that maybe from Div_par_K_Grad_par_mod with petsc operator?

When i run with the current BOUT++ commit in Hermes-3, it does not spam this.

Like the current bout++/next? Or what ever hermes-3 currently tracks in master?

Co-authored-by: Stefan Tirkas <tirkas1@llnl.gov>
@totork

totork commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Seems to work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Petsc operator error when running with more than one core

3 participants